關于Android的 DiskLruCache磁盤緩存機制原理
一、為什么用DiskLruCache
1、LruCache和DiskLruCache
LruCache和DiskLruCache兩者都是利用到LRU算法,通過LRU算法對緩存進行管理,以最近最少使用作為管理的依據(jù),刪除最近最少使用的數(shù)據(jù),保留最近最常用的數(shù)據(jù);
LruCache運用于內存緩存,而DiskLruCache是存儲設備緩存;
2、為何使用DiskLruCache
離線數(shù)據(jù)存在的意義,當無網絡或者是網絡狀況不好時,APP依然具備部分功能是一種很好的用戶體驗;
假設網易新聞這類新聞客戶端,數(shù)據(jù)完全存儲在緩存中而不使用DiskLruCache技術存儲,那么當客戶端被銷毀,緩存被釋放,意味著再次打開APP將是一片空白;
另外DiskLruCache技術也可為app“離線閱讀”這一功能做技術支持;
DiskLruCache的存儲路徑是可以自定義的,不過也可以是默認的存儲路徑,而默認的存儲路徑一般是這樣的:/sdcard/Android/data/包名/cache,包名是指APP的包名。我們可以在手機上打開,瀏覽這一路徑;
二、DiskLruCache使用
1、添加依賴
// add dependence implementation 'com.jakewharton:disklrucache:2.0.2'
2、創(chuàng)建DiskLruCache對象
/* * directory – 緩存目錄 * appVersion - 緩存版本 * valueCount – 每個key對應value的個數(shù) * maxSize – 緩存大小的上限 */ DiskLruCache diskLruCache = DiskLruCache.open(directory, 1, 1, 1024 * 1024 * 10);
3、添加 / 獲取 緩存(一對一)
/**
* 添加一條緩存,一個key對應一個value
*/
public void addDiskCache(String key, String value) throws IOException {
File cacheDir = context.getCacheDir();
DiskLruCache diskLruCache = DiskLruCache.open(cacheDir, 1, 1, 1024 * 1024 * 10);
DiskLruCache.Editor editor = diskLruCache.edit(key);
// index與valueCount對應,分別為0,1,2...valueCount-1
editor.newOutputStream(0).write(value.getBytes());
editor.commit();
diskLruCache.close();
}
/**
* 獲取一條緩存,一個key對應一個value
*/
public void getDiskCache(String key) throws IOException {
File directory = context.getCacheDir();
DiskLruCache diskLruCache = DiskLruCache.open(directory, 1, 1, 1024 * 1024 * 10);
String value = diskLruCache.get(key).getString(0);
diskLruCache.close();
}
4、添加 / 獲取 緩存(一對多)
/**
* 添加一條緩存,1個key對應2個value
*/
public void addDiskCache(String key, String value1, String value2) throws IOException {
File directory = context.getCacheDir();
DiskLruCache diskLruCache = DiskLruCache.open(directory, 1, 2, 1024 * 1024 * 10);
DiskLruCache.Editor editor = diskLruCache.edit(key);
editor.newOutputStream(0).write(value1.getBytes());
editor.newOutputStream(1).write(value2.getBytes());
editor.commit();
diskLruCache.close();
}
/**
* 添加一條緩存,1個key對應2個value
*/
public void getDiskCache(String key) throws IOException {
File directory = context.getCacheDir();
DiskLruCache diskLruCache = DiskLruCache.open(directory, 1, 2, 1024);
DiskLruCache.Snapshot snapshot = diskLruCache.get(key);
String value1 = snapshot.getString(0);
String value2 = snapshot.getString(1);
diskLruCache.close();
}
三、源碼分析

1、open()
DiskLruCache的構造方法是private修飾,這也就是告訴我們,不能通過new DiskLruCache來獲取實例,構造方法如下:
private DiskLruCache(File directory, int appVersion, int valueCount, long maxSize) {
this.directory = directory;
this.appVersion = appVersion;
this.journalFile = new File(directory, JOURNAL_FILE);
this.journalFileTmp = new File(directory, JOURNAL_FILE_TEMP);
this.journalFileBackup = new File(directory, JOURNAL_FILE_BACKUP);
this.valueCount = valueCount;
this.maxSize = maxSize;
}
但是提供了open()方法,供我們獲取DiskLruCache的實例,open方法如下:
/**
* Opens the cache in {@code directory}, creating a cache if none exists
* there.
*
* @param directory a writable directory
* @param valueCount the number of values per cache entry. Must be positive.
* @param maxSize the maximum number of bytes this cache should use to store
* @throws IOException if reading or writing the cache directory fails
*/
public static DiskLruCache open(File directory, int appVersion, int valueCount, long maxSize)
throws IOException {
if (maxSize <= 0) {
throw new IllegalArgumentException("maxSize <= 0");
}
if (valueCount <= 0) {
throw new IllegalArgumentException("valueCount <= 0");
}
// If a bkp file exists, use it instead.
//看備份文件是否存在
File backupFile = new File(directory, JOURNAL_FILE_BACKUP);
//如果備份文件存在,并且日志文件也存在,就把備份文件刪除
//如果備份文件存在,日志文件不存在,就把備份文件重命名為日志文件
if (backupFile.exists()) {
File journalFile = new File(directory, JOURNAL_FILE);
// If journal file also exists just delete backup file.
//
if (journalFile.exists()) {
backupFile.delete();
} else {
renameTo(backupFile, journalFile, false);
}
}
// Prefer to pick up where we left off.
//初始化DiskLruCache,包括,大小,版本,路徑,key對應多少value
DiskLruCache cache = new DiskLruCache(directory, appVersion, valueCount, maxSize);
//如果日志文件存在,就開始賭文件信息,并返回
//主要就是構建entry列表
if (cache.journalFile.exists()) {
try {
cache.readJournal();
cache.processJournal();
return cache;
} catch (IOException journalIsCorrupt) {
System.out
.println("DiskLruCache "
+ directory
+ " is corrupt: "
+ journalIsCorrupt.getMessage()
+ ", removing");
cache.delete();
}
}
//不存在就新建一個
// Create a new empty cache.
directory.mkdirs();
cache = new DiskLruCache(directory, appVersion, valueCount, maxSize);
cache.rebuildJournal();
return cache;
}
open函數(shù):如果日志文件存在,直接去構建entry列表;如果不存在,就構建日志文件;
2、rebuildJournal()
構建文件:
//這個就是我們可以直接在disk里面看到的journal文件 主要就是對他的操作
private final File journalFile;
//journal文件的temp 緩存文件,一般都是先構建這個緩存文件,等待構建完成以后將這個緩存文件重新命名為journal
private final File journalFileTmp;
/**
* Creates a new journal that omits redundant information. This replaces the
* current journal if it exists.
*/
private synchronized void rebuildJournal() throws IOException {
if (journalWriter != null) {
journalWriter.close();
}
//指向journalFileTmp這個日志文件的緩存
Writer writer = new BufferedWriter(
new OutputStreamWriter(new FileOutputStream(journalFileTmp), Util.US_ASCII));
try {
writer.write(MAGIC);
writer.write("\n");
writer.write(VERSION_1);
writer.write("\n");
writer.write(Integer.toString(appVersion));
writer.write("\n");
writer.write(Integer.toString(valueCount));
writer.write("\n");
writer.write("\n");
for (Entry entry : lruEntries.values()) {
if (entry.currentEditor != null) {
writer.write(DIRTY + ' ' + entry.key + '\n');
} else {
writer.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
}
}
} finally {
writer.close();
}
if (journalFile.exists()) {
renameTo(journalFile, journalFileBackup, true);
}
//所以這個地方 構建日志文件的流程主要就是先構建出日志文件的緩存文件,如果緩存構建成功 那就直接重命名這個緩存文件,這樣做好處在哪里?
renameTo(journalFileTmp, journalFile, false);
journalFileBackup.delete();
//這里也是把寫入日志文件的writer初始化
journalWriter = new BufferedWriter(
new OutputStreamWriter(new FileOutputStream(journalFile, true), Util.US_ASCII));
}
再來看當日志文件存在的時候,做了什么
3、readJournal()
private void readJournal() throws IOException {
StrictLineReader reader = new StrictLineReader(new FileInputStream(journalFile), Util.US_ASCII);
try {
//讀日志文件的頭信息
String magic = reader.readLine();
String version = reader.readLine();
String appVersionString = reader.readLine();
String valueCountString = reader.readLine();
String blank = reader.readLine();
if (!MAGIC.equals(magic)
|| !VERSION_1.equals(version)
|| !Integer.toString(appVersion).equals(appVersionString)
|| !Integer.toString(valueCount).equals(valueCountString)
|| !"".equals(blank)) {
throw new IOException("unexpected journal header: [" + magic + ", " + version + ", "
+ valueCountString + ", " + blank + "]");
}
//這里開始,就開始讀取日志信息
int lineCount = 0;
while (true) {
try {
//構建LruEntries entry列表
readJournalLine(reader.readLine());
lineCount++;
} catch (EOFException endOfJournal) {
break;
}
}
redundantOpCount = lineCount - lruEntries.size();
// If we ended on a truncated line, rebuild the journal before appending to it.
if (reader.hasUnterminatedLine()) {
rebuildJournal();
} else {
//初始化寫入文件的writer
journalWriter = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream(journalFile, true), Util.US_ASCII));
}
} finally {
Util.closeQuietly(reader);
}
}
然后看下這個函數(shù)里面的幾個主要變量:
//每個entry對應的緩存文件的格式 一般為1,也就是一個key,對應幾個緩存,一般設為1,key-value一一對應的關系
private final int valueCount;
private long size = 0;
//這個是專門用于寫入日志文件的writer
private Writer journalWriter;
//這個集合應該不陌生了,
private final LinkedHashMap<String, Entry> lruEntries =
new LinkedHashMap<String, Entry>(0, 0.75f, true);
//這個值大于一定數(shù)目時 就會觸發(fā)對journal文件的清理了
private int redundantOpCount;
下面就看下entry這個實體類的內部結構
private final class Entry {
private final String key;
/**
* Lengths of this entry's files.
* 這個entry中 每個文件的長度,這個數(shù)組的長度為valueCount 一般都是1
*/
private final long[] lengths;
/**
* True if this entry has ever been published.
* 曾經被發(fā)布過 那他的值就是true
*/
private boolean readable;
/**
* The ongoing edit or null if this entry is not being edited.
* 這個entry對應的editor
*/
private Editor currentEditor;
@Override
public String toString() {
return "Entry{" +
"key='" + key + '\'' +
", lengths=" + Arrays.toString(lengths) +
", readable=" + readable +
", currentEditor=" + currentEditor +
", sequenceNumber=" + sequenceNumber +
'}';
}
/**
* The sequence number of the most recently committed edit to this entry.
* 最近編輯他的序列號
*/
private long sequenceNumber;
private Entry(String key) {
this.key = key;
this.lengths = new long[valueCount];
}
public String getLengths() throws IOException {
StringBuilder result = new StringBuilder();
for (long size : lengths) {
result.append(' ').append(size);
}
return result.toString();
}
/**
* Set lengths using decimal numbers like "10123".
*/
private void setLengths(String[] strings) throws IOException {
if (strings.length != valueCount) {
throw invalidLengths(strings);
}
try {
for (int i = 0; i < strings.length; i++) {
lengths[i] = Long.parseLong(strings[i]);
}
} catch (NumberFormatException e) {
throw invalidLengths(strings);
}
}
private IOException invalidLengths(String[] strings) throws IOException {
throw new IOException("unexpected journal line: " + java.util.Arrays.toString(strings));
}
//臨時文件創(chuàng)建成功了以後 就會重命名為正式文件了
public File getCleanFile(int i) {
Log.v("getCleanFile","getCleanFile path=="+new File(directory, key + "." + i).getAbsolutePath());
return new File(directory, key + "." + i);
}
//tmp開頭的都是臨時文件
public File getDirtyFile(int i) {
Log.v("getDirtyFile","getDirtyFile path=="+new File(directory, key + "." + i + ".tmp").getAbsolutePath());
return new File(directory, key + "." + i + ".tmp");
}
}
DiskLruCache的open函數(shù)的主要流程就基本走完了;
4、get()
/**
* Returns a snapshot of the entry named {@code key}, or null if it doesn't
* exist is not currently readable. If a value is returned, it is moved to
* the head of the LRU queue.
* 通過key獲取對應的snapshot
*/
public synchronized Snapshot get(String key) throws IOException {
checkNotClosed();
validateKey(key);
Entry entry = lruEntries.get(key);
if (entry == null) {
return null;
}
if (!entry.readable) {
return null;
}
// Open all streams eagerly to guarantee that we see a single published
// snapshot. If we opened streams lazily then the streams could come
// from different edits.
InputStream[] ins = new InputStream[valueCount];
try {
for (int i = 0; i < valueCount; i++) {
ins[i] = new FileInputStream(entry.getCleanFile(i));
}
} catch (FileNotFoundException e) {
// A file must have been deleted manually!
for (int i = 0; i < valueCount; i++) {
if (ins[i] != null) {
Util.closeQuietly(ins[i]);
} else {
break;
}
}
return null;
}
redundantOpCount++;
//在取得需要的文件以后 記得在日志文件里增加一條記錄 并檢查是否需要重新構建日志文件
journalWriter.append(READ + ' ' + key + '\n');
if (journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
return new Snapshot(key, entry.sequenceNumber, ins, entry.lengths);
}
5、validateKey
private void validateKey(String key) {
Matcher matcher = LEGAL_KEY_PATTERN.matcher(key);
if (!matcher.matches()) {
throw new IllegalArgumentException("keys must match regex "
+ STRING_KEY_PATTERN + ": \"" + key + "\"");
}
}
這里是對存儲entry的map的key做了正則驗證,所以key一定要用md5加密,因為有些特殊字符驗證不能通過;
然后看這句代碼對應的:
if (journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
對應的回調函數(shù)是:
/** This cache uses a single background thread to evict entries. */
final ThreadPoolExecutor executorService =
new ThreadPoolExecutor(0, 1, 60L, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>());
private final Callable<Void> cleanupCallable = new Callable<Void>() {
public Void call() throws Exception {
synchronized (DiskLruCache.this) {
if (journalWriter == null) {
return null; // Closed.
}
trimToSize();
if (journalRebuildRequired()) {
rebuildJournal();
redundantOpCount = 0;
}
}
return null;
}
};
其中再來看看trimTOSize()的狀態(tài)
6、trimTOSize()
private void trimToSize() throws IOException {
while (size > maxSize) {
Map.Entry<String, Entry> toEvict = lruEntries.entrySet().iterator().next();
remove(toEvict.getKey());
}
}
就是檢測總緩存是否超過了限制數(shù)量,
再來看journalRebuildRequired函數(shù)
7、journalRebuildRequired()
/**
* We only rebuild the journal when it will halve the size of the journal
* and eliminate at least 2000 ops.
*/
private boolean journalRebuildRequired() {
final int redundantOpCompactThreshold = 2000;
return redundantOpCount >= redundantOpCompactThreshold //
&& redundantOpCount >= lruEntries.size();
}
就是校驗redundantOpCount是否超出了范圍,如果是,就重構日志文件;
最后看get函數(shù)的返回值 new Snapshot()
/** A snapshot of the values for an entry. */
//這個類持有該entry中每個文件的inputStream 通過這個inputStream 可以讀取他的內容
public final class Snapshot implements Closeable {
private final String key;
private final long sequenceNumber;
private final InputStream[] ins;
private final long[] lengths;
private Snapshot(String key, long sequenceNumber, InputStream[] ins, long[] lengths) {
this.key = key;
this.sequenceNumber = sequenceNumber;
this.ins = ins;
this.lengths = lengths;
}
/**
* Returns an editor for this snapshot's entry, or null if either the
* entry has changed since this snapshot was created or if another edit
* is in progress.
*/
public Editor edit() throws IOException {
return DiskLruCache.this.edit(key, sequenceNumber);
}
/** Returns the unbuffered stream with the value for {@code index}. */
public InputStream getInputStream(int index) {
return ins[index];
}
/** Returns the string value for {@code index}. */
public String getString(int index) throws IOException {
return inputStreamToString(getInputStream(index));
}
/** Returns the byte length of the value for {@code index}. */
public long getLength(int index) {
return lengths[index];
}
public void close() {
for (InputStream in : ins) {
Util.closeQuietly(in);
}
}
}
到這里就明白了get最終返回的其實就是entry根據(jù)key 來取的snapshot對象,這個對象直接把inputStream暴露給外面;
8、save的過程
public Editor edit(String key) throws IOException {
return edit(key, ANY_SEQUENCE_NUMBER);
}
//根據(jù)傳進去的key 創(chuàng)建一個entry 并且將這個key加入到entry的那個map里 然后創(chuàng)建一個對應的editor
//同時在日志文件里加入一條對該key的dirty記錄
private synchronized Editor edit(String key, long expectedSequenceNumber) throws IOException {
//因為這里涉及到寫文件 所以要先校驗一下寫日志文件的writer 是否被正確的初始化
checkNotClosed();
//這個地方是校驗 我們的key的,通常來說 假設我們要用這個緩存來存一張圖片的話,我們的key 通常是用這個圖片的
//網絡地址 進行md5加密,而對這個key的格式在這里是有要求的 所以這一步就是驗證key是否符合規(guī)范
validateKey(key);
Entry entry = lruEntries.get(key);
if (expectedSequenceNumber != ANY_SEQUENCE_NUMBER && (entry == null
|| entry.sequenceNumber != expectedSequenceNumber)) {
return null; // Snapshot is stale.
}
if (entry == null) {
entry = new Entry(key);
lruEntries.put(key, entry);
} else if (entry.currentEditor != null) {
return null; // Another edit is in progress.
}
Editor editor = new Editor(entry);
entry.currentEditor = editor;
// Flush the journal before creating files to prevent file leaks.
journalWriter.write(DIRTY + ' ' + key + '\n');
journalWriter.flush();
return editor;
}
然后取得輸出流
public OutputStream newOutputStream(int index) throws IOException {
if (index < 0 || index >= valueCount) {
throw new IllegalArgumentException("Expected index " + index + " to "
+ "be greater than 0 and less than the maximum value count "
+ "of " + valueCount);
}
synchronized (DiskLruCache.this) {
if (entry.currentEditor != this) {
throw new IllegalStateException();
}
if (!entry.readable) {
written[index] = true;
}
File dirtyFile = entry.getDirtyFile(index);
FileOutputStream outputStream;
try {
outputStream = new FileOutputStream(dirtyFile);
} catch (FileNotFoundException e) {
// Attempt to recreate the cache directory.
directory.mkdirs();
try {
outputStream = new FileOutputStream(dirtyFile);
} catch (FileNotFoundException e2) {
// We are unable to recover. Silently eat the writes.
return NULL_OUTPUT_STREAM;
}
}
return new FaultHidingOutputStream(outputStream);
}
}
注意這個index 其實一般傳0 就可以了,DiskLruCache 認為 一個key 下面可以對應多個文件,這些文件 用一個數(shù)組來存儲,所以正常情況下,我們都是
一個key 對應一個緩存文件 所以傳0
//tmp開頭的都是臨時文件
public File getDirtyFile(int i) {
return new File(directory, key + "." + i + ".tmp");
}
然后你這邊就能看到,這個輸出流,實際上是tmp 也就是緩存文件的 .tmp 也就是緩存文件的 緩存文件 輸出流;
這個流 我們寫完畢以后 就要commit;
public void commit() throws IOException {
if (hasErrors) {
completeEdit(this, false);
remove(entry.key); // The previous entry is stale.
} else {
completeEdit(this, true);
}
committed = true;
}
這個就是根據(jù)緩存文件的大小 更新disklrucache的總大小 然后再日志文件里對該key加入clean的log
//最后判斷是否超過最大的maxSize 以便對緩存進行清理
private synchronized void completeEdit(Editor editor, boolean success) throws IOException {
Entry entry = editor.entry;
if (entry.currentEditor != editor) {
throw new IllegalStateException();
}
// If this edit is creating the entry for the first time, every index must have a value.
if (success && !entry.readable) {
for (int i = 0; i < valueCount; i++) {
if (!editor.written[i]) {
editor.abort();
throw new IllegalStateException("Newly created entry didn't create value for index " + i);
}
if (!entry.getDirtyFile(i).exists()) {
editor.abort();
return;
}
}
}
for (int i = 0; i < valueCount; i++) {
File dirty = entry.getDirtyFile(i);
if (success) {
if (dirty.exists()) {
File clean = entry.getCleanFile(i);
dirty.renameTo(clean);
long oldLength = entry.lengths[i];
long newLength = clean.length();
entry.lengths[i] = newLength;
size = size - oldLength + newLength;
}
} else {
deleteIfExists(dirty);
}
}
redundantOpCount++;
entry.currentEditor = null;
if (entry.readable | success) {
entry.readable = true;
journalWriter.write(CLEAN + ' ' + entry.key + entry.getLengths() + '\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.write(REMOVE + ' ' + entry.key + '\n');
}
journalWriter.flush();
if (size > maxSize || journalRebuildRequired()) {
executorService.submit(cleanupCallable);
}
}
commit以后 就會把tmp文件轉正 ,重命名為 真正的緩存文件了;
這個里面的流程和日志文件的rebuild 是差不多的,都是為了防止寫文件的出問題。所以做了這樣的冗余處理;
總結:
DiskLruCache,利用一個journal文件,保證了保證了cache實體的可用性(只有CLEAN的可用),且獲取文件的長度的時候可以通過在該文件的記錄中讀取。
利用FaultHidingOutputStream對FileOutPutStream很好的對寫入文件過程中是否發(fā)生錯誤進行捕獲,而不是讓用戶手動去調用出錯后的處理方法;
到此這篇關于關于Android DiskLruCache的磁盤緩存機制原理的文章就介紹到這了,更多相關Android DiskLruCache磁盤緩存機制原理內容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關文章希望大家以后多多支持腳本之家!
相關文章
實例詳解android studio如何導入.so文件的方法
通過實例給大家詳細講解了如何在android studio如何導入.so文件以及中間遇到的問題解決辦法,需要的讀者們可以仔細學習一下。2017-12-12
Android開發(fā)實現(xiàn)的Intent跳轉工具類實例
這篇文章主要介紹了Android開發(fā)實現(xiàn)的Intent跳轉工具類,簡單描述了Intent組件的功能并結合實例形式給出了頁面跳轉、拍照、圖片調用等相關操作技巧,需要的朋友可以參考下2017-11-11
Android中Handler實現(xiàn)倒計時的兩種方式
本篇文章主要介紹了Android中Handler實現(xiàn)倒計時的兩種方式,小編覺得挺不錯的,現(xiàn)在分享給大家,也給大家做個參考。一起跟隨小編過來看看吧2017-07-07
Android使用CardView作為RecyclerView的Item并實現(xiàn)拖拽和左滑刪除
這篇文章主要介紹了Android使用CardView作為RecyclerView的Item并實現(xiàn)拖拽和左滑刪除,具有一定的參考價值,感興趣的小伙伴們可以參考一下2017-11-11
Android 自定義TextView去除paddingTop和paddingBottom
這篇文章主要介紹了Android 自定義TextView去除paddingTop和paddingBottom的相關資料,這里提供實例來幫助大家實現(xiàn)這樣的功能,需要的朋友可以參考下2017-09-09

