OKHTTP分享二快取策略
與快取有關的Header
Expires
Expires: Thu, 12 Jan 2017 11:01:33 GMT
表示到期時間,一般用在response報文中,當超過此時間響應將被認為是無效的而需要網路連線,反之直接使用快取條件GET
客戶端傳送條件get請求,如果快取是有效的,則返回304 Not Modifiled,否則才返回body。ETag
ETag是對資原始檔的一種摘要,當客戶端第一次請求某個物件,伺服器在響應頭返回
ETag: “5694c7ef-24dc”
客戶端再次請求時,通過傳送
If-None-Match:”5694c7ef-24dc”
交給伺服器進行判斷,如果仍然可以快取使用,伺服器就直接返回304 Not ModifiledVary
Vary: *
告訴客戶端和快取伺服器不要快取任何資訊
Vary: header-name, header-name, …
逗號分隔的一系列http頭部名稱,用於確定快取是否可用
作用:動態服務,防止客戶端誤使用了用於pc端的快取。即使請求的是相同資源,因Vary指定的首部欄位不同,也必須從源伺服器請求- Cache Control
客戶端可以在HTTP**請求**中使用的標準 Cache-Control 指令
Cache-Control: max-age=
Cache-Control: max-stale[=]
Cache-Control: min-fresh=
Cache-control: no-cache
Cache-control: no-store
Cache-control: no-transform
Cache-control: only-if-cached
伺服器
Cache-control: must-revalidate
Cache-control: no-cache
Cache-control: no-store
Cache-control: no-transform
Cache-control: public
Cache-control: private
Cache-control: proxy-revalidate
Cache-Control: max-age=
Cache-control: s-maxage=
1) 可快取性
public
表明其他使用者也可以利用快取。
private
表明快取只對單個使用者有效,不能作為共享快取。
no-cache
強制所有快取了該響應的快取使用者,在使用已儲存的快取資料前,傳送帶驗證的請求到原始伺服器
no-store
快取不應儲存有關客戶端請求或伺服器響應的任何內容。
only-if-cached
表明客戶端只接受已快取的響應,並且不要向原始伺服器檢查是否有更新的拷貝(相當於禁止使用網路連線)
2) 到期
max-age=<seconds>
設定快取儲存的最大週期,超過這個時間快取被認為過期(單位秒)。與Expires相反,時間是相對於請求的時間。
s-maxage=<seconds>
覆蓋max-age 或者 Expires 頭,但是僅適用於共享快取(比如各個代理),並且私有快取中它被忽略。
max-stale[=<seconds>]
表明客戶端願意接收一個已經過期的資源,且可選地指定響應不能超過的過時時間。
min-fresh=<seconds>
表示客戶端希望在指定的時間內獲取最新的響應。
3) 有效性
must-revalidate
快取必須在使用之前驗證舊資源的狀態,並且不可使用過期資源。
proxy-revalidate
與must-revalidate作用相同,但它僅適用於共享快取(例如代理),並被私有快取忽略。
immutable
表示響應正文不會隨時間而改變。資源(如果未過期)在伺服器上不發生改變,因此客戶端不應傳送重新驗證請求頭(例如If-None-Match或If-Modified-Since)來檢查更新,即使使用者顯式地重新整理頁面。
快取策略
CacheInterceptor
@Override public Response intercept(Chain chain) throws IOException {
Response cacheCandidate = cache != null
? cache.get(chain.request())
: null;
long now = System.currentTimeMillis();
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse;
...
if (cacheCandidate != null && cacheResponse == null) {
closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
}
// If we're forbidden from using the network and the cache is insufficient, fail.
if (networkRequest == null && cacheResponse == null) {
return new Response.Builder()
.request(chain.request())
.protocol(Protocol.HTTP_1_1)
.code(504)
.message("Unsatisfiable Request (only-if-cached)")
.body(Util.EMPTY_RESPONSE)
.sentRequestAtMillis(-1L)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
}
// If we don't need the network, we're done.
if (networkRequest == null) {
return cacheResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build();
}
Response networkResponse = null;
try {
networkResponse = chain.proceed(networkRequest);
} finally {
// If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null && cacheCandidate != null) {
closeQuietly(cacheCandidate.body());
}
}
// If we have a cache response too, then we're doing a conditional get.
if (cacheResponse != null) {
if (networkResponse.code() == HTTP_NOT_MODIFIED) {
Response response = cacheResponse.newBuilder()
.headers(combine(cacheResponse.headers(), networkResponse.headers()))
.sentRequestAtMillis(networkResponse.sentRequestAtMillis())
.receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
networkResponse.body().close();
// Update the cache after combining headers but before stripping the
// Content-Encoding header (as performed by initContentStream()).
cache.trackConditionalCacheHit();
cache.update(cacheResponse, response);
return response;
} else {
closeQuietly(cacheResponse.body());
}
}
Response response = networkResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
if (cache != null) {
if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
// Offer this request to the cache.
CacheRequest cacheRequest = cache.put(response);
return cacheWritingResponse(cacheRequest, response);
}
if (HttpMethod.invalidatesCache(networkRequest.method())) {
try {
cache.remove(networkRequest);
} catch (IOException ignored) {
// The cache cannot be written.
}
}
}
return response;
}
主要做三件事:
- 根據Request和之前快取的Response得到CacheStrategy
- 根據CacheStrategy決定是請求網路還是直接返回快取
- 如果2中決定請求網路,則在這一步將返回的網路響應和本地快取對比,對本地快取進行增刪改操作
CacheStrategy
public final class CacheStrategy {
/** The request to send on the network, or null if this call doesn't use the network. */
public final @Nullable Request networkRequest;
/** The cached response to return or validate; or null if this call doesn't use a cache. */
public final @Nullable Response cacheResponse;
CacheStrategy(Request networkRequest, Response cacheResponse) {
this.networkRequest = networkRequest;
this.cacheResponse = cacheResponse;
}
...
}
- 作用:將“請求”和舊的快取進行分析比較,決定是發起網路請求還是直接使用快取。具體來說,根據networkRequest和cacheResponse是否為空執行不同的動作
networkRequest | cacheResponse | 結果 |
---|---|---|
null | null | 禁止進行網路請求,但快取不存在或者過期,只能返回503錯誤 |
null | non-null | 快取可以使用,直接返回快取,不用請求網路 |
non-null | null | 快取不存在或者過期,直接訪問網路 |
non-null | non-null | 條件get,請求網路 |
CacheStrategy的加工過程
主要是讀取請求頭和響應頭中有關快取的HTTP 欄位生成CacheStrategy物件,可結合開頭與快取有關的Header檢視原始碼,這裡不再贅述
快取UML類圖
Cache
- OkHttp的快取“門面”,對外提供增刪改查方法
- 通過內部的DiskLruCache來管理快取物件
- 每一條快取記錄的key是url的md5值
OkHttp的快取檔案,如下圖
public Cache(File directory, long maxSize) {
this(directory, maxSize, FileSystem.SYSTEM);
}
//OkHttpClient指定Cache
public Builder cache(@Nullable Cache cache) {
this.cache = cache;
this.internalCache = null;
return this;
}
使用時需指定快取目錄和快取大小上限
DiskLruCache
Cache內部通過DiskLruCache管理cache在檔案系統層面的建立,讀取,自動清理等工作
public final class DiskLruCache implements Closeable, Flushable {
static final String JOURNAL_FILE = "journal";
static final String JOURNAL_FILE_TEMP = "journal.tmp";
static final String JOURNAL_FILE_BACKUP = "journal.bkp";
static final String MAGIC = "libcore.io.DiskLruCache";
static final String VERSION_1 = "1";
static final long ANY_SEQUENCE_NUMBER = -1;
static final Pattern LEGAL_KEY_PATTERN = Pattern.compile("[a-z0-9_-]{1,120}");
private static final String CLEAN = "CLEAN";//快取記錄的4種狀態
private static final String DIRTY = "DIRTY";
private static final String REMOVE = "REMOVE";
private static final String READ = "READ";
final FileSystem fileSystem;
final File directory;
private final File journalFile;//快取日誌
private final File journalFileTmp;
private final File journalFileBackup;
private final int appVersion;
private long maxSize;
final int valueCount;
private long size = 0;
BufferedSink journalWriter;
final LinkedHashMap<String, Entry> lruEntries = new LinkedHashMap<>(0, 0.75f, true);//Entry是快取檔案的描述
...
}
DiskLruCache通過journal檔案和lruEntries(LinkedHashMap)共同管理快取檔案
journal
作用:
- 在程序啟動時重建DiskLruCache(lruEntries),將磁碟中的快取檔案和url對應關係載入到記憶體中
- 記錄和跟蹤快取檔案的狀態
- 保證對快取檔案讀寫操作的原子性
- 前5行固定不變,分別為:常量:libcore.io.DiskLruCache;diskLruCache版本;應用程式版本;valueCount(表示一個Entry對應的檔案數量,在Cache中為2),空行
- 接下來每一行對應一個cache entry的一次狀態記錄,其格式為:[狀態(DIRTY,CLEAN,READ,REMOVE),key(url的md5值),檔案大小(兩個檔案:響應頭和響應體)]。中間以空格隔開。
- DIRTY 表示快取正在被插入、更新或刪除,在磁碟中操作成功後會有一條對應的CLEAN或REMOVE記錄。否則該DIRTY記錄無效(操作未成功,被異常中斷過)。相當於DIRTY對應的只是臨時檔案。
- CLEAN 表示該條快取是一個有效的記錄,可以正常讀取(get)。
- READ 表示該條快取最近被讀取過
- REMOVE 表示該條記錄對應的快取檔案已經被刪除了
DiskLruCache.Entry
private final class Entry {
final String key;
/** Lengths of this entry's files. */
final long[] lengths;
final File[] cleanFiles;
final File[] dirtyFiles;
/** True if this entry has ever been published. */
boolean readable;
/** The ongoing edit or null if this entry is not being edited. */
Editor currentEditor;
- Entry是快取檔案在檔案系統層面的引用
- key是url的md5值
- 每個Entry可以對應多個檔案,具體由DiskLruCache的valueCount決定,預設是2(響應頭和響應體)
- cleanFiles代表正常有效的可讀快取,dirtyFiles表示快取檔案正在被建立或更新(但還沒完成,只是臨時檔案),操作完成後會將dirtyFiles重新命名為cleanFiles,並將舊的cleanFiles刪除
Entry(String key) {
this.key = key;
lengths = new long[valueCount];
cleanFiles = new File[valueCount];
dirtyFiles = new File[valueCount];
// The names are repetitive so re-use the same builder to avoid allocations.
StringBuilder fileBuilder = new StringBuilder(key).append('.');
int truncateTo = fileBuilder.length();
for (int i = 0; i < valueCount; i++) {
fileBuilder.append(i);
cleanFiles[i] = new File(directory, fileBuilder.toString());
fileBuilder.append(".tmp");
dirtyFiles[i] = new File(directory, fileBuilder.toString());
fileBuilder.setLength(truncateTo);
}
}
生成Entry物件的同時,生成以url的md5命名的File物件(但快取檔案還未寫入磁碟)。檔名如下圖:
DiskLruCache的初始化
public synchronized void initialize() throws IOException {
assert Thread.holdsLock(this);
if (initialized) {
return; // Already initialized.
}
// If a bkp file exists, use it instead.
if (fileSystem.exists(journalFileBackup)) {
// If journal file also exists just delete backup file.
if (fileSystem.exists(journalFile)) {
fileSystem.delete(journalFileBackup);
} else {
fileSystem.rename(journalFileBackup, journalFile);
}
}
// Prefer to pick up where we left off.
if (fileSystem.exists(journalFile)) {
try {
readJournal();
processJournal();
initialized = true;
return;
} catch (IOException journalIsCorrupt) {
Platform.get().log(WARN, "DiskLruCache " + directory + " is corrupt: "
+ journalIsCorrupt.getMessage() + ", removing", journalIsCorrupt);
}
// The cache is corrupted, attempt to delete the contents of the directory. This can throw and
// we'll let that propagate out as it likely means there is a severe filesystem problem.
try {
delete();
} finally {
closed = false;
}
}
rebuildJournal();
initialized = true;
}
- 根據journal檔案重建lruEntries,並刪除dirty快取
- 如果journal檔案初始化失敗會重建journal檔案
private void readJournal() throws IOException {
BufferedSource source = Okio.buffer(fileSystem.source(journalFile));
try {
String magic = source.readUtf8LineStrict();
String version = source.readUtf8LineStrict();
String appVersionString = source.readUtf8LineStrict();
String valueCountString = source.readUtf8LineStrict();
String blank = source.readUtf8LineStrict();
if (!MAGIC.equals(magic)
|| !VERSION_1.equals(version)
|| !Integer.toString(appVersion).equals(appVersionString)
|| !Integer.toString(valueCount).equals(valueCountString)
|| !"".equals(blank)) {
throw new IOException("unexpected journal header: [" + magic + ", " + version + ", "
+ valueCountString + ", " + blank + "]");
}
int lineCount = 0;
while (true) {
try {
readJournalLine(source.readUtf8LineStrict());
lineCount++;
} catch (EOFException endOfJournal) {
break;
}
}
redundantOpCount = lineCount - lruEntries.size();
// If we ended on a truncated line, rebuild the journal before appending to it.
if (!source.exhausted()) {
rebuildJournal();
} else {
journalWriter = newJournalWriter();
}
} finally {
Util.closeQuietly(source);
}
}
readJournal方法開頭校驗Journal檔案的前面5行是否正確,如果不正確丟擲異常,並在之後重建Journal檔案。然後通過readJournalLine方法,逐行讀取Journal檔案的每條快取記錄,並更新對應Entry
private void processJournal() throws IOException {
fileSystem.delete(journalFileTmp);
for (Iterator<Entry> i = lruEntries.values().iterator(); i.hasNext(); ) {
Entry entry = i.next();
if (entry.currentEditor == null) {
for (int t = 0; t < valueCount; t++) {
size += entry.lengths[t];
}
} else {
entry.currentEditor = null;
for (int t = 0; t < valueCount; t++) {
fileSystem.delete(entry.cleanFiles[t]);
fileSystem.delete(entry.dirtyFiles[t]);
}
i.remove();
}
}
}
processJournal方法將DIRTY(且無對應CLEAN)的entry從記憶體和磁碟中一併刪除
Response快取的新增
//CacheInterceptor中intercept方法
if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
// Offer this request to the cache.
CacheRequest cacheRequest = cache.put(response);//快取響應頭
return cacheWritingResponse(cacheRequest, response);//快取響應體
}
響應頭的快取
@Nullable CacheRequest put(Response response) {
String requestMethod = response.request().method();
if (HttpMethod.invalidatesCache(response.request().method())) {
try {
remove(response.request());
} catch (IOException ignored) {
// The cache cannot be written.
}
return null;
}
if (!requestMethod.equals("GET")) {
// Don't cache non-GET responses. We're technically allowed to cache
// HEAD requests and some POST requests, but the complexity of doing
// so is high and the benefit is low.
return null;
}
if (HttpHeaders.hasVaryAll(response)) {
return null;
}
Entry entry = new Entry(response);
DiskLruCache.Editor editor = null;
try {
editor = cache.edit(key(response.request().url()));
if (editor == null) {
return null;
}
entry.writeTo(editor);
return new CacheRequestImpl(editor);
} catch (IOException e) {
abortQuietly(editor);
return null;
}
}
- 檢查請求頭的方法,非”get”的響應不快取
- 響應頭中包含“Vary:*”的不快取
- 通過DiskLruCache.Editor將響應頭資訊快取到磁碟中
- 生成CacheRequest物件為下一步將響應體資訊快取到磁碟中做準備
public @Nullable Editor edit(String key) throws IOException {
return edit(key, ANY_SEQUENCE_NUMBER);
}
synchronized Editor edit(String key, long expectedSequenceNumber) throws IOException {
initialize();
checkNotClosed();
validateKey(key);
Entry entry = lruEntries.get(key);
if (expectedSequenceNumber != ANY_SEQUENCE_NUMBER && (entry == null
|| entry.sequenceNumber != expectedSequenceNumber)) {
return null; // Snapshot is stale.
}
if (entry != null && entry.currentEditor != null) {
return null; // Another edit is in progress.
}
if (mostRecentTrimFailed || mostRecentRebuildFailed) {
// The OS has become our enemy! If the trim job failed, it means we are storing more data than
// requested by the user. Do not allow edits so we do not go over that limit any further. If
// the journal rebuild failed, the journal writer will not be active, meaning we will not be
// able to record the edit, causing file leaks. In both cases, we want to retry the clean up
// so we can get out of this state!
executor.execute(cleanupRunnable);
return null;
}
// Flush the journal before creating files to prevent file leaks.
journalWriter.writeUtf8(DIRTY).writeByte(' ').writeUtf8(key).writeByte('\n');
journalWriter.flush();//在jounar檔案上新增DIRTY記錄,表示該檔案正在被加入快取
if (hasJournalErrors) {
return null; // Don't edit; the journal can't be written.
}
if (entry == null) {
entry = new Entry(key);//建立和url對應的Entry物件
lruEntries.put(key, entry);//將Entry儲存到LinkedHashMap中
}
Editor editor = new Editor(entry);
entry.currentEditor = editor;
return editor;
}
- 在journal檔案上寫入DIRTY記錄,表示該條快取正在被寫入
- 新建和url對應的Entry物件,將Entry儲存到LinkedHashMap中
- 返回Editor物件,方便下一步真正寫入檔案流
DiskLruCache.Editor
public final class Editor {
final Entry entry;
final boolean[] written;
private boolean done;
Editor(Entry entry) {
this.entry = entry;
this.written = (entry.readable) ? null : new boolean[valueCount];
}
- 每一個DiskLruCache.Editor物件對應一個DiskLruCache.Entry物件
- 負責返回和DIRTY FILE對應的output stream(new sink(int index))及commit修改
Cache.Entry
Entry(Response response) {
this.url = response.request().url().toString();
this.varyHeaders = HttpHeaders.varyHeaders(response);
this.requestMethod = response.request().method();
this.protocol = response.protocol();
this.code = response.code();
this.message = response.message();
this.responseHeaders = response.headers();
this.handshake = response.handshake();
this.sentRequestMillis = response.sentRequestAtMillis();
this.receivedResponseMillis = response.receivedResponseAtMillis();
}
- 將Response中除了Responsebody外的資訊提取出來
public void writeTo(DiskLruCache.Editor editor) throws IOException {
BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));//獲取輸出流
sink.writeUtf8(url)
.writeByte('\n');
sink.writeUtf8(requestMethod)
.writeByte('\n');
sink.writeDecimalLong(varyHeaders.size())
.writeByte('\n');
for (int i = 0, size = varyHeaders.size(); i < size; i++) {
sink.writeUtf8(varyHeaders.name(i))
.writeUtf8(": ")
.writeUtf8(varyHeaders.value(i))
.writeByte('\n');
}
sink.writeUtf8(new StatusLine(protocol, code, message).toString())
.writeByte('\n');
sink.writeDecimalLong(responseHeaders.size() + 2)
.writeByte('\n');
for (int i = 0, size = responseHeaders.size(); i < size; i++) {
sink.writeUtf8(responseHeaders.name(i))
.writeUtf8(": ")
.writeUtf8(responseHeaders.value(i))
.writeByte('\n');
}
sink.writeUtf8(SENT_MILLIS)
.writeUtf8(": ")
.writeDecimalLong(sentRequestMillis)
.writeByte('\n');
sink.writeUtf8(RECEIVED_MILLIS)
.writeUtf8(": ")
.writeDecimalLong(receivedResponseMillis)
.writeByte('\n');
if (isHttps()) {
sink.writeByte('\n');
sink.writeUtf8(handshake.cipherSuite().javaName())
.writeByte('\n');
writeCertList(sink, handshake.peerCertificates());
writeCertList(sink, handshake.localCertificates());
sink.writeUtf8(handshake.tlsVersion().javaName()).writeByte('\n');
}
sink.close();
}
- 將除了響應體外的資訊通過輸出流寫入到磁碟中
private static final int ENTRY_METADATA = 0;//對應響應頭
private static final int ENTRY_BODY = 1;//對應響應體
public Sink newSink(int index) {//在valuecount為2時,index為0或1
synchronized (DiskLruCache.this) {
if (done) {
throw new IllegalStateException();
}
if (entry.currentEditor != this) {
return Okio.blackhole();
}
if (!entry.readable) {
written[index] = true;
}
File dirtyFile = entry.dirtyFiles[index];
Sink sink;
try {
sink = fileSystem.sink(dirtyFile);//獲取臨時檔案xxx.0.tmp對應的輸出流
} catch (FileNotFoundException e) {
return Okio.blackhole();
}
return new FaultHidingSink(sink) {
@Override protected void onException(IOException e) {
synchronized (DiskLruCache.this) {
detach();
}
}
};
}
}
獲取和指定File對應的輸出流
Blockquote
Sink是okio對OutputStream的封裝,可簡單理解為OutputStream,與之對應的還有source,是okio對InputStream的封裝
CacheRequestImpl(final DiskLruCache.Editor editor) {
this.editor = editor;
this.cacheOut = editor.newSink(ENTRY_BODY);//拿到xxx.1.tmp
this.body = new ForwardingSink(cacheOut) {
@Override public void close() throws IOException {
synchronized (Cache.this) {
if (done) {
return;
}
done = true;
writeSuccessCount++;
}
super.close();
editor.commit();
}
};
}
private Response cacheWritingResponse(final CacheRequest cacheRequest, Response response)
throws IOException {
// Some apps return a null body; for compatibility we treat that like a null cache request.
if (cacheRequest == null) return response;
Sink cacheBodyUnbuffered = cacheRequest.body();
if (cacheBodyUnbuffered == null) return response;
final BufferedSource source = response.body().source();//從Response中取出body
final BufferedSink cacheBody = Okio.buffer(cacheBodyUnbuffered);//通過cacheBody將檔案寫入儲存裝置中
Source cacheWritingSource = new Source() {
boolean cacheRequestClosed;
@Override public long read(Buffer sink, long byteCount) throws IOException {
long bytesRead;
try {
bytesRead = source.read(sink, byteCount);
} catch (IOException e) {
if (!cacheRequestClosed) {
cacheRequestClosed = true;
cacheRequest.abort(); // Failed to write a complete cache response.
}
throw e;
}
if (bytesRead == -1) {
if (!cacheRequestClosed) {
cacheRequestClosed = true;
cacheBody.close(); // The cache response is complete!
}
return -1;
}
sink.copyTo(cacheBody.buffer(), sink.size() - bytesRead, bytesRead);
cacheBody.emitCompleteSegments();
return bytesRead;
}
@Override public Timeout timeout() {
return source.timeout();
}
@Override public void close() throws IOException {
if (!cacheRequestClosed
&& !discard(this, HttpCodec.DISCARD_STREAM_TIMEOUT_MILLIS, MILLISECONDS)) {
cacheRequestClosed = true;
cacheRequest.abort();
}
source.close();
}
};
String contentType = response.header("Content-Type");
long contentLength = response.body().contentLength();
return response.newBuilder()
.body(new RealResponseBody(contentType, contentLength, Okio.buffer(cacheWritingSource)))
.build();
}
這一步涉及較多okio的知識,但主要意思是將ResponseBody通過寫io快取到磁碟中
提交更改
public void commit() throws IOException {
synchronized (DiskLruCache.this) {
if (done) {
throw new IllegalStateException();
}
if (entry.currentEditor == this) {
completeEdit(this, true);
}
done = true;
}
}
synchronized void completeEdit(Editor editor, boolean success) throws IOException {
Entry entry = editor.entry;
if (entry.currentEditor != editor) {
throw new IllegalStateException();
}
// If this edit is creating the entry for the first time, every index must have a value.
if (success && !entry.readable) {
for (int i = 0; i < valueCount; i++) {
if (!editor.written[i]) {
editor.abort();
throw new IllegalStateException("Newly created entry didn't create value for index " + i);
}
if (!fileSystem.exists(entry.dirtyFiles[i])) {
editor.abort();
return;
}
}
}//保證兩個Dirty File都“寫完”才提交
for (int i = 0; i < valueCount; i++) {
File dirty = entry.dirtyFiles[i];
if (success) {
if (fileSystem.exists(dirty)) {
File clean = entry.cleanFiles[i];
fileSystem.rename(dirty, clean);
long oldLength = entry.lengths[i];
long newLength = fileSystem.size(clean);
entry.lengths[i] = newLength;
size = size - oldLength + newLength;
}//將寫完的Dirty File(xxx.0.tmp)重新命名為Clean File的名字(xxx.0),並刪除Dirty File
} else {
fileSystem.delete(dirty);
}
}
redundantOpCount++;
entry.currentEditor = null;
if (entry.readable | success) {//更新journar檔案
entry.readable = true;
journalWriter.writeUtf8(CLEAN).writeByte(' ');
journalWriter.writeUtf8(entry.key);
entry.writeLengths(journalWriter);
journalWriter.writeByte('\n');
if (success) {
entry.sequenceNumber = nextSequenceNumber++;
}
} else {
lruEntries.remove(entry.key);
journalWriter.writeUtf8(REMOVE).writeByte(' ');
journalWriter.writeUtf8(entry.key);
journalWriter.writeByte('\n');
}
journalWriter.flush();
if (size > maxSize || journalRebuildRequired()) {
executor.execute(cleanupRunnable);
}
}
commit成功代表寫入快取完成
快取的清理
boolean journalRebuildRequired() {
final int redundantOpCompactThreshold = 2000;
return redundantOpCount >= redundantOpCompactThreshold
&& redundantOpCount >= lruEntries.size();
}
journal檔案的快取條目數量同時超出閾值(2000)和Entry的數量,說明journal需要重建
private final Runnable cleanupRunnable = new Runnable() {
public void run() {
synchronized (DiskLruCache.this) {
if (!initialized | closed) {
return; // Nothing to do
}
try {
trimToSize();
} catch (IOException ignored) {
mostRecentTrimFailed = true;
}
try {
if (journalRebuildRequired()) {
rebuildJournal();
redundantOpCount = 0;
}
} catch (IOException e) {
mostRecentRebuildFailed = true;
journalWriter = Okio.buffer(Okio.blackhole());
}
}
}
};
void trimToSize() throws IOException {
while (size > maxSize) {
Entry toEvict = lruEntries.values().iterator().next();
removeEntry(toEvict);
}
mostRecentTrimFailed = false;
}
在迴圈中不斷刪除“舊”檔案,直到剩餘快取檔案的大小總和小於DiskLruCache初始化時傳入的maxSize
boolean removeEntry(Entry entry) throws IOException {
if (entry.currentEditor != null) {
entry.currentEditor.detach(); // Prevent the edit from completing normally.
}
for (int i = 0; i < valueCount; i++) {
fileSystem.delete(entry.cleanFiles[i]);//刪除快取檔案
size -= entry.lengths[i];
entry.lengths[i] = 0;
}
redundantOpCount++;
journalWriter.writeUtf8(REMOVE).writeByte(' ').writeUtf8(entry.key).writeByte('\n');//更新journal
lruEntries.remove(entry.key);//移除entry
if (journalRebuildRequired()) {
executor.execute(cleanupRunnable);
}
return true;
}
重建journal
synchronized void rebuildJournal() throws IOException {
if (journalWriter != null) {
journalWriter.close();
}
BufferedSink writer = Okio.buffer(fileSystem.sink(journalFileTmp));
try {
writer.writeUtf8(MAGIC).writeByte('\n');
writer.writeUtf8(VERSION_1).writeByte('\n');
writer.writeDecimalLong(appVersion).writeByte('\n');
writer.writeDecimalLong(valueCount).writeByte('\n');
writer.writeByte('\n');
for (Entry entry : lruEntries.values()) {
if (entry.currentEditor != null) {
writer.writeUtf8(DIRTY).writeByte(' ');
writer.writeUtf8(entry.key);
writer.writeByte('\n');
} else {
writer.writeUtf8(CLEAN).writeByte(' ');
writer.writeUtf8(entry.key);
entry.writeLengths(writer);
writer.writeByte('\n');
}
}
} finally {
writer.close();
}
if (fileSystem.exists(journalFile)) {
fileSystem.rename(journalFile, journalFileBackup);
}
fileSystem.rename(journalFileTmp, journalFile);
fileSystem.delete(journalFileBackup);
journalWriter = newJournalWriter();
hasJournalErrors = false;
mostRecentRebuildFailed = false;
}
- 在journalFileTmp中新建journal,且只保留 DIRTY和CLEAN的記錄
journalFileTmp新建完成後重新命名為journalFile,並將舊的journalFile重新命名為journalFileBackup
DiskLruCache總結
- 通過LinkedHashMap實現LRU替換
- 通過journal保證Cache操作的原子性及可用性
- 每一條快取對應兩個狀態副本:DIRTY,CLEAN。CLEAN表示當前可用的Cache。DIRTY為編輯狀態的cache。由於更新和建立都只操作DIRTY狀態的副本,實現了讀和寫的分離。
- 每一個url對應四個檔案,兩個狀態(DIRY,CLEAN),每個狀態對應兩個檔案:0檔案對應儲存meta資料,1檔案儲存body資料。