JAVA I/O（三）記憶體對映檔案

Java · 發表 2019-01-30 10:20:00

摘要：《Java程式設計思想》中對記憶體對映檔案有詳細的介紹，此處僅做簡單記錄和總結。記憶體對映檔案允許建立和修改因為太大而不能放入記憶體的檔案。 1. 記憶體對映檔案簡單例項 import java.io.IOException; import java.io.RandomAccessFi...

《Java程式設計思想》中對記憶體對映檔案有詳細的介紹，此處僅做簡單記錄和總結。記憶體對映檔案允許建立和修改因為太大而不能放入記憶體的檔案。

1. 記憶體對映檔案簡單例項

import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;

public class LargeMappedFiles {

private static int LENGTH = 0x0000FFF;

public static void main(String[] args) throws IOException{
MappedByteBuffer out = new RandomAccessFile("test.dat", "rw")
　　　　　　　　　 .getChannel()
.map(FileChannel.MapMode.READ_WRITE, 0, LENGTH);

for(int i = 0; i < LENGTH; i++) {
out.put((byte)'x');
}

for(int i = LENGTH/2; i < LENGTH/2 + 6; i++) {
System.out.print((char)out.get(i));
}
}
}

輸出：

xxxxxx

通過RandomAccessFile類獲取FileChannel，使其具備讀寫功能。
通過FileChannel的map方法，獲取MappedByteBuffer，該方法包含三個引數，MapMode對映型別、開始位置、對映總數量，意味著可以對映大檔案的較小部分。
MappedByteBuffer是一個特殊的直接緩衝器，對該緩衝器的修改會反映到對應檔案中；另外，其繼承ByteBuffer，具有ByteBuffer的所有方法。

本例中首先建立MappedByteBuffer，並設定為讀寫模式；然後往緩衝器中寫入字元x；最後在檔案中間開始讀取6個字元。

2. 記憶體對映檔案原始碼

以下是FileChannel.map()方法的解釋：

 /**
* Maps a region of this channel's file directly into memory.
*
* <p> A region of a file may be mapped into memory in one of three modes:
* </p>
*
* <ul>
*
*<li><p> <i>Read-only:</i> Any attempt to modify the resulting buffer
*will cause a {@link java.nio.ReadOnlyBufferException} to be thrown.
*({@link MapMode#READ_ONLY MapMode.READ_ONLY}) </p></li>
*
*<li><p> <i>Read/write:</i> Changes made to the resulting buffer will
*eventually be propagated to the file; they may or may not be made
*visible to other programs that have mapped the same file.({@link
*MapMode#READ_WRITE MapMode.READ_WRITE}) </p></li>
*
*<li><p> <i>Private:</i> Changes made to the resulting buffer will not
*be propagated to the file and will not be visible to other programs
*that have mapped the same file; instead, they will cause private
*copies of the modified portions of the buffer to be created.({@link
*MapMode#PRIVATE MapMode.PRIVATE}) </p></li>
*
* </ul>
*
* <p> For a read-only mapping, this channel must have been opened for
* reading; for a read/write or private mapping, this channel must have
* been opened for both reading and writing.
*
* <p> The {@link MappedByteBuffer <i>mapped byte buffer</i>}
* returned by this method will have a position of zero and a limit and
* capacity of <tt>size</tt>; its mark will be undefined.The buffer and
* the mapping that it represents will remain valid until the buffer itself
* is garbage-collected.
*
* <p> A mapping, once established, is not dependent upon the file channel
* that was used to create it.Closing the channel, in particular, has no
* effect upon the validity of the mapping.
*
* <p> Many of the details of memory-mapped files are inherently dependent
* upon the underlying operating system and are therefore unspecified.The
* behavior of this method when the requested region is not completely
* contained within this channel's file is unspecified.Whether changes
* made to the content or size of the underlying file, by this program or
* another, are propagated to the buffer is unspecified.The rate at which
* changes to the buffer are propagated to the file is unspecified.
*
* <p> For most operating systems, mapping a file into memory is more
* expensive than reading or writing a few tens of kilobytes of data via
* the usual {@link #read read} and {@link #write write} methods.From the
* standpoint of performance it is generally only worth mapping relatively
* large files into memory.</p>
*
* @parammode
*One of the constants {@link MapMode#READ_ONLY READ_ONLY}, {@link
*MapMode#READ_WRITE READ_WRITE}, or {@link MapMode#PRIVATE
*PRIVATE} defined in the {@link MapMode} class, according to
*whether the file is to be mapped read-only, read/write, or
*privately (copy-on-write), respectively
*
* @paramposition
*The position within the file at which the mapped region
*is to start; must be non-negative
*
* @paramsize
*The size of the region to be mapped; must be non-negative and
*no greater than {@link java.lang.Integer#MAX_VALUE}
*
* @returnThe mapped byte buffer
*
* @throws NonReadableChannelException
*If the <tt>mode</tt> is {@link MapMode#READ_ONLY READ_ONLY} but
*this channel was not opened for reading
*
* @throws NonWritableChannelException
*If the <tt>mode</tt> is {@link MapMode#READ_WRITE READ_WRITE} or
*{@link MapMode#PRIVATE PRIVATE} but this channel was not opened
*for both reading and writing
*
* @throws IllegalArgumentException
*If the preconditions on the parameters do not hold
*
* @throws IOException
*If some other I/O error occurs
*
* @see java.nio.channels.FileChannel.MapMode
* @see java.nio.MappedByteBuffer
*/
public abstract MappedByteBuffer map(MapMode mode,
long position, long size)
throws IOException;

該方法直接將通道對應檔案的一部分對映到記憶體，並返回MappedByteBuffer
有3種模式：READ_ONLY（只讀）、READ_WRITE（讀寫）、PRIVATE（私有，用於copy-on-write）
MappedByteBuffer一旦建立，就與建立它的通道無關，即通道關閉時，不影響該緩衝器
記憶體對映需要依賴於底層作業系統；另外，對大部分作業系統，記憶體對映要比直接讀寫昂貴，故一般都對映較大的檔案。
該方法的引數包括讀寫模式（由FileChannel內部類MapMode定義，如下）、開始位置position、對映大小size

/**
* A typesafe enumeration for file-mapping modes.
*
* @since 1.4
*
* @see java.nio.channels.FileChannel#map
*/
public static class MapMode {

/**
* Mode for a read-only mapping.
*/
public static final MapMode READ_ONLY
= new MapMode("READ_ONLY");

/**
* Mode for a read/write mapping.
*/
public static final MapMode READ_WRITE
= new MapMode("READ_WRITE");

/**
* Mode for a private (copy-on-write) mapping.
*/
public static final MapMode PRIVATE
= new MapMode("PRIVATE");

private final String name;

private MapMode(String name) {
this.name = name;
}

/**
* Returns a string describing this file-mapping mode.
*
* @returnA descriptive string
*/
public String toString() {
return name;
}

}

3. 檔案加鎖

JDK1.4引入檔案加鎖機制，允許同步訪問共享資原始檔。檔案鎖對其他作業系統程序是可見的，因為Java的檔案加鎖直接對映到本地作業系統的加鎖工具。

可以通過FileChannel的tryLock()和lock()方法獲取整個檔案的FileLock。介面如下，tryLock()是非阻塞的，如果不能獲取鎖，則返回null；lock()是阻塞的，一直等待檔案鎖。另外，SocketChannel、DatgramChannel、ServerSocketChannel不需要加鎖，因為他們是從單程序實體繼承而來；並且通常不會在兩個程序間共享socket。

public abstract FileLock lock(long position, long size, boolean shared)
throws IOException;

public final FileLock lock() throws IOException {
return lock(0L, Long.MAX_VALUE, false);
}

public abstract FileLock tryLock(long position, long size, boolean shared)
throws IOException;

public final FileLock tryLock() throws IOException {
return tryLock(0L, Long.MAX_VALUE, false);
}

FileLock是對檔案某區域進行標識的（A token representing a lock on a region of a file.），可以通過FileChannel和AsynchronousFileChannel的加鎖方法建立，包含四個成員：

public abstract class FileLock implements AutoCloseable {

private final Channel channel;
private final long position;
private final long size;
private final boolean shared;

加鎖區域由size-position決定，不會根據檔案大小變化而變化。shared為共享鎖和排它鎖標識。

對對映檔案的部分加鎖

檔案對映通常應用於極大的檔案，對其一部分進行加鎖，其他程序可以對其他部分檔案進行操作。資料庫就是這樣，多個使用者可以同時訪問。下邊用2個執行緒分別對檔案不同部分加鎖。

import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.ByteBuffer;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.nio.channels.FileChannel.MapMode;
import java.nio.channels.FileLock;

/**
 * 對對映檔案加鎖
 * 對映檔案：MappedByteBuffer out = fc.map(MapMode.READ_WRITE, 0, LENGTH);
 * 加鎖：FileLock fl = fc.lock(start, end, false);fl.release();
 * @author bob
 *
 */
public class LockingMappedFIles {

private static final int LENGTH = 0x0000FFF;//128M
static FileChannel fc;

public static void main(String[] args) throws IOException{
//1.獲取讀寫FileChannel
fc = new RandomAccessFile("data.dat", "rw").getChannel();
//2.根據FileChannel獲取MappedByteBuffer，讀寫模式、全檔案
MappedByteBuffer out = fc.map(MapMode.READ_WRITE, 0, LENGTH);
//3.寫入字元x
for(int i = 0; i < LENGTH; i++) {
out.put((byte)'x');
}
//4.啟動執行緒1，對檔案的前1/3加鎖，通過緩衝器操作檔案
Thread thread1 = new LockAndModify(out, 0, LENGTH/3);
thread1.start();
//5.啟動執行緒2，對檔案的後1/3加鎖，通過緩衝器操作檔案
Thread thread2 = new LockAndModify(out, LENGTH*2/3, LENGTH);
thread2.start();

}

static class LockAndModify extends Thread{
private ByteBuffer byteBuffer;
private int start, end;

public LockAndModify(ByteBuffer byteBuffer, int start, int end) {
//記錄加鎖位置的起始位置
this.start = start;
this.end = end;
/**
* 1. 設定MappedByteBuffer的position和limit
* 2. 調slice()方法，建立新ByteBuffer，對映原ByteBuffer；其position為0，limit為緩衝器容量
*由slice()方法建立的ByteBuffer是直接的、只可讀的
*修改會對映到原ByteBuffer中
* 3. 另外，limit 和 position不可顛倒順序，否則position可能比limit大，報錯
*/
byteBuffer.limit(end);
byteBuffer.position(start);
this.byteBuffer = byteBuffer.slice();
}

public void run() {
try {
//加排它鎖
FileLock fl = fc.lock(start, end, false);
System.out.println("Locked: " + start + " to " + end);
//修改內容
while(byteBuffer.position() < byteBuffer.limit()+1) {
byteBuffer.put(byteBuffer.position(), (byte)(byteBuffer.get()+1));
}
fl.release();
System.out.println("release: " + start + " to " + end);
} catch (Exception e) {
// TODO: handle exception
}
}
}
}

執行結果，檔案data.bat中前1/3和後1/3字元變為y。

總結

1.記憶體檔案對映主要用於極大檔案的訪問和操作，可提高效能；

2. 記憶體對映檔案通過通道建立，可設定讀寫模式和限制對映區域；

3. 對檔案某區域加鎖可實現多執行緒或程序對共享資原始檔不同區域併發修改；

4. MappedByteBuffer是一種特殊的直接緩衝器，對其修改會反映到檔案中。

參考

《Java核心程式設計》

JAVA I/O（三）記憶體對映檔案

1. 記憶體對映檔案簡單例項

2. 記憶體對映檔案原始碼

3. 檔案加鎖

對對映檔案的部分加鎖

總結

參考

您可能也會喜歡…