Max memory = [-Xmx] + [-XX:MaxPermSize] + number_of_threads * [-Xss]

MaxPerm/MetaSize + MaxDirectMemorySize + Xmx（執行緒棧一般只有幾十M，可以忽略）這差不多就是JVM可以佔用的最大記憶體了

整個Java程序分為heap和non-heap兩部分，每部分有以下幾個概念：

reserved memory 是指JVM 通過mmaped PROT_NONE 申請的虛擬地址空間，在頁表中已經存在了記錄（entries），保證了其他程序不會被佔用，會page faults,committed memory 是JVM向操做系統實際分配的記憶體（malloc/mmap）,mmaped PROT_READ | PROT_WRITE,仍然會page faults 但是跟 reserved 不同，完全核心處理像什麼也沒發生一樣。used memory 是JVM實際儲存了資料（Java物件）的大小，當used~=committed的時候，heap就會grow up，-Xmx設定了上限。

關於committed,reserved以及rss之間的關係實際情況要複雜的多：

• reserved 但是沒有 committed pages 不算 rss.
• page out 的算committed，但是不算 rss.
• 已經 committed 的也不一定在rss內(committed > rss): malloc/mmap is lazy unless told otherwise. Pages are only backed by physical memory once they're accessed.

committed 可能會比init小，因為JVM可能會將記憶體還給OS，但是一定不會小於used,也就是commited是操做系統保證JVM可以使用的記憶體空間，但是不一定都使用了。init是啟動時後JVM向OS申請的記憶體，max是能夠使用的最大邊界值。注意這裡說的都是虛擬記憶體，所以理論上整個操做系統commited的記憶體為實體記憶體加上交換空間的大小，換句話說如果commited超過實體記憶體的話，多餘的部分就會被換出到磁碟。

JVM（堆）佔用的實體記憶體是跟committed相關，committed變小意味著JVM將記憶體還給OS了，則同過top命令看到的RSS會變小。

測試發現：committed後的記憶體是不會還給OS的，FullGC後used（堆記憶體）降下來了，但是隻要committed不變，佔用的RSS就不會降下來。目前用的CMS也沒有強制將記憶體還給OS的方法。Java堆佔用的實體記憶體不會超過-Xmx，但是一個程序具體佔用多少實體記憶體不等於used，也不等於commited，目前來說JVM如何向OS申請記憶體和如何將記憶體還給OS我們是不知道的。

top命令檢視到的RSS大於-Xmx設定的值，超過的部分肯定是堆外記憶體，如果大很多那說明對外記憶體使用的是有問題的。

OS

commit charge:

In thinking about virtual memory, there are two concepts that every programmer should understand: resident set size and commit charge. The second is easiest to explain: it's the total amount of memory that your program might be able to modify (ie, it excludes read-only memory-mapped files and program code). The potential commit charge for an entire system is the sum of RAM and swap space, and no program can exceed this. It doesn't matter how big your virtual address space is: if you have 2G of RAM, and 2G of swap, you can never work with more than 4G of in-memory data; there's no place to store it.

dirty page:

One final concept: pages in the resident set can be “dirty,” meaning that the program has changed their content. A dirty page must be written to swap space before its physical memory can be used by another page. By comparison, a clean (unmodified) page may simply be discarded; it will be reloaded from disk when needed. If you can guarantee that a page will never be modified, it doesn't count against a program's commit charge — we'll return to this topic when discussing memory-mapped files.

Below is a picture showing an example of a memory pool:

    +----------------------------------------------+
    +////////////////           |                  +
    +////////////////           |                  +
    +----------------------------------------------+

    |--------|
       init
    |---------------|
           used
    |---------------------------|
              committed
    |----------------------------------------------|

通過jconsole的MBean可以很方便的監控heap,noheap以及commited,used這些內容：

JVM非堆的記憶體可能會有哪些？GC,JIT,Threads,Classes and Classloaders(PermGen),NIO(direct buffer)

But besides the memory consumed by your application, the JVM itself also needs some elbow room. The need for it derives from several different reasons:

• Garbage collection. As you might recall, Java is a garbage-collected language. In order for the garbage collector to know which objects are eligible for collection, it needs to keep track of the object graphs. So this is one part of the memory lost to this internal bookkeeping. G1 is especially known for its excessive appetite for additional memory, so be aware of this.

• JIT optimization. Java Virtual Machine optimizes the code during runtime. Again, to know which parts to optimize it needs to keep track of the execution of certain code parts. So again, you are going to lose memory.

• Off-heap allocations. If you happen to use off-heap memory, for example while using direct or mapped ByteBuffers yourself or via some clever 3rd party API then voila – you are extending your heap to something you actually cannot control via JVM configuration.

• JNI code. When you are using native code, for example in the format of Type 2database drivers, then again you are loading code in the native memory.

• Metaspace. If you are an early adopter of Java 8, you are using metaspace instead of the good old permgen to store class declarations. This is unlimited and in a native part of the JVM.

虛擬記憶體不重要，尤其是在64位作業系統上，重要的是RSS，但是有時它也不一定就說明你的程式實際需要使用的記憶體。JVM佔用的實體記憶體超過-Xmx設定的值？

But RSS is also misleading, especially on a lightly loaded machine. The operating system doesn't expend a lot of effort to reclaiming the pages used be a process. There's little benefit to be gained by doing so, and the potential for an expensive page fault if the process touches the page in the future. As a result, the RSS statistic may include lots of pages that aren't in active use.

限制程序能夠使用的實體記憶體：貌似不是很容易，主要原因是程序fork子程序

• http://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process

• http://coldattic.info/shvedsky/pro/blogs/a-foo-walks-into-a-bar/posts/40

DirectByteBuffer

Examining a heap dump for java.nio.DirectByteBuffer instances should provide further insight.

-XX:MaxDirectMemorySize 可以限制JVM使用DirectMemory的大小。

jconsole non-heap memory:

Non-heap memory includes a method area shared among all threads and memory required for the internal processing or optimization for the Java VM. It stores per-class structures such as a runtime constant pool, field and method data, and the code for methods and constructors. The method area is logically part of the heap but, depending on the implementation, a Java VM may not garbage collect or compact it. Like the heap memory, the method area may be of a fixed or variable size. The memory for the method area does not need to be contiguous.

jconsole中看到的NonHeapMemory不包括direct buffer和mapped，可以通過java.nio中MBean監控，實體記憶體在linux下用top檢視，在windows下工作管理員看到有些問題

通過pmap能檢視到mapped的檔案：pamp -x :

MappedByteBuffer和DirectByteBuffer雖然是堆外的記憶體但是通過FullGC是可以“回收”的。

Garbage Collection of Direct/Mapped Buffers

That brings up another topic: how does the non-heap memory for direct buffers and mapped files get released? After all, there's no method to explicitly close or release them. The answer is that they get garbage collected like any other object, but with one twist: if you don't have enough virtual memory space or commit charge to allocate a direct buffer, that will trigger a full collection even if there's plenty of heap memory available. Normally, this won't be an issue: you probably won't be allocating and releasing direct buffers more often than heap-resident objects. If, however, you see full GC's appearing when you don't think they should, take a look at your program's use of buffers.

使用Direct buffer的場景：

In fact, the only reason that I can see for using direct buffers in a pure Java program is that they won't be moved during garbage collection. If you've read my article on reference objects, you'll remember that the garbage collector compacts the heap after disposing of dead objects. If you have large blocks of heap memory allocated as buffers, they may get moved as part of compaction, and no matter how fast your CPU, that takes time; it's not something you want to do on every full collection. Since the direct buffer lives outside of the heap, it isn't affected by collections. On the other hand, every data access is a JNI call. Only benchmarking will tell you whether this helps or hurts your particular application.

What's using my native memory?

Once you have determined you are running out of native memory, the next logical question is: What's using that memory? Answering this question is hard because, by default, Windows and Linux do not store information about which code path is allocated a particular chunk of memory.

如何監控和排查non-heap 或 native memory leak

• IBM Support Assistant: https://www-01.ibm.com/marketing/iwm/iwm/web/reg/download.do?source=isa&S_PKG=isa5&lang=en_US&cp=UTF-8&dlmethod=http
• Preprocessor level: Dmalloc
• Linker level: Ccmalloc
• Runtime-linker level: NJAMD
• Emulator-based: memcheck
• JNI leaking: Valgrind memcheck http://www.oracle.com/technetwork/java/javase/memleaks-137499.html#gbyvk
• Java core file: http://www.javacodegeeks.com/2013/02/analysing-a-java-core-dump.html
• NMT Native Memory Tracking: https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html 注意目前（7u40）NMT只能用來分析HotSpot internal memory usage，不能分析第三方的JNI.
• GCMV: https://www.ibm.com/developerworks/java/jdk/tools/gcmv/ https://www.ibm.com/developerworks/community/blogs/troubleshootingjava/entry/gcmv_native_memory?lang=en

相比分析堆記憶體的洩漏，分析non-heap的要困難的多，不同的場景需要不同的工具去分析。

一些連結：

http://www.ibm.com/developerworks/linux/library/j-nativememory-linux/

http://docs.oracle.com/javase/7/docs/api/java/lang/management/MemoryUsage.html

http://stackoverflow.com/questions/561245/virtual-memory-usage-from-java-under-linux-too-much-memory-used

http://stackoverflow.com/questions/1612939/why-does-the-sun-jvm-continue-to-consume-ever-more-rss-memory-even-when-the-heap

(下面這兩個答案很有用)

https://plumbr.eu/blog/memory-leaks/why-does-my-java-process-consume-more-memory-than-xmx

http://www.importnew.com/14292.htm