chenlanjie842179335的專欄
說到C++除錯想必大家會想到一堆除錯中遇到的問題，而在我看來C++中最難也是最普遍的除錯問題就出在記憶體上。為什麼要這麼說呢？可以想想你曾經碰到過的問題，記憶體洩露應該是最普遍的，其次是記憶體越界，野指標，這些碰到哪一個都是硬點子。特別是專案規模越來越大的時候，這些問題就成為骨中釘，肉中刺，膈應的開發人員什麼想法都沒有了。
問題既然產生，那必然會有方法解決。我們從現在開始一點一點的剖析這些問題的產生原因再對症下藥，保管藥到病除啊。
1. 記憶體洩露
記憶體洩露是個老掉牙的問題，從寫程式的第一天就沒離開過我的視野範圍。有點程式基礎的人都知道它是怎麼產生的，我這裡就不羅嗦。我只介紹幾種記憶體洩露的檢查方法。
1.1 如何檢測記憶體洩露
正常情況下，我們通過Virtual Studio 生成的程式，除MFC以外是不會報告記憶體洩露的，即使你確實洩露了。那麼為什麼是除MFC應用程式以外呢？這個問題就說到了MFC應用程式嚮導都為我們生成了些什麼。
[cpp] view plaincopyprint?- #ifdef _DEBUG
- #define new DEBUG_NEW
- #endif
如果你細心的話，應該會在你的專案裡找到這麼一段話。這段話的意思是，如果是DEBUG版本，則將 new 替換成 DEBUG_NEW。
那麼DEBUG_NEW又是什麼？
我們跟過去看一下它的定義
[cpp] view plaincopyprint?- #define DEBUG_NEW new(THIS_FILE, __LINE__)
它只是在new後面加了兩個引數 THIS_FILE, __LINE__
這兩個引數都是編譯器的預定義巨集(THIS_FILE其實是重新定義的__FILE__)，分別表示，當前檔案的檔名和行號，如果你的MFC程式發生了洩露，又正好被捕獲到了，那麼output視窗中顯示的檔名和行號就是從這裡來的。
那麼new 怎麼會有引數的呢？
祕密在於MFC過載了 operator new。當然所有的記憶體分配最後都會呼叫crt的malloc進行記憶體分配。
[cpp] view plaincopyprint?- void* __cdecl operator new(size_t nSize, LPCSTR lpszFileName, int nLine)
- {
- return ::operator new(nSize, _NORMAL_BLOCK, lpszFileName, nLine);
- }
- void* __cdecl operator new[](size_t nSize, LPCSTR lpszFileName, int nLine)
- {
- return ::operator new[](nSize, _NORMAL_BLOCK, lpszFileName, nLine);
- }
- void __cdecl operator delete(void* pData, LPCSTR/* lpszFileName */,
- int/* nLine */)
- {
- ::operator delete(pData);
- }
- void __cdecl operator delete[](void* pData, LPCSTR/* lpszFileName */,
- int/* nLine */)
- {
- ::operator delete(pData);
- }
所以我們不一定要用MFC，CRT本身就自帶了記憶體洩露的檢測功能。我們需要做的，只是做一些小設定。
在CRT中我們可通過如下的函式呼叫來開啟記憶體洩露報告。
[cpp] view plaincopyprint?- #include <crtdbg.h>
- _CrtSetDbgFlag(_CrtSetDbgFlag(_CRTDBG_REPORT_FLAG) | _CRTDBG_LEAK_CHECK_DF);
如果發生了記憶體洩露則會有如下的輸出
[cpp] view plaincopyprint?- Detected memory leaks!
- Dumping objects ->
- {91} normal block at 0x00725BB0, 256 bytes long.
- Data: < > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD
- Object dump complete.
報告是有了，但是報告中沒有指明產生洩露的檔案和行號，那麼如何通過報告來找出記憶體洩露的地方呢？
可以看到，每個洩露的報告項中的最前面有一個{91}，這其實是一個分配的序號。無論你用new或者malloc都會導致這個序號增長。
當我們需要定位某個序號的記憶體洩露的時候可以通過如下程式碼
[cpp] view plaincopyprint?- _CrtSetBreakAlloc(91);
當程式分配到91塊記憶體的時候就會出現ASSERT斷言，暫停程式的執行。這樣做有一個好處，就是可以通過函式呼叫堆疊還原洩露時的現場，從而更具體的分析洩露出現的原因。
那麼CRT真的無法像MFC一樣打印出洩露的檔名和行號嗎？其實這是一個很簡單的事情，我們只要通過
[cpp] view plaincopyprint?- new(_NORMAL_BLOCK, __FILE__, __LINE__) char[256]
來分配記憶體就可以得到像MFC的記憶體洩露檢測報告。其實MFC也是通過CRT做的。現在我們看到的記憶體洩露檢測報告應該是這樣的
[c-sharp] view plaincopyprint?- Detected memory leaks!
- Dumping objects ->
- d:/developed/directxlearn/directxlearn/directxlearn.cpp(21) : {91} normal block at 0x002B5BB0, 256 bytes long.
- Data: < > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD
- Object dump complete.
如此我們已經可以讓記憶體洩露檢測報告像MFC一樣打印出檔名和行號了。
3. 更深入的討論
CRT是如何將檔名和行號儲存下來的？又是如何列印到我們的Output視窗的呢？
我們先看一下反彙編以後的new 程式碼
[cpp] view plaincopyprint?- mov eax,dword ptr [`wWinMain'::`2'::__LINE__Var (2D801Ch)]
- add eax,3
- push eax
- push offset string "d://developed//directxlearn//direct"... (2D6960h)
- push 1
- push 100h
- call operator new[] (2D12EEh)
- add esp,10h
- mov dword ptr [ebp-0F8h],eax
從反彙編出來的程式碼中我們可以看到，檔名和行號都是儲存在程式資料段中的，傳入operator new 的只是檔名的地址而已。跟蹤進入call operator new[] 我們可以看到如下程式碼
[cpp] view plaincopyprint?- void *__CRTDECL operator new[](
- size_t cb,
- int nBlockUse,
- constchar * szFileName,
- int nLine
- )
- _THROW1(_STD bad_alloc)
- {
- void *res = operator new(cb, nBlockUse, szFileName, nLine );
- RTCCALLBACK(_RTC_Allocate_hook, (res, cb, 0));
- return res;
- }
operator new 的程式碼如下
[cpp] view plaincopyprint?- void *__CRTDECL operator new(
- size_t cb,
- int nBlockUse,
- constchar * szFileName,
- int nLine
- )
- _THROW1(_STD bad_alloc)
- {
- /* _nh_malloc_dbg already calls _heap_alloc_dbg in a loop and calls _callnewh
- if the allocation fails. If _callnewh returns (very likely because no
- new handlers have been installed by the user), _nh_malloc_dbg returns NULL.
- */
- void *res = _nh_malloc_dbg( cb, 1, nBlockUse, szFileName, nLine );
- RTCCALLBACK(_RTC_Allocate_hook, (res, cb, 0));
- /* if the allocation fails, we throw std::bad_alloc */
- if (res == 0)
- {
- staticconst std::bad_alloc nomem;
- _RAISE(nomem);
- }
- return res;
- }
可以看到，最終還是呼叫了_nh_malloc_dbg.如果我們繼續跟蹤下去的話會找到真正分配記憶體的地方，這裡我著重講一下CRT的記憶體結構。和很多記憶體池一樣，CRT也在分配記憶體的前後加入了一些標記和內容。其中最重要的就是一個_CrtMemBlockHeader 的結構。
[cpp] view plaincopyprint?- typedefstruct _CrtMemBlockHeader
- {
- struct _CrtMemBlockHeader * pBlockHeaderNext;
- struct _CrtMemBlockHeader * pBlockHeaderPrev;
- char * szFileName;
- int nLine;
- #ifdef _WIN64
- /* These items are reversed on Win64 to eliminate gaps in the struct
- * and ensure that sizeof(struct)%16 == 0, so 16-byte alignment is
- * maintained in the debug heap.
- */
- int nBlockUse;
- size_t nDataSize;
- #else /* _WIN64 */
- size_t nDataSize;
- int nBlockUse;
- #endif /* _WIN64 */
- long lRequest;
- unsigned char gap[nNoMansLandSize];
- /* followed by:
- * unsigned char data[nDataSize];
- * unsigned char anotherGap[nNoMansLandSize];
- */
- } _CrtMemBlockHeader;
這個結構是一個雙向連結串列，分別記錄了前一個和後一個分配的記憶體塊的首地址。同時，該結構也記錄了產生這塊記憶體的具體檔案和行號。其次是記憶體的型別，以及請求的大小，以及分配記憶體時的分配序號。最後是一個4位元組的上溢保護，通過這四個位元組可以檢測出大部分由負序數導致的陣列上溢問題。
[cpp] view plaincopyprint?- blockSize = sizeof(_CrtMemBlockHeader) + nSize + nNoMansLandSize;
從記憶體實際分配位元組數的計算中我們還可以看出，除請求的記憶體量和_CrtMemBlockHeader所佔用的記憶體量以外，還有一個4位元組的下溢位元組保護。上溢和下溢的保護位元組一般被初始化為0xFD。
初始化記憶體資訊的程式碼如下
[cpp] view plaincopyprint?- pHead->pBlockHeaderNext = _pFirstBlock;
- pHead->pBlockHeaderPrev = NULL;
- pHead->szFileName = (char *)szFileName;
- pHead->nLine = nLine;
- pHead->nDataSize = nSize;
- pHead->nBlockUse = nBlockUse;
- pHead->lRequest = lRequest;
- /* link blocks together */
- _pFirstBlock = pHead;
- /* fill in gap before and after real block */
- memset((void *)pHead->gap, _bNoMansLandFill, nNoMansLandSize);
- memset((void *)(pbData(pHead) + nSize), _bNoMansLandFill, nNoMansLandSize);
- /* fill data with silly value (but non-zero) */
- memset((void *)pbData(pHead), _bCleanLandFill, nSize);
如此，在最終生成記憶體洩露檢測報告的時候就可以根據記憶體塊資訊來得到檔名和行號了。
4. CRT的缺陷
CRT檢查記憶體洩露的方法是比較直觀的，但是也相應的存在缺陷。
首先，是你new的時候或者malloc的時候不可能都是用特殊版本的函式呼叫，只能通過定義巨集來實現。巨集這個東西我比較討厭，因為在來來回回的包含中你不能確定哪些new 被替換了。所以一旦有沒有被巨集替換過的new那麼你的報告就會出現一些沒有地址和行號的記憶體洩露報告。
其次，CRT的記憶體洩露報告沒有呼叫堆疊，有時候洩露很可能是一些和呼叫順序相關的臨界條件引起的，至少碰到這種情況就比較難查。
最後，假如你的程式中存在靜態物件，恰好你的靜態物件被析構的時候是在記憶體檢測報告完成之後，那麼記憶體檢測報告就會發生誤報。
以上種種，都促使我尋找一種更先進的檢查記憶體洩露的方法。
5. Virutal Leak Detected
初見VLD的時候我認為沒有比他更好的記憶體洩露檢測方法了。從技術的角度講，它所做的工作有點像黑客乾的事情，因為它用到了一種技術——DLL補丁。
說起DLL補丁我們還要先講一下EXE程式和DLL之間的關係，以及EXE如何呼叫DLL
當主程式載入DLL到自己的程序地址空間之前，作業系統首先要將整個DLL檔案載入到記憶體中，然後根據需要重新對映動態連結庫的地址，之後調整匯出符號表中入口函式的地址，使之指向正確的函式入口。如果你跟蹤彙編程式碼的話，會發現你Call 一個DLL的匯出函式或者類函式的時候其實是先到了一個全是jmp指令的地方，然後才到達正確的程式碼地址。這個全是jmp指令的地方就是匯出函式表。如果我們修改了表中的跳轉地址的話，當你去call 的時候就會跳轉到一個你指定的地方，比如說一個鉤子函式。但是，這種修改跳轉表的方法只對當前程式例項有效。
而VLD的核心思想是，攔截所有的記憶體分配函式的入口地址，使之轉移到我們的入口函式中記錄一些內容，比如呼叫時的指令指標EIP的值等。
雖然看起來很複雜，但是vld的使用時非常簡單的我們只需要在工程中設定vld.lib匯入庫的地址，在某一個頭檔案中包含vld.h就可以了。之後我們將vld.ini放在執行目錄下
vld.ini有如下配置
[cpp] view plaincopyprint?- There are a several configuration options that control specific aspects of VLD's operation. These configuration options are stored in the vld.ini configuration file. By default, the configuration file should be in the Visual Leak Detector installation directory. However, the configuration file can be copied to the program's working directory, in which case the configuration settings in that copy of vld.ini will apply only when debugging that one program.
- VLD
- This option acts as a master on/off switch. By default, this option is set to "on". To completely disable Visual Leak Detector at runtime, set this option to "off". When VLD is turned off usingthis option, it will do nothing but print a message to the debugger indicating that it has been turned off.
- AggregateDuplicates
- Normally, VLD displays each individual leaked block in detail. Setting this option to "yes" will make VLD aggregate all leaks that share the same size and call stack under a single entry in the memory leak report. Only the first leaked block will be reported in detail. No other identical leaks will be displayed. Instead, a tally showing the total number of leaks matching that size and call stack will be shown. This can be useful if there are only a few sources of leaks, but those few sources are repeatedly leaking a very large number of memory blocks.
- ForceIncludeModules
- In some rare cases, it may be necessary to include a module in leak detection, but it may not be possible to include vld.h in any of the module's sources. In such cases, this option can be used to force VLD to include those modules in leak detection. List the names of the modules (DLLs) to be forcefully included in leak detection. If you do use this option, it's advisable to also add vld.lib to the list of library modules in the linker options of your project's settings.
- Caution: Use this option only when absolutely necessary. In some situations, use of this option may result in unpredictable behavior including false leak reports and/or crashes. It's best to stay away from this option unless you are sure you understand what you are doing.
- MaxDataDump
- Set this option to an integer value to limit the amount of data displayed in memory block data dumps. When this number of bytes of data have been dumped, the dump will stop. This can be useful if any of the leaked blocks are very large and the debugger's output window becomes too cluttered. You can set this option to 0 (zero) if you want to suppress data dumps altogether.
- MaxTraceFrames
- By default, VLD will trace the call stack for each allocated block as far back as possible. Each frame traced adds additional overhead (in both CPU time and memory usage) to your debug executable. If you'd like to limit this overhead, you can define this macro to an integer value. The stack trace will stop when it has traced this number of frames. The frame count may include some of the "internal" frames which, by default, are not displayed in the debugger's output window (see TraceInternalFrames below). In some cases there may be about three or four "internal" frames at the beginning of the call stack. Keep this in mind when usingthis macro, or you may not see the number of frames you expect.
- ReportEncoding
- When the memory leak report is saved to a file, the report may optionally be Unicode encoded instead of using the default ASCII encoding. This might be useful if the data contained in leaked blocks is likely to consist of Unicode text. Set this option to "unicode" to generate a Unicode encoded report.
- ReportFile
- Use this option to specify the name and location of the file in which to save the memory leak report when using a file as the report destination, as specified by the ReportTo option. If no file is specified here, then VLD will save the report in a file named "memory_leak_report.txt" in the working directory of the program.
- ReportTo
- The memory leak report may be sent to a file in addition to, or instead of, the debugger. Use this option to specify which type of destination to use. Specify one of "debugger" (the default), "file", or "both".
- SelfTest
- VLD has the ability to check itself for memory leaks. This feature is always active. Every time you run VLD, in addition to checking your own program for memory leaks, it is also checking itself for leaks. Setting this option to "on" forces VLD to intentionally leak a small amount of memory: a 21-character block filled with the text "Memory Leak Self-Test". This provides a way to test VLD's ability to check itself for memory leaks and verify that this capability is working correctly. This option is usually only useful for debugging VLD itself.
- SlowDebuggerDump
- If enabled, this option causes Visual Leak Detector to write the memory leak report to the debugger's output window at a slower than normal rate. This option is specifically designed to work around a known issue with some older versions of Visual Studio where some data sent to the output window might be lost if it is sent too quickly. If you notice that some information seems to be missing from the memory leak report, try turning this on.
- StackWalkMethod
- Selects the method to be used for walking the stack to obtain call stacks for allocated memory blocks. The default"fast" method may not always be able to successfully trace completely through all call stacks. In such cases, the "safe" method may prove to be more reliable in obtaining the full stack trace. The disadvantage with the "safe" method is that it is significantly slower than the "fast" method and will probably result in very noticeable performance degradation of the program being debugged. In most cases it should be okay to leave this option set to "fast". If you experience problems getting VLD to show call stacks, you can try setting this option to "safe".
- If you do use the "safe" method, and notice a significant performance decrease, you may want to consider using the MaxTraceFrames option to limit the number of frames traced to a relatively small number. This can reduce the amount of time spent tracing the stack by a very large amount.
- StartDisabled
- Set this option to "yes" to disable memory leak detection initially. This can be useful if you need to be able to selectively enable memory leak detection from runtime, without needing to rebuild the executable; however, this option should be used with caution. Any memory leaks that may occur before memory leak detection is enabled at runtime will go undetected. For example, if the constructor of some global variable allocates memory before execution reaches a subsequent call to VLDEnable, then VLD will not be able to detect if the memory allocated by the global variable is never freed. Refer to the following section on controlling leak detection at runtime for details on using the runtime APIs which can be useful in conjunction with this option.
- TraceInternalFrames
- This option determines whether or not all frames of the call stack, including frames internal to the heap, are traced. There will always be a number of frames on the call stack which are internal to Visual Leak Detector and C/C++ or Win32 heap APIs that aren't generally useful for determining the cause of a leak. Normally these frames are skipped during the stack trace, which somewhat reduces the time spent tracing and amount of data collected and stored in memory. Including all frames in the stack trace, all the way down into VLD's own code can, however, be useful for debugging VLD itself.
我們先看一下VLD的初始化過程，這有助於我們加深對VLD工作機制的理解。我這裡使用的是vld 1.9h的版本，支援VS2008及以下的編譯器。最新的2.0a版本已經可以支援vs2010
6. 非洩露記憶體增長
什麼是非洩露記憶體增長？舉例來說，你的程式有一個列表，所有已分配的記憶體都被記錄在這個列表中，但是列表中的記憶體在某些情況下沒有被刪除。所以，未釋放的記憶體越來越多，直到最後記憶體分配失敗。但是，你使用之前的方法檢查記憶體洩露卻發現，並沒有任何日誌。原來，在你正常退出程式的時候列表中未被釋放的記憶體已經被你挨個釋放了。
實際情況要比這個複雜的多，也隱晦的多。這種情況也是記憶體洩露的一種，屬於執行時洩露，它的危害更為嚴重。對於這種問題的追查也是一件很惱人的事情。那麼從現在開始，讓這麼麻煩的問題見鬼去吧。
6.1 UMDH簡介
UMDH 是windows debug tools 下的一款命令列工具，它的全名是User-Mode Dump Heap 這個工具會分析當前程序在堆上分配的記憶體，並有兩種模式
1. 程序分析模式，這個模式會對程序分配的每一塊記憶體做記錄，其中包含分配的記憶體大小；記憶體分配地址；記憶體分配時的函式呼叫堆疊等。
2. 日誌分析模式，該模式會比較幾個不同的日誌，找出記憶體增長的地方。
在使用UMDH做分析之前我們要先做一些準備工作。
首先，開啟程序的棧捕捉標誌。這個步驟通過一行命令來完成
gflags /i ImageName +ust
這個命令隻影響新啟動的程序，對已經在執行狀態的程序不起作用。
其次，安裝windows 的 symbol檔案。如果不需要的話可以不用安裝。
最後，設定環境變數
set _NT_SYMBOL_PATH=Path
通過這幾個步驟後我們才可以使用UMDH來對程序記憶體做分析。
首先，我們先啟動目標程序，稍等一會兒，讓程序進入穩定的執行狀態。
之後我們通過命令列 umdh -p:2230 -f:dump_allocations.txt 對程序進行分析。
其中-p後面的數字是程序號， -f 後面跟的是日誌檔案的檔名。
等待一段時間後我們就會收集到一個檔案，裡面記錄了一些記憶體的分配資訊。
之後我們重複以上步驟幾次，最好給檔名編個號。比如 dump1.txt dump2.txt
最後，我們通過命令列對剛才的檔案進行分析
umdh -v dump1.txt dump2.txt > memleak.txt
這樣我們得到了一個描述記憶體增長的日誌檔案memleak.txt類似如下的格式
+ 5320 (f110 - 9df0) 3a allocs BackTrace00053
Total increase == 5320
ntdll!RtlDebugAllocateHeap+0x000000FD
ntdll!RtlAllocateHeapSlowly+0x0000005A
ntdll!RtlAllocateHeap+0x00000808
MyApp!_heap_alloc_base+0x00000069
MyApp!_heap_alloc_dbg+0x000001A2
MyApp!_nh_malloc_dbg+0x00000023
MyApp!_nh_malloc+0x00000016
MyApp!operator new+0x0000000E
MyApp!LeakyFunc+0x0000001E
MyApp!main+0x0000002C
MyApp!mainCRTStartup+0x000000FC
KERNEL32!BaseProcessStart+0x0000003D