The two primary tools for debugging memory issues are memory and valgrind. Tcl and extensions based on sampleextension provide a valgrind target which executes the test suite under Valgrind's "memcheck" tool.
When Tcl is compiled with --enable-symbols=mem, it places guards that are sensitive to misuse around allocated memory. This means that some errors which might not normally produce segmentation faults now will, making it easier to track them down.
A Tcl built with --enable-symbols=mem also includes memory, which can be used to investigate memory issues.
Given these facts, the typical commands to build Tcl for Valgrinding purposes are:
./configure CFLAGS=-DPURIFY --enable-symbols && make clean && make valgrind
tests/all.tcl replaces exit with an empty procedure. This makes the Valgrind reports cleaner. To do the same thing when debugging your own Tcl script:
proc exit args {}According to de, on POSIX systems, ensuring that the LANG environment variable is set to POSIX may reduce some debugging "noise":
export LANG=POSIX
tools/valgrind_suppress specifies call chains that allocate memory which is intended to remain allocated for the lifetime of the program, and also memory allocations that are not Tcl's concern. These allocations are ignored by Valgrind when reporting potential issues.
Address Sanitizer can provide a good picture of memory errors, but you need to build with PURIFY and no memory debugging. The following should do the trick:
export CFLAGS="-DPURIFY -fsanitize=address" export LDFLAGS="-lasan" ../unix/configure --enable-symbols make tcltest
On Fedora, be sure to install the compiler-rt package first. Other sanitizers can also be enabled with similar flags.
Rolf Ade 2002-04-14:
For linux, the mostly praised purify (and, as far as I'm aware, also most of the other commercial memory debugger) is not available; but Insure++ is, and it's plenty good enough. http://www.cs.colorado.edu/~zorn/MallocDebug.html lists some memory debugging tools (commercial ones and free ones); but then, so does http://phaseit.net/claird/comp.software.testing/mem_test.html , and the latter is more current.
I've tried some of the free tools and ended up in using mpatrol http://www.cbmamiga.demon.co.uk/mpatrol/ , because it not only provides extensive debugging, profiling and tracing capabilities to help fix memory allocation errors, but also can help pinpoint memory leaks with their associated symbolic stack tracebacks. Another advantage is you don't need to add special memory debugging related code to your code and that you don't have to link against some special libraries. The biggest drawback is that, depending on which debugging options you use, the speed of your application may slow down dramatically, so that memory debugging of the core or of an extension with a long running script may become very time expensive (or even virtually impossible).
(The following is as of tcl8.4a4 and on Linux.) If you really want to memory debug the tcl core, you should build tcl with the -DPURIFY define (as mentioned above), with debugging symbols included and statically linked.
To do this, cd to the unix directory of your tcl source distribution. If you have the results of a prior compilation laying around, first do a
make clean
Then do
./configure --enable-symbols --disable-shared
After configure has finished, edit the produced Makefile. Search for CFLAGS, and add -DPURIFY to it.
After that, do the usual
make
This should result in a (because of the static build unusual big) tclsh binary, that runs without dependences to libtcl, from anywhere, you move it. You don't need to do a make install, nor you should do this, because this would overwrite your normal tcl installation. (Or, if you insist in installing your static tclsh build, use the --prefix option in the ./configure call above, to point the installation to a location thats comfortable for you.)
If your mpatrol installation was successful (for how to do that, see the really extensive mpatrol documentation, included in the distribution), you now could start debugging. Do
mpatrol --dynamic /path/to/your/static/build/tclsh testscript.tcl
This does produce a log file, named 'mpatrol.<processID>.log', in which you find the debugging information. Either with additional options or via a config file you could customize, what memory related operations mpatrol checks and / or logs. For example, if you want to check for memory leaks, you could use:
mpatrol -g --dynamic --show-unfreed --leak-table tclsh testscript.tcl
This gives you a log file, with a summary of all not freed memory and a stack trace of every not freed allocation. To get the exact lines of code for the calls in the stack trace, use the 'mpsym' tool, that's included in the mpatrol distribution (you need to have gdb installed, for this to work):
mpsym /path/to/your/static/build/tclsh testscript.tcl <mpatrol-log>
See the extensive mpatrol documentation, for which memory debugging options are available and what they do.
More probably, you will debug this great C coded tcl extension, you're writing (or this third party extension, that causes you trouble with your scripts).
For that, you only need a static tcl build (as described above), if you are suspect, that your use of some tcl API's has triggered a otherwise not spotted memory problem of the core - which is, with all respect, not that likely. If a tclsh with an extension loaded crashes or seems to leek memory, it may be wise, first to suspect the extension code. (If your analysis really shows, that the problem is in deed in the tcl core, don't hesitate to fill up a bug report. But first, do really analyze, and try hard, to track it down, and don't forget, to double check.)
Even without the need of a static tcl build it is helpful to compile tcl with the above mentioned -DPURIFY. This reduces the "noise" of false alarms and may help you, to detect your real problem(s) faster.
You get the most useful debugging information, if you use a custom tclsh with the extension in question compiled in. Even in these days of loadable extensions (which are normally recommended!) a lot, if not all extensions have a build target, that build such a custom tclsh. (And at least just for this debugging needs, extension maintainer should still provide such a build target.) Also helpful is, of course, to build the custom tclsh with debugging information included (This is done by adding the -g option to the compiler options).
If the extension in question really doesn't provide a custom tclsh build, it is normally very easy to build it by yourself. You find some information about this at Building a custom tclsh.
To actually doing the memory debugging just use the mpatrol and mpsym tools, as shown above, but only with your custom tclsh with your extension compiled in instead of the standard tclsh.
AMG: Recent Tcl doesn't attempt to clean up all its memory on exit because that takes time and serves no purpose. However, this interferes with valgrind's leak detection. Setting the TCL_FINALIZE_ON_EXIT environment variable is supposed to restore full cleanup, though I couldn't get it to work for me. Compiling with -DPURIFY works much better, but it seems to have the side effect of making Tcl unload all dynamic libraries on exit, which prevents valgrind from showing the names of functions in loaded libraries.
If you're debugging a library loaded by Tcl and not Tcl itself, you can just use this suppression file:
{
Tcl allocation 1
Memcheck:Leak
match-leak-kinds: possible
fun:malloc
fun:GetBlocks
fun:TclpAlloc
}
{
Tcl allocation 2
Memcheck:Leak
match-leak-kinds: possible
fun:malloc
fun:TclThreadAllocObj
}
{
Tcl allocation 3
Memcheck:Leak
match-leak-kinds: possible
fun:malloc
fun:TclpAlloc
}
{
linker uninitialized branch at startup
Memcheck:Cond
fun:index
fun:expand_dynamic_string_token
fun:_dl_map_object
fun:map_doit
fun:_dl_catch_error
fun:do_preload
fun:dl_main
fun:_dl_sysdep_start
fun:_dl_start
obj:/lib64/ld-2.19.so
}That last entry isn't for Tcl, but it's useful to me on my Slackware64 system, so I'm leaving it in for now.
(author unknown, dated 2002-08-02) (MS pleads guilty)
The recent report on the valgrind memory debugger [L1 ] prompted me to try it on the tcl sources - help out the poor Jeff, the only purify licensee among us.
This is what I did after downloading and installing valgrind on my linux/i386 laptop:
proc exit args {} /opt/valgrind/bin/valgrind -v --leak-check=yes --num-callers=10 \
--logfile-fd=9 --leak-resolution=high --show-reachable=no \
./tcltest ../tests/all.pfy 9>valgrind.out
The output is
==19434== valgrind-1.0.0, a memory error detector for x86 GNU/Linux. ==19434== Copyright (C) 2000-2002, and GNU GPL'd, by Julian Seward. ==19434== Startup, with flags: ==19434== --suppressions=/opt/valgrind/lib/valgrind/default.supp ==19434== -v ==19434== --leak-check=yes ==19434== --num-callers=10 ==19434== --logfile-fd=9 ==19434== --leak-resolution=high ==19434== --show-reachable=no ==19434== Reading suppressions file: /opt/valgrind/lib/valgrind/default.supp ==19434== Reading syms from /CVS/tcl_SF_clean/unix/tcltest ==19434== Reading syms from /lib/ld-2.2.4.so ==19434== Reading syms from /opt/valgrind/lib/valgrind/valgrind.so ==19434== Reading syms from /lib/libdl-2.2.4.so ==19434== Reading syms from /lib/i686/libm-2.2.4.so ==19434== Reading syms from /lib/i686/libc-2.2.4.so ==19434== Estimated CPU clock rate is 598 MHz ==19434== ==19434== ==19434== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) ==19434== malloc/free: in use at exit: 3930 bytes in 22 blocks. ==19434== malloc/free: 136637 allocs, 136615 frees, 56516658 bytes allocated. ==19434== ==19434== searching for pointers to 22 not-freed blocks. ==19434== checked 4470668 bytes. ==19434== ==19434== definitely lost: 544 bytes in 17 blocks. ==19434== possibly lost: 0 bytes in 0 blocks. ==19434== still reachable: 3386 bytes in 5 blocks. ==19434== ==19434== 544 bytes in 17 blocks are definitely lost in loss record 4 of 6 ==19434== at 0x400467C4: malloc (vg_clientfuncs.c:100) ==19434== by 0x80CF7AE: TclpAlloc (./../generic/tclAlloc.c:680) ==19434== by 0x8066041: Tcl_Alloc (./../generic/tclCkalloc.c:1002) ==19434== by 0x80BC2E9: NewVar (./../generic/tclVar.c:4266) ==19434== by 0x80B9419: TclLookupArrayElement (./../generic/tclVar.c:929) ==19434== by 0x80840B4: TclExecuteByteCode (./../generic/tclExecute.c:1775) ==19434== by 0x8082CB9: TclCompEvalObj (./../generic/tclExecute.c:1007) ==19434== by 0x80AF574: TclObjInterpProc (./../generic/tclProc.c:1074) ==19434== by 0x8061BAD: TclEvalObjvInternal (./../generic/tclBasic.c:3033) ==19434== by 0x808381C: TclExecuteByteCode (./../generic/tclExecute.c:1430) ==19434== ==19434== LEAK SUMMARY: ==19434== definitely lost: 544 bytes in 17 blocks. ==19434== possibly lost: 0 bytes in 0 blocks. ==19434== still reachable: 3386 bytes in 5 blocks. ==19434== Reachable blocks (those to which a pointer was found) are not shown. ==19434== To see them, rerun with: --show-reachable=yes ==19434== --19434-- lru: 798 epochs, 0 clearings. --19434-- translate: new 14353 (218894 -> 3060822), discard 0 (0 -> 0). --19434-- dispatch: 39900000 basic blocks, 14078/524777 sched events, 248895 tt_fast misses. --19434-- reg-alloc: 5203 t-req-spill, 573378+33302 orig+spill uis, 77003 total-reg-r. --19434-- sanity: 1014 cheap, 41 expensive checks.
Notes
UPDATE: "if it is too good to be true ... it is probably not true"
When run as above, valgrind will not monitor children processes. But tests are normally run in separate processes, so that I was not monitoring the memory usage of the actual tests.
With the option --trace-children=yes, or running the tests in the same process, the slowdown is enormous - append.test takes 75x longer under valgrind.
Anyone know if there are commonly available tclkit's with the memory command, etc. turned on?
NEM I doubt it. tclkit is meant as a deployment solution, so it is unlikely you would want debugging turned on in this situation.
[Talk about array statistics.]