Updated 2018-06-22 17:09:26 by pooryorick

See Also  edit

Hacking on The Core
Contains information useful to extension writers as well.

Description  edit

[As of 8.4, add "-DPURIFY" to CFLAGS to change core allocation from pool- to single-object-allocation. tclguy might change the label someday.]

[... -DTCL_MEM_DEBUG ...]

If you're using tclsh or another shell driven by Tcl_Main(), and you're using Tcl 8.4a4 or later, then you can reduce the number of memory leak false alarms by ending your script with:
   proc exit args {}

At least on Linux (and likely depending on your libc version) it may also helpful to set the LANG environment variable to POSIX, if it's not already set to this. If you use the bash sh, simply do
    export LANG=POSIX

This has further reduced the debugging "noise" for me.


Unfortunately, even compiled with -DPURIFY and with the (very helpful) exit redefinition from above, tcl8.4a4 does not completely clean up his memory usage at program end. For details see http://sourceforge.net/tracker/index.php?func=detail&aid=543549&group_id=10894&atid=110894

At least some memory debugger will report this as memory leaks. This is, although true, not a real problem, because it's not possible to 'exploit' this for scripts, that results in an always growing tclsh process. It's safe to ignore them. The trick is, of course, to know, which not freed mallocs could be ignored. I'm really not experienced enough, to give a clear rule.

(Without the exit redefinition trick from above the "noise" of harmless (ie not "exploitable") not freed memory allocations are hundreds of mallocs.)


For linux, the mostly praised purify (and, as far as I'm aware, also most of the other commercial memory debugger) is not available; but Insure++ is, and it's plenty good enough. http://www.cs.colorado.edu/~zorn/MallocDebug.html lists some memory debugging tools (commercial ones and free ones); but then, so does http://phaseit.net/claird/comp.software.testing/mem_test.html, and the latter is more current.

I've tried some of the free tools and ended up in using mpatrol http://www.cbmamiga.demon.co.uk/mpatrol/, because it not only provides extensive debugging, profiling and tracing capabilities to help fix memory allocation errors, but also can help pinpoint memory leaks with their associated symbolic stack tracebacks. Another advantage is you don't need to add special memory debugging related code to your code and that you don't have to link against some special libraries. The biggest drawback is that, depending on which debugging options you use, the speed of your application may slow down dramatically, so that memory debugging of the core or of an extension with a long running script may become very time expensive (or even virtually impossible).

(The following is as of tcl8.4a4 and on Linux.) If you really want to memory debug the tcl core, you should build tcl with the -DPURIFY define (as mentioned above), with debugging symbols included and statically linked.

To do this, cd to the unix directory of your tcl source distribution. If you have the results of a prior compilation laying around, first do a
    make clean

Then do
    ./configure --enable-symbols --disable-shared

After configure has finished, edit the produced Makefile. Search for CFLAGS, and add -DPURIFY to it.

After that, do the usual

This should result in a (because of the static build unusual big) tclsh binary, that runs without dependences to libtcl, from anywhere, you move it. You don't need to do a make install, nor you should do this, because this would overwrite your normal tcl installation. (Or, if you insist in installing your static tclsh build, use the --prefix option in the ./configure call above, to point the installation to a location thats comfortable for you.)

If your mpatrol installation was successful (for how to do that, see the really extensive mpatrol documentation, included in the distribution), you now could start debugging. Do
    mpatrol --dynamic /path/to/your/static/build/tclsh testscript.tcl

This does produce a log file, named 'mpatrol.<processID>.log', in which you find the debugging information. Either with additional options or via a config file you could customize, what memory related operations mpatrol checks and / or logs. For example, if you want to check for memory leaks, you could use:
    mpatrol -g --dynamic --show-unfreed --leak-table tclsh testscript.tcl

This gives you a log file, with a summary of all not freed memory and a stack trace of every not freed allocation. To get the exact lines of code for the calls in the stack trace, use the 'mpsym' tool, that's included in the mpatrol distribution (you need to have gdb installed, for this to work):
    mpsym /path/to/your/static/build/tclsh testscript.tcl <mpatrol-log>

See the extensive mpatrol documentation, for which memory debugging options are available and what they do.

More probably, you will debug this great C coded tcl extension, you're writing (or this third party extension, that causes you trouble with your scripts).

For that, you only need a static tcl build (as described above), if you are suspect, that your use of some tcl API's has triggered a otherwise not spotted memory problem of the core - which is, with all respect, not that likely. If a tclsh with an extension loaded crashes or seems to leek memory, it may be wise, first to suspect the extension code. (If your analysis really shows, that the problem is in deed in the tcl core, don't hesitate to fill up a bug report. But first, do really analyze, and try hard, to track it down, and don't forget, to double check.)

Even without the need of a static tcl build it is helpful to compile tcl with the above mentioned -DPURIFY. This reduces the "noise" of false alarms and may help you, to detect your real problem(s) faster.

You get the most useful debugging information, if you use a custom tclsh with the extension in question compiled in. Even in these days of loadable extensions (which are normally recommended!) a lot, if not all extensions have a build target, that build such a custom tclsh. (And at least just for this debugging needs, extension maintainer should still provide such a build target.) Also helpful is, of course, to build the custom tclsh with debugging information included (This is done by adding the -g option to the compiler options).

If the extension in question really doesn't provide a custom tclsh build, it is normally very easy to build it by yourself. You find some information about this at Building a custom tclsh.

To actually doing the memory debugging just use the mpatrol and mpsym tools, as shown above, but only with your custom tclsh with your extension compiled in instead of the standard tclsh.

Rolf Ade

I recently ran valgrind [1] on the still unreleased tcl8.4b2. First impression of valgrind is very good. More on what I did at valgrind. MS

Anyone know if there are commonly available tclkit's with the memory command, etc. turned on?

NEM I doubt it. tclkit is meant as a deployment solution, so it is unlikely you would want debugging turned on in this situation.

[Talk about array statistics.]