Tclbench is a set of tools to compare performances between different Tcl and/or Tk versions.
Tclbench is a cvs module of the tcllib sourceforge project.
cvs -z3 -d:pserver:[email protected]:/cvsroot/tcllib co -P tclbench
The runbench.tcl application provides the overall management for the execution of benchmarks. If no specific benchmark files are provided on the command line itself the application will use all directory containing the application itself. If files are provided all files matching the glob pattern *tk* are assumed to be benchmarks for the Tk toolkit.
The following options are recognized by the application:
Kroc - 23/06/2004 : I compared two recents 8.4.6 Tclkit versions, a standard and an upx one, and an older 8.4.0. The bench were launched on three differents computers, running nothing else.
Tclkit 8.4.0 -> 08 min 48 sec 00 min 41 sec 10 min 12 sec + 24 % Tclkit 8.4.6 -> 07 min 33 sec 00 min 40 sec 08 min 13 sec + 0 % Tclkit 8.4.6 upx -> 09 min 03 sec 02 min 36 sec 11 min 39 sec + 42 %
Tclkit 8.4.0 -> 08 min 01 sec 01 min 35 sec 09 min 36 sec + 6 % Tclkit 8.4.6 -> 07 min 32 sec 01 min 31 sec 09 min 03 sec + 0 % Tclkit 8.4.6 upx -> 10 min 06 sec 05 min 32 sec 15 min 38 sec + 72 %
Tclkit 8.4.0 -> 06 min 16 sec 01 min 03 sec 07 min 19 sec + 33 % Tclkit 8.4.6 -> 04 min 31 sec 01 min 00 sec 05 min 31 sec + 0 % Tclkit 8.4.6 upx -> 05 min 05 sec 02 min 04 sec 07 min 09 sec + 30 %
First thing: a recent Tclkit version is always fastest than an older one, whatever platform and computer running it. So we could say TCT do their job ;^) jcw explained me upx results are bad because it is slower on startup. Anyway, the smaller the computer is, the slower upx was compared to standard Tclkit.
ferrieux - 20/10/2010: measuring perf effects in TEBC et al, I found it necessary to do some extra work to _denoise_ tclbench output:
if {$res*$iter<100000} { set iter [expr {int(200000.0/$res)}] set code [catch {uplevel \#0 [list time $opts(-body) $iter]} res] set res [lindex $res 0] }
The effect of these 3 tricks is a much greater stability. Remaining noise no longer occurs from one run to another, but rather exposes sensitivity to function offsets, instruction cache associativity, etc. Here again, other "spacers" sprinkled over the library have drastic effects. However, optimizing such things across processors and compilers is a daunting task. The bottom line is that tclbench can be used as a reliable speed comparison tool, but that an observed slowdown or speedup may be a very indirect consequence of code changes.