Version 22 of tclbench

Updated 2010-04-03 20:06:06 by dkf

This is a cvs module of the tcllib sourceforge project.

Tclbench is a set of tools to compare performances between different Tcl and/or Tk versions.

SYNOPSIS

runbench.tcl ?-option value ...? ?file ...?

DESCRIPTION

The runbench.tcl application provides the overall management for the execution of benchmarks. If no specific benchmark files are provided on the command line itself the application will use all directory containing the application itself. If files are provided all files matching the glob pattern *tk* are assumed to be benchmarks for the Tk toolkit.

The following options are recognized by the application:

-help
Causes the application to print a list of the recognized option plus short explanations of their meaning
-errors boolean
Takes a boolean value (0 or 1). True means that errors are thrown, else they are ignored.
-iterations count
The value of this option is a positive integer number and declares how often the body of benchmarks should be executed to get accurate timing results. This is a global maximum value which can be overidden by the individual benchmarks.
-minversion version
Declares the minimum version of tcl interpreters the application is allowed to use in the benchmarking.
-maxversion version
Declares the maximum version of tcl interpreters the application is allowed to use in the benchmarking.
-rmatch regexp
Restricts the benchmarking to benchmarks whose description matches the regexp pattern specified as the value of this option. If both -match and -rmatch are specified benchmarks have to fulfill both conditions to be executed.
-match pattern
Restricts the benchmarking to benchmarks whose description matches the glob pattern specified as the value of this option. If both -match and -rmatch are specified benchmarks have to fulfill both conditions to be executed.
-normalize version
Takes a version number as value and causes the application to normalize the timing values to the value of the given version.
-notcl
Forces the application to skip all tcl related benchmarks. Takes no additional value.
-notk
Forces the application to skip all tk related benchmarks. Takes no additional value.
-output style
Declares the style to use when generating the output for the benchmarks. Accepts text, csv and list. text is the default.
-paths list
Accepts a list of paths to search for tcl interpreters. If no paths are specified the paths in the environment variable PATH are used.
-verbose
Activates the output of interim status info. Takes no additional value

Kroc - 23/06/2004 : I compared two recents 8.4.6 Tclkit versions, a standard and an upx one, and an older 8.4.0. The bench were launched on three differents computers, running nothing else.

  • Laptop with Crusoe 600 MHz processor, 192 Mb RAM running linux 2.6.6 (full results here [L1 ]) :
TCL tests : TK tests : Total
Diff. :
 Tclkit 8.4.0     ->      08 min 48 sec         00 min 41 sec         10 min 12 sec         + 24 %
 Tclkit 8.4.6     ->      07 min 33 sec         00 min 40 sec         08 min 13 sec         +  0 %
 Tclkit 8.4.6 upx ->      09 min 03 sec         02 min 36 sec         11 min 39 sec         + 42 %
  • PC with Celeron 600 MHz processor, 128 Mb RAM running Windows 98 SE (full results here [L2 ]) :
TCL tests : TK tests : Total
Diff. :
 Tclkit 8.4.0     ->      08 min 01 sec         01 min 35 sec         09 min 36 sec         +  6 %
 Tclkit 8.4.6     ->      07 min 32 sec         01 min 31 sec         09 min 03 sec         +  0 %
 Tclkit 8.4.6 upx ->      10 min 06 sec         05 min 32 sec         15 min 38 sec         + 72 %
  • PC with Celeron 2400 MHz processor, 512 Mb RAM running Windows XP Family (full results here [L3 ]) :
TCL tests : TK tests : Total
Diff. :
 Tclkit 8.4.0     ->      06 min 16 sec         01 min 03 sec         07 min 19 sec         + 33 %
 Tclkit 8.4.6     ->      04 min 31 sec         01 min 00 sec         05 min 31 sec         +  0 %
 Tclkit 8.4.6 upx ->      05 min 05 sec         02 min 04 sec         07 min 09 sec         + 30 %

First thing: a recent Tclkit version is always fastest than an older one, whatever platform and computer running it. So we could say TCT do their job ;^) jcw explained me upx results are bad because it is slower on startup. Anyway, the smaller the computer is, the slower upx was compared to standard Tclkit.