Version 17 of tclbench

Updated 2004-06-25 07:13:59 by kroc

This is a cvs module of the tcllib sourceforge project.

Tclbench is a set of tools to compare performances between different Tcl and/or Tk versions.

SYNOPSIS

 runbench.tcl ?-option value ...? ?file ...?

DESCRIPTION

The runbench.tcl application provides the overall management for the execution of bench�� marks. If no specific benchmark files are provided on the command line itself the application will use all directory containing the application itself. If files are provided all files matching the glob pattern *tk* are assumed to be benchmarks for the Tk toolkit.

The following options are recognized by the application:

-help
Causes the application to print a list of the recognized option plus short explanations of their meaning
-errors
Takes a boolean value (0 or 1). True means that errors are thrown, else they are ignored.
-iterations
The value of this option is a positive integer number and declares how often the body of benchmarks should be executed to get accurate timing results. This is a global maximum value which can be overidden by the individual benchmarks.
-minversion
Declares the minimum version of tcl interpreters the application is allowed to use in the benchmarking.
-maxversion
Declares the maximum version of tcl interpreters the application is allowed to use in the benchmarking.
-rmatch
Restricts the benchmarking to benchmarks whose description matches the regexp pattern specified as the value of this option. If both -match and -rmatch are specified benchmarks have to fulfill both conditions to be executed.
-match
Restricts the benchmarking to benchmarks whose description matches the glob pattern specified as the value of this option. If both -match and -rmatch are specified benchmarks have to fulfill both conditions to be executed.
-normalize
Takes a version number as value and causes the application to normalize the timing values to the value of the given version.
-notcl
Forces the application to skip all tcl related benchmarks. Takes no additional value.
-notk
Forces the application to skip all tk related benchmarks. Takes no additional value.
-output
Declares the style to use when generating the output for the benchmarks. Accepts text, csv and list. text is the default.
-paths
Accepts a list of paths to search for tcl interpreters. If no paths are specified the paths in the environment variable PATH are used.
-verbose
Activates the output of interim status info. Takes no additional value

Kroc - 23/06/2004 : I've done a TclBench.kit (v 1.15) to test Tclkit performance. It includes Tcllib 1.6.1 and Tclx 8.4 for linux and win32 but it should work on any platform.

DOWNLOAD

http://www.kroc.tk/tclkit/TclBench_1.15.2.kit

SYNOPSIS

 TclBench.kit ?-option value ...? ?file ...?

DESCRIPTION

Similar to runbench.tcl above, except:

-paths
looks for tclkit interpreters too.
-output
added file style to output results to benchmark.txt instead of stdout.

Example of use

I compared two recents 8.4.6 Tclkit versions, a standard and an upx one, and an older 8.4.0. The bench were launched on three differents computers, running nothing else. The command used was:

 TclBench_1.15.2.kit -verbose -paths ./ -output file
  • Laptop with Crusoe 600 MHz processor, 192 Mb RAM running linux 2.6.6 (full results here [L1 ]) :
TCL tests : TK tests : Total
Diff. :
 Tclkit 8.4.0     ->      08 min 48 sec         00 min 41 sec         10 min 12 sec         + 24 %
 Tclkit 8.4.6     ->      07 min 33 sec         00 min 40 sec         08 min 13 sec         +  0 %
 Tclkit 8.4.6 upx ->      09 min 03 sec         02 min 36 sec         11 min 39 sec         + 42 %
  • PC with Celeron 600 MHz processor, 128 Mb RAM running Windows 98 SE (full results here [L2 ]) :
TCL tests : TK tests : Total
Diff. :
 Tclkit 8.4.0     ->      08 min 01 sec         01 min 35 sec         09 min 36 sec         +  6 %
 Tclkit 8.4.6     ->      07 min 32 sec         01 min 31 sec         09 min 03 sec         +  0 %
 Tclkit 8.4.6 upx ->      10 min 06 sec         05 min 32 sec         15 min 38 sec         + 72 %
  • PC with Celeron 2400 MHz processor, 512 Mb RAM running Windows XP Family (full results here [L3 ]) :
TCL tests : TK tests : Total
Diff. :
 Tclkit 8.4.0     ->      06 min 16 sec         01 min 03 sec         07 min 19 sec         + 33 %
 Tclkit 8.4.6     ->      04 min 31 sec         01 min 00 sec         05 min 31 sec         +  0 %
 Tclkit 8.4.6 upx ->      05 min 05 sec         02 min 04 sec         07 min 09 sec         + 30 %

First thing: a recent Tclkit version is always fastest than an older one, whatever platform and computer running it. So we could say TCT do their job ;^) jcw explained me upx results are bad because it is slower on startup. Anyway, the smaller the computer is, the slower upx was compared to standard Tclkit.


category package | category performance