Version 11 of Tcl Benchmarks

Updated 2002-09-11 06:23:37

Jeffrey Hobbs: There have been discussions about the changing relative speed of Tcl over the years (primarily rotating around the addition of the byte-code compiler - faster, but then also the changes to support i18n - slower).

I created the tclbench [L1 ] application to get a better understanding of what has truly happened to performance over the years, with the forward looking view of finding the pain points and fixing them. Anyone can get tclbench from CVS [L2 ] to do relative benchmarks themselves, but below I've posted the results from a full run over the benchmark suite. All builds represent the final code of that version.

A few have noted that these numbers aren't the easiest to read overall, which I can agree with. Take a look at the Tcl Normalized Benchmarks that set 8.4 as a norm of 1.0 to compare against.

If you've got BLT handy, then here is a little code for Plotting Tcl Benchmarks.

The speed enhancements in 8.4 are primarily achieved by work done by myself and Eric Melski at Ajuba and continued at ActiveState [L3 ]. Addition optimizations are also done by Miguel Sofer.

For those interested in the use of threads for concurrent processing, please see the Tcl Benchmarks with Threads.

The numbers are microseconds, but you should only look at them relative to each other. Lower is better. There is some fluctuation expected as Tcl's 'time' does wall timing, not true cpu time. In any case, these were done on a dual P3-550 with 512MB physmem SuSE 7.3 box. Dual doesn't make any difference - all interpreters were compiled static with no threads.


 TCL_INTERP: 1:8.4.0 2:8.3.4 3:8.2.3 4:8.0.5 5:7.6p2
 STARTED 2002-09-10 22:46:33 (runbench.tcl v1.14)
 Benchmark 1:8.4.0 /home/jeffh/install/linux-ix86/bin/tclsh8.4
 bbccdeeefghkllmmmmpprrssstuvw 00:05:51 elapsed
 Benchmark 2:8.3.4 /home/jeffh/install/linux-ix86/bin/tclsh8.3
 bbccdeeefghkllmmmmpprrssstuvw 00:08:28 elapsed
 Benchmark 3:8.2.3 /home/jeffh/install/linux-ix86/bin/tclsh8.2
 bbccdeeefghkllmmmmpprrssstuvw 00:08:39 elapsed
 Benchmark 4:8.0.5 /home/jeffh/install/linux-ix86/bin/tclsh8.0
 bbccdeeefghkllmmmmpprrssstuvw 00:07:08 elapsed
 Benchmark 5:7.6p2 /home/jeffh/install/linux-ix86/bin/tclsh7.6
 bbccdeeefghkllmmmmpprrssstuvw 00:20:26 elapsed
 000 VERSIONS:                               1:8.4.0 2:8.3.4 3:8.2.3 4:8.0.5 5:7.6p2
 001 BASE64 decode 10                            436     498     466     472     -=-
 002 BASE64 decode 100                          3539    3944    3884    3939     -=-
 003 BASE64 decode 1000                        34455   38611   37932   38538     -=-
 004 BASE64 decode 10000                      343203  383174  385355  387199     -=-
 005 BASE64 decode2 10                           433     488     457     462     -=-
 006 BASE64 decode2 100                         3496    3871    3758    3864     -=-
 007 BASE64 decode2 1000                       33941   38024   36519   37648     -=-
 008 BASE64 decode2 10000                     338565  377294  372193  376979     -=-
 009 BASE64 decode3 10                           173     288     274     234     -=-
 010 BASE64 decode3 100                         1194    2209    2045    1765     -=-
 011 BASE64 decode3 1000                       11318   21411   19678   17008     -=-
 012 BASE64 decode3 10000                     112723  216587  196838  170605     -=-
 013 BASE64 encode 10                            235     359     338     388     -=-
 014 BASE64 encode 100                          1833    2968    2956    3449     -=-
 015 BASE64 encode 1000                        17810   29235   28959   34248     -=-
 016 BASE64 encode 10000                      176749  292249  289969  340641     -=-
 017 BASE64 encode2 10                           224     350     321     372     -=-
 018 BASE64 encode2 100                         1760    2923    2807    3348     -=-
 019 BASE64 encode2 1000                       17119   28467   27288   33099     -=-
 020 BASE64 encode2 10000                     169688  283957  272785  330257     -=-
 021 BASE64 encode3 10                           122     184     192     154     -=-
 022 BASE64 encode3 100                          575    1170    1273     986     -=-
 023 BASE64 encode3 1000                        4946   10911   11946    9175     -=-
 024 BASE64 encode3 10000                      48212  109527  119610   91049     -=-
 025 BIN bitset-v1 1000 chars                  13562   16178   15683   14116     -=-
 026 BIN bitset-v1 5000 chars                  65697   79293   78416   70536     -=-
 027 BIN bitset-v1 10000 chars                131239  158357  158299  141484     -=-
 028 BIN bitset-v2 1000 chars                   7888   10162    9099    8111     -=-
 029 BIN bitset-v2 5000 chars                  38797   50351   45213   40465     -=-
 030 BIN bitset-v2 10000 chars                 77573  102252   90780   80709     -=-
 031 BIN bitset-v3 1000 chars                   2073    4773    4614    4044     -=-
 032 BIN bitset-v3 5000 chars                   9260   24100   22496   20034     -=-
 033 BIN bitset-v3 10000 chars                 18503   48328   46049   40658     -=-
 034 BIN c scan, 1000b                           588     402     390     307     -=-
 035 BIN c scan, 5000b                          1986    1895    1841    1464     -=-
 036 BIN c scan, 10000b                         3700    4246    3650    2967     -=-
 037 CATCH error, complex                         29      32      31      27      31
 038 CATCH no catch used                           7       7       6       6      14
 039 CATCH return error                           29      31      29      25      30
 040 CATCH return except                           7       7       6       6      18
 041 CATCH return ok                               7       7       6       6      21
 042 DATA access in a list                       630    2044    1787    1719   40475
 043 DATA access in an array                    1000    2059    2074    1830    7173
 044 DATA create in a list                       562    2179    2163    2060    6550
 045 DATA create in an array                    1786    2687    2614    3264    8380
 046 ENC iso2022-jp, gets                      10718    9536    9913     -=-     -=-
 047 ENC iso2022-jp, read                      10552    9379   10013     -=-     -=-
 048 ENC iso2022-jp, read & size               10607    9411    9684     -=-     -=-
 049 ENC iso8859-2, gets                         949     621     577     -=-     -=-
 050 ENC iso8859-2, read                         784     452     441     -=-     -=-
 051 ENC iso8859-2, read & size                  791     462     451     -=-     -=-
 052 EVAL cmd and mixed lists                   1846    1416    3860    2713     564
 053 EVAL cmd eval as list                         8      10      29      26      11
 054 EVAL cmd eval as string                      28      25      30      26      12
 055 EVAL cmd eval in list obj var                 8      11      30      27      15
 056 EVAL list cmd and mixed lists              1837    1419    3885    2714     566
 057 EVAL list cmd and pure lists                227     235    3867    2712     566
 058 EXPR $a != $b int                             5       6       5       5      41
 059 EXPR $a != $b str (!= len)                   11      16      14      13      60
 060 EXPR $a != $b str (== len)                   14      16      14      13      59
 061 EXPR $a == $b int                             5       6       5       5      40
 062 EXPR $a == $b str (!= len)                   11      16      14      13      58
 063 EXPR $a == $b str (== len)                   14      16      14      13      60
 064 EXPR braced                                  11      12      12      10      79
 065 EXPR fifty operands                          19      21      21      18      84
 066 EXPR incr with expr                           3       3       3       3      15
 067 EXPR incr with incr                           3       5       4       3      11
 068 EXPR inline                                   7      13      13      11      36
 069 EXPR one operand                              3       3       3       3      13
 070 EXPR ten operands                             6       6       6       5      27
 071 EXPR unbraced                               125      90      91      74      76
 072 FCOPY binary: 160K                         8480    8526    8534     -=-     -=-
 073 FCOPY encoding: 160K                      31317    8586    8281     -=-     -=-
 074 FCOPY std: 160K                            8216    8621    8314    8746    8935
 075 GCCont_cpb::cGCC 50                         273     583     611     509     -=-
 076 GCCont_cpb::cGCC 500                       1137    3489    3891    3130     -=-
 077 GCCont_cpb::cGCC 5000                      9754   33212   36715   29462     -=-
 078 GCCont_cpbre1::cGCC 50                      372     394     -=-     -=-     -=-
 079 GCCont_cpbre1::cGCC 500                    2439    2473     -=-     -=-     -=-
 080 GCCont_cpbre1::cGCC 5000                  23118   23284     -=-     -=-     -=-
 081 GCCont_cpbre2::cGCC 50                      305     324     -=-     -=-     -=-
 082 GCCont_cpbre2::cGCC 500                    2244    2272     -=-     -=-     -=-
 083 GCCont_cpbre2::cGCC 5000                  21629   21740     -=-     -=-     -=-
 084 GCCont_cpbrs2::cGCC 50                       97     348     345     -=-     -=-
 085 GCCont_cpbrs2::cGCC 500                     375    2595    2588     -=-     -=-
 086 GCCont_cpbrs2::cGCC 5000                   3137   24962   25022     -=-     -=-
 087 GCCont_cpbrs::cGCC1 50                      170     533     531     376     -=-
 088 GCCont_cpbrs::cGCC1 500                     476    2920    2918     706     -=-
 089 GCCont_cpbrs::cGCC1 5000                   3494   26516   26585    3952     -=-
 090 GCCont_cpbrs::cGCC2 50                      136     582     579     355     -=-
 091 GCCont_cpbrs::cGCC2 500                     404    3043    3028     638     -=-
 092 GCCont_cpbrs::cGCC2 5000                   3044   27373   27372    3304     -=-
 093 GCCont_cpbrs_trap::cGCC 50                  354     377     376     229     -=-
 094 GCCont_cpbrs_trap::cGCC 500                2369    2476    2475    1197     -=-
 095 GCCont_cpbrs_trap::cGCC 5000              22485   23353   23411   10827     -=-
 096 GCCont_expr::cGCC 50                        265     249     247     -=-     -=-
 097 GCCont_expr::cGCC 500                      3092    3209    3208     -=-     -=-
 098 GCCont_expr::cGCC 5000                   399559  416633  425794     -=-     -=-
 099 GCCont_i::cGCC1 50                          443     461     476     429     -=-
 100 GCCont_i::cGCC1 500                        4018    4139    4349    3967     -=-
 101 GCCont_i::cGCC1 5000                      39715   40404   43683   39504     -=-
 102 GCCont_i::cGCC2 50                          145     223     242     197     -=-
 103 GCCont_i::cGCC2 500                        1066    1743    2055    1692     -=-
 104 GCCont_i::cGCC2 5000                      10271   16970   20205   16570     -=-
 105 GCCont_i::cGCC3 50                          131     211     232     187     -=-
 106 GCCont_i::cGCC3 500                         913    1624    1931    1573     -=-
 107 GCCont_i::cGCC3 5000                       8714   15629   19261   15302     -=-
 108 GCCont_r1::cGCC 50                          620     686     724     578     -=-
 109 GCCont_r1::cGCC 500                        5857    6587    7017    5570     -=-
 110 GCCont_r1::cGCC 5000                      58152   66481   70427   55707     -=-
 111 GCCont_r2::cGCC 50                          517     470     550     433     -=-
 112 GCCont_r2::cGCC 500                        4894    4241    5128    4083     -=-
 113 GCCont_r2::cGCC 5000                      47641   42473   51190   40716     -=-
 114 GCCont_r3::cGCC 50                          529     483     555     434     -=-
 115 GCCont_r3::cGCC 500                        4872    4379    5212    4040     -=-
 116 GCCont_r3::cGCC 5000                      48309   43732   51795   40306     -=-
 117 GCCont_rsf1::cGCC 50                        180     240     291     -=-     -=-
 118 GCCont_rsf1::cGCC 500                      1059    1589    2194     -=-     -=-
 119 GCCont_rsf1::cGCC 5000                     9814   15145   21173     -=-     -=-
 120 GCCont_rsf2::cGCC1 50                       130     181     200     -=-     -=-
 121 GCCont_rsf2::cGCC1 500                      624    1051    1313     -=-     -=-
 122 GCCont_rsf2::cGCC1 5000                    5524    9677   12271     -=-     -=-
 123 GCCont_rsf2::cGCC2 50                       114     147     168     -=-     -=-
 124 GCCont_rsf2::cGCC2 500                      593     957    1198     -=-     -=-
 125 GCCont_rsf2::cGCC2 5000                    5348    8997   11555     -=-     -=-
 126 GCCont_rsf3::cGCC 50                        116     147     168     -=-     -=-
 127 GCCont_rsf3::cGCC 500                       594     955    1198     -=-     -=-
 128 GCCont_rsf3::cGCC 5000                     5350    9001   11547     -=-     -=-
 129 GCCont_turing::cGCC 50                       80     142     140     -=-     -=-
 130 GCCont_turing::cGCC 500                     285     835     833     -=-     -=-
 131 GCCont_turing::cGCC 5000                   2330    7827    8028     -=-     -=-
 132 HEAPSORT size 10                            275     379     364    4509    2680
 133 HEAPSORT size 50                           2286    3020    2870   26405   22377
 134 HEAPSORT size 100                          5477    7165    6928   55668   53408
 135 HEAPSORT2 size 10                           238     -=-     -=-     -=-     -=-
 136 HEAPSORT2 size 50                          1913     -=-     -=-     -=-     -=-
 137 HEAPSORT2 size 100                         4514     -=-     -=-     -=-     -=-
 138 IF 1/0 check                                  4       8       6       6      27
 139 IF else true al                              14      15      14      12      44
 140 IF else true numeric                          6       8       7       7      38
 141 IF elseif true al                            11      15      14      12      43
 142 IF elseif true numeric                        6       8       8       7      39
 143 IF if false al/al                            10      10       9       8      32
 144 IF if false al/num                            8       9       8       7      53
 145 IF if false num/num                           5       6       6       6      33
 146 IF if true al                                 5      10       9       8      34
 147 IF if true al/al                              5      11      10       8      33
 148 IF if true num/num                            5       7       6       6      32
 149 IF if true numeric                            5       6       6       5      30
 150 IF multi 1st true                             5       6       6       6      50
 151 IF multi 2nd true                             6       8       7       7      53
 152 IF multi 9th true                            11      16      15      14      90
 153 IF multi default true                        11      16      15      14      92
 154 KLIST shuffle0 llength 1                     50      76      70      66    8.0+
 155 KLIST shuffle0 llength 10                   116     227     213     197    8.0+
 156 KLIST shuffle0 llength 100                  890    1842    1753    1568    8.0+
 157 KLIST shuffle0 llength 1000               10530   19753   19083   16924    8.0+
 158 KLIST shuffle0 llength 10000             146661  237772  231811  206514     -=-
 159 KLIST shuffle1-s llength 1                   44      47      45      34     120
 160 KLIST shuffle1-s llength 10                 194     219     212     210     924
 161 KLIST shuffle1-s llength 100               2535    2804    2596    2299   15627
 162 KLIST shuffle1-s llength 1000            268694  273514  243721  239235  936457
 163 KLIST shuffle1a llength 1                    50      56      52      40     153
 164 KLIST shuffle1a llength 10                  190     238     245     233    1203
 165 KLIST shuffle1a llength 100                1750    2188    2331    2150   19837
 166 KLIST shuffle1a llength 1000              17419   21931   23392   21587 1150255
 167 KLIST shuffle1a llength 10000            175102  220280  234814  219602     -=-
 168 KLIST shuffle2 llength 1                     50      65      62      66     154
 169 KLIST shuffle2 llength 10                   199     282     268     351     984
 170 KLIST shuffle2 llength 100                 1844    2558    2424    3162    9395
 171 KLIST shuffle2 llength 1000               18538   25822   24183   31487   93957
 172 KLIST shuffle2 llength 10000             205823  276485  260922  340704     -=-
 173 KLIST shuffle3 llength 1                     44      57      54      44     120
 174 KLIST shuffle3 llength 10                   152     229     238     234     880
 175 KLIST shuffle3 llength 100                 1354    2056    2171    1972   10597
 176 KLIST shuffle3 llength 1000               15197   22003   22845   20579  348281
 177 KLIST shuffle3 llength 10000             368559  441199  460021  432061     -=-
 178 KLIST shuffle4 llength 1                     46      58      57      46     128
 179 KLIST shuffle4 llength 10                   161     248     253     248     975
 180 KLIST shuffle4 llength 100                 1437    2211    2303    2124   14367
 181 KLIST shuffle4 llength 1000               13903   21684   22974   20816  664493
 182 KLIST shuffle4 llength 10000             141074  218045  232679  210536     -=-
 183 KLIST shuffle5-s llength 1                   18      22      21      18      90
 184 KLIST shuffle5-s llength 10                 127     144     129     109     642
 185 KLIST shuffle5-s llength 100               1438    1614    1362    1114    7219
 186 KLIST shuffle5-s llength 1000             76835   78658   60202   60174  207552
 187 KLIST shuffle5a llength 1                    21      25      23      20     104
 188 KLIST shuffle5a llength 10                  122     139     131     113     770
 189 KLIST shuffle5a llength 100                1112    1243    1168    1020    8792
 190 KLIST shuffle5a llength 1000              12346   13483   12825   12075  268428
 191 KLIST shuffle5a llength 10000            338446  351454  341989  341270     -=-
 192 KLIST shuffle6 llength 1                      5     -=-     -=-     -=-     -=-
 193 KLIST shuffle6 llength 10                    58     -=-     -=-     -=-     -=-
 194 KLIST shuffle6 llength 100                  519     -=-     -=-     -=-     -=-
 195 KLIST shuffle6 llength 1000                5199     -=-     -=-     -=-     -=-
 196 KLIST shuffle6 llength 10000              53487     -=-     -=-     -=-     -=-
 197 LIST append to list                           7      16      14      13      11
 198 LIST concat APPEND 2x10                      42      47      45      36      21
 199 LIST concat APPEND 2x100                    251     254     251     205      65
 200 LIST concat APPEND 2x1000                  2343    2333    2374    1934     463
 201 LIST concat APPEND 2x10000                25592   25900   26423   22152    4846
 202 LIST concat CONCAT 2x10                      12      15      32      26      25
 203 LIST concat CONCAT 2x100                     19      22     203     163      86
 204 LIST concat CONCAT 2x1000                   109     112    2006    1660     638
 205 LIST concat CONCAT 2x10000                 1023    1025   21986   18507    6632
 206 LIST concat EVAL/LAPPEND 2x10                26      28      49      44      35
 207 LIST concat EVAL/LAPPEND 2x100               37      39     122     104     135
 208 LIST concat EVAL/LAPPEND 2x1000             154     155     859     723    1064
 209 LIST concat EVAL/LAPPEND 2x10000           1748    1848    9407    7095   11091
 210 LIST concat FOREACH/LAPPEND 2x10             22      48      46      41      59
 211 LIST concat FOREACH/LAPPEND 2x100           104     332     317     289     365
 212 LIST concat FOREACH/LAPPEND 2x1000          938    3141    3060    2775    3377
 213 LIST concat FOREACH/LAPPEND 2x10000        9653   31650   30865   28339   34018
 214 LIST concat SET 2x10                         29      32      30      25      22
 215 LIST concat SET 2x100                       209     207     207     169      70
 216 LIST concat SET 2x1000                     2014    1990    2005    1629     509
 217 LIST concat SET 2x10000                   21951   21838   22021   18244    5355
 218 LIST exact search, first item                 6       7       6       5     431
 219 LIST exact search, last item                 36      32      31      29     448
 220 LIST exact search, middle item               15      14      13      11     439
 221 LIST exact search, non-item                  99      92      91      86     471
 222 LIST exact search, typed item                29      32      31      29     448
 223 LIST exact search, untyped item              35      32      31      29     448
 224 LIST index first element                      4       6       5       5     143
 225 LIST index last element                       4       6       5       5     176
 226 LIST index middle element                     4       6       5       5     159
 227 LIST insert an item at "end"                113     114     115     110     384
 228 LIST insert an item at middle               122     120     120     119     238
 229 LIST insert an item at start                120     121     121     118     222
 230 LIST iterate list                          1462    4129    3645    3423  352676
 231 LIST join list                              607     613     614     389    1123
 232 LIST large, early range                      13      15      13      11     174
 233 LIST large, late range                       13      14      13      12     183
 234 LIST length, pure list                        4       7       6       5     501
 235 LIST list                                    16      24      22      21      59
 236 LIST remove first element                   120     120     121     119     190
 237 LIST remove in mixed list                   136     134    1106     130     236
 238 LIST remove last element                    118     118     120     117     254
 239 LIST remove middle element                  118     122     120     118     236
 240 LIST replace first el with multiple         122     122     122     121     233
 241 LIST replace first element                  113     115     114     111     222
 242 LIST replace in mixed list                  130     128    1095     124     240
 243 LIST replace last el with multiple          116     116     116     113     397
 244 LIST replace last element                   113     115     114     110     389
 245 LIST replace middle el with multiple        117     120     119     118     321
 246 LIST replace middle element                 113     114     115     111     312
 247 LIST replace range                          112     115     114     109     281
 248 LIST small, early range                       8       9       9       7     147
 249 LIST small, late range                        8       9       9       7     175
 250 LIST sort                                  1578    1599    1560    1508    1550
 251 LIST sorted search, first item                7       7       6       5     431
 252 LIST sorted search, last item                 8      32      31      29     448
 253 LIST sorted search, middle item               7      14      12      11     439
 254 LIST sorted search, non-item                  7      91      91      86     470
 255 LIST sorted search, typed item                7      32      31      29     448
 256 LIST typed sort                            1379    1378    1336    1091    5495
 257 LOOP for (to 1000)                          741    1613    1568    1527   12011
 258 LOOP for, iterate list                     1255    3009    2716    2525  147329
 259 LOOP for, iterate string                   2007    4043    4299    3447   37376
 260 LOOP foreach, iterate list                  681    1193    1174    1091    3312
 261 LOOP foreach, iterate string               1022    1407    2138    1857    3038
 262 LOOP while (to 1000)                        741    1612    1568    1527   12200
 263 LOOP while 1 (to 1000)                      692    1654    1546    1446   13261
 264 MAP ([chars])-case regsub                   174     178     178      34      54
 265 MAP http mapReply                        169383  164815  164656  153677   22481
 266 MAP regsub -nocase, no match               1875    1660    1652     932    1236
 267 MAP regsub 1 val                            341    2680    2669     434     865
 268 MAP regsub 1 val -nocase                   1093    2674    2679     594    1073
 269 MAP regsub 2 val                            868   11858   11810    1097    1892
 270 MAP regsub 2 val -nocase                   2216   11855   11825    1434    2279
 271 MAP regsub 3 val                           1143   16166   16186    1498    2490
 272 MAP regsub 3 val -nocase                   3127   16163   16182    1982    3024
 273 MAP regsub 4 val                           1493   20410   20453    1881    3077
 274 MAP regsub 4 val -nocase                   4039   20424   20448    2510    3759
 275 MAP regsub short                             26     111     109      26      56
 276 MAP regsub, no match                        303    1662    1652     602     799
 277 MAP string -nocase, no match               2115   11902   11913   nomap   nomap
 278 MAP string 1 val                            312    4092    4142   nomap   nomap
 279 MAP string 1 val -nocase                   1082    6447    6430   nomap   nomap
 280 MAP string 2 val                            807    4993    5024   nomap   nomap
 281 MAP string 2 val -nocase                   2078    9332    9381   nomap   nomap
 282 MAP string 3 val                            998    5942    5989   nomap   nomap
 283 MAP string 3 val -nocase                   2712   12165   12190   nomap   nomap
 284 MAP string 4 val                           1267    6431    6467   nomap   nomap
 285 MAP string 4 val -nocase                   3337   14175   14065   nomap   nomap
 286 MAP string short                             19      22      21   nomap   nomap
 287 MAP string, no match                        488    5919    5980   nomap   nomap
 288 MAP |-case regsub                            85      86      84      26      41
 289 MAP |-case strmap                            20      26      25   nomap   nomap
 290 MATRIX mult 5x5                             367     719     699     590    3487
 291 MATRIX mult 10x10                          2377    4373    4296    3597   25932
 292 MATRIX mult 15x15                          7612   13683   13441   11256   90871
 293 MATRIX transposition-0                     3426   11025   10575    8666   28465
 294 MATRIX transposition-1                     2398    7663    6598    6039  108401
 295 MD5 msg len 10                             1411    1434    1334    1311     -=-
 296 MD5 msg len 100                            2402    2517    2328    2329     -=-
 297 MD5 msg len 1000                          16353   17927   16536   16743     -=-
 298 MD5 msg len 10000                        156623  172929  159633  161055     -=-
 299 MD5-2 msg len 10                            653     591     565     490     -=-
 300 MD5-2 msg len 100                           942     891     866     761     -=-
 301 MD5-2 msg len 1000                         4931    5089    5033    4489     -=-
 302 MD5-2 msg len 10000                       45051   47405   47112   41906     -=-
 303 MTHD array stored proc call                   5       5       4       4     -=-
 304 MTHD call absolute                            9      16      12      13     -=-
 305 MTHD call relative                           15      16      13      13     -=-
 306 MTHD direct ns proc call                      3       3       3       3     -=-
 307 MTHD imported ns proc call                    3       3       3       3     -=-
 308 MTHD indirect proc eval                      13      14      32      30     -=-
 309 MTHD indirect proc eval #2                   26      25      31      29     -=-
 310 MTHD inline call                              1       2       1       1     -=-
 311 MTHD interp alias proc call                   5      10       9       4     -=-
 312 MTHD ns lookup call                          46      48      67      56     -=-
 313 MTHD switch method call                      20      21      40      39     -=-
 314 PARSE html form upload (7820)             10459   11905   11549   10826     -=-
 315 PARSE html form upload (974068)         1278019 1455554 1421775 1340903     -=-
 316 PROC do-nothing, no args                      3       3      14       2       1
 317 PROC do-nothing, one arg                      3       3       3       2       4
 318 PROC empty, no args                           1       3      15       2       4
 319 PROC empty, use args                          1       4       4       3       6
 320 PROC explicit return                          3       4       3       3       6
 321 PROC explicit return (2)                      3       4       3       3       5
 322 PROC explicit return (3)                      3       4       3       3       5
 323 PROC heavily commented                        3       3       2       2     194
 324 PROC implicit return                          3       3       3       3       8
 325 PROC implicit return (2)                      3       3       3       3       6
 326 PROC implicit return (3)                      3       3       3       2       4
 327 PROC local links with global                544     664     658     640    1574
 328 PROC local links with upvar                 492     594     587     585    1584
 329 PROC local links with variable              479     562     549     341    1555
 330 RE 1-char long-end                           45      45      45     586     624
 331 RE 1-char long-end catching                  79      97      95     613     650
 332 RE 1-char long-middle                        29      31      29     298     336
 333 RE 1-char long-middle catching               64      82      82     326     363
 334 RE 1-char long-start                         13      16      14      11      48
 335 RE 1-char long-start catching                48      67      66      39      74
 336 RE 1-char short                              13      16      14      11      13
 337 RE 1-char short catching                     48      66      66      38      39
 338 RE basic                                     12      15      12      11      13
 339 RE basic catching                            53      71      70      38      38
 340 RE c-comment long                            55      58      56      85     124
 341 RE c-comment long catching                  172     187     188     122     197
 342 RE c-comment long nomatch                    77      80      81      17      90
 343 RE c-comment long nomatch catching           80      80      83      20      98
 344 RE c-comment long pmatch                     86      87      86     557     633
 345 RE c-comment long pmatch catching            88      89      88     560     641
 346 RE c-comment many *s                         89      91      89     159     235
 347 RE c-comment many *s catching               273     297     296     203     357
 348 RE c-comment nomatch                         18      21      19       9      13
 349 RE c-comment nomatch catching                20      23      20      12      21
 350 RE c-comment simple                          33      35      32      20      25
 351 RE c-comment simple catching                124     142     139      50      52
 352 RE count all matches                         99      98     285     240     491
 353 RE extract all matches                      120     116     347     287     646
 354 RE literal regexp                            21      23      20      15      16
 355 RE n-char long-end                           43      46      46     663     702
 356 RE n-char long-end catching                  89     107     106     691     731
 357 RE n-char long-middle                        29      32      30     337     375
 358 RE n-char long-middle catching               74      92      91     365     403
 359 RE n-char long-start                         14      17      14      11      50
 360 RE n-char long-start catching                58      77      76      39      78
 361 RE n-char short                              14      17      14      11      13
 362 RE n-char short catching                     57      75      75      40      37
 363 RE static long match                         15      34      33      23      59
 364 RE static long nomatch                       24      48      47      25      60
 365 RE static short match                         4      15      14       6       8
 366 RE static short nomatch                       4      10       8       5       8
 367 RE var-based regexp                          21      24      22      16      20
 368 READ 595K, gets                          176939  266765  216253  116650  215808
 369 READ 595K, read                           50795   55929   57094  219396  239918
 370 READ 595K, read & size                    50883   56123   57252   17582  240694
 371 READ 3050b, gets                            983    1449    1189     664    1214
 372 READ 3050b, read                            339     346     343     159     262
 373 READ 3050b, read & size                     354     357     353     164     275
 374 READ bin 595K, gets                      183532  278429  227663  116683  215652
 375 READ bin 595K, read                       12082   11949   13417  220957  239312
 376 READ bin 595K, read & size                12133   11957   13453   17608  239425
 377 READ bin 3050b, gets                       1020    1516    1251     691    1224
 378 READ bin 3050b, read                        146     153     146     174     271
 379 READ bin 3050b, read & size                 160     162     155     179     282
 380 SHA sha1 msg len 10                        3207    3747    3568    3904     -=-
 381 SHA sha1 msg len 100                       5828    7067    6626    7327     -=-
 382 SHA sha1 msg len 1000                     42977   52089   49769   55529     -=-
 383 SHA sha1 msg len 10000                   414992  515575  488466  539124     -=-
 384 SHA sha1DF msg len 10                      1239    2187    2014    1781     -=-
 385 SHA sha1DF msg len 100                     2352    4204    3888    3433     -=-
 386 SHA sha1DF msg len 1000                   17879   32552   30029   26609     -=-
 387 SHA sha1DF msg len 10000                 174755  318100  296115  260303     -=-
 388 STARTUP time to launch tclsh              12828   10978   11390   11814    5768
 389 STR append                                   50      39      37      29     246
 390 STR append (1KB + 1KB)                       19      31      30      24     149
 391 STR append (1MB + (1b + 1K + 1b) * 100)    7922   15594   15636   15601  114920
 392 STR append (1MB + 1KB * 20)                6921   14508   14693   14711  110836
 393 STR append (1MB + 1KB * 1000)             14026   24267   23503   23059  153275
 394 STR append (1MB + 1KB)                     6688   14374   14588   14638  109988
 395 STR append (1MB + 1MB * 3)                29739   37254   36668   37158  324695
 396 STR append (1MB + 1MB * 5)                36933   74074   73918   74430  425502
 397 STR append (1MB + 2b * 1000)               7777   18513   18092   17988  116498
 398 STR append (10KB + 1KB)                      45      90      85      79     650
 399 STR first (failure)                          42      35      33      33     189
 400 STR first (failure) utf                      42     133     129      31     188
 401 STR first (success)                           6       9       7       6     151
 402 STR first (success) utf                       6      10       9       6     151
 403 STR first (total failure)                    36      16      15      15     181
 404 STR first (total failure) utf                35      18      16      15     179
 405 STR index 0                                   6      10       9       7     177
 406 STR index 100                                 6      10       9       7     177
 407 STR index 500                                 6      10       9       7     177
 408 STR info locals match                       438     515     502     481    1429
 409 STR last (failure)                           56      64      62      49     189
 410 STR last (success)                            6     132     131       6     192
 411 STR last (total failure)                     43      51      50      35     180
 412 STR length (==4010)                           4       6       6       5     177
 413 STR length of a LIST                          4       6       5       5     172
 414 STR length static str                         4       6       5       5      20
 415 STR match, complex (failure)                 86     626     626     265     403
 416 STR match, complex (success early)            5      13      12       8     148
 417 STR match, complex (success late)           101     637     636     269     407
 418 STR match, complex (total failure)           70     615     615     261     397
 419 STR match, exact (failure)                    3       5       4       4       7
 420 STR match, exact (success)                    3       6       5       4       7
 421 STR match, exact -nocase (failure)            4       6       5     ERR     ERR
 422 STR match, exact -nocase (success)            7       8       7     ERR     ERR
 423 STR match, simple (failure)                   4       5       4       4       9
 424 STR match, simple (success)                   4       5       4       4      10
 425 STR range, index 100..200 of 4010             9      11      10       7     182
 426 STR repeat, 4010 chars * 10                  88      89     179     216    1796
 427 STR repeat, 4010 chars * 100               1671    1579    3497    3581   15518
 428 STR repeat, abcdefghij * 10                   8       8      11      43     163
 429 STR repeat, abcdefghij * 100                 26      27      49     344    1337
 430 STR repeat, abcdefghij * 1000               209     214     407    3351   13191
 431 STR replace, equal replacement               38     177     175      26     401
 432 STR replace, longer replacement              40     177     175      26     401
 433 STR replace, no replacement                  37     176     176      26     400
 434 STR reverse iter/append, 10 c                46      87      83      64     176
 435 STR reverse iter/append, 10 uc               44      87      81      63     177
 436 STR reverse iter/append, 100 c              305     696     654     497    1654
 437 STR reverse iter/append, 100 uc             300     681     639     506    1659
 438 STR reverse iter/append, 400 c             1170    2750    2559    1958    9689
 439 STR reverse iter/append, 400 uc            1146    2684    2497    1963    9725
 440 STR reverse iter/set, 10 c                   44      63      63      52     183
 441 STR reverse iter/set, 10 uc                  41      62      63      52     182
 442 STR reverse iter/set, 100 c                 381     511     529     440    1804
 443 STR reverse iter/set, 100 uc                378     500     536     439    1811
 444 STR reverse iter/set, 400 c                1645    2121    2181    1769   11851
 445 STR reverse iter/set, 400 uc               1609    2086    2183    1778   11813
 446 STR reverse recursive, 10 c                 118     149     136     105     328
 447 STR reverse recursive, 10 uc                112     149     137     105     327
 448 STR reverse recursive, 100 c               1140    1473    1379    1077    3478
 449 STR reverse recursive, 100 uc              1095    1899    1806    1080    3475
 450 STR reverse recursive, 400 c               4838    6122    5775    4543    8.0+
 451 STR reverse recursive, 400 uc              5746   14150   13860    4578    8.0+
 452 STR split iter, 4000 uchars                4333    5905    8609    7424   12479
 453 STR split iter, 4010 chars                 4286    5846    8598    7439   12338
 454 STR split iter, rand 100 c                  251     288     230     196     -=-
 455 STR split iter, rand 1000 c                1511    1894    2178    1870     -=-
 456 STR split iter, rand 10000 c              11372   15127   22075   18622     -=-
 457 STR split, 4000 uchars                     1709    1192    3877    3030    1061
 458 STR split, 4010 chars                      1658    1152    3882    3066    1030
 459 STR split, rand 100 c                       178     162     108      84     -=-
 460 STR split, rand 1000 c                      859     730    1007     775     -=-
 461 STR split, rand 10000 c                    4802    3487   11001    7561     -=-
 462 STR str $a eq $b                              7      16      15      14      54
 463 STR str $a eq $b (same obj)                   7      16      15      14      53
 464 STR str $a equal ""                           5      11      11       8      63
 465 STR str $a ne $b                              7      17      15      15      49
 466 STR str $a ne $b (same obj)                   6      17      15      14      50
 467 STR str num == ""                             6       8       9       8      63
 468 STR string compare                            7      13      11      11      21
 469 STR string compare ""                         8      11      11      12      39
 470 STR string compare long                     131     132     129     129     661
 471 STR string compare long (same obj)            5     131     129     129     664
 472 STR string compare mixed long               187     136     134     133     664
 473 STR string compare uni long                 130     150     148     128     662
 474 STR string equal ""                           7      11      11      12      39
 475 STR string equal long (!= len)               11     142     141     139     725
 476 STR string equal long (== len)              100     135     133     132     721
 477 STR string equal long (same obj)              7     134     132     132     726
 478 STR string equal mixed long                   7     135     132     132     723
 479 STR string equal uni long                   113     153     151     132     720
 480 STR/LIST length, obj shimmer               1211    1212    1206       8     652
 481 SWITCH 1st true                              18      16      15      14      48
 482 SWITCH 2nd true                              19      17      15      14      47
 483 SWITCH 9th true                              22      19      18      16      48
 484 SWITCH default true                          23      19      18      17      49
 485 TRACE all set (rwu)                          18      15      15      14      24
 486 TRACE no trace set                           18      15      15      14      24
 487 TRACE read                                   18      15      15      13      23
 488 TRACE unset                                  18      15      15      14      23
 489 TRACE write                                  18      15      15      13      24
 490 UNSET catch var !exist                       34      36      35      31      21
 491 UNSET catch var exists                        5       5       5       5      12
 492 UNSET info check var !exist                   6       5       5       5      13
 493 UNSET info check var exists                   8       8       7       7      15
 494 UNSET nocomplain var !exist                   5      36      34      30      22
 495 UNSET nocomplain var exists                   5       5       5       5      13
 496 UNSET var exists                              4       5       4       4       7
 497 VAR 'array set' of 100 elems                 97     154     158     140     244
 498 VAR 100 'set's in array                      70      81      79      80     232
 499 VAR access global                             7      16      14      12      21
 500 VAR access local proc arg                     4       5       5       5      19
 501 VAR access locally set                        4       5       5       5      18
 502 VAR access upvar                              8      18      17      15      23
 503 VAR mset                                     10      16      14      14      55
 504 VAR mset (foreach)                            6       8       7       7      30
 505 VAR ref absolute                             61     326     330     265    8.0+
 506 VAR ref local                                31      68      66      61    8.0+
 507 VAR ref variable                             35     163     160     138    8.0+
 508 VAR set array element                         7       8       6       6       9
 509 VAR set scalar                                3       3       3       3       8
 510 WORDCOUNT wc1                              2851    3090    2983    1131    1839
 511 WORDCOUNT wc2                               776    1741    1740     671    1016
 512 WORDCOUNT wc3                               712    1651    1661     593     734
 512 BENCHMARKS                              1:8.4.0 2:8.3.4 3:8.2.3 4:8.0.5 5:7.6p2
 FINISHED 2002-09-10 23:37:05

JH: The numbers below by Kevin are for the previous set of numbers, and need updating. The only thing likely to change though is to show that 8.4 is pulling ahead in performance.

Kevin Kenny: For those that insist on a single figure of merit, I took these numbers, normalized them to a scale of 8.4a3 == 1.0, and then took the harmonic mean of all benchmarks.


    Version   Relative Speed   Comments

     7.6p2         0.125       The last version without bytecode
                               compilation

     8.0.5         0.736       Bytecode compilation initially sped
                               things up by a factor of about 6.

     8.2.3         0.600       The cost of internationalization was
                               that things got about 20% slower.

     8.3.1         0.606       Not much change here....

     8.4a3         1.000       The reference against which the
                               others were measured.  Many more
                               operations are bytecoded, which
                               gains about 50% in speed.

As with all benchmarks, take these numbers with a spoonful of salt!