Itcl overhead

This page is about benchmarking Tcl's proc vs Itcl-based code to see how much overhead Itcl adds and how to make its performance better.

I decided to compare plain Tcl recursive calling vs Itcl inheriting, using itcl chain and delegating.

If Itcl class has no variables and procs do not use any variables:

 ITcl: Chain 10 level deep      51.347
 ITcl: Chain 11 level deep      55.769
 ITcl: Chain 12 level deep      61.921
 ITcl: Chain 13 level deep      72.064
 ITcl: Chain 14 level deep      76.513
 ITcl: Chain 15 level deep      77.732
 ITcl: Chain 16 level deep      84.874
 ITcl: Chain 17 level deep      91.932
 ITcl: Chain 18 level deep     121.868
 ITcl: Chain 19 level deep      114.08
 ITcl: Chain 20 level deep     108.025
 ITcl: Delegate 10 level deep   32.479
 ITcl: Delegate 11 level deep   36.436
 ITcl: Delegate 12 level deep   39.776
 ITcl: Delegate 13 level deep   41.741
 ITcl: Delegate 14 level deep   45.113
 ITcl: Delegate 15 level deep   49.032
 ITcl: Delegate 16 level deep   52.265
 ITcl: Delegate 17 level deep   54.443
 ITcl: Delegate 18 level deep   93.945
 ITcl: Delegate 19 level deep   71.611
 ITcl: Delegate 20 level deep    70.01
 ITcl: Inherit 10 level deep    25.959
 ITcl: Inherit 11 level deep    28.699
 ITcl: Inherit 12 level deep    31.088
 ITcl: Inherit 13 level deep    33.196
 ITcl: Inherit 14 level deep    35.765
 ITcl: Inherit 15 level deep    38.178
 ITcl: Inherit 16 level deep    44.797
 ITcl: Inherit 17 level deep    43.085
 ITcl: Inherit 18 level deep    54.318
 ITcl: Inherit 19 level deep    47.477
 ITcl: Inherit 20 level deep    50.636
 Tcl:  Recursive 10 level deep  10.403
 Tcl:  Recursive 11 level deep  11.743
 Tcl:  Recursive 12 level deep  12.324
 Tcl:  Recursive 13 level deep  14.245
 Tcl:  Recursive 14 level deep  14.721
 Tcl:  Recursive 15 level deep  16.241
 Tcl:  Recursive 16 level deep  16.997
 Tcl:  Recursive 17 level deep  17.485
 Tcl:  Recursive 18 level deep  18.763
 Tcl:  Recursive 19 level deep  22.154
 Tcl:  Recursive 20 level deep  21.667

So it seems that inheriting and calling methods from the same object seem to be the fastest way and using chain is the worst way.

But things change when we define 20 variables at each level - so a 20-level-deep test actually increments 19 * 20 = 380 variables:

 ITcl: Chain 10 level deep     104.602
 ITcl: Chain 11 level deep     112.974
 ITcl: Chain 12 level deep     126.078
 ITcl: Chain 13 level deep     138.874
 ITcl: Chain 14 level deep     147.253
 ITcl: Chain 15 level deep     161.369
 ITcl: Chain 16 level deep     173.584
 ITcl: Chain 17 level deep     188.603
 ITcl: Chain 18 level deep     198.702
 ITcl: Chain 19 level deep     242.112
 ITcl: Chain 20 level deep     226.422
 ITcl: Delegate 10 level deep   59.411
 ITcl: Delegate 11 level deep   68.117
 ITcl: Delegate 12 level deep   70.796
 ITcl: Delegate 13 level deep   78.662
 ITcl: Delegate 14 level deep   84.386
 ITcl: Delegate 15 level deep   90.361
 ITcl: Delegate 16 level deep   96.495
 ITcl: Delegate 17 level deep  103.415
 ITcl: Delegate 18 level deep  109.857
 ITcl: Delegate 19 level deep  116.133
 ITcl: Delegate 20 level deep  121.538
 ITcl: Inherit 10 level deep    75.653
 ITcl: Inherit 11 level deep   105.711
 ITcl: Inherit 12 level deep    92.505
 ITcl: Inherit 13 level deep   104.654
 ITcl: Inherit 14 level deep   107.463
 ITcl: Inherit 15 level deep   120.157
 ITcl: Inherit 16 level deep   150.662
 ITcl: Inherit 17 level deep    134.11
 ITcl: Inherit 18 level deep   144.124
 ITcl: Inherit 19 level deep   154.595
 ITcl: Inherit 20 level deep   163.931
 Tcl:  Recursive 10 level deep  61.822
 Tcl:  Recursive 11 level deep  68.536
 Tcl:  Recursive 12 level deep  79.564
 Tcl:  Recursive 13 level deep  82.756
 Tcl:  Recursive 14 level deep  93.726
 Tcl:  Recursive 15 level deep  96.324
 Tcl:  Recursive 16 level deep 102.629
 Tcl:  Recursive 17 level deep 109.943
 Tcl:  Recursive 18 level deep 118.434
 Tcl:  Recursive 19 level deep 123.855
 Tcl:  Recursive 20 level deep 129.817

The most interesting part is that delegates now seem to work much faster than the other ones. Still, 25% performance loss using inheritance and chain having the biggest loss. The interesting part is that delegates now caught up with procs.

Now, for 50 variables (so 1000 variable references with 20th level):

 ITcl: Chain 10 level deep     172.473
 ITcl: Chain 11 level deep     194.999
 ITcl: Chain 12 level deep     244.775
 ITcl: Chain 13 level deep     242.553
 ITcl: Chain 14 level deep     273.848
 ITcl: Chain 15 level deep     304.169
 ITcl: Chain 16 level deep      337.07
 ITcl: Chain 17 level deep      370.41
 ITcl: Chain 18 level deep     432.148
 ITcl: Chain 19 level deep     466.136
 ITcl: Chain 20 level deep      510.95
 ITcl: Delegate 10 level deep   98.282
 ITcl: Delegate 11 level deep  113.818
 ITcl: Delegate 12 level deep   118.42
 ITcl: Delegate 13 level deep  128.619
 ITcl: Delegate 14 level deep  140.015
 ITcl: Delegate 15 level deep  153.629
 ITcl: Delegate 16 level deep  171.915
 ITcl: Delegate 17 level deep  175.932
 ITcl: Delegate 18 level deep  193.936
 ITcl: Delegate 19 level deep  212.568
 ITcl: Delegate 20 level deep   259.58
 ITcl: Inherit 10 level deep   146.439
 ITcl: Inherit 11 level deep   164.215
 ITcl: Inherit 12 level deep   183.119
 ITcl: Inherit 13 level deep   199.634
 ITcl: Inherit 14 level deep    232.69
 ITcl: Inherit 15 level deep   269.736
 ITcl: Inherit 16 level deep   270.989
 ITcl: Inherit 17 level deep   336.297
 ITcl: Inherit 18 level deep    338.85
 ITcl: Inherit 19 level deep   369.274
 ITcl: Inherit 20 level deep   423.055
 Tcl:  Recursive 10 level deep 130.904
 Tcl:  Recursive 11 level deep 156.326
 Tcl:  Recursive 12 level deep 160.326
 Tcl:  Recursive 13 level deep 179.524
 Tcl:  Recursive 14 level deep 189.009
 Tcl:  Recursive 15 level deep 208.528
 Tcl:  Recursive 16 level deep 219.666
 Tcl:  Recursive 17 level deep 232.773
 Tcl:  Recursive 18 level deep 253.428
 Tcl:  Recursive 19 level deep 287.717
 Tcl:  Recursive 20 level deep 304.345

This is very interesting because the test claims that delegates using Itcl can actually be faster than non-OO programming - which usually should not be the case.

The tclbench-based test that did the trick:

 package require Itcl

 set itclvars 0
 set minlevel 10
 set maxlevel 20
 set iter 10000

 proc ::level1 {a b c d e} {
 }

 itcl::class ::class1 {
    public method level1 {a b c d e} {
         return 1
    }
 }
 itcl::class ::classI1 {
    inherit ::class1
 }

 itcl::class ::classD1 {
    inherit ::class1
 }

 itcl::class ::classC1 {
    public method level {a b c d e} {
         return 1
    }
 }

 for {set i 2} {$i <= $maxlevel} {incr i} {
    set varcode ""
    set varproccode ""
    set varitclcode ""
    set varlist [list]
    for {set j 0} {$j < $itclvars} {incr j} {
         set vn v${i}c${j}
         lappend varlist $vn
         append varcode "protected variable $vn 0" \n
         set ::$vn 0
         append varproccode "incr ::$vn" \n
         append varitclcode "incr $vn" \n
    }

    set ip [expr {$i-1}]

    proc ::level$i {a b c d e} "$varproccode \; return \[::level$ip \$a \$b \$c \$d \$e\]"

    itcl::class ::classD$i "
         $varcode
         public method level$i \{a b c d e\} \{
             $varitclcode
             return \[::oD$ip level$ip \$a \$b \$c \$d \$e\]
         \}
    "

    itcl::class ::classI$i "inherit ::classI$ip
         $varcode
         public method level$i \{a b c d e\} \{
             $varitclcode
             return \[level$ip \$a \$b \$c \$d \$e\]
         \}
    "

    itcl::class ::classC$i "inherit ::classC$ip
         $varcode
         public method level \{a b c d e\} \{
             $varitclcode
             return \[chain \$a \$b \$c \$d \$e\]
         \}
    "
 }

 for {set i 1} {$i <= $maxlevel} {incr i} {
    ::classI$i ::oI$i
    ::classD$i ::oD$i
    ::classC$i ::oC$i
 }

 if {[catch {
    ::oI$maxlevel level$maxlevel 1 2 3 4 5
    ::oD$maxlevel level$maxlevel 1 2 3 4 5
    ::oC2 level 1 2 3 4 5
    ::level2 1 2 3 4 5
 }]} {
    puts stderr $::errorInfo
    exit 1
 }
 for {set i $minlevel} {$i <= $maxlevel} {incr i} {
    bench -desc "Tcl:  Recursive [format %2d $i] level deep" -iter $iter \
         -body "::level$i a b c d e"

    bench -desc "ITcl: Inherit [format %2d $i] level deep" -iter $iter \
         -body "::oI$i level$i a b c d e"

    bench -desc "ITcl: Delegate [format %2d $i] level deep" -iter $iter \
         -body "::oD$i level$i a b c d e"

    bench -desc "ITcl: Chain [format %2d $i] level deep" -iter $iter \
         -body "::oC$i level a b c d e"
 }

If someone wishes to make improvements to the code, please also update the benchmarks.