Arjen Markus (25 october 2015) Operations on arrays of numbers are quite common, for instance in statistical analyses. A programming environment/language like MATLAB heavily depends on the possibility to apply numerical operations on arrays without going through a for-loop explicitly. Within Tcl we have Christian Gollwitzer's VecTcl extension and of course the BLT package has such features too.

I wanted to see if it was possible to use plain Tcl to achieve the same functionality. Perhaps not as fast and efficient as a compiled extension, but it may be useful to have a pure Tcl implementation.

The idea is this: suppose you have two lists of values and you want to calculate the element-by-element sum of these two. The ordinary way would be something like:

set x {1 2 3 4 5 6} set y {2 3 4 5 6 1} set sum {} foreach x1 $x y1 $y { lappend sum [expr {$x1+$y1}] }

resulting in a list {3 5 7 9 11 7}. If we could define a procedure that simply takes the expression and applies it to the elements of the two lists, we could dispense with the foreach loop:

set x {1 2 3 4 5 6} set y {2 3 4 5 6 1} set sum [map {$x+$y}]

The code below achieves just that. It is not as robust as one might want:

- Internally it uses variables with the pattern "_letters_" - if you use such names in the expression, there may be a name clash.
- There is no check (yet) that the lists involved have the same length.
- It is assumed that a variable referred to in the given expression has a scalar value or is a valid list of numbers.
- There is no provision for missing values - an empty element or a NaN value.

A more peculiar limitation is that the expression needs to be in braces:

map $x+$y

and even

map $x

will not be evaluated correctly. I know of no way to detect if a value is already a list to avoid shimmering. The command [string is list $value] may or may not be capable of that.

Anyway, the procedure below can be used as a template for a whole host of array operations. To name a few:

- Sum a list (or an expression applied to lists) under a given condition. Like: sum all positive values
- Count the number of values that conform to a given condition
- Filter on a condition

As an illustration, a possible implementation for the conditional summing of a list of numbers is also given.

proc map {_expr_} { # # Extract the variables from the expression # set _nvars_ [lsort -unique [regexp -all -inline {\$[a-zA-Z_0-9]+} $_expr_]] set _vars_ {} set _values_ {} foreach _v_ $_nvars_ { set _v_ [string range $_v_ 1 end] upvar 1 $_v_ _${_v_}_ lappend _values_ [set _${_v_}_] if { [llength [set _${_v_}_]] > 1 } { upvar 1 $_v_ _${_v_}_ lappend _vars_ _${_v_}_ append _forvars_ "$_v_ \$_${_v_}_ " } else { upvar 1 $_v_ $_v_ lappend _vars_ $_v_ } } apply [list $_vars_ [string map [list EXPR $_expr_ FOREACH $_forvars_] { set _result_ {} foreach FOREACH { lappend _result_ [expr {EXPR}] } return $_result_ }]] {*}$_values_ } proc sum {_expr_ {_filter_ 1}} { # # Extract the variables from the expression # set _nvars_ [lsort -unique [regexp -all -inline {\$[a-zA-Z_0-9]+} "$_expr_ $_filter_"]] set _vars_ {} set _values_ {} foreach _v_ $_nvars_ { set _v_ [string range $_v_ 1 end] upvar 1 $_v_ _${_v_}_ lappend _values_ [set _${_v_}_] if { [llength [set _${_v_}_]] > 1 } { upvar 1 $_v_ _${_v_}_ lappend _vars_ _${_v_}_ append _forvars_ "$_v_ \$_${_v_}_ " } else { upvar 1 $_v_ $_v_ lappend _vars_ $_v_ } } apply [list $_vars_ [string map [list EXPR $_expr_ FOREACH $_forvars_ FILTER $_filter_] { set _result_ 0 foreach FOREACH { if { FILTER } { set _result_ [expr {$_result_ + EXPR}] } } return $_result_ }]] {*}$_values_ } # Simple test # set a 10 set x {1 2 3 4 5 6} set y {2 3 4 5 6 1} puts [map {$x + $a * $y}] puts [map {$x * $y}] puts [map {cos($x / double($y))}] puts [sum {$x} {$x > $y || $x > 3}]

Measurements of the performance of this procedure show that there is a small overhead in comparison to a direct implementation with a dedicated [foreach] loop:

set x [lrepeat 10000 [expr {0.1}]] set y [lrepeat 10000 [expr {0.2}]] puts [time { map {$x + $y} } 100] proc sum2 {x y} { set r {} foreach x $x y $y { lappend r [expr {$x + $y}] } return $r } puts [time { sum2 $x $y } 100]

resulted in:

2848.96 microseconds per iteration 2828.11 microseconds per iteration