## Tcl-only attempt at array operations

Arjen Markus (25 october 2015) Operations on arrays of numbers are quite common, for instance in statistical analyses. A programming environment/language like MATLAB heavily depends on the possibility to apply numerical operations on arrays without going through a for-loop explicitly. Within Tcl we have Christian Gollwitzer's VecTcl extension and of course the BLT package has such features too.

I wanted to see if it was possible to use plain Tcl to achieve the same functionality. Perhaps not as fast and efficient as a compiled extension, but it may be useful to have a pure Tcl implementation.

The idea is this: suppose you have two lists of values and you want to calculate the element-by-element sum of these two. The ordinary way would be something like:

```set x {1 2 3 4 5 6}
set y {2 3 4 5 6 1}

set sum {}
foreach x1 \$x y1 \$y {
lappend sum [expr {\$x1+\$y1}]
}
```

resulting in a list {3 5 7 9 11 7}. If we could define a procedure that simply takes the expression and applies it to the elements of the two lists, we could dispense with the foreach loop:

```set x {1 2 3 4 5 6}
set y {2 3 4 5 6 1}

set sum [map {\$x+\$y}]```

The code below achieves just that. It is not as robust as one might want:

• Internally it uses variables with the pattern "_letters_" - if you use such names in the expression, there may be a name clash.
• There is no check (yet) that the lists involved have the same length.
• It is assumed that a variable referred to in the given expression has a scalar value or is a valid list of numbers.
• There is no provision for missing values - an empty element or a NaN value.

A more peculiar limitation is that the expression needs to be in braces:

`   map \$x+\$y`

and even

`   map \$x`

will not be evaluated correctly. I know of no way to detect if a value is already a list to avoid shimmering. The command [string is list \$value] may or may not be capable of that.

Anyway, the procedure below can be used as a template for a whole host of array operations. To name a few:

• Sum a list (or an expression applied to lists) under a given condition. Like: sum all positive values
• Count the number of values that conform to a given condition
• Filter on a condition

As an illustration, a possible implementation for the conditional summing of a list of numbers is also given.

```proc map {_expr_} {
#
# Extract the variables from the expression
#
set _nvars_  [lsort -unique [regexp -all -inline {\\$[a-zA-Z_0-9]+} \$_expr_]]
set _vars_   {}
set _values_ {}

foreach _v_ \$_nvars_ {
set _v_ [string range \$_v_ 1 end]
upvar 1 \$_v_ _\${_v_}_
lappend _values_  [set _\${_v_}_]

if { [llength [set _\${_v_}_]] > 1 } {
upvar 1 \$_v_ _\${_v_}_
lappend _vars_    _\${_v_}_
append _forvars_ "\$_v_ \\$_\${_v_}_ "
} else {
upvar 1 \$_v_ \$_v_
lappend _vars_ \$_v_
}
}

apply [list \$_vars_ [string map [list EXPR \$_expr_ FOREACH \$_forvars_] {
set _result_ {}
foreach FOREACH {
lappend _result_ [expr {EXPR}]
}
return \$_result_
}]] {*}\$_values_
}

proc sum {_expr_ {_filter_ 1}} {
#
# Extract the variables from the expression
#
set _nvars_  [lsort -unique [regexp -all -inline {\\$[a-zA-Z_0-9]+} "\$_expr_ \$_filter_"]]
set _vars_   {}
set _values_ {}

foreach _v_ \$_nvars_ {
set _v_ [string range \$_v_ 1 end]
upvar 1 \$_v_ _\${_v_}_
lappend _values_  [set _\${_v_}_]

if { [llength [set _\${_v_}_]] > 1 } {
upvar 1 \$_v_ _\${_v_}_
lappend _vars_    _\${_v_}_
append _forvars_ "\$_v_ \\$_\${_v_}_ "
} else {
upvar 1 \$_v_ \$_v_
lappend _vars_ \$_v_
}
}

apply [list \$_vars_ [string map [list EXPR \$_expr_ FOREACH \$_forvars_ FILTER \$_filter_] {
set _result_ 0
foreach FOREACH {
if { FILTER } {
set _result_ [expr {\$_result_ + EXPR}]
}
}
return \$_result_
}]] {*}\$_values_
}

# Simple test
#
set a 10
set x {1 2 3 4 5 6}
set y {2 3 4 5 6 1}

puts [map {\$x + \$a * \$y}]
puts [map {\$x * \$y}]
puts [map {cos(\$x / double(\$y))}]

puts [sum {\$x} {\$x > \$y || \$x > 3}]```

Measurements of the performance of this procedure show that there is a small overhead in comparison to a direct implementation with a dedicated [foreach] loop:

```set x [lrepeat 10000 [expr {0.1}]]
set y [lrepeat 10000 [expr {0.2}]]

puts [time { map {\$x + \$y} } 100]

proc sum2 {x y} {
set r {}
foreach x \$x y \$y {
lappend r [expr {\$x + \$y}]
}
return \$r
}

puts [time { sum2 \$x \$y } 100]```

resulted in:

```2848.96 microseconds per iteration
2828.11 microseconds per iteration```

 Category Mathematics