VecTcl

VecTcl is a package for efficiently doing numerical processing in Tcl using a natural syntax close to that of NumPy and Matlab.

It was presented at EuroTcl in 2014: PDF of presentation slides here

The internal representation of values is syntactically compatible with lists (and nested lists, and …) but is optimised internally for the case where all elements are of the same, numeric type.

Examples

In VecTcl, there is no distinction between a Tcl list and a vector or matrix. They are created by setting a variable with a list of doubles:

# create a vector
set x { 1 2 3 }
# create a matrix
set A {{1.0 2.0 3.0} {4.0 5.0 6.0} {7.0 8.0 9.0}}

Of course, list commands such as list, lappend, linsert, lrepeat etc. can also be used. To evaluate an expression involving vector operations, pass the expression to vexpr:

vexpr { A*x } ;# compute the matrix-vector product
# 14.0 32.0 50.0

In order for this to work, you must first load the package and import the commands:

package require vectcl
namespace import vectcl::*

Vectors can contain integers, floating-point values or complex numbers:

set x {1 2 3} ;# an integer vector
set y {2.0 3.0 5.0} ;# a floating-point vector
set z {0+1i 2+3.5i 3.0+0i} ;# a complex vector

VecTcl includes support for linear equation solving

vexpr { x = A\y ;# solve A x = y for x
    # in the least squares sense if m>n
}

array slicing, shaping and reductions

# define a vector with 3 elements 
set x {1 2 3}
# ...and a 3x2 matrix 
set A {{2.0 3.0} {5.0 6.0} {7.0 8.0}}

# replace column 1 in A with {9 10 11}
# indices start from 0
vexpr { A[:,1] = {9 10 11} }
# { {2.0 9.0} {5.0 10.0} {7.0 11.0} }

# create a matrix with columns x and x.^2
vexpr { A=hstack(x, x.^2) }
# {1.0 1.0} {2.0 4.0} {3.0 9.0}
vexpr { sum(x.^2)}
# 14.0

and elementary transcendental functions

vexpr { sinh(2+3i) } ;# complex hyperbolic sine
# -3.5905645899857794+0.5309210862485197i

Any Tcl command can be called as a function

set x {1 2 3}
vexpr { n=llength(x); puts(n) } 
# writes 3 to stdout
# Caveat: llength(x) is inefficient, it
# involves a conversion to a list. Use rows(x) instead.

Not only short expressions are supported. Looping and branching make it possible to write larger math functions in a single expression

vexpr { 
    for i=1:5 {
        if i!=2 {
            puts(i)
        }
    }
} 

A second command, vproc defines a procedure fully in terms of a VecTcl expression

vproc rms {x} {
    # compute the root mean square
    xm=mean(x)
    sqrt(mean((x-xm).^2))
} 

Vector expressions are compiled into Tcl procedures; the curious can peek into the compiler output

vectcl::compile {
    x, y = list(y, x) ;# swap x and y
    A= -3*x
}
# this outputs:
upvar 1 y y
upvar 1 x x
upvar 1 A A
set __temp1 [list [set y] [set x]]
lassign $__temp1 x y

set A [numarray::neg [numarray::* 3 [set x]]]

Undefined values

VecTcl can also work with unknowns. A missing value is represented with the word 'NaN':

set v {1 2 3 4 NaN 6 7 NaN}
vectcl::vexpr {v*2}
# results in "2.0 4.0 6.0 8.0 NaN 12.0 14.0 NaN"

set x {1 NaN 3}
set A {{1.0 2.0 3.0} {4.0 5.0 6.0} {7.0 8.0 9.0}}
vectcl::vexpr "A * x"
# results in "NaN NaN NaN"

set x {1 2 3}
set A {{1.0 2.0 3.0} {4.0 5.0 6.0} {7.0 NaN 9.0}}
vectcl::vexpr "A * x"
# results in "14.0 32.0 NaN"

This feature is implicitly documented here: https://auriocus.github.io/VecTcl/design/50.html . A more complex example using NaN is here: https://auriocus.github.io/VecTcl/using_vectcl_for_arrays.html .

So, you can use this in calculations where the NaN does not lead to an overall impossible calculation and still get sensible results. This means, the sum or the mean e.g. cannot be calculated because VecTcl does not know what to do when summing up numbers and NaN:

set v {1 2 3 4 NaN 6 7 NaN}
vectcl::vexpr {sum(v)}
# results in "NaN"
vectcl::vexpr {mean(v)}
# results in "NaN"

To make such operations work, you would need to define your own version of 'sum()' and 'mean()', explicitly telling VecTcl what to do when a NaN is encountered. For this, you need to know how to test for a NaN. You cannot just compare each number with the string "NaN" ... In VecTcl, the fact is used that NaN is not the same as any other value, NaN itself included. So you can produce a vector telling you which elements are NaN by comparing the vector with itself and then treat the elements as you wish:

set v {1 2 3 NaN 5 NaN}
vectcl::vexpr {nan_mask = v != v}
# results in nan_mask = 0 0 0 1 0 1
# telling you where the NaNs are

With this, we can make a VecTcl function to calculate the mean from a list also having NaN elements:

vproc mean_with_nan {array} {
   #
   # a VecTcl procedure to compute the mean of a vector possibly having NaN elements
   #
   isNumber = array == array
   length = shape(isNumber)
   j = 0
   for i=0:length-1 {
      if (isNumber[i] == 1) {j = j + array[i]}
   }
   if (j == 0) {
      j
   } else {
      j/sum(isNumber)
   }
}

# Let us test this:
set v {1 2 3 NaN 5 NaN}
vectcl::vexpr {mean_with_nan(v)}
# results in 2.75

Extended Examples

Identifying duplicate photographs
A simple algorithm to identify duplicate photographs using fingerprinting.

More Information