Extending math functions to support lists

Some languages (e.g. GNU Octave) support handling lists (vectors) with their built-in math operators. An example is GNU Octave that allows doing that:

% multiply and add scalar value to all list elements
y = 0.5 * [0 1 2 3] + 1.0
% -> y is [1.0 1.5 2.0 2.5] here

% compute median absolute deviation (MAD)
x = [0 1 2 3 4]
y = median(abs(x - median(x)))
% -> y is 1 here

Could Tcl do the same? This page collects some thoughts.

Status quo

Some Tcl math functions seem to have (limited) capabilities to handle lists. For example min() and max():

% expr {min(4, 2, 3, 5)}
# -> 2

% set x {4 2 3 5}
% expr {min($x)}
# -> expected floating-point number but got "4 2 3 5"

Some operators fully support lists:

% expr {3 in {4 2 3 5}}
-> 1

% expr {1 in {4 2 3 5}}
-> 0

% set x {4 2 3 5}
% expr {3 in $x}
-> 1

Most math functions and operators however do not support lists. Good news is new math functions can be added to the ::tcl::mathfunc namespace and existing ones can be redefined to support lists.

A step forward

As an example consider computing the median absolute deviation (MAD). Computing it requires a median() and abs() function and a substraction operator that can handle lists.

First replacing abs():

proc ::tcl::mathfunc::abs {x args} {
   set y {}
   foreach p [list {*}$x {*}$args] {
      if {$p < 0} {
         lappend y [expr {-$p}]
      } else {
         lappend y [expr {$p + 0.0}]
      }
   }
   return $y
}

This is the result:

% expr {abs(-2, 1, -inf, 0)}
# -> 2 1 Inf 0

% set x {-1 4 -5 3}
% expr {abs($x)}
# -> 1 4 5 3

% expr {abs(-42)}
# -> 42

Next a median() math function is required:

proc ::tcl::mathfunc::median {x args} {
   # expand x to allow handling a list argument
   set p [lsort -real [list {*}$x {*}$args]]
   set n [llength $p]
   set i [expr {$n >> 1}]
   if {0 != ($n & 1)} {
      set y [lindex $p $i]
   } else {
      # for an even number of elements the median is
      # the average of the two center elements
      set a [lindex $p $i]
      set b [lindex $p [expr {$i - 1}]]
      set y [expr {0.5 * ($a + $b)}]
   }
   return $y
}

This is the result:

% expr {median(1,2,3,4,5)}
# -> 3

% set x {1 2 3 4 5}
% expr {median($x)}
# -> 3

Unfortunately the substraction operator doesn't handle list arguments and redefining it in ::tcl::mathop namespace does not work either (why?).

According to the reference manual :

"(...) renaming, reimplementing or deleting any of the commands in the namespace does not alter the way that the expr command behaves, and nor does defining any new commands in the ::tcl::mathop namespace."

In other words the following does not work:

% set mad [expr {median(abs($x - median($x)))}]
# -> can't use non-numeric string as operand of "-"

Adding a substraction function is a possible workaround:

proc ::tcl::mathfunc::sub {x k} {
   set y {}
   foreach p [list {*}$x] {
      lappend y [expr {$p - $k}]
   }
   return $y
}

The result is pretty much this:

% set x {0 1 2 3 4}
% set mad [expr {median(abs(sub($x, median($x))))}]
# -> mad is 1 here

Extending math functions

Single argument functions

Most math functions take a single argument:

abs	acos	asin	atan
bool	ceil	cos	cosh
double	entier	exp	floor
int	isqrt	log	log10
round	sin	sinh	sqrt
tan	tanh	wide

Adding list handling capabilities is easy for these math functions:

The single argument math function is moved to another namespace using rename.
A proc that can handle list arguments is added to ::tcl::mathfunc namespace. It forwards to the old math function to do the actual processing.

This is an example replacing the sin math function:

rename ::tcl::mathfunc::sin ::tcl::mathjunk::sin

proc ::tcl::mathfunc::sin {x args} {
   set y {}
   foreach p [list {*}$x {*}$args] {
      lappend y [::tcl::mathjunk::sin $p]
   }
   return $y
}

The result of this is:

% expr {sin(0.0, 0.5, 1.0, 1.5)}
# -> 0.0 0.479425538604203 0.8414709848078965 0.9974949866040544

% set x {0.0 0.5 1.0 1.5}
% expr {sin($x)}
# -> 0.0 0.479425538604203 0.8414709848078965 0.9974949866040544

To replace multiple functions with minimal work the following may be useful:

# replace multiple (single argument) functions
# here: sin, cos, tan, ...
foreach n {sin cos tan} {
   rename ::tcl::mathfunc::$n ::tcl::mathjunk::$n
   proc ::tcl::mathfunc::$n {x args} {
      set whoami [namespace tail [dict get [info frame 0] proc]]
      set y {}
      foreach p [list {*}$x {*}$args] {
         lappend y [::tcl::mathjunk::$whoami $p]
      }
      return $y
   }
}

Two argument functions

Some math functions take two arguments:

atan2	fmod	hypot	pow

atan2

fmod

hypot

pow

This is a pow function replacement that can handle lists:

rename ::tcl::mathfunc::pow ::tcl::mathjunk::pow

# valid argument combinations are:
#
# a) pow(scalar, scalar)
#    y = a ** b
# b) pow(vector, scalar) (N > 1)
#    [a_1 a_2 ... a_N] ** b
#    -> y = [a_1**b a_2**b ... a_N**b]
# c) pow(scalar, vector) (N > 1)
#    a ** [b_1 b_2 ... b_N]
#    -> y = [a**b_1 a**b_2 ... a**b_N]
# d) pow(vector, vector) (N > 1)
#    [a_1 a_2 ... a_N] ** [b_1 b_2 ... b_N]
#    -> y = [a_1**b_1 a_2**b_2 ... a_N**b_N]

proc ::tcl::mathfunc::pow {a b} {
   set na [llength $a]
   set nb [llength $b]
   set y {}
   if {$na == $nb} {
      foreach aa [list {*}$a] bb [list {*}$b] {
         lappend y [::tcl::mathjunk::pow $aa $bb]
      }
   } else {
      if {$na == 1} {
         foreach bb [list {*}$b] {
            lappend y [::tcl::mathjunk::pow $a $bb]
         }
      } else {
         if {$nb == 1} {
            foreach aa [list {*}$a] {
               lappend y [::tcl::mathjunk::pow $aa $b]
            }
         } else {
            error "dimension mismatch"
         }
      }
   }
   return $y
}

The result is:

% set x {1 2 3 4 5 6}
% expr {pow(2,8)}
# -> 256

% expr {pow(2,$x)}
# -> 2.0 4.0 8.0 16.0 32.0 64.0

% expr {pow($x,2)}
# -> 1.0 4.0 9.0 16.0 25.0 36.0

% expr {pow($x,$x)}
# -> 1.0 4.0 27.0 256.0 3125.0 46656.0

Fixing min() and max()

Although min() and max() support lists they only do so when the list is split into multiple arguments. Fixing this:

rename ::tcl::mathfunc::min ::tcl::mathjunk::min
rename ::tcl::mathfunc::max ::tcl::mathjunk::max

proc ::tcl::mathfunc::min {x args} {
   return [::tcl::mathjunk::min {*}$x {*}$args]
}

proc ::tcl::mathfunc::max {x args} {
   return [::tcl::mathjunk::max {*}$x {*}$args]
}

With this lists are fully supported:

% expr {min(7, 4, 9, 3)}
# -> 3
% expr {max(7, 4, 9, 3)}
# -> 9

% set x {4 2 5 3}
% expr {min($x)}
# -> 2
% expr {max($x)}
# -> 5

Rethinking addition

Mixing scalar and list arguments can be tricky. The following function sketches this for addition (for sure it can be optimized further):

proc ::tcl::mathfunc::add {a args} {
   # If there's a single arg add all its elements
   if {[llength $args] == 0} {
      set y 0
      foreach p $a {
         incr y $p
      }
      return $y
   }
   # Two or more args
   set na [llength $a]
   set b [lindex $args 0]
   set nb [llength $b]
   set c [lrange $args 1 end]
   set nc [llength $c]
   if {$na == 1} {
      # 1st arg is NOT a list
      if {$nb == 1} {
         # 2nd arg is NOT a list
         # -> remaining args are expected to be non-list
         set y 0
         foreach p [list $a $b {*}$c] {
            incr y $p
         }
      } else {
         # 2nd arg IS a list
         # -> add to each element of this list
         if {$nc > 0} {
            error "Invalid arguments"
         }
         set y {}
         foreach p $b {
            lappend y [expr {$a + $p}]
         }
      }
   } else {
      if {$nb == 1} {
         # 1st arg IS a list, 2nd arg is NOT a list
         # -> add to each element of list
         if {$nc > 0} {
            error "Invalid arguments"
         }
         set y {}
         foreach p $a {
            lappend y [expr {$b + $p}]
         }
      } else {
         # 1st and 2nd arg are lists
         # -> add element-by-element
         if {$na != $nb} {
            error "List length must be identical"
         }
         set y {}
         foreach p $a q $b {
            lappend y [expr {$p + $q}]
         }
      }
   }
   return $y
}

This is what the result looks like

% expr {add(1,2,3,4,5,6)}
# -> 21
% set x {1 2 3 4 5 6}
% expr {add($x)}
# -> 21

% expr {add($x,$x)}
# -> 2 4 6 8 10 12
% expr {add({6 5 4 3 2 1},$x)}
# -> 7 7 7 7 7 7

% expr {add(1,$x)}
# -> 2 3 4 5 6 7
% expr {add($x,1)}
# -> 2 3 4 5 6 7

Other languages

Python does not support list arguments for math functions but has special syntax for processing lists. This is an example:

>>> y = [x**x for x in [1,2,3,4,5,6]]
>>> y
[1, 4, 27, 256, 3125, 46656]

Category Mathematics