ensemble with parameters

A feature which is new in Tcl 8.6a3 is that namespace ensembles can have -parameters. This page explains this feature and shows some applications.


Explanation

What is a parameter?

In mathematics (and the like, e.g. physics), one sometimes distinguishes between two types of arguments for functions: "normal" arguments, and parameters. The parameters may be separated from the normal arguments by a semicolon (e.g. f (x ; q)) or bar (e.g. F(q,r | x)), but the most common is probably that the parameter appears as a subscript of the main function symbol, such as the α on Bessel's J function in [1 ]. When implemented in tcllib, this α is the first of two arguments of the math::special::Jn command, but it is usually only the second argument x that is considered a normal argument of "the Bessel J function" — α is merely a parameter.

The difference between parameters and other arguments is ultimately in the eye of the beholder, but some common distinguishing characteristics are:

  • Parameters tend to not change as much as normal arguments.
  • Parameters can be of a different type than ordinary variables (e.g. if one is doing calculus, then integer arguments may be put aside as parameters, since it doesn't make sense to differentiate with respect to them).
  • The special case of the function that one gets by assigning fixed values to the parameters is still useful to reason about — perhaps even more useful than the general function.

In the case of math::special::Jn, all of these can be used to label the first argument as a parameter and the second as a normal argument.

Parameters in Tcl

In some languages, syntax enforces an explicit distinction between parameters and normal arguments. In for example [2 ] (PDF document), it says:

It is important to define the mathematically natural concept, rather than the computationally natural one. This may sound obvious, but we will give an illustration: the Bessel functions. Consider the Bessel function (J_ν(z) for simplicity: the same points are true for other functions). This could be defined, as it is in all numerical libraries, as a function (ℂ×ℂ)→ℂ, […] but it makes more sense to define it as ℂ→(ℂ→ℂ), […] so that we can say that J(ν) satisfies Bessel's equation, rather than having to say that λx.J(ν,x) satisfies Bessel's equation.

Tcl is not one of these languages, since our best counterpart of "function" in the above sense is the command prefix. A practical definition of "parameter" in Tcl is argument included in the command prefix, since additional arguments provided when calling the command prefix are merely appended to the list of words that are found in it, and passed along to the base command without notice of where they came from; this (or things very much like it) is sometimes called currying [3 ] (see also Custom curry). As of Tcl 8.5, you can use list to curry any command (or command prefix), and the only cost of doing so is that you have to remember to use {*} when calling the result of this currying, since it is a command prefix rather than a command.

There was however in Tcl 8.5 one class of commands which wasn't possible to bundle with parameters in a command prefix (or at least not very useful to do so), namely namespace ensembles. The reason for this was that the subcommand name of an ensemble was always the second word in the command; if you regard it as a parameter, then you're always calling the same subcommand, so you might just as well let your command prefix be what this command+subcommand combination is mapped to by the ensemble. This shouldn't be taken as saying a command+subcommand combination is never a useful command prefix (they sometimes are), rather the point is that there was no way to bundle a data parameter for an ensemble into a command prefix without also hardwiring the subcommand used.

Ensemble parameters

The solution TIP#314 [4 ] provides to this problem is simply to add another option -parameters for namespace ensembles, which controls which word in the command is considered the subcommand name that the ensemble should map to something else. The value of this option is a list of "parameter names", and the ensemble command will step over as many arguments as there are elements in this list before picking one as the subcommand name to dispatch on. The stepped-over arguments are the ensemble parameters, which will appear before the arguments beyond the subcommand name in the target command for the ensemble call.

Example (again about Bessel functions, which is a bit of a stretch, but not entirely pointless):

 namespace eval Bessel {
    proc J {n x} {
       # Compute Jn(x)
    }
    proc J_zero {n k} {
       # Compute k'th positive zero of Jn
    }
    proc Y {n x} {
       # Compute Yn(x)
    }
    proc Y_zero {n k} {
       # Compute k'th positive zero of Yn
    }
    namespace export *
    namespace ensemble create -parameters n
 }

After this, one can say

 set prefix [list Bessel $n]

to create a command prefix for "order n Bessel functions",

 {*}$prefix J $x

to compute Jn(x),

 {*}$prefix J_zero 4

to compute the fourth zero of Jn (this zero tends to determine the frequency of the fourth overtone of a wave propagating on a circular domain, which is involved in the reason why dinner-gongs and bells have a ringing sound rather a uniform tone), and so on.

(DKF: Note that interp alias can make these prefixes easier to use. For example, instead of set pfx {Bessel 7}, you'd do interp alias {} Bessel7 {} Bessel 7 or something like that.)Lars H: Yes, although it is important to note that you don't have to fold every command prefix into an alias; it's just something you may do when it is convenient.


Applications

It may be noted that all of the following can be done without ensembles-with-parameters — if one needs to, any given ensemble command can be emulated using a proc with a large switch in the body — but ensembles help to improve code quality and maintainability (e.g. it is easier to add new subcommands).

Templates for abstract data types

One pattern for abstract data types in Tcl is to collect every operation on data of the abstract type into an ensemble; the best example of this is probably the dict command, but it is certainly not the only one. What makes dict an abstract data type in this sense is that as long as you only operate on (which includes creating) dictionaries using subcommands of dict then it doesn't matter how the data is encoded; it suffices that you know its semantics. This is the hallmark of an abstract data type.

Conceptually, dictionaries are associative arrays [5 ]. Internally they are implemented using hashes, but it is a classic topic in computer science courses that they can alternatively be implemented using binary trees or (more recently) skiplists, and in many communities outside Tcl these alternatives are hailed as superior because they keep the keys sorted; one can efficiently implement operations such as "get maximal key" or "get previous key". (Part of this opinion may however rather be founded in a lack of good hashtable and hashfunction implementations in these communities, combined with the fact that the theory of hashing is not as elementary as the theory of binary trees.) Sorting is however not without a price: one must always provide a function for comparing keys.

This brings us to the application of ensembles with parameters. A command for binary tree associative arrays could be defined for example as

 namespace eval BinaryTreeAssArray {
    namespace ensemble create -parameters sortcmd\
      -subcommands {append create exists filter for get\
      incr info keys lappend merge remove replace set\
      size unset update values with\
      maxkey minkey nextkey prevkey}
 }

where the first three lines of subcommands provide the same API as the dict command, and the last line provide extra functionality derived from having the keys sorted in the tree. The general syntax of this command is

BinaryTreeAssArray sortcmd subcommand ?arg ...?

but one would typically want to bundle the sortcmd with the BinaryTreeAssArray. A mydict command which orders keys using string compare could for example be created as

 interp alias {} mydict {} BinaryTreeAssArray {string compare}

and a versdict which orders keys using package vcompare would be defined as

 interp alias {} versdict {} BinaryTreeAssArray {package vcompare}

A command floatdict with floating-point numbers as keys could be defined through

 interp alias {} floatdict {} BinaryTreeAssArray {::apply {{a b} {expr {$a<$b ? -1 : $a>$b ? 1 : 0}} ::}}

and so on.

This is the pattern for code where you know you'll be handling keys of a particular type, but there is also the possibility of more abstract code which operates on data using a command providing a known API but not caring about its implementation. An example of this could be a parray-type command for abstract dictionary values:

 proc pAnyDict {cmdp dict} {
    set maxlen 0
    foreach key [{*}$cmdp keys $dict] {
       if {[string length $key]>$maxlen} then {set maxlen [string length $key]}
    }
    {*}$cmdp for {key value} $dict {
       puts [format {%*s = %-s} $maxlen $key $value]
    }
 }

This can be called as

 pAnyDict {::BinaryTreeAssArray {package vcompare}} $versdict_value

or as

 pAnyDict ::versdict $versdict_value

depending on what is appropriate. The former form gets more important in situations where it is some third piece of code that combined the sorting function with the BinaryTreeAssArray ensemble because it saw a need in combining the two.


Lightweight OO system

NEM One application (less important now there is TclOO) is for the creation of lightweight OO systems:

namespace eval person {
    namespace export *
    namespace ensemble create
    namespace ensemble create -command ::person::instance \
            -parameters self

    proc create {self name} {
        set self ::$self
        interp alias {} $self {} ::person::instance $self
        set $self [dict create name $name]
        return $self
    }

    proc name self {
        upvar 1 $self selfdict
        dict get $selfdict name
    }

    proc say {self msg} {
        puts "[$self name] says '$msg'"
    }
}

person create harry "Harry Potter"
harry say "Hello, World!"

TOOT will also benefit immensely from this.


White-box OO system

This is sketched in the TIP. The ensemble is in this case the class, or at least the dispatch command for the class. Objects are command prefixes which carry their state in the parameters.

Pro
The "objects" are values, so they can be stored in lists, dicts, and other data structures. Creation and destruction is extremely cheap.
Con
The "objects" have no control over when they are copied or destroyed, so they can't own anything that requires lifetime management.

An unusual trait (but hard to classify as pro or con) is that objects are immutable, so any mutating method (such as configure) must be implemented as returning a new object with the wanted change; the old object may still be around. Also unusual is that one should prefix with a {*} when calling an object, so the general syntax is

{*}$object method ?arg ...?

rather than the

$object method ?arg ...?

of the standard OO systems.

A minimal class (just providing getting and setting options) under such a system is

namespace eval minimal {
    # Object commands:
    namespace export {[a-z]*}
    namespace ensemble create -command [namespace current]::Dispatch\
      -parameters optdict

    proc cget {optdict name} {dict get $optdict $name}
    proc cgetall {optdict} {return $optdict}
    proc configure {optdict args} {
        foreach {key value} $args {
            if {![dict exists $optdict $key]} then {
                return -code error "Unknown option \"$key\", must\
                 be: [join [lsort [dict keys $optdict]] {, }]"
            }
        }
        list [namespace which Dispatch] [dict replace $optdict {*}$args]
    }
    
    # Class commands (for now, just create):
    namespace ensemble create -map {create Create}
    proc Create {args} {
        list [namespace which Dispatch] [dict create {*}$args]
    }
}

This behaves as

 % set obj [minimal create -foo 1 -bar 2]
 ::minimal::Dispatch {-foo 1 -bar 2}
 % {*}$obj cget -foo
 1
 % set obj2 [{*}$obj configure -foo 3]
 ::minimal::Dispatch {-foo 3 -bar 2}
 % {*}$obj cgetall
 -foo 1 -bar 2
 % {*}$obj2 cgetall
 -foo 3 -bar 2
 % set obj3 [{*}$obj configure -baz 1]
 Unknown option "-baz", must be: -bar, -foo

RS 2008-10-20: Isn't that what TOOT is all about?

Lars H: Yes, although TOOT tries harder to look like a standard OO system. TOOT has also evolved during its time on this wiki, so it might worth asking which TOOT? TOOT with {*} is perhaps that which comes closest.