**What is a command prefix?** A '''command prefix''' is a prefix of a command (in the sense of rule 2 of the [dodekalogue], i.e., a sequence of words) that is constructed with the expectation that zero or more arguments will be appended to it and that the resulting command will be evaluated. This is very often done several times for each command prefix, with the arguments completing the command being different for each evaluation. The classical (although not very efficient) example of this is the '''-command''' option of [lsort], which takes a command prefix as argument. This command prefix is supposed to take two arguments and compare them according to some custom order, returning -1, 0, or 1 according to how the comparison came out. The equally classical example of such a command option is [string compare], so one can say lsort -command {string compare} {a B c D 0} which however is just the same as `lsort {a B c D 0}`. A more interesting choice is [package vcompare], which compares version numbers: % lsort -command {package vcompare} {2.0.1 3 1.10 1.9 2a0} 1.9 1.10 2a0 2.0.1 3 The '''-dictionary''' option of [lsort] is rather close, but it wouldn't get the alpha and beta versions right: % lsort -dictionary {2.0.1 3 1.10 1.9 2a0} 1.9 1.10 2.0.1 2a0 3 In order to see (at least some of) how the sorting gets done, we can define a procedure that calls some other command prefix to do the actual sorting, writes the result to [stdout], and then returns it: proc showsort {prefix a b} { set res [{*}$prefix $a $b] puts "$a [expr {$res<0 ? "<" : $res>0 ? ">" : "="}] $b" return $res } For example, % lsort -command {showsort {string compare}} {a B c D 0} a > B c > D B < D a > D a < c B > 0 0 B D a c % lsort -command {showsort {package vcompare}} {2.0.1 3 1.10 1.9 2a0} 2.0.1 < 3 1.10 > 1.9 2.0.1 > 1.9 2.0.1 > 1.10 1.9 < 2a0 1.10 < 2a0 2.0.1 > 2a0 1.9 1.10 2a0 2.0.1 3 This particular technique — to let one or more of the words in a command prefix themselves be command prefixes that handle some subtask — can become very powerful if you have several command prefixes that do related things, much for the same reasons as pipelines make the Unix command-line powerful. **Command prefix flavours** Modern command prefixes are typically lists, i.e., they are supposed to be called as {*}$prefix $arg1 $arg2 ... but classical command prefixes has sometimes rather been called effectively as eval $prefix [list $arg1 $arg2 ...] These differ with respect to how they interpret characters that have special meaning in scripts but not in lists — the [eval] variant treats the ''prefix'' more as a "script prefix" than a list prefix. The most common use of this is in an [idiom] for [trace] callbacks where one isn't interested in the extra arguments, and ends the ''prefix'' with "`; #`" to end the command and have the arguments that are appended ignored as part of a comment: trace add variable localGuardVariable unset "rename someCommand {} ; #" The more modern way to achieve this effect is however to use [apply]: trace add variable localGuardVariable unset [list ::apply {args {rename someCommand ""} ::}] ''[NEM] (corrected by [Lars H]): Alternatively (and pre-[apply]) you can use a simple wrapper command that discards any other arguments:'' proc discard {script args} {eval $script} trace add variable foo unset [list discard {rename someCommand ""}] [[Find and include links to discussion of non-list command prefix flavours — not all of these are the same, either.]] **Command prefixes in comparison** In general, one uses [list] to form a command prefix and [{*}] to expand it, but beyond that: * [interp alias] can be used to turn any command prefix into a named command. * [apply] can be used to turn any lambda into a command prefix. The main difference between '''[script]s''' and command prefixes is that the latter typically take input (the arguments appended), whereas scripts do not. Hence if you need to communicate any information in to a script when evaluating it, you need to supply that through the context (e.g. store in variables with predetermined names). The main difference between '''named [command]s''' and command prefixes is that command prefixes can contain additional data. For example, in a '''[socket] -server''' callback one could use the same proc to handle connections for several ports, and use additional arguments to specify the exact protocol to use for incoming connections. Concretely: proc foo::incoming {secure sock clientaddr clientport} { fconfigure $sock -translation binary if {$secure} then {tcl::import $sock} # Do normal handshake } socket -server {foo::incoming 0} 3000 ; # Unsecure port socket -server {foo::incoming 1} 3001 ; # Secure port This difference is less pronounced at the C side, since e.g. [Tcl_CreateObjCommand] takes a ''clientData'' argument that it passes on to the C function actually implementing the Tcl command created. Data arguments embedded in a command prefix are thus a way to achieve on the Tcl side what has always been straightforward on the C side. From a usage point of view, a '''[lambda]''' is like a script that takes arguments, so it could be thought of as overcoming the disadvantage scripts have relative to command prefixes. In doing so, they also make heavy use of variable substitution however, which places them at the opposite end of the spectrum from command prefixes (or scripts) with embedded data. '''[Lambda]s''' are ''syntactically'' the odd man out in this collection, as they are not partial scripts. They are also special in that they provide their own local environment, but it is more often scripts that stand out as different in that respect, since a proc used as basis for a named command or command prefix also provides a local environment. %| | Script | Command | Lambda | Command prefix |% &| Needs Tcl version | Any | Any | 8.5+ | Any |& &| Can take arguments | No | Yes | Yes | Yes |& &| Difficulty embedding data | Medium | Impossible | High | Low |& &| Lifetime semantics | value | needs cleanup | value | value |& &| Encapsulation | Poor | Good | Excellent | Good |& ---- [NEM] Note that a command prefix can be handled much more efficiently than a general script. Under the hood a command prefix can be resolved to the Tcl_Command that implements it and this can then be invoked directly with little or no interpretive overhead. A general script however must be evaluated every time. While the parsing can be cached, there is still some overhead to resolve the command afresh each time. This is why things like '''lsort -command''' are slow: while they are only really used with command prefixes they are actually documented as treating the argument as a script, and so need to call eval. Hopefully this can be changed at some point. Any new commands should take command prefixes and invoke them as such (using [{*}]). Ideally, Tcl needs an [invoke] command that does the same as [uplevel] but treats its argument as a command prefix rather than a general script. [MS] Please note the special case of scripts which are ''canonical lists'', ie, which was generated using [list] or any other of the list operations. The core recognizes that these are pre-parsed single commands and optimizes accordingly (if still not optimally) in many cases including [eval], [uplevel] and [namespace eval]. It would be possible to do things so that '''lsort -command''' and other such cases (notably traces) also recognize this special structure. Actually, it is not a matter of the structure being recognized, we just need not to spoil it when present. ''2008-09-19: just checked that lsort does the right thing already'' [NEM] Does this mean that code that calls [uplevel] in a loop on the same command should be roughly as efficient as C code calling Tcl_EvalObjv? [NEM] (Regarding lsort doing the right thing). OK, if lsort is able to cache the command resolution then surely the following two commands should be roughly equivalent in performance? ====== set xs [genlist 10000] ;# random list of chars from a-z time { lsort $xs } 100 ;# => 5246 microseconds per iteration set cmd [list string compare] time { lsort -command $cmd $xs } 100 ;# => 121838 microseconds per iteration ====== It seems to me that we must still be getting interpreter overhead in that loop, because a C coded sort calling a C coded compare should not be that slow, should it? [MS] of course there is command invocation overhead! The command is cached, but we are still calling Tcl_EvalObjv at each iteration, and checking that the cached command is valid, and resetting the interp's result, and (if a proc) pushing a CallFrame, and initializing local vars after checking that the number is ok, and checking the result for errors, and communicating the result with a new Tcl_Obj in the interp's result, and ... What I meant is: it is a command not a script, it is ''not reparsed'' at each iteration, the cached Command is reused. [NEM]: Much of this can be avoided, I think, especially with a native comparison command. I created a stripped down version of lsort specialised just for the -command usage (all other options removed). I then rearranged things so that it looks up the command at the start and then calls the objProc directly for comparisons. This reduces the runtime by approximately 1/2, but still nowhere near '''lsort -integer''' (this is sorting a 10000 element list of random ints in range 0-99, tests repeated 100 times): ====== fastsort : 22872.614190000004 microseconds per iteration lsort/cmd : 46619.153609999994 microseconds per iteration lsort : 3249.8842 microseconds per iteration ====== (Code available on request). Profiling, it seems that >20% of the time is spent in three functions: Tcl_GetIntFromObj, Tcl_SetObjResult, Tcl_GetObjResult. It seems having to pass the comparison result through the interpreter result object is the real performance killer. Indeed, if I cut out the trip through the interpreter result (handing int value via a C global var for just this test), then I can knock that time in half again (now only 4x slower than -integer): ====== fastsort : 12132.10835 microseconds per iteration ====== The other bottleneck is Tcl_GetIntFromObj, but that seems unavoidable. '''[DGP]''' Take a look at TclGetNumberFromObj for a possible route to improvement. ---- !!!!!! %| [Category Concept] | [Category Glossary] |% !!!!!!