Purpose: to document the use of the new '''namespace ensemble''' functions. http://tip.tcl.tk/112 ---- Other pages discussing applications of ensembles are: * [option parsing using ensemble] * [Using namespace ensemble without a namespace] * [data is code] ---- From the Tcl 8.5 namespace manpage http://www.tcl.tk/man/tcl8.5/TclCmd/namespace.htm: **Ensembles** The namespace ensemble is used to create and manipulate ensemble commands, which are commands formed by grouping subcommands together. The commands typically come from the current namespace when the ensemble was created, though this is configurable. Note that there may be any number of ensembles associated with any namespace (including none, which is true of all namespaces by default), though all the ensembles associated with a namespace are deleted when that namespace is deleted. The link between an ensemble command and its namespace is maintained however the ensemble is renamed. Three subcommands of the namespace ensemble command are defined: '''namespace ensemble create''' ''?option value ...?'': Creates a new ensemble command linked to the current namespace, returning the fully qualified name of the command created. The arguments to namespace ensemble create allow the configuration of the command as if with the namespace ensemble configure command. If not overridden with the -command option, this command creates an ensemble with exactly the same name as the linked namespace. See the section ENSEMBLE OPTIONS below for a full list of options supported and their effects. '''namespace ensemble configure''' ''command ?option? ?value ...?'': Retrieves the value of an option associated with the ensemble command named command, or updates some options associated with that ensemble command. See the section ENSEMBLE OPTIONS below for a full list of options supported and their effects. '''namespace ensemble exists''' ''command'': Returns a boolean value that describes whether the command command exists and is an ensemble command. This command only ever returns an error if the number of arguments to the command is wrong. When called, an ensemble command takes its first argument and looks it up (according to the rules described below) to discover a list of words to replace the ensemble command and subcommand with. The resulting list of words is then evaluated (with no further substitutions) as if that was what was typed originally (i.e. by passing the list of words through Tcl_EvalObjv) and returning the result of the command. Note that it is legal to make the target of an ensemble rewrite be another (or even the same) ensemble command. The ensemble command will not be visible through the use of the uplevel or info level commands. ***Ensemble Options*** The following options, supported by the namespace ensemble create and namespace ensemble configure commands, control how an ensemble command behaves: -map: When non-empty, this option supplies a dictionary that provides a mapping from subcommand names to a list of prefix words to substitute in place of the ensemble command and subcommand words (in a manner similar to an alias created with [interp alias]; the words are not reparsed after substitution). When this option is empty, the mapping will be from the local name of the subcommand to its fully-qualified name. Note that when this option is non-empty and the -subcommands option is empty, the ensemble subcommand names will be exactly those words that have mappings in the dictionary. -prefixes: This option (which is enabled by default) controls whether the ensemble command recognizes unambiguous prefixes of its subcommands. When turned off, the ensemble command requires exact matching of subcommand names. -subcommands: When non-empty, this option lists exactly what subcommands are in the ensemble. The mapping for each of those commands will either whatever is defined in the -map option, or to the command with the same name in the namespace linked to the ensemble. If this option is empty, the subcommands of the namespace will either be the keys of the dictionary listed in the -map option or the exported commands of the linked namespace at the time of the invocation of the ensemble command. -unknown: When non-empty, this option provides a partial command (to which all the words that are arguments to the ensemble command, including the fully-qualified name of the ensemble, are appended) to handle the case where an ensemble subcommand is not recognized and would otherwise generate an error. When empty (the default) an error (in the style of Tcl_GetIndexFromObj) is generated whenever the ensemble is unable to determine how to implement a particular subcommand. See UNKNOWN HANDLER BEHAVIOUR for more details. The following extra option is allowed by '''namespace ensemble create''': -command: This write-only option allows the name of the ensemble created by '''namespace ensemble create''' to be anything in any existing namespace. The default value for this option is the fully-qualified name of the namespace in which the namespace ensemble create command is invoked. The following extra option is allowed by '''namespace ensemble configure''': -namespace: This read-only option allows the retrieval of the fully-qualified name of the namespace which the ensemble was created within. ***Unknown Handler Behavior*** If an unknown handler is specified for an ensemble, that handler is called when the ensemble command would otherwise return an error due to it being unable to decide which subcommand to invoke. The exact conditions under which that occurs are controlled by the -subcommands, -map and -prefixes options as described above. To execute the unknown handler, the ensemble mechanism takes the specified -unknown option and appends each argument of the attempted ensemble command invocation (including the ensemble command itself, expressed as a fully qualified name). It invokes the resulting command in the scope of the attempted call. If the execution of the unknown handler terminates normally, the ensemble engine reparses the subcommand (as described below) and tries to dispatch it again, which is ideal for when the ensemble's configuration has been updated by the unknown subcommand handler. Any other kind of termination of the unknown handler is treated as an error. The result of the unknown handler is expected to be a list (it is an error if it is not). If the list is an empty list, the ensemble command attempts to look up the original subcommand again and, if it is not found this time, an error will be generated just as if the -unknown handler was not there (i.e. for any particular invocation of an ensemble, its unknown handler will be called at most once.) This makes it easy for the unknown handler to update the ensemble or its backing namespace so as to provide an implementation of the desired subcommand and reparse. When the result is a non-empty list, the words of that list are used to replace the ensemble command and subcommand, just as if they had been looked up in the -map. It is up to the unknown handler to supply all namespace qualifiers if the implementing subcommand is not in the namespace of the caller of the ensemble command. Also note that when ensemble commands are chained (e.g. if you make one of the commands that implement an ensemble subcommand into an ensemble, in a manner similar to the [text] widget's tag and mark subcommands) then the rewrite happens in the context of the caller of the outermost ensemble. That is to say that ensembles do not in themselves place any namespace contexts on the Tcl call stack. Where an empty -unknown handler is given (the default), the ensemble command will generate an error message based on the list of commands that the ensemble has defined (formatted similarly to the error message from Tcl_GetIndexFromObj). This is the error that will be thrown when the subcommand is still not recognized during reparsing. It is also an error for an -unknown handler to delete its namespace. <> **Discussion** [LV] Anyone feel up to writing an example of using this facility to add a new command to one of the Tcl core namespaces, so that the command was available? Perhaps adding a new math function to [expr]? [Lars H]: There is one such example (although rather off-topic) in [Fun with Chinese Characters]. ---- ''[escargo] 28 Mar 2005'' - Reading the description above about the '''-command''' option, and seeing it described as a "write-only option" seems to me to be a problem. Does it make sense to specify an option whose value cannot be read? Is there some other way to determine what value was passed via -command? [WHD]: In order to retrieve any of the options, you need the name of the ensemble, i.e., the command name. So reading the value of -command would just give you back the command name you already had. ''[escargo] 29 Mar 2005'' - OK. So it's not so much a write-only option as an option that would be pointless to read. (Unless there was something that could be renamed, and you wanted to find its original name, which may not be the case here.) ---- [DKF]: One key feature of 8.5's ensembles is that they can be [bytecode] compiled in some circumstances. ---- **Emulation** [Lars H]: Does anyone feel up to writing a [namespace ensemble] emulation for Tcl 8.4? The following is a start, but it lacks most of the features. ====== proc "namespace ensemble" {subcmd args} { switch -- $subcmd "configure" { error "namespace ensemble configure not supported (sorry!)" } "exists" { error "namespace ensemble exists not supported (sorry!)" } "create" {} default { error "unknown subcommand $subcmd, must be create" } array set Opt $args foreach o {-map -prefixes -subcommands -unknown} { if {[info exists Opt($o)]} then { error "$o option not supported (sorry!)" } } set ns [uplevel 1 {::namespace current}] if {![info exists Opt(-command)]} then { set Opt(-command) $ns } elseif {![string match ::* $Opt(-command)]} then { set Opt(-command) ${ns}::$Opt(-command) } interp alias {} $Opt(-command) {} ::ensemble_dispatch_0 $ns } proc ensemble_dispatch_0 {ns subcmd args} { set cmdL [list] foreach pat [namespace eval $ns {::namespace export}] { eval [list lappend cmdL] [ namespace eval $ns [list ::info commands $pat] ] } set matchL [lsearch -all -inline -glob $cmdL\ [string map {* \\* [ \\[ ] \\] ? \\?} $subcmd]*] if {[llength $matchL] == 1} then { uplevel 1 [list ${ns}::[lindex $matchL 0]] $args } elseif {[llength $matchL] > 1} then { error "ambiguous subcommand \"$subcmd\": matches\ [join [lsort -dictionary $matchL] ", "]" } else { error "unknown subcommand \"$subcmd\": must be\ [join [lsort -dictionary $cmdL] ", "]" } } ====== ---- [kruzalex] Playing a little bit with emulation I got a following extension with examples: ====== proc "namespace ensemble" {subcmd args} { switch -- $subcmd "exists" { error "namespace ensemble exists not supported (sorry!)" } "[subst $subcmd]" { } "create" {} default { error "unknown subcommand $subcmd, must be create" } if {[string equal $subcmd "configure"]} { array set Opt [lrange $args 1 end] } else { array set Opt $args } foreach o {-prefixes -subcommands -unknown} { if {[info exists Opt($o)]} then { error "$o option not supported (sorry!)" } } if {[string equal $subcmd "configure"]} { set ns [lindex $args 0] } else { set ns [uplevel 1 {::namespace current}] } if {![info exists Opt(-command)]} then { set Opt(-command) $ns } elseif {![string match ::* $Opt(-command)]} then { set Opt(-command) ${ns}::$Opt(-command) } if {[info exists Opt(-map)]} { interp alias {} $ns {} namespace inscope $ns if {[string equal $subcmd "configure"]} { foreach {func arg} [lindex $args 2] { namespace eval $ns {} $ns proc $func {} $arg } } else { foreach {func arg} [lindex $args 1] { $ns proc $func "{[info args ${ns}::${func}] [lindex $arg 1]}" [info body ${ns}::${func}] } } } interp alias {} $Opt(-command) {} ::ensemble_dispatch_0 $ns } proc ensemble_dispatch_0 {ns subcmd args} { set cmdL [list] set nsL [list] foreach pat [namespace eval $ns {::namespace export}] { eval [list lappend cmdL] [ namespace eval $ns [list ::info commands $pat] ] } set matchL [lsearch -all -inline -glob $cmdL\ [string map {* \\* [ \\[ ] \\] ? \\?} $subcmd]*] if {[llength $matchL] == 1} then { uplevel 1 [list ${ns}::[lindex $matchL 0]] $args } elseif {[llength $matchL] > 1} then { error "ambiguous subcommand \"$subcmd\": matches\ [join [lsort -dictionary $matchL] ", "]" } else { if [namespace exists ${ns}::${subcmd}] { foreach pat [namespace eval $ns {::namespace export}] { if {[llength [namespace eval $ns [list ::info commands $pat]]] == 0} { eval [list lappend nsL] $pat } } if {[llength [lsearch -all $nsL $subcmd]] == 0} { error "ERROR:\"$subcmd\" not known" } else { uplevel 1 [list ${ns}::${subcmd}::${args}] } } else { if ![namespace exists $ns] { error "unknown subcommand \"$subcmd\": must be\ [join [lsort -dictionary $cmdL] ", "]" } else { if [llength $args] { uplevel 1 [list ${ns}::${subcmd} $args] } else { uplevel 1 [list ${ns}::${subcmd}] } } } } } namespace eval carrot { namespace export foo bar potato "namespace ensemble" create ;# Creates command ::carrot proc foo {} {puts 1} ;# Exported proc bar {} {puts 2} ;# Exported proc boo {} {puts 3} ;# Not exported namespace eval turnip { ;# Not exported namespace export alpha proc alpha {} {puts 4} ;# Exported proc beta {} {puts 5} ;# Not exported "namespace ensemble" create } namespace eval potato { proc north {x} {puts 6,$x} ;# Not exported proc south {x} {puts 7,$x} ;# Not exported # Notice we resolve names locally, and Tcl metachars are not special "namespace ensemble" create -map { north {north [$meta[$chars} south {south [$meta[$chars} } } } % puts [carrot foo] 1 % puts [carrot bar] 2 % puts [carrot b] 2 % puts [carrot turnip alpha] ERROR: "turnip" not known % puts [carrot::turnip alpha] 4,[$meta[$chars % puts [carrot::turnip::alpha] 4,[$meta[$chars % puts [carrot potato north] 6,[$meta[$chars % puts [carrot::potato north] 6,[$meta[$chars % puts [carrot::potato::north] 6,[$meta[$chars % puts [carrot potato north 6] 6, 6 % puts [carrot potato south] 7,[$meta[$chars % rename ::carrot::potato ::spud % spud north 6,[$meta[$chars % spud south 7,[$meta[$chars % "namespace ensemble" configure spud -map { % north {puts NORTH} south {puts SOUTH} % } % spud north NORTH % spud south SOUTH ====== ---- [Lars H], 2008-07-25 (improved 08-18): Revisiting this issue, I came up with the following ''very crude'' (but quick) way of emulating an ensemble before Tcl 8.5. Instead of ====== namespace eval ::foo { namespace export * namespace ensemble create } ====== do ====== interp alias {} ::foo {} namespace inscope ::foo [lreplace x 0 0] ====== Creating further aliases in the foo namespace can even be used to emulate the effect of the -map option! The reason this works is that a call such as ====== foo bar baz ====== gets mapped to ====== namespace inscope ::foo {} bar baz ====== which in turn behaves as the ====== ::foo::bar baz ====== call one would expect from the '''bar''' subcommand of the '''foo''' namespace ensemble, because the [namespace inscope] call is equivalent to each of ====== namespace eval ::foo {} [list bar baz] namespace eval ::foo [concat {} [list bar baz]] namespace eval ::foo [concat [list bar baz]] namespace eval ::foo [list bar baz] ====== The business with `[[lreplace x 0 0]]` is to generate the empty list as a [pure list], since an impure "" here would lead to [shimmering]. There are however several respects in which this doesn't behave as a namespace ensemble: 1. There is no way to provide only some commands as subcommands — every command that works unqualified in the '''foo''' namespace becomes a subcommand. This includes the ''global'' Tcl commands! 2. There is no "unknown subcommand" error. 3. If you use [set], [unset], [for], [close], or the like as subcommand names, and implement some subcommands using procs in the '''foo''' namespace, then you need to qualify by :: every use of the ordinary Tcl commands that has the same name as one of your subcommands. This is error-prone. 5. [namespace inscope] pushes a context on the stack, so if the implementation of a normal namespace ensemble would `upvar 1` or `uplevel 1`, an implementation using this hack would need to do `upvar 2` or `uplevel 2`, respectively. It does however have the advantage over a separate dispatch [proc] that it does not increase the -level needed for [return], which prior to 8.5 could never be higher than 1. '''Warning:''' If you try the above in Tcl 8.5.0–8.5.4, then it crashes due to a bug [http://sourceforge.net/tracker/index.php?func=detail&aid=2055782&group_id=10894&atid=110894] in [Tcl_ConcatObj], but (provided you don't mind the shimmering) you can still experiment with it if you replace the `[[lreplace x 0 0]]` by e.g. `\n` (any other noncanonical representation of the empty list should work too). The [upvar] problem can furthermore be worked around by doing the [namespace inscope] ''in another [interp]reter!'' Concretely: ====== % interp create helper -safe helper % interp alias {} ::foo helper namespace inscope ::foo \n ::foo % interp alias helper ::foo::bar {} puts "I'm foo bar." ::foo::bar % interp alias helper ::foo::baz {} puts "I'm foo baz." ::foo::baz % foo bar I'm foo bar. % foo baz I'm foo baz. % proc pickup {varname} {upvar 1 $varname var; puts "The caller's $varname is $var."} % interp alias helper ::foo::pickup {} pickup ::foo::pickup % proc wrap {script} {eval $script} % set a 2; wrap {set a 3; foo pickup a} The caller's a is 3. ====== ---- **Extending ensembles** [GJS]: Someone on comp.lang.tcl wanted a way to just use "namespace ensemble add" (which doesn't exist) so I created a small package to take care of this. My original goal was to make it work with namespace, so "namespace ensemble add" could be used, but I had some trouble getting namespace to work when it was renamed and it is not an ensemble; so the commands are "nse add map", "nse add subcommand", "nse remove map", "nse remove subcommand", and "nse map". ====== #Namespace Ensemble Package #Gary J S #usage: # nse add subcommand namespace dict # dict is a dictionary value, keys are the command names keyValues are the # procs. This command adds the commands to the -subcommand option of the # ensemble. The ensemble must already exist before adding commands. # nse add map namespace dict # Same as "subcommand", but adds the commands to the -map option. # nse remove subcommand namespace list # list is a list of commands to remove from the -subcommand option of the # ensemble. There is no error if the command does not exist, it is simply not # removed. # nse remove map namespace list # Same as "subcommand" but removes the commands from the -map option. # nse map namespace # Creates an ensemble consisting of all procs and namespaces in namespace. package require Tcl 8.5 namespace eval ::nse { namespace eval add { proc subcommand {namespace dict} { #create a map of all commands if {[string length [namespace ensemble configure $namespace -subcommand]]} { #create a map of current commands in ensemble set map [dict create {*}[namespace ensemble configure $namespace -subcommand]] } #add new values to the map dict for {cmd proc} $dict { dict set map $cmd $proc } #remap the ensemble namespace ensemble configure $namespace -subcommand $map } proc map {namespace dict} { #create a map of all commands if {[string length [namespace ensemble configure $namespace -map]]} { #create a map of current commands in ensemble set map [dict create {*}[namespace ensemble configure $namespace -map]] } #add new values to the map dict for {cmd proc} $dict { dict set map $cmd $proc } #remap the ensemble namespace ensemble configure $namespace -map $map } } namespace eval remove { proc subcommand {namespace commands} { #create a map of current commands if {[string length [namespace ensemble configure $namespace -subcommand]]} { #create a map of current commands in ensemble set map [dict create {*}[namespace ensemble configure $namespace -subcommand]] } else {return} #remove commands from the mapping set map [dict remove $map {*}$commands] #remap the ensemble namespace ensemble configure $namespace -subcommand $map } proc map {namespace commands} { #create a map of current commands if {[string length [namespace ensemble configure $namespace -map]]} { #create a map of current commands in ensemble set map [dict create {*}[namespace ensemble configure $namespace -map]] } else {return} #remove commands from the mapping set map [dict remove $map {*}$commands] #remap the ensemble namespace ensemble configure $namespace -map $map } } proc map {namespace} { #create an ensembe, if it does not already exist if {![namespace ensemble exists $namespace]} { namespace eval $namespace {namespace ensemble create} } #turn all child namespaces into ensembles foreach ns [namespace children $namespace] { #create an ensembe, if it does not already exist if {![namespace ensemble exists $ns]} { namespace eval $ns {namespace ensemble create} } map $ns } #add all commands to ensemble, unless they start with a _ foreach cmd [info commands ${namespace}::*] { if {[string index [namespace tail $cmd] 0] != "_"} { ::nse::add::map $namespace [dict create [namespace tail $cmd] $cmd] } } } proc _init {args} { #map the namespace map ::nse namespace ensemble configure ::nse } } nse::_init #package info for pkg_mkIndex package provide NSE 0.3 ====== ---- **Extending ensembles generically** [Twylite] 2013-03-08: I was looking for a generic way to add a new subcommand to an existing ensemble. This can be tricky because: * An ensemble with no -map and no -subcommands exposes as its subcommands all commands exported from its namespace; * An ensemble a -map and -subcommands has exactly the listed subcommands, and they are looked up in the -map or - if not found there - in the commands exported from the ensemble's namespace. * An ensemble a -map and NO -subcommands has exactly the subcommands indicated by the -map. * Each value in a -map is a [command prefix], not a command name. The new subcommand being added could be implemented as an anonymous proc. Extending an ensemble that has a -map is fairly straightforward: alter the -map, and update the -subcommands list if it is not empty. If the ensemble does not have a -map, then it can be extended by adding an exported command (with the name of the subcommand) to the ensemble's namespace, and updating the list of -subcommands if not empty. But there are corner cases: we may want the subcommand to be linked to a differently named command, or to a command in a different namespace, or to a multi-word command prefix; or the target command not be exported from the ensemble's namespace. To resolve these cases we have several options: * Dynamically create ${ns}::${subcommand} as a proc or 'interp alias' that will forward to the desired command. This doesn't work if we're trying to link the subcommand to a command prefix and ${ns}::${subcommand} already exists. * Add the target command to the namespace's export list. This has unintentional side-effects for users of 'namespace import'. * Promote the ensemble to have a -map. This affects code that expects to extend the ensemble by adding new exported commands to the appropriate namespace. I have chosen the latter approach: it handles all corner cases, and breaks in more predictable ways than the other hacks. ====== # Find all exported commands for the calling namespace proc nsExported {} { set ns [uplevel 1 { ::namespace current }] concat {*}[lmap pattern [namespace eval $ns {::namespace export}] { lsearch -all -inline [info commands ${ns}::*] ${ns}::${pattern} }] } proc nsExtendEnsemble {ensembleCmd subcommand cmdPrefix} { set ensemble [uplevel 1 [list ::namespace which $ensembleCmd]] set nsconfig [namespace ensemble configure $ensemble] set map [dict get $nsconfig -map] set subcmds [dict get $nsconfig -subcommands] # Handle addition to command map in three cases: if { [dict size $map] > 0 } { # CASE 1: If there's a map, add to it. dict set map $subcommand $cmdPrefix } else { # CASE 2: If there's no map and this is a trivial expansion then we're # done. A trivial expansion is where cmd is a single-word command with # name $subcommand in the ensemble's namespace, and is already exported set ns [dict get $nsconfig -namespace] set needCmd ${ns}::${subcommand} set nsExportedCmds [namespace eval $ns { ::nsExported }] if { ($cmdPrefix ne $needCmd) || ($cmdPrefix ni $nsExportedCmds) } { # CASE 3: If there's no map and this is a non-trivial expansion, then # promote the ensemble to a map, and add the new subcommand. # The map will comprise all subcommands (that's either the subcommands # list if non-empty, or all exported commands), and we can them empty # the subcommand list. foreach c [expr { $subcmds ne {} ? $subcmds : $nsExportedCmds }] { dict set map [namespace tail $c] ${ns}::[namespace tail $c] } dict set map $subcommand $cmdPrefix set subcmds {} } } # Handle addition to the list of subcommands if { ($subcmds ne {}) && ($subcommand ni $subcmds) } { lappend subcmds $subcommand } # Update the ensemble, and we're done namespace ensemble configure $ensemble -map $map -subcommands $subcmds return {} } ====== [RS] 2013-12-05 - [Introspection] of an ensemble goes a little bit clumsier than we are used to, in Tcl: ====== % namespace ensemble configure dict -map ====== append ::tcl::dict::append create ::tcl::dict::create exists ::tcl::dict::exists filter ::tcl::dict::filter for ::tcl::dict::for get ::tcl::dict::get incr ::tcl::dict::incr info ::tcl::dict::info keys ::tcl::dict::keys lappend ::tcl::dict::lappend map ::tcl::dict::map merge ::tcl::dict::merge remove ::tcl::dict::remove replace ::tcl::dict::replace set ::tcl::dict::set size ::tcl::dict::size unset ::tcl::dict::unset update ::tcl::dict::update values ::tcl::dict::values with ::tcl::dict::with The following delivers nothing: % namespace ensemble configure dict -subcommands So here is a shortcut that does what I expected: % proc ens_subcmds cmd {dict keys [namespace ensemble configure $cmd -map]} % ens_subcmds dict append create exists filter for get incr info keys lappend map merge remove replace set size unset update values with <> **See also** * [namespace] * [wrapping commands] * [ensemble extend] * [A plugin replacement for proc supporting lambdas and ensembles] (assumes -subcommands and -map both empty) * [SubCommands and named arguments] <> Command