Error processing request

Parameters

Body

Error

Unknow state transition: LINE -> END

-code

1

-level

0

-errorstack

INNER {returnImm {Unknow state transition: LINE -> END} {}} CALL {my render_wikit {Local Packages and Transparent Namespaces} {[SLB] Small Tcl scripts are easy to write with short procedure names
placed in the root namespace. As scripts become larger, breaking them
into relatively self-contained components which can be re-used and
avoiding name conflicts between those components becomes important.
This is normally done with namespaces and packages.

While the namespace and package mechanisms work reasonably well,
they do add clutter to scripts. [http://www.tcl.tk/cgi-bin/tct/tip/189.html]
has made it easier to write packages but you do still need to choose
a unique name for each package. Sometimes it just seems easier
to do something like:

        source [file dirname [info script]]/OtherScript.tcl

and bypass the use of packages altogether.

Yet packages have benefits. This page  explores an alternative that
gives some of the benefits of packages and namespaces but with less clutter
and reduced risk of name collisions...

The alternative is based on a 'loader' command which is used thus:

        loader use ?-force? ?-cmd cmd? ?-import? moduleName.tcl

Its behaviour is somewhat similar to both 'source' and 'package require'.

The full pathname for moduleName.tcl is obtained (see later).
If the module has not already been loaded, the pathname is sourced. This is done
within the context of a dynamically allocated namespace. Thus a procedure
definition 'proc a {} { ... }' within moduleName.tcl will automatically
be distinguished from any other definition of 'proc a'.

Provided the moduleName.tcl is sourced without error (or if it was previously
loaded successfully), the commands it exports are made available to the client
script. This can be done directly by importing the commands into the client's
namespace (if the -import switch was specified) or by adding an ensemble
command to the client's namespace that contains the exported commands.

The mechanism for determining the full pathname of a module loaded by 'loader use'
is of course based on a search path of directories. However, it does not require
a single global search path. Instead, 'loader use' obtains the namespace from
which it is called and uses that to determine the full pathname of the file which was sourced
when the namespace was allocated. This pathname can have its own unique search path
of directories. If the referenced module name is not found within the module's
own search path, its parent directory is considered. The mechanism does not
look for modules in the parent directory directly, instead it checks whether
a search path has been defined for the directory. If it has that search path is
used. The search then proceeds through all parent directories up to the top level.

Most directories will have no associated search path, so use of a deep directory
hierarchy will not trigger any additional file i/o. Some directories will specify
their own pathname as the only entry in their search path. A few may have a search path
containing other directories.

Each time a module is loaded, the parent directory is added to its own search path.
This is a bit ad-hoc but seems useful.

Modules can be made globally available by placing their directory in the search path
for directory '/'. This requires a little special handing on Windows, where
pathnames such as 'C:/' are considered top level directories. That special
handling is built into the mechanism and so is transparent to its clients.

The -force switch allows a module to be reloaded. In such cases, it is reloaded
into the same namespace that was previously allocated for it.Commands that are not contained within a namespace allocated by 'loader use'
can load modules either by specifying the absolute path of the module or by
referencing modules accessible from the search path for directory '/'.


The result of all this is that you can create a collection of modules which reference
each other using short descriptive names such as Process or Launcher. Such modules
can then be placed unchanged into a 'package' (though not a traditional Tcl package)
called MyPackage.
Clients of MyPackage will reference the modules using names such as
MyPackage/Process or MyPackage/Launcher. That larger system could itself
be turned into a package, and the original components might then be known to
its clients as BigPackage/MyPackage/Process and BigPackage/MyPackage/Launcher.

One limitation of the mechanism compared to traditional packages is that there's no support for
version checking. Of course a module can provide a version checking command that clients
can call once the module has been loaded.

Some other disadvantages:

   * reports in $errorInfo are less meaningful since they reference the dynamically allocated namespaces. We mitigate this by providing 'loader errorInfo', which translates the namespace names into file names. However, Tk's background error reporting uses $errorInfo.
   * you can't copy and paste procedures into wish/tclsh console, since procs no longer specify the namespace they belong to.
   * module authors must make rigorous use of  'namespace current' or 'namespace code' when defining  callbacks.

See also [An Alternative to Namespace] for another approach to avoiding hardcoded
namespaces.

**Implementation**

'''Implementation'''
 # Uses 'namespace ensemble'.
 
 namespace eval ::Loader {} {
    array set map {}
    variable nextId 0
 }
 
 proc ::Loader::findFile {clientNs fileName} {
 # Find the absolute pathname corresponding to $fileName.
 
    if {[file pathtype $fileName] eq "relative"} {
    
       variable files
       if {[info exists files($clientNs)]} {
          set clientFile $files($clientNs)
       } else {
          # Namespace was not allocated by Loader::use, just use the global
          # search path.
          set clientFile /
       }
             
       set child $clientFile
       variable searchDirs
 
       while {true} {
 
          foreach dir [getSearchDirs $child] {
             if {[file exists $dir/$fileName]} {
                return [file normalize $dir/$fileName]
             }
          }
 
          set parent [file dirname $child]
          if {$parent eq $child} {
             break;
          }
          
          set child $parent
       }
       
       if {$child ne "/"} {
       
          # On Windows, need to consider / as a special case since file dirname C:
          # yields C:.
          foreach dir [getSearchDirs /] {
             if {[file exists $dir/$fileName]} {
                return [file normalize $dir/$fileName]
             }
          }
          
          
       }
       
       error "Cannot find file $fileName in search path for $clientFile"
    }
    
    return [file normalize $fileName]
 }
 
 proc ::Loader::getSearchDirs {dir} {
    variable searchDirs
    set dirs {}
    catch {set dirs $searchDirs($dir)}
    return $dirs
 }
 
 proc ::Loader::setSearchDirs {dir dirs} {
 # Associate directory $dir with list $dirs.
 # When scripts located inside $dir attempt to locate
 # a script via 'loader use', the directories in $dirs
 # will be searched in the attempt to locate the script.
 
    variable searchDirs
    set searchDirs($dir) $dirs
 }
 
 proc ::Loader::addSearchDir {markedDir searchDir} {
 # Adds a directory to the search path for $dir.
 # This is a no-op if $searchDir is already in the search path.
 
    # Hack for Windows, we treat '/' as the root directory, i.e.
    # the location for the root directory search path. However,
    # in windows, each drive letter is a root directory.
    if {$markedDir ne "/"} {
       set markedDir [file normalize $markedDir]
    }
    
    set searchDir [file normalize $searchDir]
    set searchDirs [getSearchDirs $markedDir]
    foreach dir $searchDirs {
       if {$dir eq $searchDir} {
          return
       }
    }
    
    lappend searchDirs $searchDir
    setSearchDirs $markedDir $searchDirs
 }
 
 proc ::Loader::errorInfo {} {
 # Returned a transformed $errorInfo with dynamically allocated
 # namespaces replaced with the associated file names.
 
    variable files
    set result ""
    foreach line [split $::errorInfo \n] {
       if {   [regexp { ( *)\(procedure "(::dynamic[0-9]+)::(.*)" line ([0-9]+)\)$} $line dummy space ns proc lineNum]
           && [info exists files($ns)]} {
          
          append result "${space}(procedure \"$proc\" line $lineNum - from file $files($ns))"
       
       } else {
          append result $line\n
       }
    }
    return $result
 }
 
 proc ::Loader::use {args} {
 # Enables use of a module, loading it if necessary.
 # -import:  imports all commands of the loaded module into the client's namespace
 # -cmd cmd: defines cmd as an ensemble through which the client can
 #           access the commands of the loaded module
 # -force    force loading of the module even if it was loaded previously.
 
    set extraArgs {}
    set mode static
    set argNum 0
    
    if {[llength $args] eq 0} {
       error "wrong # args: should be \"loader use\" ?switches? fileName\""
    }
    
    set fileName [lindex $args end]
    
    set argNum 0
    set mode noop
    set force false
    while {$argNum < [llength $args]-1} {
       set arg [lindex $args $argNum]
       switch -- $arg {
       
          -cmd {
             set mode cmd
             incr argNum
             set cmd [lindex $args $argNum]
          }
          
          -import {
             set mode import
          }
          
          -force {
             set force true
          }
         
          default {
             lappend extraArgs $arg
          }
       }
       incr argNum
    }
    
    set clientNs [uplevel 1 namespace current]
 
    set fileName [findFile $clientNs $fileName]
    variable nextId
    variable namespaces
    variable files
 
    set alreadyLoaded [info exists namespaces($fileName)]
    
    if {$alreadyLoaded} {
       set pkgNs $namespaces($fileName)
    } else {
       set pkgNs ::dynamic$nextId
       set namespaces($fileName) $pkgNs
       set files($pkgNs) $fileName
       incr nextId
    }
 
    if {!$alreadyLoaded || $force} {
       # We knowthe directory contains at least one module so we make the directory
       # part of the search path for all scripts under the directory. This means
       # that all modules can load modules in the same directory using the leaf file name.
       set dir [file dirname $fileName]
       addSearchDir $dir $dir 
       
       uplevel #0 namespace eval $pkgNs [list source $extraArgs $fileName]
    }
    
    switch -- $mode {
       import {
          namespace eval $clientNs [list namespace import ::${pkgNs}::*]
       }
       
       cmd {
          
          if {![regexp  :: $cmd]} {
             # Normal case, client has specified the command within its namespace,
             # so concatenate them together.
             set cmd ${clientNs}::$cmd
          }
          # else client has specified the cmd name within the namespace hierarchy.
 
          namespace eval $pkgNs [list namespace ensemble create -command $cmd]
       }
       
       noop {
       }
       
       default {
          error "Internal error - unknown mode $mode"
       }
    }
 }
 
 namespace eval ::Loader {
    namespace export {[a-z]*}
    namespace ensemble create -command ::loader
 }
 

'''Example usage'''

### toplevel.tcl

 source Loader.tcl
 loader addSearchDir / [pwd]
 loader use -cmd m1 SubDir/Module1.tcl
 
 m1 a
 
### SubDir/Module1.tcl
  loader use -cmd m2 Module2.tcl

 proc a {} {
    puts "This is proc a from Module1 defined in namespace [namespace current]"
    m2 a
 }
 
 proc b {} {
    puts "This is proc b from Module1 defined in namespace [namespace current]"
 
 }
 
 namespace export {[a-z]*}


### SubDir/Module2.tcl
  
 # Note: Module1.tcl and Module2.tcl both use each other. This is not
 # necessarily desirable but illustrates the flexibility of the mechanism.
 loader use -cmd m1 Module1.tcl

 proc a {} {
    puts "This is proc a from Module2 defined in namespace [namespace current]"
    m1 b
 }
 
 namespace export {[a-z]*}
======
} regexp2} CALL {my render {Local Packages and Transparent Namespaces} {[SLB] Small Tcl scripts are easy to write with short procedure names
placed in the root namespace. As scripts become larger, breaking them
into relatively self-contained components which can be re-used and
avoiding name conflicts between those components becomes important.
This is normally done with namespaces and packages.

While the namespace and package mechanisms work reasonably well,
they do add clutter to scripts. [http://www.tcl.tk/cgi-bin/tct/tip/189.html]
has made it easier to write packages but you do still need to choose
a unique name for each package. Sometimes it just seems easier
to do something like:

        source [file dirname [info script]]/OtherScript.tcl

and bypass the use of packages altogether.

Yet packages have benefits. This page  explores an alternative that
gives some of the benefits of packages and namespaces but with less clutter
and reduced risk of name collisions...

The alternative is based on a 'loader' command which is used thus:

        loader use ?-force? ?-cmd cmd? ?-import? moduleName.tcl

Its behaviour is somewhat similar to both 'source' and 'package require'.

The full pathname for moduleName.tcl is obtained (see later).
If the module has not already been loaded, the pathname is sourced. This is done
within the context of a dynamically allocated namespace. Thus a procedure
definition 'proc a {} { ... }' within moduleName.tcl will automatically
be distinguished from any other definition of 'proc a'.

Provided the moduleName.tcl is sourced without error (or if it was previously
loaded successfully), the commands it exports are made available to the client
script. This can be done directly by importing the commands into the client's
namespace (if the -import switch was specified) or by adding an ensemble
command to the client's namespace that contains the exported commands.

The mechanism for determining the full pathname of a module loaded by 'loader use'
is of course based on a search path of directories. However, it does not require
a single global search path. Instead, 'loader use' obtains the namespace from
which it is called and uses that to determine the full pathname of the file which was sourced
when the namespace was allocated. This pathname can have its own unique search path
of directories. If the referenced module name is not found within the module's
own search path, its parent directory is considered. The mechanism does not
look for modules in the parent directory directly, instead it checks whether
a search path has been defined for the directory. If it has that search path is
used. The search then proceeds through all parent directories up to the top level.

Most directories will have no associated search path, so use of a deep directory
hierarchy will not trigger any additional file i/o. Some directories will specify
their own pathname as the only entry in their search path. A few may have a search path
containing other directories.

Each time a module is loaded, the parent directory is added to its own search path.
This is a bit ad-hoc but seems useful.

Modules can be made globally available by placing their directory in the search path
for directory '/'. This requires a little special handing on Windows, where
pathnames such as 'C:/' are considered top level directories. That special
handling is built into the mechanism and so is transparent to its clients.

The -force switch allows a module to be reloaded. In such cases, it is reloaded
into the same namespace that was previously allocated for it.Commands that are not contained within a namespace allocated by 'loader use'
can load modules either by specifying the absolute path of the module or by
referencing modules accessible from the search path for directory '/'.


The result of all this is that you can create a collection of modules which reference
each other using short descriptive names such as Process or Launcher. Such modules
can then be placed unchanged into a 'package' (though not a traditional Tcl package)
called MyPackage.
Clients of MyPackage will reference the modules using names such as
MyPackage/Process or MyPackage/Launcher. That larger system could itself
be turned into a package, and the original components might then be known to
its clients as BigPackage/MyPackage/Process and BigPackage/MyPackage/Launcher.

One limitation of the mechanism compared to traditional packages is that there's no support for
version checking. Of course a module can provide a version checking command that clients
can call once the module has been loaded.

Some other disadvantages:

   * reports in $errorInfo are less meaningful since they reference the dynamically allocated namespaces. We mitigate this by providing 'loader errorInfo', which translates the namespace names into file names. However, Tk's background error reporting uses $errorInfo.
   * you can't copy and paste procedures into wish/tclsh console, since procs no longer specify the namespace they belong to.
   * module authors must make rigorous use of  'namespace current' or 'namespace code' when defining  callbacks.

See also [An Alternative to Namespace] for another approach to avoiding hardcoded
namespaces.

**Implementation**

'''Implementation'''
 # Uses 'namespace ensemble'.
 
 namespace eval ::Loader {} {
    array set map {}
    variable nextId 0
 }
 
 proc ::Loader::findFile {clientNs fileName} {
 # Find the absolute pathname corresponding to $fileName.
 
    if {[file pathtype $fileName] eq "relative"} {
    
       variable files
       if {[info exists files($clientNs)]} {
          set clientFile $files($clientNs)
       } else {
          # Namespace was not allocated by Loader::use, just use the global
          # search path.
          set clientFile /
       }
             
       set child $clientFile
       variable searchDirs
 
       while {true} {
 
          foreach dir [getSearchDirs $child] {
             if {[file exists $dir/$fileName]} {
                return [file normalize $dir/$fileName]
             }
          }
 
          set parent [file dirname $child]
          if {$parent eq $child} {
             break;
          }
          
          set child $parent
       }
       
       if {$child ne "/"} {
       
          # On Windows, need to consider / as a special case since file dirname C:
          # yields C:.
          foreach dir [getSearchDirs /] {
             if {[file exists $dir/$fileName]} {
                return [file normalize $dir/$fileName]
             }
          }
          
          
       }
       
       error "Cannot find file $fileName in search path for $clientFile"
    }
    
    return [file normalize $fileName]
 }
 
 proc ::Loader::getSearchDirs {dir} {
    variable searchDirs
    set dirs {}
    catch {set dirs $searchDirs($dir)}
    return $dirs
 }
 
 proc ::Loader::setSearchDirs {dir dirs} {
 # Associate directory $dir with list $dirs.
 # When scripts located inside $dir attempt to locate
 # a script via 'loader use', the directories in $dirs
 # will be searched in the attempt to locate the script.
 
    variable searchDirs
    set searchDirs($dir) $dirs
 }
 
 proc ::Loader::addSearchDir {markedDir searchDir} {
 # Adds a directory to the search path for $dir.
 # This is a no-op if $searchDir is already in the search path.
 
    # Hack for Windows, we treat '/' as the root directory, i.e.
    # the location for the root directory search path. However,
    # in windows, each drive letter is a root directory.
    if {$markedDir ne "/"} {
       set markedDir [file normalize $markedDir]
    }
    
    set searchDir [file normalize $searchDir]
    set searchDirs [getSearchDirs $markedDir]
    foreach dir $searchDirs {
       if {$dir eq $searchDir} {
          return
       }
    }
    
    lappend searchDirs $searchDir
    setSearchDirs $markedDir $searchDirs
 }
 
 proc ::Loader::errorInfo {} {
 # Returned a transformed $errorInfo with dynamically allocated
 # namespaces replaced with the associated file names.
 
    variable files
    set result ""
    foreach line [split $::errorInfo \n] {
       if {   [regexp { ( *)\(procedure "(::dynamic[0-9]+)::(.*)" line ([0-9]+)\)$} $line dummy space ns proc lineNum]
           && [info exists files($ns)]} {
          
          append result "${space}(procedure \"$proc\" line $lineNum - from file $files($ns))"
       
       } else {
          append result $line\n
       }
    }
    return $result
 }
 
 proc ::Loader::use {args} {
 # Enables use of a module, loading it if necessary.
 # -import:  imports all commands of the loaded module into the client's namespace
 # -cmd cmd: defines cmd as an ensemble through which the client can
 #           access the commands of the loaded module
 # -force    force loading of the module even if it was loaded previously.
 
    set extraArgs {}
    set mode static
    set argNum 0
    
    if {[llength $args] eq 0} {
       error "wrong # args: should be \"loader use\" ?switches? fileName\""
    }
    
    set fileName [lindex $args end]
    
    set argNum 0
    set mode noop
    set force false
    while {$argNum < [llength $args]-1} {
       set arg [lindex $args $argNum]
       switch -- $arg {
       
          -cmd {
             set mode cmd
             incr argNum
             set cmd [lindex $args $argNum]
          }
          
          -import {
             set mode import
          }
          
          -force {
             set force true
          }
         
          default {
             lappend extraArgs $arg
          }
       }
       incr argNum
    }
    
    set clientNs [uplevel 1 namespace current]
 
    set fileName [findFile $clientNs $fileName]
    variable nextId
    variable namespaces
    variable files
 
    set alreadyLoaded [info exists namespaces($fileName)]
    
    if {$alreadyLoaded} {
       set pkgNs $namespaces($fileName)
    } else {
       set pkgNs ::dynamic$nextId
       set namespaces($fileName) $pkgNs
       set files($pkgNs) $fileName
       incr nextId
    }
 
    if {!$alreadyLoaded || $force} {
       # We knowthe directory contains at least one module so we make the directory
       # part of the search path for all scripts under the directory. This means
       # that all modules can load modules in the same directory using the leaf file name.
       set dir [file dirname $fileName]
       addSearchDir $dir $dir 
       
       uplevel #0 namespace eval $pkgNs [list source $extraArgs $fileName]
    }
    
    switch -- $mode {
       import {
          namespace eval $clientNs [list namespace import ::${pkgNs}::*]
       }
       
       cmd {
          
          if {![regexp  :: $cmd]} {
             # Normal case, client has specified the command within its namespace,
             # so concatenate them together.
             set cmd ${clientNs}::$cmd
          }
          # else client has specified the cmd name within the namespace hierarchy.
 
          namespace eval $pkgNs [list namespace ensemble create -command $cmd]
       }
       
       noop {
       }
       
       default {
          error "Internal error - unknown mode $mode"
       }
    }
 }
 
 namespace eval ::Loader {
    namespace export {[a-z]*}
    namespace ensemble create -command ::loader
 }
 

'''Example usage'''

### toplevel.tcl

 source Loader.tcl
 loader addSearchDir / [pwd]
 loader use -cmd m1 SubDir/Module1.tcl
 
 m1 a
 
### SubDir/Module1.tcl
  loader use -cmd m2 Module2.tcl

 proc a {} {
    puts "This is proc a from Module1 defined in namespace [namespace current]"
    m2 a
 }
 
 proc b {} {
    puts "This is proc b from Module1 defined in namespace [namespace current]"
 
 }
 
 namespace export {[a-z]*}


### SubDir/Module2.tcl
  
 # Note: Module1.tcl and Module2.tcl both use each other. This is not
 # necessarily desirable but illustrates the flexibility of the mechanism.
 loader use -cmd m1 Module1.tcl

 proc a {} {
    puts "This is proc a from Module2 defined in namespace [namespace current]"
    m1 b
 }
 
 namespace export {[a-z]*}
======
}} CALL {my revision {Local Packages and Transparent Namespaces}} CALL {::oo::Obj3943673 process revision/Local+Packages+and+Transparent+Namespaces} CALL {::oo::Obj3943671 process}

-errorcode

NONE

-errorinfo

Unknow state transition: LINE -> END
    while executing
"error $msg"
    (class "::Wiki" method "render_wikit" line 6)
    invoked from within
"my render_$default_markup $N $C $mkup_rendering_engine"
    (class "::Wiki" method "render" line 8)
    invoked from within
"my render $name $C"
    (class "::Wiki" method "revision" line 31)
    invoked from within
"my revision $page"
    (class "::Wiki" method "process" line 56)
    invoked from within
"$server process [string trim $uri /]"

-errorline

4