Wikit run in a Slave Interpreter on TclHttpd

"WhipIt" - Wikit run in a Slave Interpreter on TclHttpd

Since I read the comments below this became an itch I just had to scratch! Jeff Smith

Extracted from the How to grow your own Wiki page.

MDD: What would be really nice would be if someone would create a Starkit that combines tclhttpd and the wikit into one turnkey wiki server. Performance should be better, since you wouldn't have to use CGI.

11apr03 jcw - Yes, that would be cool, but it may need some work (either to make it properly keep state with repeated requests, or to make it work in a slave interp which gets re-inited on each access).

To try "WhipIt" create a directory and place in it whipit.kit, available from [L1 ]. Download wikit.kit from [L2 ] and place this in the same directory.

To launch on unix

 # ./tclkit whipit.kit

On Windows

from a dos window

 C:\tclkitsh whipit.kit

from windows explorer

 drag and release whipit.kit on tclkit

Point your browser to


BAJ uses whipit and finds it very useful, but just discovered that the history feature of wikit (enabled with "WIKIT_HIST" is broken by whipit (the puts that should write to a disk file ends up being remapped to the WktCGI_Puts command in script below).

Jeff Smith 22 November 2004 - Thanks for the report! I have never used the history feature. If you have a fix can you post it here?

BAJ 22 November 2004 - I think I have a fix now that involves only aliasing the "cgi_puts" command. I'll do a little more testing and then post it.

Jeff Smith 30 June 2005 - I have adjusted the code and now history and caching are working. This is reflected in the code below.

The code below is in the custom directory inside the whipit.kit starkit. To enable history or caching you need to unwrap the starkit and adjust the code to suit.

************* whipit.tcl ****************************

 # Create a cache and hist directory incase it is needed.
 file mkdir [file dir $starkit::topdir]/cache
 file mkdir [file dir $starkit::topdir]/hist

 set WktDir [file dir $starkit::topdir]/wikit.kit

 if {[file exists $WktDir]} {
     # Mount the wikit startkit
     vfs::mk4::Mount [file dir $starkit::topdir]/wikit.kit [file dir $starkit::topdir]/wikit.kit

     # Append wikit library from the mounted wikit starkit to the auto_path
     lappend auto_path [file dir $starkit::topdir]/wikit.kit/lib

 Url_PrefixInstall /wikit [list wikitProc /wikit]

 proc wikitProc {prefix sock suffix} {
         upvar #0 Httpd$sock data
         global env auto_path WktDir

      # Set the CGI environment variables

      Cgi_SetEnv $sock $prefix
      set env(PATH_INFO) $suffix

      # Various WIKIT environment variables are set

      # This sets the WIKIT_BASE environment variable dynamically based on
      # the host portion of the url typed in the browser. This is good if
      # you are not going to have caching enabled. If caching will be
      # enabled I would set it to an IP address or FQDN Hostname.
      set env(WIKIT_BASE) http://$env(HTTP_HOST)$prefix/

      # Uncomment the following if caching or history is required
      # set env(WIKIT_HIST) [file dir $starkit::topdir]/hist
      # set env(WIKIT_CACHE) [file dir $starkit::topdir]/cache

      # This sets the WIKIT_CSS environment variable
      # set env(WIKIT_CSS) /whipit2-css.css

      append data(query) ""

      # Use the Session Module in TclHttpd to create a slave interpreter.
      # A session state array is also created but we don't use it. This
      # maybe of some use if you have a need for a session based wiki.
      # To emulate a CGI environment for Wikit, a slave interpreter is created
      # and then destroyed per request.

      # Destroy any old sessions that are laying around. Interpreters that have
      # been created but for some reason not destroyed. In this instance
      # 2 minutes is the setting.

      Session_Reap 120 Wkt

      # Create a new session.

      set session [Session_Create Wkt 0]

      if {![file exists $WktDir]} {
          append NoWikit "<H2>Download a copy of \
                          <A HREF=\"\">wikit.kit</A> \
                           and place it in the same directory as whipit.kit</H2>\n"
          Httpd_ReturnData $sock text/html $NoWikit
          Session_Destroy $session

      if {[info exists env(WIKIT_CACHE)]} {
          if {[file exists $env(WIKIT_CACHE)$suffix.html]} {
              Httpd_ReturnFile $sock text/html $env(WIKIT_CACHE)$suffix.html
              Session_Destroy $session

      set env(REDIRECT_URL) $prefix$suffix
      set html [WktProcess $session $data(query)]

      Httpd_ReturnData $sock text/html $html
      Session_Destroy $session

 proc WktProcess {session query} {
      upvar #0 Session:$session state
      set interp $state(interp)
      global auto_path WktDir

      set WktCGI_LineNumber 0

      set html ""

      # Redirect the read and puts in the slave interpreter
      # that runs Wikit.

      interp eval  $interp {rename puts real_puts}
      interp alias $interp puts {} WktCGI_Puts $interp
      interp hidden $interp
      interp eval $interp {rename read real_read}
      interp alias $interp read {} WktCGI_Read $interp $query

      # Fool wikit to think it is running in it own Starkit so we can copy
      # the first ten pages from wikidoc.tkd

      interp eval $interp [list namespace eval starkit set topdir $WktDir]

      # Following taken from httpd.tcl for getting these variables into
      # a slave interpreter.
      # Transfer the scalar global variables

      foreach var {::v ::auto_path} {
         $interp eval [list set $var [set $var]]

      # Procedure that is run when the slave interpreter does a read.

      proc WktCGI_Read {interp query args} {

            if { [llength $args] == 2 && [lindex $args 0] == "stdin" } {
                return [$interp eval subst $query]
            } else {
                return [$interp eval real_read $args]

      # Precedure that is run when the slave interpreter does a puts.

      proc WktCGI_Puts {interp args} {
            upvar 1 WktCGI_LineNumber WktCGI_LineNumber
            upvar 1 html html

           if {[string match "-nonewline" [lindex $args 0]]} {
               set flag -nonewline
               set args [lrange $args 1 end]
           } else {
               set flag ""
           if {[llength $args] == 1} {
               set chan stdout
               incr WktCGI_LineNumber
               if {$WktCGI_LineNumber <= 3} {
                   return ""
               } else {
                   append html [lindex $args end]\n
           } elseif {[llength $args] == 2} {
               return [$interp eval real_puts $flag $args]
           } else {
               return [$interp error "wrong # args: should be \"puts ?-nonewline? ?channelId? string\""]

      # Setup and run Wikit.

      $interp eval {
                    set argv0 tclkit
                    set argv {}
                    set argc 0

                    package require cgi
                    #cgi_debug -on

                    # Work around for getting lassign into the slave interpreter.
                    # From "Practical Programming in Tcl and Tk" Third Edition
                    # by Brent B. Welch page 131.

                    proc lassign {valueList args} {
                         if {[llength $args] == 0 } {
                             error "wrong # args: lassign list varname ?varname ..?"
                         if {[llength $valueList] == 0} {
                             #Ensure one trip through the foreach loop
                              set valueList [list {}]
                         uplevel 1 [list foreach $args $valueList {break}]
                         return [lrange $valueList [llength $args] end]

                    # Source the starkit
                    package require app-wikit
       return $html


MHo This approach has one drawback: only one Wiki(-database) can be hosted with this method...

Jeff Smith 6 April 2006 - Just thinking out loud with a few modification this should be achieveable. You could setup a Url_PrefixInstall for each Wiki you wish to run. Then set argv of the wiki database you wish to access based on the prefix variable. To get history and caching working for each Wiki you could set WIKIT_HIST and WIKIT_CACHE based on the prefix variable also.

MHo 2006/05/12 Yes, this is the way I currently doing things with Wikit - haven't tried whipit.kit yet. I'm interested in including this feature into my single starpack descriped in Tclhttpd Winservice. Some questions arise:

  • The above patch seems to be already included in [L3 ] - is it right?
  • Haven't worked out yet if it is possible to include Wikit.kit itself into my starpack (nested VFS) - I don't think it will work, so I possibly have to include wikit unwrapped into my starpack, which is more difficult and makes further updating harder...
  • Also, the directories used for the tkd, chache and hist must be configurable because under Windows data directories are always on different path then program directories.
  • My current Domain handler for supporting multi-wikis is listed somewhere in the middle of Wikit configuration. I will try to merge everything together to support multiple Wikis out the of the box with my Tclhttpd Winservice.

Category Wikit - Category TclHttpd