mod_tcl: Tcl inside Apache

(According to https://tcl.apache.org/ the mod_tcl and Websh projects are no longer supported (2023). Rivet is still developed and supported.)

Notes on a talk given by Olly Stephens at Tcl2008.

mod_tcl is one of three technologies available through http://tcl.apache.org/ . All three added to support Apache's httpd daemon for authoring dynamic web pages in Tcl. All three can be embedded inside the Apache server itself as a persistent interpreter. The others, Websh, and Rivet focus on providing a good framework for generating HTML pages (classic way to mix HTML and a dynamic language). mod_tcl is a little different. mod_tcl focuses on exposing the guts of the request workflow to Tcl. Not as feature rich as the others for page generation, but capable of doing other things too.

mod_tcl is dangerously close to the deadpool. No major releases or code changes since 2002 (version 1.0). However, COMPANY: ARM use mod_tcl in one of our core customer facing websites. Used to manage the delivery of large datasets to partners. Over the years he's tweaked changed, refined the code enough that it is almost a complete rewrite. Lots of additional functionality. "About time I contributed it back." (This talk is the first step of that.)

Apache request flow:

  • Apache splits request processing into a number of phases
  • Within each phase, modules can hook in their own handlers
  • Handlers can either handle the phase or decline, in which case it filters through to the next one
  • rivet/websh focus on the response phase
  • mod_tcl lets you provide handlers for any of the request phases you want (ultimate in flexibility; however, very low-level--you need to know what you are doing)

Apache Directives

  • Basic syntax in Apache config file is: TclphaseHandler funcname [file]
  • When Apache enters phase: file is sourced if it hasn't been already or has changed (file is local to Apache root unless absolute); namespace is based on file path
  • funcname is called with no arguments (assumed to be within the namespace if not qualified)
  • Special case for response phase (file is assumed to be the file mapped to by the URI)
  • Samples:
 TclTranslateHandler translate tcllib/mapper.tcl
 TclLogHandler ::myapp::log_access
 TclResponseHandler page

Example 1: response handling

conf/httpd.conf:

 <FilesMatch "\.tcl$">
     SetHandler tcl-script
     TclResponseHandler response
 </FilesMatch>

htdocs/test.tcl:

 proc response {} {
     apache::request content_type text/html
     apache::send_http_header
     apache::rputs "<html>"
     ...
     apache::rputs "</html>"
     return $::apache::OK
 }

Example 2: authentication

conf/httpd.conf:

 <Location /secret>
     AuthType Basic
     AuthName "My Secret Stuff"
     Require valid-user
     TclAuthenHandler authen tcllib/auth/handlers.tcl
     TclAuthzHandler authz tcllib/auth/handlers.tcl
 </Location>

tcllib/auth/handlers.tcl:

 proc authen {} {
     foreach {rc pw} [apache::get_basic_auth_pw] break
     if {$rc != $::apache::OK} {return $rc}
     if {$pw eq [apache::request user]} {return $::apache::OK}
     apache::note_basic_auth_failure
     return $::apache::HTTP_FORBIDDEN
 }

 proc authz {} {
     if {[apache::request user] eq "olly"} {
         return $::apache::OK
     } else {return $::apache::HTTP_FORBIDDEN}
 }

Additional modules:

  • mod_tcl has hooks to allow other modules to interact with it
  • mod_tcl_io: wires standard streams (stdin, stdout, stderr) into Apache; provides the ability to write input and output filters in Tcl
  • mod_tcl_apreq: hooks apreq parser (parameters, forms, uploads) into Tcl; useful for response handlers--same library that Rivet uses
  • mod_tcl_ssl: Makes SSL parameters available to Tcl
  • mod_tcl_apr: Makes various useful apr and apr_util functions available to Tcl; should really be a normal Tcl library (not a mod_tcl extension)

Example 3: uploading via PUT

conf/httpd.conf:

 <Location /uploads>
     TclTransHandler translate tcllib/putfile.tcl
     TclResponseHandler response
 </Location>

tcllib/putfile.tcl:

 proc translate {} {
     apache::request filename tcllib/putfile.tcl
     return $::apache::OK
 }

 proc response {} {
     if {[apache::request method] ne "PUT"} {
         error "not a PUT request"
     }
     set nm [file join $::TMPDIR [file tail [apache::request uri]]]
     set fd [open $nm w]
     set sz [fcopy stdin $fd]
     close $fp
     apache::request content_type text/plain
     puts "uplaoded $nm ($sz bytes)"
     return $::apache::OK
 }

Example 4: downloading (ranges)

conf/httpd.conf:

 <Location /downloads>
     TclTransHandler translate tcllib/getfile.tcl
 </Location>

tcllib/getfile.tcl:

 proc translate {} {
     foreach {key fnm typ} [spool_file [apache::request uri]] break
     apache::request filename $fnm
     apache::handler type [namespace code "set_type $typ"]
     apache::handler log [namespace code "log_dnld $key [clock sec]"]
     return $::apache::OK
 }

 proc set_type {typ} {
     apache::request content_Type $typ
 }

 proc log_dnld {key start} {
     log_download $key [apache::request bytes_sent] \
                       [expr [clock seconds] - $start]
 }

(Note: Examples are (per Olly) abbreviated untested samples to fit within the constraints of a single slide, and also subject to my transcription errors--MC.)

Interpreter Handling:

  • Apache can be configured to handle concurrency differently: prefork (default on Linux), threaded (default on Windows), worker (default on Solaris). Historically mod_tcl was prefork only; can work with all types now
  • One Apache server can handle many different web sites
  • mod_tcl maintains a pool of interpreters (each virtual server has its own pool), no cross pollution of interpreters
  • Assigns an interpreter to a request on demand (i.e., lazily); once assigned same interpreter is used throughout the lifetime of a single request

Implementation challenges? APR tables. They are like dictionaries, but can have duplicate keys. They are used a lot for Apache structures so needed an interface.

Current status:

  • About to release code as a candidate 2.0
  • to a branch within the Apache svn repository
  • Just waiting on clearance from my employers (review scheduled with corporate legal department in two weeks)
  • Code is proven in production use
  • Currently in prefork mode on Solaris
  • Yet to try a Windows build, but it should work
  • Would like to add/create some other modules: mod_tcl_rivet or mod_tcl_websh (factoring out common code of course); mod_tcl_dav would be interesting (but hasn't had a business need for it yet); splitting out and improving apr interface (because there's lots of useful things in it)
  • Also very interested in mod_parrot (because he uses Perl and Python extensively too)