Version 3 of Templates and subst

Updated 2009-01-31 03:36:05 by andy

APW 2007-06-07 The contents of this page has been moved from TemplaTcl: a Tcl Template engine to here, to be able to reference that code. It is an example on how to use a simple templating language which can have Tcl code interspersed.

03jun07 jcw - But I don't see the point in embedding an escape layer using XML. Here's a visually simpler approach

  % if {$a != $b} {
      <h3> abc </h3>
      <li> [expr {$a != 1 ? $var1 : $var2}] </li>
  % } else {
      <li> [myobj getResult] </li>
      <table>
         <tr> Hello there </tr>
      </table>
  % }

What you get is the ability to use Tcl's subst where convenient and an outer-level line-by-line escape for larger amounts of code. Just like in Tcl, where we use both if's and expr's.

The example is contrived and has relatively much flow control, so it doesn't come out as clearly as real-world examples would. The above isn't an arbitrary choice btw - it's what Mason uses. As it turns out, less than a dozen lines of Tcl are needed to convert the above into a normal Tcl script which consists mostly of subst calls. In fact, I use a second level of <%...%> escapes as well for structuring the entire files into "components". This is the sort of thing Mason has worked out long ago, driven by practical use, not mere preference of a specific technology or language construct.

Before taking this discussion further, my suggestion would be to take some notation and run with it for a while to see just well it works out. You can't discuss these design choices without using them in several decently-sized real-world website applications. Well, not meaningfully anyway - IMO.

APW That approach looks very good to me and makes sense in that the final HTML layout is still visible in the template! Thanks to jcw for bringing that in.

jcw Here's the code for that approach:

  proc substify {in {var OUT}} {
    set pos 0  
    foreach pair [regexp -line -all -inline -indices {^%.*$} $in] {
      lassign $pair from to
      set s [string range $in $pos [expr {$from-2}]]
      append script "append $var \[" [list subst $s] "]\n" \
                    [string range $in [expr {$from+1}] $to] "\n"
      set pos [expr {$to+2}]
    }
    set s [string range $in $pos end]
    append script "append $var \[" [list subst $s] "]\n"
  }

You give it a template, and it returns a script you can eval to apply that template. I turn such scripts into procs, but that's just one way to use this.

APW Great work jcw, that's exactly, what I am looking for.

FF I noticed a little problem with switch:

 % switch what {
 % case1 {
 something
 % }
 % what {
 something other
 % }
 % }

seems that two adjacent % lines generate an empty append line, and a construct like switch don't like that

APW Here is a slightly modified version of substify which should handle switch correctly:

  proc substify {in {var OUT}} {
    set script ""
    set pos 0
    foreach pair [regexp -line -all -inline -indices {^%.*$} $in] {
      foreach {from to} $pair break
      set s [string range $in $pos [expr {$from-2}]]
      if {[string length $s] > 0} {
        append script "append $var \[" [list subst $s] "]\n"
      }
      append script "[string range $in [expr {$from+1}] $to]\n"
      set pos [expr {$to+2}]
    }
    set s [string range $in $pos end]
    if {[string length $s] > 0} {
      append script "append $var \[" [list subst $s] "]\n"
    }
    return $script
  }

Good catch, thanks! -jcw

FYI, fixed a missing init of $script in the above (when $in is empty).

merlin, 2007-11-02, 17:00 GMT: I think, it is simple to use a native tcl interpreter to parse templates. We have a simple template syntax, which is easy to convert ("compile" a template) into proper and correct tcl code. Doing subst on that code ("fetch" a template), we get a rendered page.

Compiling can be done without any variables set, which are used in the template. "Fetching" requires these variables to be set in the current scope (else we get a tcl error, about using unset variables).

  # ====== Compiler ======
  # compilng section block
  proc parse_section {arg} {return "\[_section [lindex $arg 0] [parse [lindex $arg 1]] \{"}
  proc parse_end_section {} {return "\}\]"}

  # compiling if-else blocks
  proc parse_if {arg} {return "\[if [parse $arg] \{_echo \""}
  proc parse_else {arg} {return "\"\} else \{_echo \""}
  proc parse_end_if {} {return "\"\}\]"}

  # curly braces
  proc parse_lbr {arg} {return "\\\{"}
  proc parse_rbr {arg} {return "\\\}"}

  # Determines block {...} type, passed to this proc as str parameter
  # and calls a proc to change that content into complement tcl-code.
  # Returns a new line to substitute {...} block
  proc parse {str} {
          if [string equal -length 1 $str \$] {
                  # Block starting with $ is a command to put variable value
                  if [regexp {\$(.+)\[(\d+)\]} $str trash vh vi] {
                          # If there are square brackets with an index in there, like [3],
                          # replacing that recursively with call to list index
                          set str "\[lindex [parse "$$vh"] $vi\]"
                  }
          } elseif [string equal -length 1 $str /] {
                  # Block starting with / is an end for some block.
                  # It can't contain any parameters
                  set cmd [string range $str 1 end]
                  set str [parse_end_$cmd]
          } else {
                  if [regexp {^(\w+)(?: (.+))?$} $str trash cmd arg] {
                          # If the block begins with the word, that word is considered as
                          # command name, the rest is parameters of that command
                          set str [parse_$cmd $arg]
                  }
                  # !TODO!: allow "else" with expressions parsing
          }
          return $str
  }

  # Find and replace any {...} to [parse ...], then subst the result
  proc compile {tpl} {
          upvar template template

  # --- Save ", ], [, $ between blocks ---
          # Find coordinates of that "betweens"
          set beg 0
          set escapes {}
          while {[regexp -start $beg -indices {\{([^\}]+)\}} $tpl res]} {
                  set escapes [concat $beg [expr [lindex $res 0]-1] $escapes]
                  set beg [expr [lindex $res 1]+1]
          }
          set escapes [concat $beg [expr [string length $tpl]-1] $escapes]
          # $escapes now contain a reversed list of blocks to escape

          # Escape rules
          set escamap {\" \\\\\\\" \$ \\\\\\\$ \[ \\\\\\\[ \] \\\\\\\]}
          # " (extra \ for subst)

          foreach {i j} $escapes {
                  set tpl [string replace $tpl $i $j [string map $escamap [string range $tpl $i $j]]]
          }

          set template $tpl

          # Regexp below finds any {...} and replaces them to [parse ...].
          # then subst calls there parses, and strips some backslashes
          uplevel {return [subst [regsub -all {\{([^\}]+)\}} $template {[parse {\1}]}]]}
  # !TODO! rewrite a compile to more clean code,
  # allowing a nested {} in the {...} blocks,
  # which will enable syntax like {section i {a b c}}<li>{$i}</li>{/section}
  }

  # ====== "Fetcher" ======
  proc _echo {val} {return $val}
  proc _section {name list block} {
          set result {}
          foreach $name $list {append result [subst $block]}
          return $result
  }

  proc fetch {ctpl} {
    upvar template template
    set template $ctpl
    uplevel {return [subst $template]}
  }

Usage of this code is demonstrated below (assuming code is in the file cp.tcl):

  source "cp.tcl"

  # A template
  set tpl {Some{if $e}<ul>{section i $cycle}<li>"{$i}"</li>{/section}</ul>{else}"{$cycle[1]}"{/if}tails}

  # Data for that template
  set e 1
  set cycle {one two three}

  # ====== Call of compiler and fetcher ======
  puts "Source template: $tpl"
  puts "Compiled template: [compile $tpl]"
  puts "Filled-in block: [fetch [compile $tpl]]"

Everything (fetch, compile) is done with subst =)


AMG: [substify] is great stuff. Thanks, JCW! See Web Templating for more information on how I use it.