FF 2007-30-05 - The discussion evolved in ToW Tcl on WebFreeWay has generate some interest, and here is an effort for making a minimal template engine. I would like to use this page for collecting ideas from people. Fell free to change everything, fix bugs and revolutionize the concept - it is the purpose of the page.
Here's how it is (initially) supposed to work:
first, let's create a template file (templtest.tmpl):
<table><%
for {set i 0} {$i < 4} {incr i} { %>
<tr>
<td><%= $i %></td>
</tr><%
} %>
</table>now run it:
source TemplaTcl TemplaTcl::parseFile templtest.tmpl TemplaTcl::dump
and here's the output that gets produced:
<table>
<tr>
<td>0</td>
</tr>
<tr>
<td>1</td>
</tr>
<tr>
<td>2</td>
</tr>
<tr>
<td>3</td>
</tr>
</table>Description: really, this engine it works by converting the whole template into a Tcl script, and then running it. This approach maybe it has pro and cons. I see the only pro that is very flexible and it leaves you maximum freedom of doing what you want, dealing with any data and putting how much code you want in the template (although the template should possibly hold minimum code - according to MVC, just the presentation logic, no code related to controller logic, state persistance, or data-retrieval logic).
The above example generates this "middle" code:
puts -nonewline <table>
for {set i 0} {$i < 4} {incr i} {
puts -nonewline {
<tr>
<td>}
puts -nonewline $i
puts -nonewline {</td>
</tr>}
}
puts -nonewline {
</table>
}(you can see that by replacing eval with puts)
With the last update I changed a few things: now the template's Tcl code runs into a safe interp (see Safe Interps). I added a new markup command: <%@ ... %> which is for setting parser/interp options (that is: before the parser, and before running the interpreter). So now in the safe interp you won't have source command anymore, but you could do:
<%@ include = poo.tcl %>
also another option I added is:
<%@ printCommand = append htmlContent %>
which is for replacing the puts -nonewline command used to spool the output.
#!/usr/bin/env tclsh
# tcllib required
package require Tcl 8.2
package require struct
namespace eval TemplaTcl {
variable data
variable mode
variable modeprev
variable options
variable ttInterp
proc parseFile {file} {
# read the template into $rawl - list of chars
set fh [open $file r]
set raw [read $fh]
close $fh
return [parse $raw]
}
proc parse {template} {
variable data
variable mode
variable modeprev
variable options
variable ttInterp
set mode raw
mode raw
::struct::queue cc
# create and setup a safe interpreter
# for running template's tcl code
catch {if [interp exists $ttInterp] {
interp delete $ttInterp}}
set ttInterp [interp create -safe]
interp share {} stdout $ttInterp
set options(printCommand) "puts -nonewline"
set rawl [split $template {}]
foreach ch $rawl {
# we work char-by-char :|
cc put $ch
# max block to compare (<%=) is 3 chars long:
if {[cc size] >= 3} {
set s3 [join [cc peek 3] {}]
set s2 [join [cc peek 2] {}]
if {$mode == "raw"} {
if {$s3 == "<%="} {
# <%= is a shorthand for puts ...
cc get 3; mode code; buf "$options(printCommand) "; continue
} elseif {$s3 == "<%@"} {
# <%@ is for setting preprocessor options
cc get 3; mode opt; continue
} elseif {$s2 == "<%"} {
# <% indicates begin of a code block
cc get 2; mode code; continue
}
} elseif {$mode == "code"} {
if {$s2 == "%>"} {
# and %> is the end of code block
cc get 2; mode raw; continue
}
} elseif {$mode == "opt"} {
# option parser
if {$s2 == "%>"} {
cc get 2;
parseOptions $data(buf:opt)
set data(buf:opt) {}
mode raw; continue
}
}
buf [cc get]
}
}
# finish working on the queue:
while {[cc size] > 0} {
buf [cc get]
}
mode flush
}
proc parseOptions {o} {
set optlist [split $o "\n;"]
foreach opt $optlist {
set pair [split $opt =]
set opt_ [string trim [lindex $pair 0]]
if {$opt_ == {}} continue
setOption $opt_ [string trim [lindex $pair 1]]
}
}
proc setOption {o v} {
variable options
variable ttInterp
switch $o {
printCommand {set options($o) $v}
include {$ttInterp invokehidden source $v}
default {return -code error -errorinfo "Unknown option: $o"}
}
}
proc mode {m} {
# used internally by parse - switches mode and stuff...
variable data
variable mode
variable modeprev
set newm {}
switch $m {code - raw - opt {set newm $m}}
if {$newm != {}} {
set modeprev $mode
set mode $newm
set data(buf:$mode) {}
}
if {$m == "flush"} {set modeprev $mode ; set mode _}
if {$mode != $modeprev} {
lappend data(out) [list $modeprev $data(buf:$modeprev)]
set data(buf:$modeprev) {}
}
}
proc buf {ch} {
# used internally by parse - put $ch in the right buffer
variable data
variable mode
append data(buf:$mode) $ch
}
proc dump {} {
# run the template script
variable data
variable options
variable ttInterp
set tclBuf ""
foreach l $data(out) {
set t [lindex $l 0]
set d [lindex $l 1]
switch $t {
raw {append tclBuf "$options(printCommand) [list $d]\n"}
code {append tclBuf "$d\n"}
}
}
$ttInterp eval $tclBuf
}
}escargo - I think it's a mistake to have parse read a file; it would be potentially more flexible to use if it just took a string to process as input, and let the caller determine where the string comes from. FF - good point! (changed)
APW Just found textutil::expander[L1 ] package in tcllib, which is included in ActiveTcl 8.4.14.0. Have the feeling it does something similar, maybe it's worth looking at it.
FF but is not much versatile. Just tried this:
% package require textutil::expander
% ::textutil::expander myexp
% set a 64
% myexp expand {[if {$a > 50} {] put some text [} {] put some other text [}]}
Error in macro:
[if {$a > 50} {] put some text [} {] put some other text [}]
--> missing close-bracketand I realized I feel well with TemplaTcl O:)
APW I think if you look closer to the manual you will see that you have to set the left and right bracket command i.e. "<%" and "%>". Next the token directly following the left bracket is interpreted as the command for handling the contents, so if you have "<%= hello %>" you will need a proc with name "=" for handling the code inside the macro.
% package require textutil::expander
% ::textutil::expander te
% te lb "<%"
% te rb "%>
% proc = {args} { return $args }
% te expand {this is a text which says <%= hello %> to you}
this is a text which says hello to youAdditionally for constructs like a "if" clause it is possible to push and pop the context and at the end to evaluate the code in the macro, normally you will need some tokens for marking begin and end of macro, means a construct like:
<% code if { ...} { %>
# some other code here
<% /code %>
proc code {args} {
te cpush ....
# some code here
}
proc /code {args} {
set str [te cpop ...]
# evaluate the code of the macro and set result
return result
}You can also define the number of passes to run for the same text to be able to do nested expansion if you use the expand.tcl directly from the author: William Duquette [L2 ]
escargo - I think you want to give some thought into what scope the template code executes at. Right now it seems to be run in proc dump. Two likely alternatives are to run it in the caller's scope or at global scope.
FF running in the caller scope might seem a convenient idea, but can potentially generate trouble, since the template is a tcl script that would run just between TemplaTcl::dump and the next command, potentially altering/damaging the caller's scope (if there are subsequent commands after the TemplaTcl::dump command); in order to make it more strict, I make it run in global scope (hoping that global is not polluted or has variable that can make the application unstable); but ideally it should run in a separate interp, copying the needed vars into that interpreter. Don't you think so?
escargo - Running in a safe interp would be the right thing, but the question might be what data can the code executing as part of template processing access (or access easily). One could reasonable argue that it should only access values provided by a supplied data object. I could see wanting to supply maybe an environment data object as well (for access to info like date and time, host name, user name, and other info that would not be part of the data coming out of a table to be published).
FF added safe interp, and also a new markup command for special use (see above). Just I don't know how to export variables to the new interp...(?) just with eval and set?