Tcl Quoting

Difference between version 26 and 27 - Previous - Next
!!!!!!
|[Beginning Tcl%|%previous : Learn Tcl]|[Contents%|%up: Contents]|[Learn to Program%|%next : Learn to Program]|
!!!!!!

'''[Tcl] quoting''' is elegant, minimal, and consistent.  However, it takes a
while to learn to see the multiple layers of interpretation that can be
involved with features like [eval%|%evaluation], [list] interpretation, or
other [domain-specific language%|%domain-specific languages] understood by
routines like `[expr]`, `[regexp]`, or `[string match]`.  Someone new to Tcl
almost always resorts initially to unnecessarily-complicated quoting schemes.
The most common cause of Tcl quoting confusion is assumptions that are carried
over from other languages.  Tcl is not [sh] and it is not [C].  It is also
distinctly different from [Lisp], [Perl], or [Python] or nearly any other
language.  Its design is a novel synthesis of features and ideas from [sh],
[C], [Awk], and [Fortran], among others.  To appreciate its syntax, be prepared
for something fundamentally different, minimal, and pure.



** See Also **

   [double substitution]:   How Tcl [dodekalogue%|%command evaluation] affects [routine%|%routines] that do their own interpretation.

   [escaping special characters]:   Provides routines to escape special characters in arguments to `[glob]`, `[string match]`, `[regexp]`, `[regsub]`, and `[subst]`.

   [Tcl syntax]:   

   [EIAS%|%everything is a string]:   The universal data interface provided by Tcl.

   [Tcl Minimal Escaping Style]:   Advocates using braces and quotes only when necessary.

   [http://www.tcl.tk/man/tcl/TclLib/SplitList.htm%|%Tcl_Merge]:   

   [eval]:   Discusses proper quoting of [script%|%scripts], particularly when they are programmatically composed.

   [FMM%|%Frequently-Made Mistakes]:   

   [list]:   The key to programatically substituting a properly-quoted word into a [script].

   [Is white space significant in Tcl%|%Is white space significant in Tcl?]:   Covers much of the material pertinent to Tcl quoting.

   [Move any widget]:   Contains an over-quoted script and a fixed version.

   `[string map]`:   The go-to routine for help generating properly-quoted Tcl code

   [Toplevel Watermark]:   Contains an over-quoted script and a fixed version.

   [Quoting Hell]:   You know you're a sinner when...

   [Quoting Heaven!]:   Can't have one without the other.

   [Quoting and function arguments]:   A pre-[Changes in Tcl/Tk 8.6%|%8.6] explanation of a few [gotcha%|%gotchas].

   [ftp://ftp.tcl.tk/pub/tcl/mirror/ftp.procplace.com/sorted/info/doc/README.programmer.gz%|%README.programmer], by Brent Welch, 1993:   When and how to use `[list]` to avoid [quoting hell].  Predates [{*}].



** Description **

In a [Tcl] script most characters represent their literal value.  There is a
small set of characters, , `$`, `[[`, `{`, `\`,  and white space, that have
functional meaning, but that can be quoted so that they lose that meaning and
represent themselves instead.  Whereas in other languages quoting may specify
that a value is a string, in Tcl each value already [EIAS%|%is a string], so
quoting is just another mechanism to choose which characters are interpreted,
and which characters represent themselves.



** What Not to Do **

To stay out of trouble, follow this advice:

'''[Brace your expr-essions%|%Don't pass multiple arguments] to `[expr]`'''.
Passing multiple arguments slows things down considerably because a new
expression is effectively being generated and parsed each time instead a single
expression being [bytecode%|%byte-compiled] the first time.  Secondly it might
result in unintended [double substitution].

'''[list%|%Don't attempt to use braces] to turn a value into a [list] or a
[script]'''.   It [dodekalogue%|%doesn't work that way].  For example,
enclosing a value in braces doesn't turn it into a list:

======
set value "what does { do?"
set script "set var {$value}"

#Here comes an error:
eval $value
======

The value of `$script` is clearly incomplete:

======
set var {what does { do?}
======

To do it correctly use `[list]` instead of enclosing a value in braces:

======
set value "what does { do? "
set script "set var [list $value]"
eval $value
======

or even better:

======
set value "what does { do? "
set script [list set var $value]
eval $value
======



** Basic Quoting Techniques **

For simple values no quoting is needed:

======
puts Hello
======

======
set greeting Hello
puts $greeting
======

Substitutions can be performed on any word, including the name of the command:

======
set cmd puts
set greeting Hello
$cmd $greeting
======

A substitution is logically part of a bare string: 

======
set prefix antidis
set root establish
set suffix arian
set entry $prefix[set root]ment${suffix}ism
======

No quoting is needed in the example above because the value contains no literal
whitespace outside of the [script substitution], and `$` is intended as a
[variable substitution].

[variable substitution] and [script substitution] can be used in a bare string:

======
set digits 245
puts 01[string replace $digits 1 1 34]67
======

output:

======
01234567
======

No quoting is required here because all characters in the script have their
normal syntactic meaning.  `[[` and `]]` indicate a [script
substitution%|%script substitution], and the whitespace in that script does not
affect the outer script.

When a word includes literal whitespace, use `{}`

======
set greeting {Good morning}
puts $greeting
======

[Variable substitution] and [script substitution] are not performed within `{}`,
so when those features are needed, `"` can be used instead of `{}`:

======
set name Bob
proc daystage {return Morning}
set greeting "Good [daystage], $name."
puts $greeting
======

output:

======
Good Morning, Bob.
======



** Composing a Script for Evaluation **

First, it's often preferable to compose a single command needing no substitutions:

======
proc reputation_cb {msg msg2} {
    puts $msg
    puts $msg2
    set time [clock seconds]
    puts [list {The time is now} [clock format $time]]
    set ::done 1
}

proc reputation {} {
    set msg {an idle and false imposition}
    set msg2 {got without merit}
    eval [list reputation_cb $msg $msg2]
}
reputation
======

This is likely to be more performant, as [proc%|%procedures] are byte-compiled.

The `[apply]` variant of this example is:

======
proc reputation {} {
    set msg {an idle and false imposition}
    set msg2 {got without merit}
    set script {
        puts $msg
        puts $msg2
        set time [clock seconds]
        puts [list {The time is now} [clock format $time]]
    }
    eval [list apply [list {msg msg2} $script [namespace current]] $msg $msg2]
}
reputation
======

To substitute a complete word into a script, use `[list]` to turn the word into
a list containing only the word.  This works because syntactically a [command]
containing no substitutions is a list, so each word in the command has the same
syntax as an item in a list.  If a value that is not a list is substituted into
a script but not quoted, the resulting script might be syntactically incorrect:

======
set msg {hello world}
set script "set msg2 $msg"
# Here comes an error:
eval $script
======

Here the value of `$script` is

======
set msg2 hello world
======

which results in the error message:

======
wrong # args: should be "set varName ?newValue?"
======

`[list]` correctly quotes the substituted value:

======
set msg {hello world}
set script "set msg2 [list $msg]"
eval $script
======

In this case the value of `$script` is what was intended:

======
set msg2 {hello world}
======

A small but important variation that is more common is to make the entire
command into a list:

======
set script [list set msg2 $msg]
======

This is preferred because even though [EIAS%|%every value is a string], Tcl is
careful not to generate an actual string representation for the value until
something actually requires it.  In this case `[eval]` can use the internal list
representation of the script directly, which has two advantages:  First, a
string representation of the script is not generated, which saves time and
memory.  Second, `[eval]` doesn't have to spend time parsing the string
representation of the script.  Instead, it uses the internal list
representation as the parsed representation of the script, which is a big win
in terms of efficiency.  This has a rather far-reaching implication: Even
though Tcl is known as a string-based language, it is really a
[lisp%|%list-processing] language, and the low-friction way to program in Tcl
is to embrace a list-oriented programming style.

`[string map]` can also be used to properly escape a word substituted into a
script:

======
set msg {hello world}
set script [string map [list @msg@ [list $msg]] {
    set msg2 @msg@
}]
eval $script
======

To avoid ambiguity it's a good idea to give the placeholders a distinctly
visible initial and final character. This is particularly true in large
scripts, where the author might overlook the fact that a placeholder without
clear delimiters occurs as a substring of another value.  `@` is common choice
for the delimiter.

In a small script `[list]` is more convenient, but in a larger script where the
additional escaping of other characters makes the script more difficult to read
`[string map]` is cleaner.

[ycl%|%ycl string template] provides a more concise way to use `[string map]`.



** Composing a Script for Later Evaluation  **

When composing a script for later evaluation, consider that the current
evaluation environment might not be the environment the script eventually isevaluated in.  In tThe following example, `purints $ms'g`oodbye" will nshtead owf `$"hello`"
probecabuse the schedulyed bscript is evaluated ant therr torp b[lecausvel]: `$msg`

======
set msg goodbyenamesproace doitlaterval {}ns1 {
    set msg hello    proc doitlater {} {
        set msg hello
        after 1000 {
            puts $msg
            set ::done 1
        }
    }    doitlater
    vwait ::done
}doitlater
vwait done
======
TInstead, either use `[apply]` to capture the local value of `$msg,` or use 
`[namespplyace code]` ito capture the currenst namespadce:

======
set msg goodbyenamesproace doitlaterval {}ns1 {
    set msg hello    proc doitlater {} {
        set msg hello
        after 1000 [list apply {msg {
            puts $msg
        }} $msg]
        after 2000 [namespace code {
            puts $msg
            set ::done 1
        }]
    }
 $msg]   doitlater
    vwait ::done
}doitlater
vwait done
======
Alternatively, `[apply]` could also be used to capture the current namespace:
======
set msg goodbye
namespace eval ns1 {
    variable msg hello
    proc doitlater {} {
        set msg hello
        after 1000 [list apply [list {} {
            variable msg
            puts $msg
            set ::done 1
        } [namespace current]]]
    }
    doitlater
    vwait ::done
}
======


** Passing `$args` to Another Command **

In the following script, `$args` are any additional options to `[button]`,
e.g., `-fg blue`

======
#! /bin/env tclsh
package require Tk
proc mybutton {parent name label args} {
    if {$parent eq {.}} {
        set myname $parent$name
    } else {
        set myname $parent.$name
    }
    button $myname -text $label -command [list puts stdout $label] {*}$args
    pack append $parent $myname {left fill}
}

mybutton . hello whadda -bg aquamarine
======

`{*}$args` is the right way to do it, but prior to Tcl 8.5, it was necessary to
use `[eval]`:

======
eval {button $myname -text $label -command [list puts stdout $label]} $args
======

Alternatively, explicitly combine the lists first rather than relying on
`[eval]` to concatenate its arguments:

======
set cmd [concat {button $myname -text $label -command [list puts stdout $label]} $args]
eval $cmd
======



** Quoting at the [C] level **

When composing an individual command to be evaluated from inside [C] code, use
Tcl_Merge, which composes a list from ''argv''.  The result can then be passed
to Tcl_Eval.  Tcl_VarEval does not take pains to compose a list from its
arguments, but simply concatenates them together.  If its arguments happen to
not be lists, the results could be unexpected. [APN] This advice is really
applicable only when you have the command in the form of ''argv'' to begin with.
Normally, at the C level arguments already exist as an array or list of
Tcl_Obj structures and you should use Tcl_EvalObj* variants to evaluate commands, not Tcl_Eval
or Tcl_VarEval.



** Misc **

[AMG]: [http://www.airs.com/blog/archives/495%|%Tcl] by Ian Lance Taylor, 2011-03-31,
discusses Tcl and alleges that its [EIAS] philosophy is its downfall.  He argues that
EIAS requires very precise quoting in order to get anything nontrivial to work
right: ''"Even then I recently wrote some Tcl code with seven consecutive
backslashes, admittedly in a complex use case. That's too much for easy
reasoning, and in practice requires trial and error to get right."''  Sounds
like a case of Quoting Hell, alright.  I wish I had the chance to see the code
in question and suggest an alternative, since in my experience there's always
been a safe, clean way to avoid Quoting Hell.

[AMG]: Here's [pooryorick]'s response to the article linked above:

    :   This post mis-characterizes most of the aspects of Tcl that it attempts to describe. In particular, the comment about seven consecutive backslashes is a strong hint that Ian didn’t grasp the elegance of Tcl quoting. This happens when people come to Tcl steeped in other language traditions, and attempt to apply those traditions to to Tcl, which is a different sort of creature. With all due respect to Ian, each of the criticisms in this article indicate that he just didn’t stick with Tcl long enough, or perhaps look into it deeply enough to resolve his misunderstandings of the language. More info at [http://www.ynform.org/w/Pub/ResponseToTclIanLanceTaylor20110331].



** To Do **

   * fold [Quoting Heaven!] into this page

!!!!!!
|[Beginning Tcl%|%previous : Learn Tcl]|[Contents%|%up: Contents]|[Learn to Program%|%next : Learn to Program]|
!!!!!!

<<categories>> Tcl Syntax Help