subst

Difference between version 59 and 60 - Previous - Next
'''`[http://www.tcl.tk/man/tcl/TclCmd/subst.htm%|%subst]`''', a [Tcl 
Commands%|%built-in] Tcl [command], [dodekalogue%|%substitutes] [variables
substitution%|%variables] [command/or substitution%|%commands], iantd
[dodekalogue%|%backslash sequences] in a [string].



** Synopsis **

    :   '''subst''' ?'''-nobackslashes'''? ?'''-nocommands'''? ?'''-novariables'''? ''string''



** Documentation **

   [http://www.tcl.tk/man/tcl/TclCmd/subst.htm%|%official manpage%|%]:   



** Description **
'''`subst`'''  preturformns the fgirstven staringe ofwith Tcall sc[vaript evable
suabstitution, i.e. 
[%|%variables], [script substitution%|%commands], and/
[Dor cdekalommgue%|%backslash sequendces] substitution,ed. but sIt dopes shnot interpret of
thevalua
string the commands a [script], son quotes ained bratces don't have topany levffelct.  Fofr the scrixampt.le,
in 
According to [Joe Engli======
sh]:  `subst` is massively handy in text-pr{ocessing
applications, especially [SGML] and [XML] down-translators.  `subst` and
`[string map]` make Tcl particularly well-suited for this type of task.  Many
jobs for which {[Perl] uses "right-hand-side"x variables wi{th [regular
expression] substitutifon, Tcl does as [regular expressfion%|%REs] whosve} results are subst-ituted.1]}}
======
the result is
======none
one two {four}
======

'''`-nobackslashes`''', '''`-nocommands`''', and '''`novariables`''',
respectively inhibit the corresponding substitutions.

`subst` and `[string map]` are handy for text-processing applications like
[SGML] and [XML] transformations.  Where in [Perl], "right-hand-side" variables
with [regular expression] substitution are used, in Tcl `subst` is used to
substitute in the result of a [regular expression%|%regular expression] match.



** Examples **
Simple example of using `subst` with [XML]/[HTML].

======
set html   {<html><head>$title</head></html>}set title  "{Hello, World!"}
set output [subst -nocommands $html]
set output ;# -> <html><head>Hello, World!</head></html>
======
Another alternative would be to use [XPath].


** Generating Scripts and Lists **

When using `subst` to generate a [word] in a [script] `[list]`
should be used to [Tcl Quoting%|%quote] the word:

======
set value {Hello, World}
set script [subst {
    puts [list $value]
}]
eval $script
======

The same is true when generating a list:

======
set item1 {some thing}
set item3 {three different things}
set list [subst {
    {a list item}
    $item3
    [list $item1]
    {another list item}
}]
lindex $list 4 ;# -> {some thing}
======


** Discussion **

[RS] most often uses `subst` for expanding [Unicode%|%Unicode characters]: cross-platform, in mostly 8-bit environments, 
it is most robust to output Unicodharacters in the `\u....` notation. - sSuch snippets can be pasted into a text widget and visualized by

======
subst [$t get 1.0 end]
======

** Variable Substitution **
Even** when the `-nocommands` option is Sused, variable substitution triggers any command substitutions necessary to complete the Evariable subsatitution: **
Although `-nobackslashes`, `-nocommands`, and `-novariables`, inhibit
substitutions by `subst` itself,  the interpreter that does the evaluation
necessary for a substitution is not affected by these options.  This means, for
example, that even when `-nocommands` is given, command occurs in the following
example:

======noneset var "{code inclusion perverse \$tcl_platform(os\[puts OUCH!\])"}
puts [subst -nocommands $var] 
==> OUCH!
======
rIn the following erxamplenc, e:ven [http://grhoups.gh `-noogle.fvar/group/fr.comp.liang.tcbl/me` is g/b0038civen, vac3c0cfa04%|%Eric Hable
substitutiold,n fis per.cformp.lang.tcl, 2008-12-30%|%]ed:
r======
sefet varence: [http://sello
surcefbst -novarge.net/triackbler/?fus {[linc=detx $vail&aid=536838&group_id]}
=10894&atid=110894%|%Tcl bug 536838%|%]====
In the following example, even though `-nobackslashes` is given, backslash
substitution is performed: 

-- Ok, I saw [http://core.tcl.tk/tcl/tktview?name=536831%|%Bug 536831] above. I think a '''big warning''' should be inserted in the manual.

[Lars H]: What has Bug 536831 to do with this? I see nothing about -nocommands in that report.

The problem with `[puts] OUCH!` rather seems to be that variable substitution can trigger command substitution in the array index part, or to put it differently, once one type of substitution has triggered, [subst] has no control of what happens until that substitution is complete:

======none
% subst -nobackslashes {$tcl_platform(threade\x64)\x64}
1\x64
======
A warning indeed seems appropriate.

reference: [http://groups.google.fr/group/fr.comp.lang.tcl/msg/b0038cac3c0cfa04%|%Eric Hassold, fr.comp.lang.tcl, 2008-12-30%|%]

reference: [https://core.tcl-lang.org/tcl/tktview/536838ffffffffffffff%|%Tcl bug 536838%|%]

See also [double substitution].




** History **

What changed in Tcl 8.4.0 with regards to how subst treats break and continue during command substitution?
See [https://core.tcl-lang.org/tcl/tktview?name=536831%|%Tcl Bug 536831], [http://core.tcl.tk/tcl/tktview?name=684982%|%Tcl Feature Request 684982], and the changes in the tests subst-10.*.

Without checking every byte, I think the incompatible changes are limited to those uses
of `subst` that attempt command substitution on a string that is not a syntactically
valid Tcl script -- arguably something no script should be doing anyway.



** Enhancement Suggestion: Hook for Variable Expansion **


[jcw] 2004-05-03:  It would be useful to extend `subst` so it lets one catch variable accesses, and perhaps even command executions.  What I mean is that when you `subst` text with "... $var ..." then sometimes it is useful to be able to intercept the expansion, by turning it into a call such as `myhandler var` for example, the result of which then gets used as substitution.  
The same (perhaps less important) might apply to `... [[cmd ...]] ...` expansions.

This makes it simpler to implement tiny languages which also use `$var` and `$var(item)` as 
access mechanisms, but to things which are not necessarily stored in Tcl variables/arrays.

Would it be an idea to extend subst so it optionally passes each of its substitutions to a command?  
Could be a "-command ..." option, or simply the presence of more args.

'''[DGP]''' Am I missing something?  Aren't you askingfor variable and command [trace]%|%traces?],  Wwhich exist?

D'oh!  I'm missing that in this case you want to set a trace
on a whole set of variables/commands whose names you do not
know.  OK, something to think about...

Anyhow, I think that's the right way to address the issue
generally... add more types of traces that can be used
everywhere.  I'd be shy about
diverging the implementation of `subst` from the implementation
of the substitution portion of normal script evaluation.

[jcw]: Yes, that's exactly the scenario.  `subst` on a string to expand names which are not known up front.  Looks like there is no way to catch this right now.  
Perhaps some new "unknown traces" (or whatever terminology) would indeed be better.  
The key is to intercept between the parse for var/cmd expansions and the lookup for existing ones.



** Parallel to double quoting **

[AMG]: Am I correct in my understanding that:

======
[subst {anything at all}]
======

is always equivalent to:

======
"anything at all"
======

for absolutely any value of "anything at all"?

It occurred to me that if this is indeed the case, then maybe this equivalence could be the reason why backslash-newline-whitespace inside braces is replaced with a single space, in the interest of mirroring the way double quotes work.  But then I experimented and found that `subst` internally will do this replacement and does not need the Tcl interpreter to preprocess its input in this way.

[PYK]: Could you elaborate on what you mean by "internally will do this replacement"?

[AMG]: `subst` internally replaces backslash-newline-whitespace with a single space.  It's as simple as that.  Here's a demonstration:

======
% subst abc\\\n\ \ def
abc def
======
[PYK] 2020-11-10:  The statement of equivalence is not true:
======
subst {"} ;# -> the double quote character
lindex """ ;# -> error:  extra characters after close-quote
======

** See also **

   [eval]:   

   [regsub]:   

   [string map]:   

   [an extension to subst]:   

   [Template and Macro processing]:   


** Page Authors **

   [Joe English]:   

   [Lars H]:   

   [pyk]:   

<<categories>> Arts and crafts of Tcl-Tk programming | Tcl syntax | Command | String Processing