{expand} was the syntax originally accepted for word expansion, but that was ultimately scrapped in favor of {*}. See TIP 157 for the details.
[expand syntax ...]
NEM 2006-11-03: Rather unexpectedly, the TCT have decided to change the syntax to {*}, in TIP 293 .
AMG: {expand} is now a syntax error: "extra characters after close-brace".
Here is a list of wiki pages that should be eventually updated: [L1 ].
NEM: Here's an attempt at explaining {expand} in Tcl 8.5. The easiest way to explain is perhaps with an example. Consider the following code, which may be used for launching a browser on Windows:
eval exec [auto_execok start] [list $url]
The reason that the [eval] is there is that [auto_execok] may return a list which represents the command to be run. This list needs to be "flattened" for exec. For instance, let's say [auto_execok start] returned:
% auto_execok start c:/windows/command.com /c start
If you just did:
exec [auto_execok start] $url
You would get an error about "can't find 'c:/windows/command.com /c start' - no such file or directory" or something similar. In order to flatten this list, Tcl before 8.5 requires use of [eval] (or more arcane uplevel equivalents) which does this for you (as it concats its arguments together before evaluation).
So, why not just say
eval exec [auto_execok start] $url
Because the $url may contain spaces, so we need to protect it so that it doesn't get flattened. So, for pre Tcl 8.5 programmers, the safe way to code is via the use of [list]:
eval exec [auto_execok start] [list $url]
There are lots of gotchas with quoting arguments correctly for [eval]. So, to solve this problem, 8.5 introduced the {expand} syntax, which inline expands arguments in a command invocation. What this means is that {expand}$list becomes word word word... at invocation time. So, we can now rewrite our example (in 8.5) as:
exec {expand}[auto_execok start] $urlwhich is clearer, safer (by avoiding eval) and better all round.
RS: {expand} is, so to speak, the inverse of list. It is not a command (because every command returns one single word), but an extension to the Tcl syntax. Consider:
{expand}[list 1 2 3]produces the three words
1 2 3
And for using {expand}, it can replace many occurrences of eval used.
eval pack [winfo children .]
is equivalent to
pack {expand}[winfo children .]But for code to be given to others (which might still run 8.4, 8.3 or older), best avoid {expand} for a while...
LES: Would someone remind us again why this command had to be braced? Why {expand} instead of expand? Why not expand [list 1 2 3] ?
MG: I think it's because {expand} isn't actually a command, like list, or pack, etc. It's more of an option. expand [list 1 2 3] would result in an 'invalid command name' error. It's a little confusing, I think, since it seems to be different to all other aspects of the Tcl syntax. . .
DKF: Actually, it's because expansion is not command-like behaviour (i.e. the result of a command is never broken up into multiple words by the Tcl interpreter itself.) Allowing commands to specify that they return an expanding list (and no, I don't know of any way to limit the effects to just some magical expand command) causes all sorts of problems, potentially including security faults when handing values between interpreters. And it actually makes the Endecalogue much more complex than the {expand} version, as it requires a lot more complexity to be added to the existing rules. By using a new piece of core language syntax, we side-step these problems and focus on giving simple expansion control to the programmer at the point when the programmer actually wants it (you could do it before with eval but it was very error-prone.)
SLB: The syntax was controversial. One discussion of the subject is recorded in DISCUSSION: Argument Expansion Syntax. See especially the comments from MS towards the end.
Because backward compatibility (a virtue to be sure) overrode aesthetics (a higher virtue, IMO).
Hmmmmm. Those who find
... [expand [list 1 2 3]] ...
superior aesthetically should, CL believes, re-read the discussion and Miguel's comments. The introcution of {expand} is not some sort of reactionary defense of abstract compatibility principles; instead, it preserves valuable syntactic simplicity.
I agree that the functionality warranted new syntax. It's just that the chosen syntax is exceptionally ugly, and it seemed that pretty much everyone agreed that it was ugly, but it was backwards compatible. But since I've engaged in this argument before and lost, there's no point in having it again. It's also the reason why I'm not putting my name on this.
Although, given that Tcl's big claim to fame is having almost no syntax (Tcl: Just 11 syntax rules!), I'm still curious as to how a 12th syntatctic rule (Tcl: Now with almost 10% more syntax!) makes it into a minor revision.
"... with difficulty", CL judges. Was it the right thing to do? I won't be secure in that until we have a few years' of experience with it.
MG: My two cents :) I think it's an awful shame that something so against-the-grain had to be added to make this possible, but I do definitely see the uses for it (even though I've not used it myself, still being on 8.4). As far as I can see, there was absolutely no way to achieve this effect without a change in syntax; I personally think that it's worth it, for all the hassle it saves with quoting for eval, etc.
DKF: FWIW, I believe that a sub-optimal syntax was selected and that it should have been done as part of Tcl 9.0 (but I didn't really want to see 8.5.) But other people disagreed with me on these matters, and I didn't (and don't) want to reopen those arguments (if you do, shout at Jeff :^). I will absolutely stand by the semantics of {expand} though; that's exactly right.
CMcC: IMHO {expand}$x is simply ugly (or, rather, complexly ugly), and sticks out like a wart on the otherwise flawless body of Tcl :) I would have preferred a composite brace style, such as {$a}@ or similar or grave accent:
`$a`.
LES: Ouch. What a can of worms I have opened. I was afraid it could happen. ... Honestly, I am pretty confident that the remarkable amount of heated debate over this intriguing issue by so many smart and experienced senior geeks has resulted in the most appropriate solution. However, I have read a lot (including lots of c.l.t., but not all) of that discussion and still can't find answers to this little idea that tickles the back of my mind: {expand} is mostly intended to solve a problem that almost invariably affects exec, isn't that right? Didn't someone suggest exec -expand when this discussion started? What was wrong with that idea? I don't remember ever seeing the argument that dismissed it.
RHS: In the way of a minor answer to the above, the ability to expand a list into separate arguments comes up in a variety of places, not just with exec. To add to that, a -expand flag would not totally satisfy the situation, even if it existed for every command. Keep in mind that {expand} allows one to specify which lists need to be expanded, rather than having all arguments expanded.
LES: Thanks. And what was wrong with creating an expand $list command? Or maybe concat -expand concat -recurse?
% concat hey {{1 2 3}} {a} {{{aa ab ac} {ba bb bc}}} mom
hey {1 2 3} a {{aa ab ac} {ba bb bc}} mom
% concat -recurse hey {{1 2 3}} {a} {{{aa ab ac} {ba bb bc}}} mom
hey 1 2 3 a aa ab ac ba bb bc momaotto: Does this mean that every argument has to be checked on {expand}?? I think this is very slow because you check thousends of words to get one {expand}. I vote for a expand command and a new obj type expandList which acts like {expand}
DKF: The current behaviour is actually very fast and causes no performance degradation at all when not used (the parser is slightly more complex, but only in what was previously an error-case). It's also been perf-analyzed so as to get an implementation technique that balances speed with maintainability. (The cost of looking for {expand} is tiny because Tcl already has to look at all those characters.)
FWIW, I wanted
`$x
or
`[[x y z]]
but that was a 9.0-only solution and many people didn't want to go to 9.0 for various reasons.
WHD: Is there any reason that your
`$x
notation couldn't be adopted as a synonym for {expand}$x in 9.0? It's certainly nicer to look at.
RS: Gentlemen: all hopes for an expand command are in vain, because every command returns one word as result, while {expand} will result in zero or more words. This requires a syntax change in Tcl - it just can't be done with the model we have so far. Nobody will be forced to use {expand} - it's up to each coder whether he prefers eval or {expand}.
LES: Many commands return a list. Good enough, IMEHO. So long as it's ONE single list, with no other lists in it.
RS: Sure enough, but a list is one word. The problem arises when you want a list to be spliced, e.g.
file copy [glob *.tcl] targetdir
This case requires either eval or {expand} (if you don't have one single Tcl file :)
Why should an individual developer {expand} rather than eval, when the former loses so much version compatibility? Donal leads off with an apt example:
set retList {
foo bar
[exit] $spong
}You can make this work with eval,
eval kooProc [lrange $retList 0 end]
but it's unsightly and inefficient of cycles.
Lars H: A better eval form is to use linsert:
eval [linsert $retList 0 kooProc]
This avoids the shimmering that is the reason for the above inefficiency. (Some people may of course criticise it for being even more unsightly.)
Sarnold: Some funny stuff we could do with expand (IMHO it could work properly):
set btprop [list -background #aaa -foreground #ffeeaa -relief flat]
button .button1 -text "Proceed" {expand}$btprop
pack .button1
button .button2 -text "Blah" {expand}$btprop
pack .button2The code above could be a quick and dirty way of creating styles in your GUIs. (just for fun)
SS 2005-01-11: I see there is some idea about adding
`
as alternative syntax of {expand}. I think it's very important to select just one way to do it. What is better between this two? I think, {expand} is better. We are not generally going to use {expand} every ten lines of code. It's a useful but relatively rare syntax, so to be clear and explicit wins over to be short. But then, we may wonder if there are alternatives that are better than both {expand} and
`
. Maybe there are: for example {} is not as subtle as
`
, it's visible enough, still not as verbose as {expand}. {} is less explicit, but after all, no one will get the sense of {expand} just looking at it... (my concern against
`
is that it is hard to see if you look at the source code), on the other hand {} scales well against a future where we find that, contrary to our expectations, we are using {expand} a lot in normal code. So I vote for {}, but my vote counts as zero :) If you agree with me, say it here or try to express your opinion in the chat with TCT members. Also I'm not sure if we are still in time to change it or not.
JDW 2005-01-11: I think {expand} syntax is fairly ugly, but the semantic content is very useful. How about some variation on ${listVar} like $@{listVar} or perhaps ${@listVar}? If @ is a problem, pick another symbol. If ${...} is a problem, how about $(listVar) or ^{listVar} or similar?
SS: The problem is about listVar: {expand} does not work just with variables, but with any word. For example this is valid Tcl8.5 code:
{expand}{puts hello}DKF: Semantically and structurally, {expand} is right on the money. It does exactly what is needed and is implemented exactly right. Syntactically, it sucks.
{expand} seems conceptually related to lisp macros, something like ,@ Could this syntax be made extensible and more general-purpose? Not that it would really be useful for much, but I could see {eval}$foo being a synonym for [eval $foo], for example. Likewise, I could see uses for lisp style read-macros or forth style immediate words, but certainly not before 9.0.
MG 2004-08-15: Assuming you mean something like
set foo [list puts -nonewline [list "My message"]]
{expand}$fooI think it would work anyway, doing
puts -nonewline "My message"
Is that what you mean?
escargo 2005-08-16: I was thinking that {expand} reminded me of a Forth IMMEDIATE word, one that gets applied before other transformations. Commands only return a single thing, while {expand} removes one layer of list wrapping. As the example using for below shows, the results are not what one always expects because each command is free to interpret its arguments as it chooses.
MG: I'm still using Tcl 8.4, so can't check, but what would 8.5 do if you have, for example,
set someVar {expand}Would it do The Right Thing, and actually set the variable to expand, or would it expand nothing and set the variable empty? And would
set someVar {{expand}}be a reasonable work-around to it, or would that effect the nesting-depth (in terms of lists) when the var was set? Or would it still try and expand? In reply to what SS said before, would trying to use some characters that aren't used elsewhere be better? Off the top of my head, I can't think of anywhere Tcl gives any significance to ~ or
`
- maybe
`expand`
(rather than just
`
) might be more user-friendly, without confusion of where {expand} is a braced string and where it's the new syntax?
DKF: Both those examples would work exactly as in 8.4 and before. The expand syntax is like this:
set someVar {expand}$varContainingAListTry entering that in 8.4 and see why the syntax was chosen... ;^)
MG *grins* In that case, it sounds good to me. It's a real shame to have to change the syntax, IMHO, but well worth it for what we'll get out of {expand}. Now I've just gotta wait for 8.5 to be released and used commonly so I can start using it... :)
LES 2005-08-015:
% set var {{a b c} {d e {foo bar hey} f} {g h i}}
{a b c} {d e {foo bar hey} f} {g h i}
% foreach i $var {puts $i}
a b c
d e {foo bar hey} f
g h i
% foreach i {expand}$var {puts $i}
a
b
cUh?! I REALLY don't know what just happened here. :-(
MG: I think (I'm on 8.4, so this is a best guess) that the problem is that the {expand} here basically turns this into:
foreach i {a b c} {d e {foo bar hey} f} {g h i} {puts $i}Which takes advantage of foreach's other syntax,
foreach $varlist1 $valuelist1 $varlist2 $valuelist2 $commands
So, it sets $i to each of a, b and c, each of $d, $e, ${foo bar hey} and $f to g, h and i.
The correct way to do what you want would probably be (but untested, as I don't have 8.5)
foreach i [list {expand}$var] {puts $i}A/AK: Using sugar for forward-compatible {*} may be of interest for those who want to give {expand} a try without upgrading to Tcl 8.5.
IL: Whether or not it would have been possible, it simply destroys the Tcl aesthetic, and frankly opens the door for future "ugly" inclusions into the language imho. I strongly suggest some rethinking towards this.
interp alias {} eval {} expandsIL, do you realize that the above isn't even close to a proper replacement for {expand}? I also whether this opens the door for 'future "ugly" inclusions'. As much debate as this issue generated, I doubt anyone is anxious to go through that again.
Oops, I had it backwards. I just did a little reading on how old this controversy is, so I'll stop there. Sorry it ended up this way, for what it's worth.
jcw 2005-11-07: Warning: can-of-worms ahead!
I'd like to see {expand} dropped from 8.5, and replaced by the following three conventions instead:
It's a strenuous compromise at best, but so be it.
And while we're at it, I'd like to see $[...] become shorthand for [expr {...}], if that's possible. Maybe even omit $ characters inside this syntactic construct, but that might be asking too much.
To shed its macro-processor-like image in the eyes of Tcl-newcomers, there is no way around it: a certain amount of algebraic-ish notation is needed. Iteration and indexing are cornerstone operations in a procedural language such as Tcl.
LES: All of these are already possible with unknown, Salt and Sugar etc... Actually, I already have my own shortcuts for llength and lindex. And I am a newbie. I don't see the point in changing Tcl's syntax in favor any particular personal preference.
AMG: Why is it necessary to abbreviate expand, [llength], and [lindex]? My guess: because we use the latter two so often with [for], and then we abbreviate the former for symmetry. My answer to that: don't use [for]! :^)
You say "x" is only "<int>" or "end-<int>". Must <int> be "immediately" specified, or can it be the result of a substitution? For instance, {$x} or {[expr {$u + $v}]}. If the answer is "can be result of substitution", what happens if the substitution yields * or # or expand or end-1? This can's getting pretty wormy, but who knows, maybe they're gummy worms. :^)
Would {x} support TIP 176 [L2 ]? Example: {5+6}. If you really want worms, you can go so far as to support full expr notation plus an optional "end-" prefix. At this point I know for certain we've gone too far. I'm not sure how long ago we went to plaid [L3 ], but... it happened.
As for $[...], I like it (of course). :^) But the omission of internal $'s has me wondering. It's something we can do by changing the expr language rather than the Tcl language, but I don't want see the two drift apart when it comes to quoting and substitution. Also I have in mind the case of an array named the same as a function: how will expr know which one is being used?
escargo 2005-11-08: If you are going to start introducing shorthand, I would want them to be more useful than tying them to lists. I very much like the uniform treatment of operators in Icon, where the size operator (*) applies to structures, strings, character sets, etc. Tcl's lack of uniformity for accessing values is sometimes an annoyance, but one I've learned to live with. There's not as much polymorphism as there could be, but there are also reasons for it. (Since everything is a string, you do need to distinguish between getting a range of characters from a list in string form versus getting a range of list elements; you can't expect identical notation to treat the two semantically different operations the same.)
PWQ 2005-11-09: We could stop all the debate by just allowing users to replace parts of Tcl such as the parser with their own code and do their own thing, but this approach is far too radical for mainstream Tcl developers. It seems that there must be one approach and one approach only to any given feature. Just expanding the ability of interp alias would go a long way to implementing some of the above.
DKF: IMO, if you replace the parser with a completely different one, you get a different language that's related to Tcl, but not Tcl. Such as Jim or Hecl. :-) Though Jeff is considering some stuff that way (struct syntax for dicts or somesuch; I've not thought through the details.)
PWQ: The language was changed to allow $var instead of set var, so what is the difference? Aliases allow the language to be changed. Like I said, too radical.