Note that
eval exec $program $args
is not an adequate substitute for
eval [linsert $args 0 exec $program]
or
exec $program {}$argssince it breaks when $program contains spaces.
JR: Question: is
eval exec {$program} $argsequivalent to the linsert variant or the new syntax?
Answer: It is equivalent if $args is a canonical list. Or more generally if $args is a list that do not contain eval-unsafe stuff like semicolons or newlines.
Option 4: leading @: Larry Smith If you favor a leading character then "@" is easy to see and read, much more so than `.
Option 4: $(<whatever you want expanded>): Larry Smith Personally, I prefer marking a particular string or set of strings for an additional parsing/substitution pass using something that is reminicent of the current syntax. $([cmd]) or $([cmd1] [cmd2]) marks things clearly as being expanded. After all, it is the $ that signifies substitution, not the {}.
RS Since some 8.4.x, $(a) looks for the element "a" in the "anonymous array" named "", a feature used by Stooop, and also in not few of my scripts.
Jacob Levy July 28, 2003: What the example above tries to achieve seems to be possible without any new syntax, using a procedure called apply, which behaves as follows:
set l {a b c d e}
apply list $l $l
==> {a b c d e a b c d e}The definition of apply is that its first argument is a function that is applied to rest of the arguments which are evaluated and their values are collected into a list.
The definition of apply is probably something like:
proc apply {fn args} {
set l ""
foreach a $args {
set l [lconcat $l $a]
}
return [eval $fn $l]
}
proc lconcat {pre el} {
set l $pre
foreach e $el {lappend l $e}
return $l
}Now there may be more to TIP #144 that is not covered by this; however, at least for the given example, no new syntax is needed for implementing the functionality in Tcl.
DGP For this specific example, [apply] would be enough, yes.
When you extend things to get a command that can solve all of the problems addressed by the proposed syntax, you end up with the [expand] command proposed in TIP 103.
TIP 103 was considered and rejected.
Can we move on now please?
DGP For that matter, [apply] will not even work for the [exec] example above (when $program contains spaces).
Jacob Levy Fixed the $args handling, above. To handle $prog with spaces in it, you'd need to pass [list $prog] to the invocation of apply.
TP I like the apply option best. I would suggest that the apply command could have it's own mini-language (ala format, regexp, etc) using the {} or ` formatting. E.g.
apply {somecommand {}$expand_this $but_not_this}DGP The "special command with its own mini-language" approach was TIP 103.
TP I went back to read TIP 103 and the wandering discussion in the TCT mail archive. I'd still vote against including some special syntax in Tcl 8.5, 9.0, 10.0, ..., where some command or modified eval could do the job. I just don't favor including some hack that was previously illegal syntax ( {}$foo or `[bar] ) in the base language, or any other non-backward compatible syntax feature. Just call me an OF [L1 ].
KBK (2003-07-28) I know that I have code that will break with the backquote syntax.
Setok I know I probably bore people already with my ramblings on the caret expansion syntax but I find it pleasing. Note that expr does not allow double-carets (no logical XOR). So:
file join ^^$FilePath
or
exec $program ^^$args
The reason I find it nice is because it actually looks like the operation it is doing. It, to me, feels like real syntax instead of a hack.
The other option I suggested a couple of times was the ability for a command to affect its calling command line. This is somewhat akin to upvar and uplevel in how dangerous it can be, but allows macro-style functionality into Tcl. Other benefits are that no new syntax is added -- only a new capability provided to commands and procedures. I do realise it's somewhat complex.
MS notes that this second option does add new syntax, or rather modifies the existing syntax (which is even worse IMHO). This behaviour is forbidden in the Endekalogue by a combination of rules 2, 3, 6, 10 and/or 11 (details depend on the specific example).
(Continuing from a thread on comp.lang.tcl)
I think that paired backquotes is a good marker for argument expansion. The desired operation is to unlist something, so having a paired operator seems fitting. In sh, a backquoted string is not a grouping mechanism, but an expansion mechanism. In fact, to be completely analogous to shell programming, a backquoted string would mean to evaluate the backqoted string and retokenize the result. Hmm...I like this. Essentially, the backquotes would be markers more like square braces than like a dollar sign.
Fleshing this out, the examples from Tip #144 would become:
destroy `winfo children .` button .b `subst $stdargs` -text $mytext -bd $border exec $prog `set opts1` `getMoreopts` $file1 $file2
(Note that there's a couple of ways to get an expanded variable dereferences. Good thing or bad thing?)
I think this would be very understandable for anybody with a shell programming background, and tcl certainly has a lot of shell heritage. And IMO, it's *a lot* cleaner. Given that tcl is built around the notion of command evaluation, extending that concept seems like a good thing. I mean, the language doesn't even have infix math operators, and I see proposals for extending the {} syntax with indexes, ranges, etc. That's so far removed from my conception of "the Tcl Way" that it's hard to fathom. After all, wasn't even the dollar sign variable dereference syntax controversial back in the day? "Why do you need a special syntax? You should just use the set command!"
Scott Gargash
DKF: My opinion is that adding a rule to the Endekalog is the way forward (it adds a new class of capability and #144 is about right on the semantic behaviour) but I'm not 100% happy with either {} (too easy to generate from a typo) or backtick (not compatable.)
Joe Mistachkin: For the record, I think adding new global syntax to Tcl for this type of feature is a big mistake. I favor something along the lines of lconvert, [apply], [expand], or [return -code expand].
Here is some Tcl code that implements the required functionality (I whipped this up really quickly): New expand (Moved)
JR: I'll raise my hand in the purist (aka old fart) camp saying adding new syntax is a big mistake. I don't see this proposal so much as adding new syntax as adding new semantics. Its not just recognizing a new character, its fundamentally changing how things are processed. Things are no longer parsed exactly once, now in some cases they would be reparsed. This is not a piece of syntax that could be implemented by rewriting it to a command the way $ can (i.e., $foo can be exactly rewritten to [set foo] with no change in meaning). I think this ability to rewrite bits of syntax to commands in this way is a good thing. If it actually was implemented this way, you could change the semantics of the syntactic bits. The es shell [L2 ] implements things this way.
RS gets a bit tired of how arguments expand on this page ;)
MAK has successfully implemented [expand <list>] (though, as lexpand) in a pretty simple manner using a new Tcl_Obj type that's identical to the list object and doing the expansion in TclEvalObjvInternal(), for those that are interested but didn't think it could be done. Some test vectors would be welcome. You can find the patch for 8.4.4 at sourceforge [L3 ]. (I should note that this is NOT an implementation of TIP #103, but a commandified/non-syntactic version of TIP #144.)
Joe Mistachkin I did this same thing months ago as lconvert. The problem that I ran into was that it didn't interact well with the bytecode compiler. Perhaps others could shed some light on this subject?
MAK Admittedly, I hadn't seen lconvert until it was mentioned in the poll (after I'd done lexpand). The main difference from what I gather between the two is that lconvert tries to some degree to treat the to-be-expanded list as a first class object and you have to thus use it with variables and pass variable references, while lexpand is set up to only work in-line (thus allowing it to use the result of other calls directly), though I could be wrong. The benefit of lexpand over lconvert is thus that you can still do one-liners and the implementation is a fair bit simpler.
Unfortunately, as Joe English pointed out over on sourceforge, it does have some issues with bytecode compiled commands. I believe it to be fairly simple to address, though perhaps not as neatly as lexpand currently is. Let me use Joe's example to explain:
# Doesn't work (with the first version)
proc test {} {
set l [list x y z]
set x [list a b c [lexpand $l] d e f]
return $x
}I believe the problem here is simply that the list command (as well as some others), when compiled, don't call TclEvalObjvInternal(), so expansion is never attempted. If it were some other defined procedure (for example) then it would make that call and would be expanded. I think it just requires adding some additional places (in and around TclExecuteByteCode) where expansion will occur, but not any more significant than will have to take place with the {} prefix proposal (unless the intent is for the presence of {} to the parser not to compile a part of the script at all, I suppose). I plan to look into it soon. I think for the most part, all that needs to happen is for the top of the stack to be expanded before use. Take, oh, INST_EXPR_STK for example:
case INST_EXPR_STK:
objPtr = stackPtr[stackTop];
Tcl_ResetResult(interp);
DECACHE_STACK_INFO();
result = Tcl_ExprObj(interp, objPtr, &valuePtr);
...Add some macro or function so that it becomes:
case INST_EXPR_STK:
objPtr = Tcl_ExpandObj(stackPtr[stackTop]);
Tcl_ResetResult(interp);
DECACHE_STACK_INFO();
result = Tcl_ExprObj(interp, objPtr, &valuePtr);
...Unless objPtr requires expansion, it would just be the same as stackPtr[stackTop]]; if it does, it'd be expanded before passing to Tcl_ExprObj(). INST_LIST would take on something similar to what's going on in TclEvalObjvInternal() perhaps, etc. Or, I suppose the list object might just need to be modified to handle the expansions its self. Bottom line: I don't think it's a reason to reject the lexpand idea, it just needs a little more work and there are lots of ways to go about it.
MS's sees three families of proposals:
Let me be more explicit wrt the third family: those proposals essentially try to obtain the expansion effect from something akin to
cmd $normalArg [doTheExpansion $expArgs] $normalArg2
This is fundamentally impossible without changing the current rules, it is new syntax.
By rules 2, 3, 6 and 11, that command contains 4 words: [cmd] will be called with three arguments. By rule 10, the interpreter cannot do the expansion in a second pass.
Note that [cmd] is free to interpret its arguments in any way it pleases (rule 2); it could do the expansion itself. In this case, the role of [doTheExpansion] is just to mark its result in some manner as "expandable". Every other command should then be taught to look for those markers and decide to use them or not ... so Tcl ceases to be typeless. [MAK This is not true in the case of lexpand. No other command ever sees the marker because the expansion takes place before the command is called, just as with {}$ et. al. Unless the presense of {}$ causes a command line to not be compiled, its implementation will almost certainly be very similar to lexpand other than the method used for the internal marking for post-evaluation expansion.] [[MS] Ah, but then this is not exploiting the freedom given by rule 2 to [cmd], it is something done before [cmd] ever sees it. This comment belongs in the previous paragraph. Which underlines my point: it does not behave as a Tcl command but rather as new syntax. So, let us not confuse the issue by using a notation that suggests it is a command and forces us to change the current syntax rules. Implementation and compilation issues are a different matter that lie in the future ...]
As an illustration of the syntactic activity of [doTheExpansion], note that
set x [cmd $arg1 $arg2 $arg3]
would never raise a "too many args" error in [set], except if "cmd" happens to be "doTheExpansion". There is no way to write a proc or C-command that raises that error. By rules 2, 3, 6 and 11, [set] is called with two arguments - and that's that.
Now, my opinion is that the third alternative should be removed from further consideration. We may chose to live with a new command, we may choose to add a new substitution rule, but let us please not change the present rules ...
[MAK In my mind, it is a new substitution rule: one that pertains to a particular object type. If the result of lexpand isn't passed to anything, expansion never occurs. Though it is possible to write a proc or C procedure, with lexpand, such that "set x [cmd ...]" raises the error: just call lexpand as the last thing in the proc (rather than return), for example -- the marked-for-expansion object result would remain marked in the default proc result.]
MS asks: "object type"? No such thing in Tcl. As I wrote in the page on lconvert:
MAK The object type distinction is just a semi-clever (ok, you don't think so) way to insure binary compatibility if added in a dot rev. There are certainly other ways to accomplish it, and it doesn't really violate the "everything is a string" thing because all the expanded elements are totally transparent. Arrays aren't strings, but they have serialization/unserialization capabilities (with array set/get). So does this (with list and lexpand)]. Oh well. I guess I'll just have to see what someone manages to come up with to implement {}$ that makes it so different (if I were implementing {}$, I'd do so almost exactly like I'm doing with lexpand).
MS Most probably. My beef is with the fact that the new syntax is disguised as a new command and requires changes (as opposed to additions) in current syntax, not with (a) the fact that there is new syntax, nor (b) how it is implemented. I have purposefully stayed clear of (a), and have not thought of (b) yet.
MAK Yes, I can understand that position. Ultimately I just want to avoid replacing one quoting hell with another (I for one will not use {} even if it's adopted, opting for a pure-command solution instead) and though it a little harsh to discount the idea entirely. With that, I'll stop polluting the page with my blathering and be quiet until I get a chance to finish the patch for BCC compatibility (feel free to move the lexpand discussions to another page if you think it'd be appropriate).