Note that 

 eval exec $program $args
is not an adequate substitute for 
   eval [linsert $args 0 exec $program]
or
  exec $program {}$args
since it breaks when $program contains spaces.


''Option 4: leading @'':
[Larry Smith]  If you favor a leading character then "@" is easy to
see and read, much more so than `.

   * So... this means [Larry Smith] has no need for the [text] widget?


''Option 4: $(<whatever you want expanded>)'':
[Larry Smith] Personally, I prefer marking a particular string or set
of strings for an additional parsing/substitution pass using something that is
reminicent of the current syntax.  $([[cmd]]) or $([[cmd1]] [[cmd2]]) marks things
clearly as being expanded.  After all, it is the $ that signifies substitution,
not the {}.

   * $([[cmd]]) is already legal syntax within variable substituition and shouldn't be mixed up with expansion.

----

[Jacob Levy] July 28, 2003: What the example above tries to achieve seems to be possible without any new syntax, using a procedure called
apply, which behaves as follows:

 set l {a b c d e}
 apply list $l $l
 ==> {a b c d e a b c d e}

The definition of apply is that its first argument is a function that is applied to rest of the arguments which are evaluated and their values
are collected into a list.

The definition of apply is probably something like:

 proc apply {fn args} {
     set l ""
     foreach a $args {
         set l [lconcat $l $a]
     }
     return [eval $fn $l]
 }

 proc lconcat {pre el} {
     set l $pre
     foreach e $el {lappend l $e}
     return $l
 }

Now there may be more to TIP #144 that is not covered by this; however, at least for the given example, no new syntax is needed for implementing the functionality in Tcl.

[DGP] For this specific example, [[apply]] would be enough, yes.

When you extend things to get a command that can solve '''all''' of
the problems addressed by the proposed syntax, you end up with the
[[expand]] command proposed in TIP 103.

TIP 103 was considered and rejected.

Can we move on now please?

[DGP] For that matter, [[apply]] will not even work for the [[exec]] example
above (when ''$program'' contains spaces).

[Jacob Levy] Fixed the $args handling, above. To handle $prog with spaces in it, you'd need to pass [[list $prog]] to the invocation of apply.

[TP] I like the '''apply''' option best.  I would suggest that the apply command could have it's own ''mini-language''
(ala format, regexp, etc) using the {} or ` formatting.  E.g.

   apply {somecommand {}$expand_this $but_not_this}

[DGP] The "special command with its own mini-language" approach
was TIP 103.

[TP] I went back to read TIP 103 and the wandering discussion in the TCT mail archive.  I'd still vote against
including some special syntax in Tcl 8.5, 9.0, 10.0,  ..., where some command or modified ''eval'' could do the job.  
I just don't favor including some hack that was previously illegal syntax ( {}$foo  or `[[bar]] ) in the base language, or
any other non-backward compatible syntax feature.  Just call me an OF [http://www.faqs.org/docs/jargon/O/old-fart.html].
 

----
[KBK] (2003-07-28) I ''know'' that I have code that will break with
the backquote syntax.

----

[Setok] I know I probably bore people already with my ramblings on the caret expansion syntax but I find it pleasing. Note that [expr] does not allow double-carets (no logical XOR). So:

  file join ^^$FilePath

or

  exec $program ^^$args

The reason I find it nice is because it actually ''looks'' like the operation it is doing. It, to me, feels like real syntax instead of a hack.

The other option I suggested a couple of times was the ability for a command to affect its calling command line. This is somewhat akin to [upvar] and [uplevel] in how dangerous it can be, but allows macro-style functionality into Tcl. Other benefits are that no new syntax is added -- only a new capability provided to commands and procedures. I do realise it's somewhat complex.

----

(Continuing from a thread on comp.lang.tcl)

I think that paired backquotes is a good marker for argument expansion. The desired operation is to unlist something, so having a paired operator seems fitting. In sh, a backquoted string is not a grouping mechanism, but an expansion mechanism. In fact, to be completely analogous to shell programming, a backquoted string would mean to evaluate the backqoted string and retokenize the result.  Hmm...I like this. Essentially, the backquotes would be markers more like square braces than like a dollar sign.   

Fleshing this out, the examples from Tip #144 would become:

   destroy `winfo children .`
   button .b `subst $stdargs` -text $mytext -bd $border
   exec $prog `set opts1` `getMoreopts` $file1 $file2

(Note that there's a couple of ways to get an expanded variable dereferences.  Good thing or bad thing?)

I think this would be very understandable for anybody with a shell programming background, and tcl certainly has a lot of shell heritage.  And IMO, it's *a lot* cleaner. Given that tcl is built around the notion of command evaluation, extending that concept seems like a good thing.  I mean, the language doesn't even have infix math operators, and I see proposals for extending the {} syntax with indexes, ranges, etc.  That's so far removed from my conception of "the Tcl Way" that it's hard to fathom.  After all, wasn't even the dollar sign variable dereference syntax controversial back in the day?  "Why do you need a special syntax? You should just use the set command!"

Scott Gargash

----
[DKF]: My opinion is that adding a rule to the Endekalog is the way forward (it adds a new class of capability and #144 is about right on the semantic behaviour) but I'm not 100% happy with either {} (too easy to generate from a typo) or backtick (not compatable.)

----

[Joe Mistachkin]: For the record, I think adding new global syntax to Tcl for this type of feature is a big mistake. I favor something along the lines of [lconvert], [[apply]], [[expand]], or [[return -code expand]].

Here is some Tcl code that implements the required functionality (I whipped this up really quickly):

----

 ## start of expand.tcl
 #
 # prototype expand command
 # Joe Mistachkin
 # 7/31/2003
 
 set expand_argument_error "wrong # args: should be \"expand ?-expand expand_list? arg ?arg ...?\""
 set expand_list_error "malformed expansion list"
 
 proc expand_processExpansionList { array_name error_name expand_list argument_count } {
   # get variable to put error message into...
   upvar "1" $error_name local_error
 
   if {$argument_count > "0"} then {
     # we are going to put the results into the caller's array.
     upvar "1" $array_name local_array
 
     # set all arguments to default state (not expanded)
     set index "0"
     while {$index < $argument_count} {
       set local_array($index) "0"
       incr index
     }
 
     # assume success until we find an actual error.
     set result "0"
 
     # are we processing a "to" range?
     set range(from) "-1"
     set range(to) "-1"
 
     set last_item ""
     foreach this_item $expand_list {
       if {[string is integer -strict $this_item] != "0"} then {
         if {$range(from) != "-1"} then {
           # mark the end of the range...
           set range(to) $this_item
 
           # make sure it's a valid range...
           if {$range(from) <= $range(to)} then {
             # mark entire range to be expanded...
             set index $range(from)
             while {$index <= $range(to)} {
               set local_array($index) "1"
               incr index
             }
 
             # reset range positions...
             set range(from) "-1"
             set range(to) "-1"
           } else {
             # nope, not a valid range.
             set local_error "not a valid range"
             set result "-1"
             break
           }
         } else {
           if {(($this_item >= "0") && ($this_item < $argument_count))} then {
             # mark this argument for expansion...
             set local_array($this_item) "1"
           } else {
             # out of bounds.
             set local_error "argument index out of bounds"
             set result "-1"
             break
           }
         }
       } else {
         switch -exact -- $this_item {
           "to"    {
             if {$range(from) == "-1"} then {
               if {[string is integer -strict $last_item] != "0"} then {
                 set range(from) $last_item
               } else {
                 # invalid start of range...
                 set local_error "invalid index for start of range"
                 set result "-1"
                 break
               }
             } else {
               # already started a range...
               set local_error "range already started"
               set result "-1"
               break
             }
           }
           "end"   {
             # mark last argument to be expanded...
             set local_array([expr {$argument_count - "1"}]) "1"
           }
           default {
             # unknown argument processing directive.
             set local_error "unknown argument processing directive"
             set result "-1"
             break
           }
         }
       }
 
       # set last item processed to current item...
       set last_item $this_item
     }
   } else {
     # NOTE: not really an error.
     set local_error "nothing to process"
     set result "0"
   }
 
   return $result
 }
 
 proc expand { args } {
   global expand_argument_error
   global expand_list_error
   #
   # check for list of things to expand...
   # (default is to expand nothing)
   #
   if {[llength $args] > "0"} then {
     switch -exact -- [lindex $args "0"] {
       "-expand" {
         # we want to expand the specified arguments.
         set expand "1"
         # next, there should be a list of things to expand.
         set index "1"
         # the option and expansion list are NOT included in the argument count.
         set argument_count [expr {[llength $args] - ($index + "1")}]
       }
       "-noexpand" {
         # we want to NOT expand the specified arguments.
         set expand "0"
         # next, there should be a list of things to NOT expand.
         set index "1"
         # the option and expansion list are NOT included in the argument count.
         set argument_count [expr {[llength $args] - ($index + "1")}]
       }
       default {
         # don't expand by default.
         set expand "0"
         # there is no list of things to expand.
         set index "-1"
         # all arguments are passed along.
         set argument_count [llength $args]
       }
     }
 
     if {(($index == "-1") || ([llength $args] > ($index + "1")))} then {
       if {$index != "-1"} then {
         # get list of args to modify.
         set expand_list [lindex $args $index]
 
         set expand_error ""
         array set expand_array {}
         if {[expand_processExpansionList expand_array expand_error $expand_list $argument_count] != "0"} then {
           error "$expand_list_error: $expand_error"
         }
       } else {
         # no arguments to modify.
         set expand_list [list]
       }
 
       # start with an empty list as the command string to evaluate.
       set result [list]
       # start just beyond the end of the expansion list (if any).
       set this_index [expr {$index + "1"}]
       while {$this_index < [llength $args]} {
         # get the translated argument index...
         if {$index != "-1"} then {
           set argument_index [expr {$this_index - ($index + "1")}]
         } else {
           set argument_index $this_index
         }
 
         if {[info exists expand_array($argument_index)] != "0"} then {
           # is the argument targeted for expansion?
           if {$expand_array($argument_index) != "0"} then {
             set do_expand [expr {$expand ? "1" : "0"}]
           } else {
             set do_expand [expr {$expand ? "0" : "1"}]
           }
         } else {
           set do_expand [expr {$expand ? "0" : "1"}]
         }
 
         if {$do_expand != "0"} then {
           # add the expanded list to the command string.
           set result [concat $result [lindex $args $this_index]]
         } else {
           # add the unmodified argument to the command string.
           lappend result [lindex $args $this_index]
         }
 
         incr this_index
       }
     } else {
       # not enough arguments.
       error $expand_argument_error
     }
   } else {
     #
     # NOTE: potential for design change here.
     #
     # return empty string when given no arguments...?
     set result ""
   }
 
   uplevel "1" $result
 }
 
 proc expand_test_proc { args } {
   puts stdout "number of args = [llength $args]"
   puts stdout "args = \{ $args \}"
 }
 
 ## end of expand.tcl

Try these examples:

 expand -expand {1 to 3 end} expand_test_proc [list argument 1] [list argument 2] [list argument 3] [list argument 4] [list argument 5]
 expand -expand {1} expand_test_proc [list this is a test.]
 expand -noexpand {1} expand_test_proc [list this is a test.]
 expand expand_test_proc [list this is a test.]

----
[RS] gets a bit tired of how arguments expand on this page ;)
----
[MAK] has successfully implemented [[expand <list>]] (though, as lexpand) in a pretty simple manner using a new Tcl_Obj type that's identical to the list object and doing the expansion in TclEvalObjvInternal(), for those that are interested but didn't think it could be done.  Some test vectors would be welcome.  You can find the patch for 8.4.4 at sourceforge [http://sourceforge.net/tracker/index.php?func=detail&aid=781929&group_id=10894&atid=310894].  (I should note that this is NOT an implementation of TIP #103, but a commandified/non-syntactic version of TIP #144.)

[Joe Mistachkin] I did this same thing months ago as [lconvert]. The problem that I ran into was that it didn't interact well with the bytecode compiler. Perhaps others could shed some light on this subject?
----
[MAK] Admittedly, I hadn't seen lconvert until it was mentioned in the poll (after I'd done lexpand).  The main difference from what I gather between the two is that lconvert tries to some degree to treat the to-be-expanded list as a first class object and you have to thus use it with variables and pass variable references, while lexpand is set up to only work in-line (thus allowing it to use the result of other calls directly), though I could be wrong.  The benefit of lexpand over lconvert is thus that you can still do one-liners and the implementation is a fair bit simpler.

Unfortunately, as [Joe English] pointed out over on sourceforge, it does have some issues with bytecode compiled commands.  I believe it to be fairly simple to address, though perhaps not as neatly as lexpand currently is.  Let me use Joe's example to explain:

 # Doesn't work (with the first version)
 proc test {} {
     set l [list x y z] 
     set x [list a b c [lexpand $l] d e f]
     return $x
 }

I believe the problem here is simply that the list command (as well as some others), when compiled, don't call TclEvalObjvInternal(), so expansion is never attempted.  If it were some other defined procedure (for example) then it would make that call and would be expanded.  I think it just requires adding some additional places (in and around TclExecuteByteCode) where expansion will occur, but not any more significant than will have to take place with the {} prefix proposal (unless the intent is for the presence of {} to the parser not to compile a part of the script at all, I suppose).  I plan to look into it soon.  I think for the most part, all that needs to happen is for the top of the stack to be expanded before use.  Take, oh, INST_EXPR_STK for example:

    case INST_EXPR_STK:
        objPtr = stackPtr[stackTop];
        Tcl_ResetResult(interp);
        DECACHE_STACK_INFO();
        result = Tcl_ExprObj(interp, objPtr, &valuePtr);
        ...

Add some macro or function so that it becomes:

    case INST_EXPR_STK:
        objPtr = Tcl_ExpandObj(stackPtr[stackTop]);
        Tcl_ResetResult(interp);
        DECACHE_STACK_INFO();
        result = Tcl_ExprObj(interp, objPtr, &valuePtr);
        ...

Unless objPtr requires expansion, it would just be the same as stackPtr[[stackTop]]]; if it does, it'd be expanded before passing to Tcl_ExprObj().  INST_LIST would take on something similar to what's going on in TclEvalObjvInternal() perhaps, etc.  Or, I suppose the list object might just need to be modified to handle the expansions its self.  Bottom line: I don't think it's a reason to reject the lexpand idea, it just needs a little more work and there are lots of ways to go about it.