Newlines in Command Evaluation

Perhaps there is a very obvious answer for this, but what is the reason why newlines need to be escaped when evaluating a command in square brackets? You could easily make it a rule that if the command is being evaluated in square brackets, that you treat newlines as whitespace.

Example:

set myList [list 
    $x
    $y
    $z
]

The Tcl interpreter could just have a flag for the depth of recursion. If it is at the top level, it will stop at a newline. Otherwise, it will search for the closing bracket.

Currently, rule 3 of the Dodekalogue states that "Words of a command are separated by white space (except for newlines, which are command separators)."

I propose to modify it to state: "Words of a command are separated by white space (except for newlines at the top level of evaluation, which are command separators)."

Additionally, rule 7, Command Substitution, could clarify that "The result of the script (i.e. the result of its last command) is substituted into the word in place of the brackets and all of the characters between them, including newlines."

I believe these changes would be back-wards compatible, and would eliminate, in most cases, the need to escape newlines.

<<FM>> It can't be backward compatible. Sometimes there is a need to use a multiline subscript, and people use it. An alternative would be to use a braced-prefix like {\\n}, to indicate to the Tcl parser how to handle newlines in the prefixed subscript.

set myList {\\n}[list 
    $x
    $y
    $z
]

<</FM>>

Rationale

I thought of this in the discussion on expr shorthand for Tcl9, but that page is getting long, so I thought I'd put it here.

The main barrier to having an easier notation for math in Tcl is that the expr command performs double-substitution, requiring that expressions are braced. So, even if you alias the expr command to the command "=", you still have to call [= {...}]. Math is enclosed within both brackets and braces.

FM proposed a special syntax [(...)] that is recognized by the Tcl interpreter as math, and does not do any substitution on the contents before passing it on to the expr engine. This eliminates the double-substitution problem. A good start.

However, because the expression contained within [(...)] is not braced, it would, with the current state of rules 3 and 7, require that one escape newlines for multi-line expressions.
<<FM>> remarks : This statement is false.

puts [(
       # comment
       1
       +
       # comment
       5
)]

is already working and is returning 6. <</FM>>

<<AMB>> Response to FM:

Cool! Glad to see it is working :)

Point of clarification: I was more saying that because there are no curly braces, your syntax (or the one I proposed here) would technically require modification to rules 3 and 7. The [(..)] notation uses square brackets, so it would require a modification to those rules, at the very least marking the exception. But I think that newlines should be allowed within square brackets, which would remove the need for the clarification.

<<FM>> : It may depend on the context (interactive context may be special), but newlines are already allowed between square brackets. Command Substitution can accept a whole script, whose subcommands may be separated by « ; » or newlines, waiting to find the corresponding closed bracket. Yes : The dodekalogue would need a new rule, becoming a tridekalogue :

mathematical expression substitution : If a word contains an open bracket immediately followed by a parenthesis « [( » then Tcl performs mathematical expression substitution. To do this it invokes the expr command parser to process the characters following the open bracket-parenthesis as a mathematical expression. This mathematical expression may contain any number of nested commands enclosed by brackets, variables, or nested mathematical expression, in following the expr mathematical expression syntax, and must be terminated by a closed parenthesis, immediately followed by a close bracket « )] ». The result of the mathematical expression is substituted into the word in place of the brackets-parenthesis block and all of the characters between them. Mathematical expression substitution is not performed on words enclosed in braces.

Rule 7 : must be modified to reference this new rule :

script substitution: If a word contains an open bracket « [ », but not followed by an open parenthesis « ( » (if so, refer to the mathematical expression substitution rule), then Tcl performs command substitution ...

Rule 3 : No changes are needed : « [(...)] » is defined as a Word in the mathematical expression substitution rule <</FM>>

<</AMB>>

This would make it impossible to have comments in an expression using this syntax. Comments in expressions, as defined in the expr documentation , start with a #, and go until the end of the line or the end of the expression, whichever comes first. A version of expr that does not do double-substitution, would not allow for comments unless the requirement to escape newlines in command evaluation was eliminated.

But if the restriction on newlines within square brackets was lifted, then [(...)] would truly be shorthand for [expr {...}].

I think we can go even further though. I propose a special syntax for math in Tcl, [=...]. It would add a rule to Tcl, creating one restriction: you cannot name a procedure starting with "=". When the Tcl interpreter is reading the name of a command, if it starts with =, it will automatically treat the rest of it as math, up until the close bracket. Additionally, I would like it to return a list, where the elements are separated by commas in the expression.

It would as shorthand as you can get for math in Tcl, I think. See below:

# Current :
.canvas create rectangle [expr {$w-100}] [expr {$h-40}] [expr {$w+100}] [expr {$h+40}] 
set y [expr {$x*2}]
set A([expr {$x*2}]) 1

# Proposed :
.canvas create rectangle {*}[=$w-100, $h-40, $w+100, $h+40]
set y [=$x*2]
set A([=$x*2]) 1

Multi-line math, with comments

proc CrossProduct {U V} {
    lassign $U x y z
    lassign $V u v w
    return [=
        # first element
        $y*$w - $v*$z,
        # second element
        $z*$u - $w*$x,
        # third element
        $x*$v - $u*$y
    ]
}

I think that it could be a nice addition to Tcl 9.1, stylistically represented as shown below XD

set version [=$Tcl*9.1]

AMB:

Upon further reflection, and because FM already has a working prototype, I am in favor of the [(...)] shorthand for math, and I see how it can be an exception to the normal [...] notation, requiring a new rule to Tcl and a modification to rule 7, as FM stated.

set version [($Tcl*9.1)]