Version 119 of expr shorthand for Tcl9

Updated 2009-03-20 14:46:05 by nem

This page collects various suggestions from the Tcl 9.0 WishList about shorthands that one could use instead of the expr command. The original numbered suggestions were

34. GPS: I would like the math commands + - * / in the standard Tcl 9.0.

and

41. Larry Smith - some special syntax to invoke expr

the discussions of which follow below (slightly reordered). Two subsequent suggestions are

Lars H - extended $ substitution as expr shorthand

and

Martin Lemburg - special quotation to allow the bytecompiler to recognize expression

NEM 2009-03-20: I've created a patch for Jacl that implements (...) as shorthand for expr {...} at [1 ]. It also includes some new commands as alternatives to if that use the syntax rather than building-in expr, so you can write things like:

proc fac n {
    when ($n <= 1) { return 1 }
    otherwise { return ($n * [fac ($n-1)]) }
}

It's all very experimental, and doesn't pass the Tcl test-suite: doesn't conflict with arrays (except the empty named array), but does conflict with some uses like [regexp (a|b|c...) ....]. Meant as a feasibility test and fun experiment rather than for serious use.


Talk

GPS: I would like the math commands + - * / in the standard Tcl 9.0. Expr is awkward, and having to use braces to workaround the byte-code compiler sucks. The issue with this as I see it is whether to make them operate on longs/ints or doubles. How can we solve this?

Lars H: You can of course have them right here and now. Just define

 proc - {term1 term2} {expr {$term1 - $term2}}
 proc + {args} {
    set res [lindex $args 0]
    foreach arg [lrange $args 1 end] {set res [expr {$res + $arg}]}
    set res
 }
 proc * {args} {
    set res [lindex $args 0]
    foreach arg [lrange $args 1 end] {set res [expr {$res * $arg}]}
    set res
 }
 proc / {numer denom} {expr {$numer / $denom}}

and you'll be able to write things like

 + 1 2 [- 3 4] [* 5 6] [/ 7 8]

for 1+2+(3-4)+(5*6)+(7/8).

The idea that one should decide whether mathematical operations operates on "longs/ints" or "doubles" shows clear signs of too much C programming. In Tcl, everything is a string.

LV In addition, with Tcl 8.5 one has the tcl::mathop namespace, from which one can import the operators as commands as well. So this wishlist item seems, at first glance, to be fulfilled now, rather than in Tcl 9.0.

ZB 23.09.2008. Personally I couldn't understand, if someone finds the solutions proposed above as "not enough". The first one looks just obvious, and usage of namespace will be especially convenient in case of math-oriented procedures. Besides: although "expr" syntax isn't especially pretty, but it's not that ugly anyway; one can live with that. Not sure, is really that "search for new delimiters" (and so on) worthy any further efforts.

peterc 2998-09-24: I wouldn't think there'd be many (any?) scripts which name their procs solely as a number. Surely a nice simple way might be to test the proc name to see if it's numeric, then if so, send it to an expr-like math proc. Then you could have more eye-friendly commands like:

 set a [12 + 5]

Martin Lemburg 17.02.2003:

In Tcl, everything is a string.? No, definitely not!

For calculations it is definitely important to handle number values only with commands that don't change the internal tcl type to string, so the exact representation is not lost!

If you don't care, if you don't have to care for the internal representation, than you can act like [i]n Tcl, everything is a string.!

We have many matrices to be multiplied and so we have to care about the internal type and not decrease the calculation's precision!

RFox

The concept of an <em>exact representation</em> is nonesense. The best you get on any computing device is an approximate fixed precision representation. In Tcl, you can set the precision to be what you want it to be, but it's still the precision of a stringized - fixed point representation that is just as exact or inexact as the binary fixed precision representation.

Lars H: That everything is a string is one of the basic principles of Tcl. It is quite true that the internal representation of real numbers currently contains more information than the string representation -- this is the subject of the A real problem page -- but IMO this is a bug in Tcl. (It can alternatively be seen as a malfeature of the tcl_precision variable.) The correct way (used by e.g. Knuth for TeX) to handle this problem is rather to make sure that the string representation of a number contains enough digits to uniquely specify all bits in the internal representation. This will mean that you generally "cannot trust the last digit" and will see numbers written with more digits than "what is really there", but since this is always "inside the program" that should not be a problem; anything shown to a user should be explicitly formatted.

Finally, if you are interested in maximal precision then you should probably read what Robert Heller writes on the subject of guard bits on the A real problem page.


Larry Smith - some special syntax to invoke expr - () would be best. This would make code much easier to read and more compact.It would be fairly easy to turn the feature on and off for backward compatibility.

This is not in the Tcl Way. If I (FW) had it my way, the correct thing to do for such a radical addition in that area would be to add math procedures to eliminate the need to use expr, for those who prefer a more purely Polish syntax.

LV In other words, FW, you would prefer to say something like

        set a [add 2 3 [divide [multiply 4 5 ] 7]]

YUCK!

FW: Well, I'm not sure what the command names would be, perhaps they would just be named after the math operators, as in GPS's suggestion in wish #34.

LV I still respond YUCK - I find reverse polish notation difficult to read and more difficult to code. I prefer the notation currently used by expr.

escargo 4 Jan 2006 - Just for the record, I must point out that this is Parsing Polish notation not rpn.


Lars H: Another possibility is to extend $ substitution. I have seen references that it was originally introduced as a shorthand for [set ...], and in view of this, it would be natural to consider making a form of it which is a shorthand for [expr {...}]. How about

 set res $={2*$a + 3*rand()}

as a shorthand for

 set res [expr {2*$a + 3*rand()}]

i.e., when $ is followed by = (a case not explicitly mentioned in the Tcl syntax rules, but in which currently no substitution will take place) and something in braces then these braces contain an expr-expression rather than the name of a variable. What one should perhaps think hard about here is the possibility to get rid of the need to put $'s before variable names in these $=-expressions. Simplicity and compatibility with expr suggest that they should be required, but forgetting the $ in expressions is probably the most common typo I make. This could be a sign that they are unnatural.


The tcl parser is a simple one - trying to replace expr ... with $= seems to me to be asking to break the parser for minimal benefit.

As for a new command to which one need only supply variable names, and not $, that too violates the tcl parser's rules - to get a reference to a variable, use the name only, but to use the value of a variable, precede the name with a $. It doesn't seem to me that the resulting command is going to be all that much beneficial, and on the contrary, it seems like it is going to make tcl slower, more complicated, and more difficult to teach and to use.


In bash one can write things like

   $(($i+1))

for what would be

   [expr {$i+1}]

in Tcl. The former currently refers to an element in the empty name array in Tcl, but if wish #39 (the empty string is no longer the name of a variable) was to be implemented, then the bash syntax (with double or single parentheses) could be used as an expr shorthand in Tcl. (In fact one could take the implementation of this syntax in Tcl 8 as an exercise in trace hacking. Set up a read trace on the empty name variable, and pass every index through expr to compute the array element value. Should work, but there would be a ridiculous amount of shimmering.) However, I think $={...} would work better syntactically than $((...)) or $(...). The best case for the latter two is the analogy with bash.

Larry Smith $(...) is not bad at all. "$(" is not currently defined so it is backward-compatible, and the () notation is natural for arithmetic expressions. It also plays well with the idea of $[] to expand the results of a command into a list. It even suggests that ${} is a cleaner and more logical way to indicate the "expand w/o eval" that we currently use "" for. This has always felt wrong because "" is otherwise meaningless in Tcl, and is frequently ignored or interpreted oddly (as in "10" == 10 in expr).

ulis Sorry, "$(" *IS* currently defined (empty named array):

  set (1+2) 4
  puts $(1+2)
  ->4

Setok -- Somehow that looks perlish. I prefer using the command: +, -, *, / etc. Result in lines like set a [+ $b $c $d]. Indeed, I use this form all the time.

Lars H: Whether it looks perlish is completely irrelevant (even though it is a classical killer argument in Tcl discussions; $ substitution got that critique too, in the beginning). What matters is whether is would be perlish, and I don't think it is.

As for having the math operators as explicit procedures, I think it would be a good idea to add a package for this to tcllib ASAP.


Sarnold In my little experience of Perl, I was surprised I could quite not force the interpreter to handle my numbers as integers. (they were automatically casted to float)

IMHO it is quite better to have both the casting operators, and the math operators as commands. I hope those changes won't make things harder to understand. IMHO it is expected from a developer to have skills with floating-point numbers and integers. Many people don't think so, but it is awfully too often required to know the limits of our machine-related number representations.


LV anyone who wants to write such a package, and submit it to tcllib, could probably get it in pretty quickly. And releases occur frequently enough that it would end up in the library distribution in a small number of months.


There is no compatibility problem, and if people like them then they could get byte-coded by Tcl 8.5. And it would still be possible to provide another shorthand like $= in Tcl 9, if some such suggestion is approved of.

Setok Well my point was not that being like perl is always necessary bad, but I find $={$a + $b} a bit strange to read, although now that I'm not so tired as when first reading this page it's not as impossible as I reckoned. As you mentioned opers-as-procs is a good compromise and I never liked expr having its own functions anyway (a [rand] procedure would be much more natural than rand()).


As long as expr continues to work, other syntactical sugar notations are easy enough to add on. I would not like to see such things become a part of the tcl.tar.gz distribution. However, if they were part of tcllib, that would be fine.

In ksh one could write things like

   $(($i+1))

or

   let $i = $i + 1

for what would be

   [expr {$i+1}]

or

   incr $i

in Tcl.

Lars H: I hope anonymous at 134.243.40.138 will refrain from making non-editorial changes to signed material in the future.


Martin Lemburg 17.02.2003:

if there is a possibility to mark expression with a special quotation to allow the bytecompiler to recognize expression, than it would easier a lot the writing of calculations.

What's about:

 set pi '4 * atan(1)';

And ... even if it's not tcl-ish to do, but if we break tcl style by using "()" for functions, why not applying the nice patch for expr to allow every proc and command to be invoked in the same way and to access variables without the "$"-sign?

Why not ...

 proc pi {} [expr {4*atan(1)}];
 proc perimeter {r} {expr {2 * r * pi()}};

or

 proc pi {} '4*atan(1)';
 proc perimeter {r} {'2 * r * pi()'};

Lars H: The "special quotation" is probably too radical to meet general approval -- cf. the hassle with {expand} (Tcl9 wish #3) -- but you're on to something concerning access to variables in expressions without using $. That would be very nice!

AJD Don't personally see any improvement in the above forms over the simple use of "$pi" or "[pi]". However, the use of argumentless functions within expr is something I've hacked around with... in particular, the case above of [expr { pi() }] to denote a constant. The core doesn't currently have any such beasts but they can be defined using the API function Tcl_CreateMathFunc. The idea being the core could define PI and other 'well known constants', saving users having to do it (possibly in a variety of places). There's even the opportunity for a teensy weensy performance gains (e.g., inline "2*pi()"). Maybe the ability to define "expr constants" could be exported to the script level... Overall though, probably doesn't buy enough beer to get taken to the bar.

Maurice.Diamantini (03 mars 2003): I'd like to see the functionality from vexpr or the vkit (is it dead ?) to be integrate into the expr command.

so that one could write

 set a [list 1.0 4.0 3.0]
 set b [list 5.5 6.6 9.8]
 set c [expr {$a*$b}]

Another amelioration I'd like in tcl9 is to extend incr, and add a few other command like mult to work with other type than int (double or list ) (il c ist the liste above:

 incr c 1    ;# add 1 to every element of the list c
 incr c 3.1  ;# add a double
 incr c $b   ;# add another list

RJM Picking up the suggestion above to use simple () parentheses: Why not doing it without preceding $? Just as "[" marks the start of a recursive script, "(" could mark the start of a math expression.

(simple example)

 set y ($x*$a + 3.5)

Note that the shortcut should only apply to expressions that can be evaluated in an expr {...} command notation. In the case the expression cannot be evaluated with {} quoting, the conventional expr command should still be applied: This is the case where STRINGS must be evaluated as a math expression. In other interpreter languages it is also usual to provide a command for such cases.

Lars H: It would certainly be a language that made sense. The big problem with making ( behave like [ (but for expressions rather than scripts) is however that it makes another character special, and thus breaks every old script in which that character wasn't protected. In particular the syntax for array elements would get into serious trouble. I suspect the change would be too great for any Tcl version increment. Math operations as explicit procedures are probably the way to go.

Larry Smith: Actually, the more I look at arrays the more I think making them part of the language was a bad idea. An array should be accessed the same way other data structures are, with normal tcl syntax:

 array set foo $bar "a string"

and

 set x [ array set foo $bar ]

or

 set x [ array get foo $bar ]

It really has no call for special syntax. It would also allow people to overload the array procs to add capabilities we don't have, but frequently want:

 array set foo $x $y $element

would permit "real" numerically-addressed arrays to use the same keywords (this example keys off of the number of args, but it could also look at the values of the passed args to decide if this is a numerically-addressed array or a hash table.

RJM: I revised my contribution above. "[" may also appear without space as to execute a script. However it does not mean that "(" as an expression shortcut should also evaluate an expression when it appears without a leading space. I suppose expression results mostly happen to appear as command arguments. When it happens to appear inside a string, the shortcut shall not apply. In such cases, the expr command should still be used. When a variable with trailing space exists, there will be a problem. But I hardly believe that anyone would define variables with trailing spaces. Hence, backward compatibility shouldn't be a problem. Moreover: To my opinion, backward compatibility is not a holy cow. It is consequent backward compatibility that made MS-windows a monster. Consider also that scripts can easily be adapted to conform a newer Tcl version, using a script.

Another idea (yep, this is a brainstorm page): When "(" is not acceptable, a new command "=" should. For evaluation purposes, enter the following proc:

 proc = {args} {expr $args}

According to my previously stated condition, this command should only work with true expressions (i.e. be equivalent to expr {text}). This way, any [= 5*1.2 + $a] inside a script text would execute the fast way round. Oops! (note few weeks later) This intention violates with Tcl rules - Tcl would do substitutions before passing the argument to the "="-command. So the fast way round is never possible with a new command. Conclusion: only a new substitution type will do (see above: $={...}, '...', $(...) and (...) ). Given the complaints with "(" I vote for $(...). Uhh, this is empty array access (see above)

Recalling backward compatibility issues, I'd recommend a little bit more courage in changing things in Tcl. expr is historically given, because everything was a string in Tcl. When no shortcut will be introduced, I believe Tcl will early or late fade out in computing history, simply because the choice in scripting languages is really large. Anybody who is selecting a scripting language will tend to not select Tcl when he/she encounters "set a [expr {....}]". This is Tcl Marketing-relevant!


RS: You can save a level in call stack by using this instead:

 interp alias {} = {} expr

(RJM: still for evaluation purpose: executes slower and does not include {} in expr).


DKF: The problem with special syntax for expressions (or at least special syntax that isn't standard command invokation or the = alias) is that this is a major departure from the existing rules; you'd be making some kinds of syntax special, and one of Tcl's key strengths is that there are no special global syntaxes. The expr syntax is always just a local syntax.

A few notes here...

  • Unbraced expr was special in the alphas of Tcl 8.0, but this was deeply unpopular with many very influential Tclers.
  • It is possible to get commands that are as efficient as the expr bytecode, or rather it would be if it wasn't for the fact that we're not very proud of the actual compiler API and so haven't exposed it publically. :^)

slebetman 15 Mar 2006: By unbraced exprs do you mean [expr $foo + $bar]? Why was special treatment of unbraced expr unpopular? Was the C code ugly? To me braced exprs are unpopular to the extent that I'm often willing to sacrifice speed rather than break my old habbit. If special treatment of unbraced exprs were already coded can't we just put it back in so that we don't need the braces to bytecompile exprs. That will also make things like let faster since currently the expr that it depends on is not bytecompiled.


RJM It can be seen, that no practicable shortcut for expr {...} seems to exist. Therefore, another approach should be discussed. Why not let appropriate commands expect an expression in one or more of its arguments, like the if command does. For example coords arguments for canvas operations. They must always be numeric, so rewriting commands involving coords to expect expressions would allow us to embed calculations within such commands in a more compact fashion, which improves readability. I also think this would not affect backwards compatibility problems, since numbers are already "expressions". Further, existing code, where [expr {....}] is already embedded, also yields no problems. I would see this idea closely related to what is written in What some command arguments expect. If the idea is interesting, an inventarisation of commands with this respect should be undertaken. Further, some new commands could be defined, such as xset/nset for expression/numeric set etc.

KPV See tip #176 [2 ]. Personally, I like what that tip proposes--it would clean up most of the cases where adding expr gets really awkward. But I've not heard much momentum behind it.


Lars H: Another TIP which should be mentioned on this page is #174 [3 ]. It is pretty much what is listed as Wish #34 above.

HGO: And yet another TIP which should be mentioned on this page is #232 [4 ]. It may help in many of the above stated cases.


AMG: What about $[...] being shorthand for [expr {...}]?

I saw mention of $[...] above, but if I read correctly it was in regards to a feature already provided by {expand}[...], so this character sequence been freed up. Does anyone currently use this sequence? At the moment it means: eval the script "..." and prepend the result with $.

We'd have to change the parser to quote between $[ and ], and it would have to handle nesting correctly. Important inconsistency: we do not do this for $(...)!

07nov05 jcw - This is not necessarily an inconsistency. [...] gets parsed as usual, the $ in front of it takes the inside (say $x) and replaces it with [list expr [list $x]]] - IOW, "$" before a [ gets interpreted as "expr" after the [ (more or less, the arg also needs to be treated as surrounded by {}'s).

AMG, 9 Dec 2005: That's not quite right... expr [list $x] and expr {$x} aren't equivalent. Surely you mean the latter.

The inconsistency I refer to is $[...] having implicit {}'s but not $(...), and you haven't addressed this. Maybe I just don't understand what you're trying to say. Care to elaborate?

Question: Should $[...] be recognized by the [expr] parser as well as the Tcl parser? I imagine $[...] inside an [expr] expression would have precisely the same effect as (...).

Larry Smith How about this idea, then:

 interp shorthand <srcproc> startchar endchar

This would allow creation of application-specific shorthand expressions. For ordinary tcl, one might declare:

 interp shorthand expr ( )

Thereafter the interpreter scanner would spot (...) and turn it into [ expr ... ].

This also allows people with full unicode editing to use things like 〚...〛, ⌊...⌋ or even <...> as shorthand for various options, which could include things like constant constructors, object calls, regexp sugar, whatever.

Lars H: The problem with this is that is changes Tcl syntax in rather unpredictable ways. list-quoting is one issue -- if list is to be usable as a generic command constructor (which is very established in Tcl) then defining "shorthands" like this requires changing the canonical representations of lists to ensure all shorthands to be quoted! That quickly becomes very messy.

Larry Smith Actually the above was rather tongue-in-cheek (and I was amazed by the variety after a very cursory look through a unicode font), but truthfully, this doesn't change tcl's syntax except to provide semantic meanings for new symbols in a context that right now only permits {...} and [...]. The symbols listed would mean nothing if they were inside pairs of the existing delimiters.

Lars H: (Sigh.) Of course they wouldn't be subjected to substitution within {} (however, they would have to be subjected to substitution inside brackets, as that is just another command -- think your syntax suggestions through before you post them!), but that's no help to you. Tcl does not have any context that "only permits {...} and [...]". "Barewords", as they are sometimes called, are legal everywhere and the string representations for lists uses them not only for very simple elements without whitespace, but in fact relies on them for the trickiest cases.

Larry Smith I do think my syntax suggestions through before posting them. Just because you don't like them is no excuse for such a remark.

As far as tcl syntax and parsing go, the scanner is the one place in the interpreter where "", [] or {} have meaning. Providing new delimiters simply converts an extremely unlikely string containing such delimiters into something tcl recognizes and executes.

 set x 〚foo〛

...under the current system sets x to 〚foo〛, true enough - but that is an extremely unlikely sort of string. Giving it a semantic - that is, extending the scanner to recognize ĺ - only breaks a very unlikely case. Frankly, if you are coding that way, you deserve to have your code broken.

Lars H: Well, a rereading with your reply in mind indicates that what I took for a lack of thought may simply have been an unclear re-presentation of the exact same ideas as that you started out with, but as it stands it really sounds as though you introduce the extra principle that "expr substitution" should not be triggered between brackets, which would render it of limited utility. Also note that brackets and braces don't have anything in common syntactically; there is no "one place" in which both are handled and (unlike brackets) braces are only special at the beginning of a word.

Regarding your argumentation:

  1. I personally don't write things like "set x 〚foo〛" right now,
  2. but people who do don't deserve to have their code broken.
  3. Indeed, the argument that this "is an extremely unlikely sort of string" is the syntax analogue of the classical argument that the government should be allowed to violate human rights: "We probably won't do it to anyone you know (and in the unlikely event that anything bad ever happens to anyone because of this then -- as is evident from us even invoking this extraordinary law -- these people obviously must have been evildoers anyway)."
  4. None of this has even begun to address the problem of list-quoting that I raised. If string representations are not changed to quote new shorthands, then you open up for attackers getting malicious code evaluated. If string representations are changed, then you have to make them interpreter-relative, but Tcl_Objs don't belong to a particular interpreter!

Sure, if you implement a Tcl/2 then you can make its syntax mutable to your heart's content, but doing so is far from free from side-effects, and you do not seem to appreciate that.


Lars H: My present (2006) opinion is that shorthands in general are unTclish. What should be done instead is to create more capable little languages, in which syntax rules could be more algebraic. Rather than

  set r <hypot($x,$y)+1>
  set x1 <$x/$r>
  set y1 <$y/$r>

where there is constant switching between Tcl and expr, one should do also the variable assignments in some expr-like language, e.g.

  calculate {
     r = hypot(x,y)+1;
     x1 = x/r;
     y1 = y/r;
  }

Larry Smith I like the idea, but the real problem here is the messy syntax that arises from the need to calculate expressions being provided to other commands. List extraction, for example, where the first index is given (j) and the end is computed: (j+5). The innocuous-looking (j+5) balloons into [ expr { $j + 5 } ] (not forgetting the need to curly brace arguments to expr).

Lars H: TIP#176 has already adressed that problem in the case of indices. For other small calculations TIP#174 provides a nice solution. That only leaves the high end of expr usage to deal with!

Larry Smith #176 works but it does put the onus on the command to provide such functionality. #174 is not, IMHO, a nice solution. It is at best a slightly less ugly one.

In all honesty this is a very restricted domain, and becomes even more so if commands can accept simple expressions like $j+1 or end-1, but it's a wart, and it adds complexity to commands to support this. General access to simple computations is really needed, but I'm begining to think expr is over-specified. Expr also supports floating point, trancendental functions and lots of other stuff not really needed to solve the problem we really are trying to deal with here.

<sarcasm> Now that we've broken down and added {expand} maybe we want to extend the idea to include {compute} - that is, expand and expr in context - where {compute}j+1 is read as [ expr { $j+1 } ]. </sarcasm>

In any case, expr is really starting to rub me the wrong way. As a means of computing indices or other such trivia, it is way over-specified. As a means of doing advanced math, it's way under-specified. As a means of doing other types of computation problems, such as matrices, vectors, or what-have-you, it fails utterly, yet it remains the repository of the logic needed to build upon and so insinuates itself into every solution, and again our code grows cluttered with [expr {...}].

I'm thinking more and more fondly of Tcl/2. And even more after this last exchange.

AM (17 march 2006) Actually a procedure like [calculate] has already been written - A little math language revisited

Gosh, it would be nice if the mechanism of expr was expanded to vectors and matrices and what not ... I have contemplated and started several experiments in that direction. All the ingredients are there. Now it is a matter of concocting the right receipe and bake the penultimate script. Main problem: time.

Sarnold (7 sept 2006) I am currently experimenting a syntax

 {math}{...} 

to be handled by a modified Jim interpreter. What I expect is RPN generated at parse-time from expr-like syntax. This means operators and math functions should be available as commands at Tcl-level.

Example:

 {math}{$i + 2 > 0}  -> [mathfunc.gt [+ $i 2]]      (gt = greater than, second arg defaults to 0)
 {math}{$j && $i>0}  -> [and {$j} {mathfunc.gt $i}] 

(and is a command, but to allow lazy evaluation, it needs braces around its args)

But I face a huge problem : I still need the expr parser because, if I wanted to bash it, I had to write {math} before each expression, including if, while, for, etc. Though, there is an advantage to this approach: you CAN redefine math operators to handle vectors, matrices, bignums, everything you need... (because +, -, *, / are commands in Jim) But don't forget it is *experimental* and does not concern Tcl but a little bit.

CMcC Mon 11th Sept 2006 sees no need for incorporating infix notation in tcl's global syntax - it's a major global change for not much local virtue and significant global cost (IMHO), I refer you to little language for a growing collection of the places in tcl where specialised parsing is contained within a linguistic {}-scope.

However, I would like to see expr syntax expanded to permit #-comments, because it would be nice to be able to comment complex if conditionals.

NEM: I actually quite like the idea of () as syntax for expr, but it would have to be done in the same way as {} and "" -- i.e., ( is only special at the start of a word. This would rule out any clash with array notation, and also with parens in ""-quoted strings (which are rife). The resulting syntax seems quite elegant for the common cases:

 lindex $xs ($ptr/2+1)
 set res ($expr + [mycommand ($subexpr)])

It doesn't really work for while or for, where the expression needs to be evaluated repeatedly (I'm assuming eager evaluation of expressions, i.e., (...) is equiv to [expr {...}]), but I can live with those as they are already concise except for the nested sub-expression case, which this proposal also nicely handles:

 for {set i 0} {$i < [lindex $foo ($x+1)]} {incr i} { ... }

You could clean up if by making it accept a plain boolean value rather than an expression. That would result in syntax like the following:

 if ($a < 12) { ... }
 if [mycmd ...] { ... }

Which both seem nicer to my eye than the current braced varieties.

Justification for adding a new global syntax? Well, firstly, I think expr is probably important enough to warrant it. Use of mathematical expressions is extremely common. If I was looking for parts of the language that justified syntax, expr would probably come after $-syntax for variables, but before either array syntax or {expand}. I also think it makes the language more newbie-friendly, as explaining expr and about always bracing expressions is perhaps less than inspiring. Finally, most of the alternatives (if you believe there is a problem to be addressed at all) seem to involve adding features to the expr little language, until it is no longer a little language, but actually quite a large one. With comments, assignment etc, the emphasis seems to be on including more and more of your source code inside expr -- the logical conclusion would be that eventually all source code is in expr syntax and Tcl becomes the little embedded language! I'd prefer to keep expr minimal and just give it some sugar.


RS: Note however that the (...) syntax isn't free for the taker any more - it rather refers to the array named "":

 % set (x+1) 5
 5
 % puts $(x+1)
 5
 % parray ""
 (x+1) = 5

At least Stooop (for which this geature was introduced) would break if (...) received a different meaning.

NEM: Note that $(...) could continue to work. Only [set (...) ...] would break. You could use [array set] to work around that (perhaps aliases to [self]?), but it would definitely require a change to Stooop. I think it would be worth it -- empty array names are cute, but little used.


CMcC (Wed 13th Sept 2006) Discussion on tkchat suggests there's a problem with some aspects of expr being made into global syntax: specifically, &&, || and ?: operators explicitly don't evaluate their second argument in some cases. This is a major departure from the tcl evaluation/substitution model.

RS: On the other hand, you sometimes need such short-circuiting - consider these not atypical cases:

 if {$d != 0 && ($n/$d) > 1} ...
 if {$d == 0 || ($n/$d) > 1} ...
 expr {$d==0? 0: $n/$d}

NEM: I don't understand. As I said, a global syntax for expr would be precisely equivalent to [expr { ... }]. Short-circuit operators would continue to work exactly as they do now in braced expressions. This is an argument against both maths operations as commands, and against changing Tcl_GetIntFromObj et al to recognise some cut-down expr syntax.


Robert Joy 2006-09-13 6:32am PST: While I respect that one might want to drive expr into a more common paradigm. I personally don't see a need to move in that direction. In some ways I prefer the use of expr as it it becomes abundantly clear that one is defining a mathmatical expression and not just creating a string that looks like an expression. For the sacrifice of a few extra characters to change to a new paradigm it's just not worth all the effort. Just my 2 cents for what it's worth and I certainly don't mean to upset anyone but I think those who originally created expr had the right idea.


WHD: For some time now, I've been using a helper command called "let". The two following statements are equivalent:

 set a [expr {$a + $b*$b + $c*$c}]

 let a {$a + $b*$b + $c*$c}

This doesn't help when you're using an expression in-line, as an argument to another command....but it makes complex computations much easier to read. - RS: Something like this?

 proc let {_var expr} {
   upvar 1 $_var var
   set var [uplevel 1 [list expr $expr]]
 }

CGM 2008-06-11: Another suggestion - use { .... }= as shorthand for expr { .... } . This is both compact and backward-compatible to the same degree as {*}, since it's currently an error "extra characters after close-brace". Using {} braces preserves the quoting semantics to avoid double-substitution of variables. There's an ambiguity over {*}= but the expand meaning should precedence here as expr {*} is an error anyway.


JMN speaking of {*} .. it seems to me that it would now be natural to use another such prefix for expr.. e.g {#}{1 + 2}

Looking above I see Sarnold has already suggested something like this with {math}{...}

JMN 2008-06-14

While at first glance this mightn't seem much better than expr {...}, I think that in the vast majority of cases the braces surrounding the actual expression could be left off simply by formatting it without whitespace.

 .eg
 {#}1+2 

If a tcl command with whitespace is included in the expression, you still wouldn't need braces.

 {#}[llength $somelist]/2

tcltkd : 2008-06-12: A classic technique:

 proc unknown {args} {uplevel 1 eval expr $args}

Is it dangerous? If is,some global interpreter flag should be employed.

NEM There's too many evals in there, so it is dangerous. A more correct version would be:

 proc unknown args { uplevel 1 [list expr $args] }

But note that it has problems with some expressions, such as && and ||:

 % [set a 1] || [set a 2]
 1
 % set a
 2

Versus using expr directly:

 % expr {[set a 1] || [set a 2]}
 1
 % set a
 1

NEM A possible approach is to write a pre-processing version of proc that implements syntax for expr:

proc func {name params body} {
    regsub -all {([^\\])\(} $body "\\1\[expr {" body
    regsub -all {([^\\])\)} $body "\\1}\]" body
    uplevel 1 [list proc $name $params $body]
}
func foo {x y} { return (1+[lindex $x ($y*4)]) }

It is a bit too eager on substitutions, requiring backslash-escaping to include a literal parenthesis anywhere, and not respecting braces. Still, it does allow a glimpse at a remarkable reimagining of Tcl:

func if {cond then {_else_ ""} args} {
    set args ([llength $args]==1 ? [lindex $args 0] : $args)
    uplevel 1 ($cond ? $then : $args)
}
func foo a {
    if ($a < 0) {
        puts Negative
    } else if ($a > 10) {
        puts Large
    } else {
        puts Small
    }
}

FM I like the concept of little language. But we could need to obtain the value of each calculation, so we should return a list. If we write :

set L [math {
   x=0;
   y=1;
   z=$y+1
}]]

we should get :

lindex $L 0 --> 0 (x value)
lindex $L 1 --> 1 (y value)
lindex $L 2 --> 2 (z value)

Then, we should be able to write in tk for instance :

wm minsize . {*}[math {[winfo screenwidth]/2];[winfo screenheight]/2}]
wm geom . [join [math {[winfo screenwidth]/2];[winfo screenheight]/2}] x]

instead of :

wm minsize . [expr {[winfo screenwidth]/2]}] [expr {[winfo screenheight]/2}]
wm geom . [expr {[winfo screenwidth]/2}]x[expr {[winfo screenheight]/2}]

or somethings like

[canvas .c] create line {*}[set L [list]; for {set i -50} {i<=50} {incr i} { lappend L {*}[math {x=0.1*$i; y=$x**2}] }; set L]

Lars H: When only one result is wanted, it would be quite a nuisance to have to lindex it out of a list of results. The approach I took in infix was to introduce , as a list-construction operation — this way one can get the effect you ask for by doing

set L [infix {} {
   x=0;
   y=1;
   z=y+1;
   x,y,z
}]]

or

set L [infix {} {
   x=0,
   y=1,
   z=y+1
}]]

FM: I found it more clear to return a list. But maybe it should be another command which "list expressions"

proc lexpr {expr} {
    namespace path {::tcl::mathfunc ::tcl::mathop}
    set L [list]
    foreach ::EXPR [split [string trim $expr] "\n\;"] {
        if { ! [expr {$::EXPR eq "" || $::EXPR eq {\{} || $::EXPR eq {\}} }] } {
            lappend L  [uplevel {
                namespace path {::tcl::mathfunc ::tcl::mathop}
                expr [subst {$::EXPR}]
            }]
        }
    }
    set L
}

Test :

set Central [dict create {*}[lexpr {
    "Rayon"; [set r 3]
    "Perimeter"; 2*[set pi [acos -1]]*$r
    "Area"; [set A [* $pi [** $r 2]]]
    "Solid Angle"; 4*$A
    "Volume"; 4*$A*$r/3
}]]

I know that you've made a lot of works on these proc infix. But I think that implementing a little langage by this way make us loosing tcl inside. Will we need to develop a new parser ?

Lars H: My point was only that with a list construction operation, you can return a list as easily as a single value. How the little language supporting that operation is implemented is a separate issue.

Let's think about Tcl. On one hand Tcl is verbose, but on the other hand programs are tiny. Why ? It's seems to me thats it's because commands could easily communicate together. I come to LOP, we could have a whole set of little langage but the problem is to be able to make them communicate together. In tcl we could write things like this :

expr {[
       set x 1
       if {$x == 1} {
           set y [expr {[info frame]*cos($x)}]
       }
       set z [expr {$y+$x+[llength [info proc ::*]]}]
       list [info level] $x $y $z [winfo children .]
      ]}

This is why Tcl is wonderfull. In fact, with the introspection facility of tcl, I think it would be possible for all this command to "know" they are in the expr context. Let push further this idea. I could implement this only this with lexpr because "info frame" tells only which proc is the caller (but not a command like expr !). Before let's do an little proc as "set" :

proc = args {
    set dico [dict create {*}[info frame -1]];
    if {[dict exist $dico proc] && [dict get $dico proc] eq "::lexpr"} {
        uplevel "set [lindex $args 0] \[expr {[lrange $args 1 end]}\]"
    } else {
        uplevel "set [lindex $args 0] {[lrange $args 1 end]}"
    }
}

This proc is looking upframe, test if it is call by lexpr (the better would be by expr) and adapt himself. let's do some trivial test :

lexpr {
    [= x 1]
    [= y 2]
    [= z $x + $y]
}
-> 1 2 3

= z $x + $y
-> 1 + 2

The proc "=" adapt himself to the caller. Imagine this kind of things could be implement with set and other command like list, so that they detect an expr context and apply expr to their argument. We could write things like this :

set Central [dict create {*}[expr {[
                                    set pi (acos(1))
                                    set r 3
                                    set P (2*$pi*$r)
                                    set A ($pi*$r**2)
                                    list Rayon $r Perimeter $P Area $A {Solid Angle} (4*$A) Volume (4*$A*$r/3)
                                   ]}]]

Another way to achieve this is to use the proc know in a modified version of expr

namespace eval ::lexpr {
    proc know {cond body} {
        proc ::lexpr::unknown {args} [string map [list @[email protected] $cond @[email protected] $body] {
            if {![catch {expr {@[email protected]}} res] && $res} {
                return [eval {@[email protected]}]
            }
        }][info body unknown]
    }

    know {[expr $args] || 1} {expr $args}

    proc let args {
        eval eval [list expr $args]
    }
    namespace unknown ::lexpr::unknown
    namespace ensemble create -map {let ::lexpr::let}
}

Test :

lexpr let {[
     set pi [acos(-1)]
     set r 3
     list Rayon $r Perimeter [2*$pi*$r] Area [set A [$pi*$r**2]] \
          {Solid Angle} [4*$A] Volume [4*$pi*$r**3/3]
]}

In this way, propagating the context, Tcl could approach a contextual grammar (here in 15 lines of code) and all Tcl commands remain inside !

Maybe a flag could be send between TclParseExpr and TclParseCommand to keep the knowledge of the context ? So, called from TclParseExpr, TclParsecommand would first try to parse a script inside it as an expression and in case of error, parse it as usuel. This could be a new concept in tcl, the ability for the parser to "know" in which context it is, and so it could be implemented for other commands. In those cases (only for expressions in a first step), {[ ... ]} mean propagate the context.

Lars H: All that is really an orthogonal idea (not tied to expr, although it can be applied to it), but anyhow… As I see it, the main problem with this kind of "adapt behaviour to context" solutions is that they don't scale at all well as the complexity (number of possible contexts) of the system grows — there are simply so many things that have to know about each other to make it all work, that it effectively becomes unextendable.

It is often possible to achieve much of the intended benefits by using a namespace as "context", i.e., having different behaviours of a command really be different commands. Besides already being supported in the language, this also scales well, because it is not based on having the computer guess (is this a command or expression or ...?) what you meant by what you wrote! It doesn't support every syntax one might imagine, but it very often supports a syntax that is as compact and expressive as the one that might be found in other languages.