**Parser For expr Command**

%|||%
&|Areas                   | Language parsing, computer arithmetic |&
&|Good if student knows   | variable, based on interests |&
&|Priority                | Low |&
&|Difficulty              | low to medium |&
&|Benefits to the student | introduction to grammar and parsing concepts |&
&|Benefits to Tcl         | Leverage ease of use of existing tools |&
&|Mentor                  | [Steve Huntley] |&

**Project Description**

[Perl] enables arbitrary precision math by means of a pragma, which overloads the standard math operators and thus transparently accepts large integers as arguments for mathematical expressions.  In this way Perl is able to do not only large integer math transparently as Tcl does, but also floating-point, rational number, vector, etc. calculations [http://perldoc.perl.org/bignum.html] [http://perldoc.perl.org/bigrat.html].

This approach could not work with Tcl, since it does not have math operators per se, but instead has the command [expr] with its own syntax for math expressions.  However, it should be straightforward to overload the expr command itself; with a command that is able to parse existing valid math expressions, and can be expanded to accept a wider range of operations.

The goal of this project would be to write a selection of parsers/lexers for mathematical expressions, and incorporate them into a replacement for the expr command, thus allowing for transparent use of a wider range of numerical types (similar to Perl's usage), and experiments on new ways to execute expressions that are currently valid, such as:

   * compiling to C statements and executing via [critcl]
   * compiling to bytecode using methods outlined by [KBK] [http://www.tclcommunityassociation.org/wub/proceedings/Proceedings-2010/KevinKenny/assembler.pdf]
   * automatic interfacing to graphing functions

The exact mix of features would be dependent on the skills and interests of the student.

Existing pure-Tcl parsing tools such as [Yeti] or [taccle] should be suitable for the task.  Evaluation of parser features and selection of tools would be part of the project.

**Requirements:**

   * flexible

**Discussion**

[LarrySmith]: incorporate the Tcl expr patch allowing us to eliminate the dereferencing operator and outer braces?  Please?  Finally?  allowing [[ expr x*x ]] instead of [[ expr {$x*$x} ]]

[AMG]: There are a lot of syntax conflicts which require variable names to be preceded by $.  As for removing the outer braces, you can do so currently, though it is unsafe and slow.

However, these two changes at the same time would avoid part of the safety problem, since [[[expr]]] (not the Tcl parser) would be performing the variable substitution.  But the speed problem would remain.  So long as there are any spaces in the expression, there's more than one argument to [[expr]], and it has to [[[concat]]] them, which makes it impossible to [bytecode] the expression.  The remaining safety issue is due to [[square bracket script substitution]], which would still be performed by the Tcl parser instead of internal to [[expr]].

Some syntax conflicts, ambiguities, and difficulties:

   * '''Arrays and functions.'''  Is `cos(-1)` calling a function "cos" with argument "-1", or is it looking for an element "-1" in an array called "cos"?
   * '''Array elements.'''  `foo(bar)`: The array is named "foo", that's no problem.  But is the element literally "bar", or is it the string stored in the variable "bar"?
   * '''Functions with an extra space.'''  It's legal to put space between the function name and the argument list, e.g. `cos (-1)`.  Is this calling "cos" or looking in the variable "cos"?
   * '''Variable names with special characters.'''  "a b" (included space) is a valid variable name, but a simple parser might see it as two separate variables separated by a space.  That's why there's "${a b}", with braces.
   * '''Variable names that look like numbers.'''  "1" is a valid variable name, so are "0xa" and "5.2e-7".  They're also valid numbers.  How can the parser tell the difference?  If it assumes the leading digit makes it a number, then there needs to be a way to force the variable interpretation, such as "${5.2e-7)".  But that brings back dollar sign notation, which defeats the purpose of your proposal and raises the safety issue again.
   * '''Variable names that look like operators.'''  "+" is also a valid variable name!
   * '''Empty string variable.'''  Believe it or not, but variables can be named empty string.  How would you take the value, other than "${}"?  Well, there's "$::", or "::" as you would have it.

Nevertheless, here's a simple example demonstrating an overloaded [[expr]] that behaves as you ask.  Note that it still has the safety problem, since variable substitution is performed by Tcl before calling [[expr]].

======
rename expr _expr; proc expr {args} {
    uplevel 1 _expr [regsub -all -nocase {[a-z:][a-z0-9_:]*\M(?!\()} [concat $args] {$&}]
}
======

This code doesn't get all cases, it doesn't support arrays, and it screws up the [?:] ternary operator.  Since it doesn't support arrays, which are used by the [[[history]]] mechanism, it won't work with an interactive Tcl session.  For interactive use, try this:

======
proc expr2 {args} {
    uplevel 1 expr [regsub -all -nocase {[a-z:][a-z0-9_:]*\M(?!\()} [concat $args] {$&}]
}
======

<<categories>>GSoC