Sugar syntax macros

Section 0 - Sugar
Section 1 - Sugar command macros
Section 2 - Sugar syntax macros (what you are reading)
Section 3 - Sugar transformers

Syntax macros and syntax sugar

Syntax macros are exactly like the command macros already described, with the only difference that every syntax macro is called for every command in the application. Actually, you can imagine a command macro as a syntax macro that works as an identity transformation if the first argument does not match a given command name.

Because syntax macros are called for every command, they have the ability to add new syntax to Tcl: they can inspect every argument of the whole script and do substitutions on it. You can do with syntax macros many syntax manipulations previously explored with unknown, with the following differences:

Expand to native code. No performance penality.
Don't need that the first argument is unknown. Always work.
The expansion happens when the procedure is created, so it's possible to use syntax macros for general static checks.

unknown is very useful to explore many Radical Language Modification ideas, but to add syntax in a reliable way and/or with acceptable performances is often impossible.

I'll show syntax macros with the implementation of the following syntax glue. Instead of writing:

puts [expr {$a+$b}]

the user will be able to write:

puts ($a+$b)

puts ( $a + $b )

or any mix. Well that's NOT A GOOD IDEA of course ;) see TIP 174, Math Operators as Commands to see another alternative to expr, that I think is a good idea, but this syntax glue it's a decent exercise for syntax macros:

sugar::syntaxmacro sugarmath args {
    set argv $args
    set start_idx -1
    set end_idx -1
    for {set i 0} {$i < [llength $argv]} {incr i} {
        set tok [lindex $argv $i]
        if {[string index $tok 0] eq {(}} {
            set start_idx $i
        }
    }
    if {$start_idx == -1} {
        return $argv
    }
    set level 0
    for {set i $start_idx} {$i < [llength $argv]} {incr i} {
        set tok [lindex $argv $i]
        foreach char [split $tok {}] {
            if {$char eq {(}} {incr level}
            if {$char eq {)}} {
                incr level -1
            }
        }
        if {$level == 0 && [string index $tok end] eq {)}} {
            set end_idx $i
        }
    }
    if {$end_idx == -1} {
        return $argv
    }
    # The syntax looks ok, perform the expansion.
    lset argv $start_idx "\[expr \{[string range [lindex $argv $start_idx] 1 end]"
    lset argv $end_idx "[string range [lindex $argv $end_idx] 0 end-1]\}\]"
    return $argv
}

That's it. With this macro the following code:

puts (($a+$b)*($a*$b))
lindex $mylist ($idx-1)

is translated to

puts [expr {($a+$b)*($a*$b)}]
lindex $mylist [expr {$idx-1}]

But you can't use it in string interpolation. Because this macro expands to a command substitution form where actually there wasn't one, it can't be used inside strings.

This will not work:

puts "($a+$b)"

Will just print ($a+$b) verbatim. Instead one should use:

puts [format ($a+$b)]

Third good thing about macros:

3) Macros allows to create syntax sugar without to cause cancer to the semicolon.

Macros are a good place where we can syntax-overload our preferite language to specialize it and solve a specific problem with less typing. If, instead, we add syntax to Tcl itself, we have a less general, more complex language, that may make a task like the creation of a macro system much more hard.

Note that syntax sugar may expand to other macros that are recursively processed, so adding syntax with macros does not limit the range of actions of other macros.

More static source code checks

Another example of syntax macro is a macro to output a warning about procedures called with a bad number of arguments. This is very short and simple and will detect a bad usage only if the called procedure is defined *before* of the bad usage (to detect bad usage of commands that are defined later, it needs to be a bit more complex and to collect unresolved calls in a global variable).

syntaxmacro aritychecker args {
    set argv $args
    set command [lindex $argv 0]
    set ns [sugar::currentProcNamespace]
    set procname $ns\::$command
    if {[catch {info args $procname} cmdargs]} {
        set procname $command
        if {[catch {info args $procname} cmdargs]} {
            return $argv
        }
    }
    set call_argnum [expr {[llength $argv] - 1}]
    # Calculate the min and max number of arguments.
    set max_argnum [llength $cmdargs]
    set min_argnum [llength $cmdargs]
    foreach a $cmdargs {
        if {[info default $procname $a _]} {
            incr min_argnum -1
        } elseif {$a eq {args}} {
            incr min_argnum -1
        }
    }
    # If the last argument is 'args', set a fake max at 1000
    if {[lindex $cmdargs end] eq {args}} {
        set max_argnum 1000
    }
    # Check if it's ok.
    if {$call_argnum < $min_argnum || $call_argnum > $max_argnum} {
        puts stderr "*** Warning, procedure '$command' may be called with bad number of arugments in [sugar::currentProcName]"
    }
    return $argv
}

This kind of macro is pretty slow (at least until the parser is implemented in C ), being called for every command in the script (possibly more times for every command, because the script is re-processed if some macro expansion happened in order to expand macros produced by macros), so it's better to use it only during development, being not useful at all in a finished program that's correct enough to don't produce warnings.

Actually, if Sugar will be of some value in my day-to-day programming (and I think it will), it will be reimplemented in C once the API is stable. But for now I'll take in pure Tcl to hack it faster if needed.

TP 2011-05-18: Fixed above macros to work with the latest version of Sugar.

Continue with Sugar transformers

Category Tutorial