Version 21 of lcomp

Updated 2010-06-14 00:31:15 by AMG

AMG I wasn't quite satisfied with [listc], Todd Coram's list comprehension procedure, so I adapted it into something far worse. Behold!

 proc lcomp {expression args} {
     # Check the number of arguments.
     if {[llength $args] < 2} {
         error "wrong # args: should be \"lcomp expression var1 list1\
             ?... varN listN? ?condition?\""
     }

     # Extract condition from $args, or use default.
     if {[llength $args] % 2 == 1} {
         set condition [lindex $args end]
         set args [lrange $args 0 end-1]
     } else {
         set condition 1
     }

     # Collect all var/list pairs and store in reverse order.
     set varlst [list]
     foreach {var lst} $args {
         set varlst [concat [list $var $lst] $varlst]
     }          

     # Actual command to be executed, repeatedly.
     set script {lappend result [subst $expression]}

     # If necessary, make $script conditional.
     if {$condition ne "1"} {
         set script [list if $condition $script]
     }

     # Apply layers of foreach constructs around $script.
     foreach {var lst} $varlst {
         set script [list foreach $var $lst $script]
     }

     # Do it!
     set result [list]
     {*}$script ;# Change to "eval $script" if using Tcl 8.4 or older.
     return $result
 }

Mainly I didn't like [listc]'s method of constructing its $foreachs variable (which I have named $script in my code), so I switched it around to be built inside-out rather than left-to-right. I start by putting together the actual command that'll be run by the tangled nest of [foreach] constructs; then I tack on [foreach]s and [if]s with the aid of [list]. Magically I no longer have to worry about matching braces, thus shortening my code into the sorry heap you see above.

Next up, I changed [eval] to [subst] and [expr], since I felt that all $expressions would use substitution and all $conditions would be boolean expressions. Especially nice is the ability to use [...] substitutions with both [subst] and [expr], so the old behavior can be had by wrapping the parameter with [ and ].

Lastly (unless I missed something), I dropped the <- noise word. I didn't much care for it, since [foreach] itself doesn't use it. You can re-add it if you like by changing "% 2" to "% 3" and "< 2" to "< 3" and the first instance of "var lst" to "var <- lst".


Examples? I'll start by listing Todd's examples modified to work with [lcomp].

 set i  [list 1 2 3 4 5 6]
 set l2 [lcomp {$i} i $i]
 puts "A copy of the list: $l2"

 set dbl [lcomp {[expr {$n*2}]} n $i]
 puts "Double values from list: $dbl"

 set evn [lcomp {$i} i $i {$i%2 == 0}]
 puts "Only even numbers: $evn"

 proc digits {str} {
     set lstr [split $str ""]
     return [lcomp {$d} d $lstr {[string is digit $d]}]
 }
 puts "Just digits from (703)-999-0012= [digits (703)-999-0012]"

 set names1 [list Todd Coram Bob Jones Tim Druid]
 set lf [lcomp {$l,$f} {f l} $names1]
 puts "From ($names1): Last,first = $lf"

 set l3 [lcomp {$f} {f l} $names1 {[string match "T*" $f]}]
 puts "From ($names1): Only names starting with 't': $l3"

 set l4 [lcomp {[list $n1 $n2]} n1 [list a b c] n2 [list 1 2 3]]
 puts "Create a matrix pairing {a b c} and {1 2 3}: $l4"

Here are some more:

 lcomp {$x} x {0 1 2 3}                           ;# 0 1 2 3
 lcomp {$y $x} {x y} {0 1 2 3}                    ;# {1 0} {3 2}
 lcomp {[expr {$x ** 2}]} x {0 1 2 3}             ;# 0 1 4 9
 lcomp {[expr {$x + $y}]} x {0 1 2 3} y {0 1 2 3} ;# 0 1 2 3 1 2 3 4 2 3 4 5 3 4 5 6
 lcomp {$x} x {0 1 2 3} {$x % 2 == 0}             ;# 0 2

Practical uses? The following deletes all images named by any elements in the images array:

 image delete {*}[lcomp {$val} {key val} [array get images]]

I'll come up with more later as I use [lcomp] more in my own code.


AMG: It's possible to modify [lcomp] to use [eval] to process $expression rather than [subst] or [expr]. [subst]-like behavior can be had with the [K*] proc found on K. Single-argument [list] or [concat] should also work. For instance,

 lcomp {K* $x} x {0 1 2 3}                        ;# 0 1 2 3
 lcomp {concat $y $x} {x y} {0 1 2 3}             ;# {1 0} {3 2}
 lcomp {expr {$x ** 2}} x {0 1 2 3}               ;# 0 1 4 9
 lcomp {expr {$x + $y}} x {0 1 2 3} y {0 1 2 3}   ;# 0 1 2 3 1 2 3 4 2 3 4 5 3 4 5 6
 lcomp {K* $x} x {0 1 2 3} {$x % 2 == 0}          ;# 0 2

Update: Instead of [K*], I suggest using single-argument [lindex]. See the [return -level 0] discussion for more information.


For the moment at least, there's a bit of list comprehension discussion at the bottom of [for]. Some day (soon?) those ideas will migrate to this page.


AMG: Here's another list comprehension I initially developed on hat0's page, in response to his musings about building a syntax for list comprehensions directly into Tcl. My intention was to demonstrate that they work fine as a command without special language support.

Unlike my earlier list comprehension posted above, this version uses [expr] to process each generated list element. Also it supports generating more than one output element for each combination of input elements, which is useful for making dicts. In addition to alternating and combinatorial iteration, it also supports parallel iteration. It's much more sugary, too--- see the words in bold in the following table. Not shown in the table, conditionals are also valid, using the if "keyword". Plus, if can be freely intermixed with for, so it may be useful to skip entire inner loops, not just individual elements.

Iteration type [foreach] example [lcomp] example
Alternating foreach {a b} $list {...} lcomp {...} for {a b} in $list
Combinatorial foreach a $list1 {foreach b $list2 {...}} lcomp {...} for a in $list1 for b in $list2
Parallel foreach a $list1 b $list2 {...} lcomp {...} for a in $list1 and b in $list2
proc lcomp {expression args} {
    set script "lappend Result \[expr [list $expression]\]"
    while {[llength $args] && [lindex $args 0] ni {for if}} {
        append script " \[expr [list [lindex $args 0]]\]"
        set args [lrange $args 1 end]
    }
    if {![llength $args]} {
        error "wrong # args: must have at least one opcode"
    }
    while {[llength $args]} {
        switch -- [lindex $args 0] {
        for {
            set nest [list foreach]
            while {[llength $nest] == 1 || [lindex $args 0] eq "and"} {
                if {[llength $args] < 4 || [lindex $args 2] ne "in"} {
                    error "wrong # operands: must be \"for\" vars \"in\" vals\
                           ?\"and\" vars \"in\" vals? ?...?"
                }
                lappend nest [lindex $args 1] [lindex $args 3]
                set args [lrange $args 4 end]
            }
        } if {
            if {[llength $args] < 2} {
                error "wrong # operands: must be \"if\" condition"
            }
            set nest [list if [lindex $args 1]]
            set args [lrange $args 2 end]
        } default {
            error "bad opcode \"[lindex $args 0]\": must be \"for\" or \"if\""
        }}
        lappend structure $nest
    }
    foreach nest [lreverse $structure] {
        set script [concat $nest [list $script]]
    }
    set Result {}
    eval $script
    return $Result
}

To demonstrate its usage, I will first rework all the examples previously given on this page.

set l {1 2 3 4 5 6}
puts "A copy of the list: [lcomp {$i} for i in $l]"
puts "Double values from list: [lcomp {$n * 2} for n in $l]"
puts "Only even numbers: [lcomp {$i} for i in $l if {$i % 2 == 0}]"
proc digits {str} {
    lcomp {$d} for d in [split $str ""] if {[string is digit $d]}
}
puts "Just digits from (703)-999-0012= [digits (703)-999-0012]"
set names1 {Todd Coram Bob Jones Tim Druid}
puts "From ($names1): Last,first = [lcomp {"$l,$f"} for {f l} in $names1]"
puts "From ($names1): Only names starting with 't':\
    [lcomp {$f} for {f l} in $names1 if {[string match T* $f]}]"
puts "Create a matrix pairing {a b c} and {1 2 3}:\
    [lcomp {[list $n1 $n2]} for n1 in {a b c} for n2 in {1 2 3}]"
lcomp {$x} for x in {0 1 2 3}                         ;# 0 1 2 3
lcomp {[list $y $x]} for {x y} in {0 1 2 3}           ;# {1 0} {3 2}
lcomp {$x ** 2} for x in {0 1 2 3}                    ;# 0 1 4 9
lcomp {$x + $y} for x in {0 1 2 3} for y in {0 1 2 3} ;# 0 1 2 3 1 2 3 4 2 3 4 5 3 4 5 6
lcomp {$x} for x in {0 1 2 3} if {$x % 2 == 0}        ;# 0 2
image delete {*}[lcomp {$val} for {key val} in [array get images]]

Now I will show off some new features:

lcomp {$key} {$val} for key in {a b c} and val in {1 2 3} ;# a 1 b 2 c 3

I guess that one example sums it up pretty well.

Both list comprehensions on this page have the drawback (or feature, depending on your preference) of running in a new stack frame. This means the iteration variables don't collide with existing variables (but also can't be examined afterwards). I tried to use uplevel to fix this, but that created a new problem: I definitely do not want the Result temporary variable to be in the caller's stack frame, especially in the case of nested [lcomp] invocation. Since I couldn't figure out a good solution to that problem, I gave up. Ideas, anyone?