Version 0 of accumulate and collect

Inlining Loops

Very often, when constructing data sturctures (especially lists) in a loop, I've had to write something like this:

  set result {}
  foreach x $input {
    lappend result [process $x]
  }

that is, declare a variable before the loop and manually appending results to that variable inside the loop. I thought, there must be a better way to do this. Well, there is map and filter from functional programming. They'll handle the common cases. But sometimes I really need the extra power of foreach. Wouldn't it be nice if foreach behave like map and filter'?

So I started thinking about writing a custom control structure. But rather than just another looping construct like foreach or map I wanted a control structure that can be applied to any and all loops. This is what I ended up with:

  set result [accumulate {
    foreach x $input {
      collect [process $x]
    }
  }]

I'm calling this accumulate and collect. The accumulate function simply evals the string passed to it constructing a list from values collected by the collect function. Basically, this is an encapsulation of the set..lappend idiom above.

The accumulate function is nestable such that:

  accumulate foreach x {1 2 3} {
    collect [accumulate foreach y {a b c} {
      collect "$x$y"
    }]
  }

would return:

  {1a 1b 1c} {2a 2b 2c} {3a 3b 3c}

This is great. It allows me to treat loops like functions that return lists. And I don't have to declare pesky temporary variables!

Then I realized something:

A Fancy List Generator

This is another problem I keep facing. I've long wished that tcl had a list generator where I don't need to backslash escape all the time. I hate having to do:

  set foo [list \
    a 1 \
    b 2 \
    c 3 \
  ]

Apart form all the '\' looking ugly, it is also error prone. I often forget to add a '\' and tcl complains about 'c' being an invalid command name.

Of course I could use {} to generate lists:

  set foo {
    a 1
    b 2
    c 3
  }

but this doesn't work if I need to perform variable substitution. And doing it with "" is not only hard to read but much more error prone than list!

While playing with accumulate..collect I realized that it is just a fancy list generator. Is this the list generator that I've been wanting for so long? Lets try it out:

  set foo [accumulate {
    collect a; collect 1;
    collect b; collect 2;
    collect $argv;         # Yes, variable substitutions work.
    collect [glob *];      # It works! And look, comments!
  }]

Wow, like so many things in tcl this came as a complete surprise. I've been waiting for something like this for so long! But the syntax looks a bit cumbersome. Let's see if we can remedy that:

  interp alias {} List {} accumulate
  interp alias {} : {} collect

  set foo [List {
    : a
    : b
    : c
    : $argv
    : [glob *]
    : [List {   #nested!
      : 1
      : 2
      : 3
      # and even plays well with [list]:
      : [list x y z]
    }]
  }]

I love it!

Implementation

So, without further ado, here's the implementation of accumulate and collect:

set accumulator {}
proc accumulate {args} {
        if {[llength $args] == 1} {
                set args [lindex $args 0]
        }

        lappend ::accumulator {}
        set code [catch {uplevel 1 $args} result]
        switch -- $code {
                0 {}
                3 break
                4 continue
                default {
                        set ::accumulator [lrange $::accumulator 0 end-1]
                        return -code $code $result
                }
        }
        set ret [lindex $::accumulator end]
        set ::accumulator [lrange $::accumulator 0 end-1]
        return $ret
}

proc collect {value} {
        set acc [lindex $::accumulator end]
        lappend acc $value
        lset ::accumulator end $acc
}