Version 23 of lmap

Updated 2014-02-03 19:38:25 by PeterLewerin

lmap - Iterate over all elements in one or more lists to generate a new list

(added to Tcl 8.6 by TIP #405 ); a “collecting foreach”.

lmap varname list body
lmap varlist1 list1 ?varlist2 list2 ...? body

The lmap command implements a loop where the loop variable(s) take on values from one or more lists. In the simplest case there is one loop variable, varname, and one list, list, that is a list of values to assign to varname. The body argument is a Tcl script. For each element of list (in order from first to last), lmap assigns the contents of the element to varname as if the lindex command had been used to extract the element, then calls the Tcl interpreter to execute body. It builds up a list of the results of evaluating body.

In the general case there can be more than one value list (e.g., list1 and list2), and each value list can be associated with a list of loop variables (e.g., varlist1 and varlist2). During each iteration of the loop the variables of each varlist are assigned consecutive values from the corresponding list. Values in each list are used in order from first to last, and each value is used exactly once. The total number of loop iterations is large enough to use up all the values from all the value lists. If a value list does not contain enough elements for each of its loop variables in each iteration, empty values are used for the missing elements.

The break and continue statements may be invoked inside body, with the same effect as in the for command (in both cases, no value is accumulated for this time round the list; in the case of break, the construction of the list finishes). Lmap returns the accumulated list (unless there's an error).


Richard Suchenwirth 2005-04-02 - lmap is a "collecting foreach" which returns a list of its results. In Jim it is built in, but it can be easily had in pure Tcl:

proc lmap {_var list body} {
    upvar 1 $_var var
    set res {}
    foreach var $list {lappend res [uplevel 1 $body]}
    set res
}

Several usage examples are at Multiplication tables. Lmap is a compromise between Tcl and the classical functional programming function map, in that it takes a "quasi-lambda" which is split up into the _var name and the body arguments. However, this style is well-known from foreach, and somehow reads better:

lmap i {1 2 3 4 5} {expr $i*$i}

vs.

map [lambda i {expr $i*$i}] {1 2 3 4 5}

Jim's lmap uses continue to skip the accumulation of the current iteration, so it works like map and filter at the same time. In Jim lmap supports multiple lists in input like the real foreach, so you can do interesting things like this:

. lmap a {1 2 3} b {A B C} {list $a $b}
{1 A} {2 B} {3 C}

Multiple lists + accumulation + continue to skip makes it also somewhat similar to list comprehension (but simpler to use in the my (SS) opinion).


A cute variation is fmap (influenced by ApplyAll in Backus' FP; Joy has a comparable operator in cleave) which maps a list of functions on one argument:

proc fmap {functions x}  {lmap f $functions {$f $x}}

Then we can write a file reader like this:

proc << filename  {lindex [fmap {read close} [open $filename]] 0}

NEM I like that one! There is an mmap function that I wrote with Monadic TOOT which is similar to lmap (look about 1/3 way down that page). Instead of using continue, it uses a maybe monad to decide which results to accumulate. (Actually, it's quite general, and any monad could be used).


iu2 2009-10-15, I like to eliminate the "helper variables":

proc lmap {list body} {
    upvar 1 0 var  ;# $0 will be available automatically!
    set res {}
    foreach var $list {lappend res [uplevel 1 $body]}
    set res
 }

So we can write

lmap {1 2 3 4 5} {expr $0 * $0}

As I often replace the original list variable with the mapped list, instead of

set list [lmap $list {expr $0 * $0}]

I can do

lmap! list {expr $0 * $0}

where lmap! is

proc lmap! {listvar body} {
    upvar 1 $listvar res
    set res [lmap $res $body]
}

Finally, the current tcl constructs are just fine for a one liner

set list2 {}; foreach x $list {lappend list2 [expr {$x * $x}]}

PL: a replacement for lmap that allows multiple varname--listval pairs:

package require Tcl 8.5
proc lmap args {
    set body [lindex $args end]
    set args [lrange $args 0 end-1]
    set n 0
    set pairs [list]
    foreach {varname listval} $args {
        upvar 1 $varname var$n
        lappend pairs var$n $listval
        incr n
    }
    set temp [list]
    foreach {*}$pairs {
        lappend temp [uplevel 1 $body]
    }
    set temp
}

Note that the above command uses the {*} thingy, so it's still no good for Tcl 8.4 users: one more implementation is perhaps needed.

package require Tcl 8.4
proc lmap args {
    set body [lindex $args end]
    set args [lrange $args 0 end-1]
    set n 0
    set pairs [list]
    foreach {varname listval} $args {
        upvar 1 $varname var$n
        lappend pairs var$n $listval
        incr n
    }
    set temp [list]
    eval foreach $pairs [list {
        lappend temp [uplevel 1 $body]
    }]
    set temp
}

DKF: Note that lmap imposes quite a cost:

% time {lmap i {1 2 3 4 5} {expr {$i*$i}}} 100000
24.6734621 microseconds per iteration
% time {set res {}; foreach i {1 2 3 4 5} {lappend res [expr {$i*$i}]};set res} 100000
7.2637871 microseconds per iteration
% time {apply {l {set res {}; foreach i $l {lappend res [expr {$i*$i}]};set res}} {1 2 3 4 5}} 100000
2.75048408 microseconds per iteration

So... lmap is 9 times slower than inlining it (the use of apply shows that the effect of compilation of foreach is a fair part of it, but that cuts both ways).

DKF: Note that there's a proposal to implement this (in a manner much like the Jim version), which will not incur the performance cost mentioned above. (The cost was largely due to the fact that using a procedure like that defeated efficient handling of variables, together with some overhead due to stack frame handling.)


Googie - 2012-10-03: As this command is currently being implemented for 8.6 I feel a need to express my concern. When I first saw lmap being mentioned my first thought was "oh, the string map for lists, great!", which is a wrong interpretation given what actually the command does. I think that more people will catch themselves with the same impression. I really think that the functionality of this command is expressed much better by lapply name, wouldn't you agree? I know it's a little late for changing any plans for 8.6, but still - worth of consideration.

Same applies for proposed dict map -> dict apply.

DKF: We can't make everyone happy, and this page indicates that majority opinion is that it's a code-driven transformation.


AMG: Does the 8.6 [lmap] catch [return] calls, or will a [return] cause [lmap]'s caller to return?

AMG: I just checked... the latter is the case.

  Remove list element not fitting a condition

HaO 2013-11-25: To get only the elements which fulfill a certain condition, one may use:

lmap v $list {
        if {!<condition>} continue
        set v
        }

For example to get all numbers:

% lmap v {1 2 3 a b c} {
        if {![string is entier $v]} continue
        set v
        }
1 2 3


AMG: See also: lcomp.