list , a built-in Tcl routine, creates a list.
list creates a new list and appends each arg, in order, as an item in the list. If no args are given, the new list is empty.
lappend and lset are the only 2 list routines which use the name of a variable containing a list rather than the list value itself.
In general, a list is an ordered tuple of values. In other languages this is sometimes known as an array or vector. list and related routines format a list such that it can be interpreted as the words of a command, making Tcl homoiconic (see "lists vs commands" below).
In contrast with other languages, in Tcl a list is not a data structure, but a string that is in a format derived from the rules of Tcl. The various list routines understand that format, and implement operations that can be perfomed on the list. For performance, those routines cache a list data structure for future use, but this is an implementation detail. The list format and the routines that process it comprise an abstract data type.
Because a list is a string, string operations can be applied to it, but the result might no longer be a valid list. See "lists vs strings" below.
In a script, it is customary for a literal value to be written in list format, e.g. the argument specification in a procedure definition, but it's safer to use list routines when building more complex lists. These routines ensure that characters like { or " are properly quoted:
list look, Ma! a \{ list with `" weird characters \} in itA list is parsed exactly as a script is parsed, except that $, [, and semicolon have no special meaning, newline is just another whitespace character, {*} is not allowed, and backslash-newline substitution is not performed on words in braces. A script that contains no script substitutions and does not use {*} is a valid list composed of all the words of all the commands in the script. Conversely, any valid list is a valid script containing at zero or more commands separated by the newline character.
When Tcl rules are stripped of those parts that refer to various aspects of evaluation, they also serve to specify the format of a list. Here are the Tcl rules that do not apply to the processing of a list:
In all other respects, list evaluation is identical to script evaluation. double quotes, braces, and backslash substitution are all processed as usual.
The empty string is a list that contains no words.
list adds braces and/or backslashes as needed to ensure that the resulting list is a single command whose words contain no substitutions. The result can be evaluated as more efficiently because the step of detecting and processing variable and command substitutions can be skipped.
Canonical lists are described later. Enclosing a canonical list in braces results in a list containing exactly one word whose value is the original list, but enclosing an arbitrary string of characters in braces does not necessarily result in a valid list, even if string itself is a valid list. For example, } is a valid list, but {}} is not. The canonical list form of } is \} and enclosing it in braces produces {\}}, which is a valid list.
One of the keys to successfully working with lists in Tcl is to understand that although braces ({ and }) are often used to delimit individual words in lists, braces do not mean "list". They are simply an escaping mechanism for some of Tcl's special characters, e.g., whitespace. As DGP put it, "Don't imagine that braces have magic list-ifying powers."
Except for the empty string, double quotes and braces are syntactic sugar for backslash escaping. For example, although braces are a very convenient way to represent nested lists, it's possible to represent the same nested lists without braces:
puts [lindex {a {b c {d e f g}}} 1 2] ;# -> d e f g
puts [lindex a\ b\\\ c\\\ d\\\\\\\ e\\\\\\\ f\\\\\\\ g 1 2] ;# -> d e f gNot that one would want to write code like with all those backslashes. It just illustrates the point that when interpreting a string as a list, braces merely escape the normal interpretation of whitespace as the delimiter between items in the list. They are only indirectly involved, by virtue of their effect on whitespace, in the interpretation of a value as a nested list. Braces by themselves do not mean "list".
The definition of the canonical list format is recursive: A list is a sequence of lists separated by whitespace. Therefore, list can be used to prepare a value for insertion into a list:
set part {five six}
set list "{one two} {three four} [list $part] {seven eight}"
llength $list; #-> 4However, since manipulating strings is expensive, it's generally preferable to use procedures such as linsert and lappend to modify lists.
This recursive definition only applies to the format. The definition of the abstract data type is slightly different: A list is a sequence of values. Procedures like lindex that implement operations on the abstract data type perform the needed quoting and unquoting to produce their results.
This page discusses both the string format of a list as well as operations of a list as an abstract data type. In the rules of Tcl, "word" has a special technical definition, and that definition carries over to the format of a list. On this page, "word" is used in this technical sense to refer specifically to a substring string that represents an item in a list. In other places it's used in the common non-technical sense. The applicable meaning should be apparent in each context.
Because whitespace delimits words in both commands and lists, every valid list is a valid script. However, not every valid script is a valid list, because the list format does not include script substitution:
% llength {set b [list $one "$two $three"]}
list element in quotes followed by "]" instead of spaceThe ] at the end of the command is a violation of the double quotes rule, which states that " terminates a quoted word. A space could be added to turn it into a valid list:
% llength {set b [list $one "$two $three" ]}
6Although as a script this is a command composed of three words, as a list there are 6 words because there is no script substitution. Furthermore, variable substitution is not performed, so the fifth word in the list is $two $three.
The empty string is an empty list:
% set list [list]
% llength $list
0
% set list {}
% llength $list
0
% set list ""
% llength $list
0A string containing nothing but whitespace is also an empty list:
% set list { }
% llength $list
0
% set list \n
% llength $list
0
set list \t\n\t
llength $list ;# -> 0A string that contains no whitespace is often, but not always a list (containing one word):
% set list hello
% llength $list
1
% set list he\{llo
% llength $list
1
% set notalist \{hello
% llength $notalist
unmatched open brace in listThe following is a list that contains one item, which is the empty string:
% set list {{}}
% llength $list
1
% set list [list {}]
llength $list ;# -> 1Whitespace separates words in a list:
% set list {1 2}
% llength $list
2
% set list "1 2"
% llength $list
2Here is a list that contains two items, both of which are empty strings:
% set list {{} {}}
% llength $list
2List routines parse strings into lists in much the same way that Tcl parses strings into commands. In the following example, llength parses the string into a list, and the backlash-space sequence results in the first character of the second word being a space character:
% set list {1 \ 2}
% llength $list
2It is very common, and perfectly acceptable, to use braces instead of list when writing a list:
% set list {one two three}
% llength $list
3But in Tcl, braces are simply a means of escaping whitespace and other special characters in strings. They are a nice way to write out lists, but Tcl itself doesn't equate braces with lists. Any quoting can be used to make a well-formed list, and there are an infinite number of ways to produce a string that is the same well-formed list:
% set list "one two three" % llength $list 3 % set list one\ two\ three % llength $list 3 % set list one\x20two\x20three % llength $list 3 % set list one\ttwo\tthree\t % llength $list 3 #... and on and on ...
When formatting a list as a string, list will escape values where necessary:
% list \n\{
\n\{All lists are strings, but not all strings are well-formed lists. The various list routines expect their arguments to be well-formed lists.
To check whether a string is a list:
string is list $somevariable
Alternatively:
catch {llength $somevariable}When doing experiments to understand lists, it is a good idea to first assign the values in question to variables before operating on them, since it is hard to keep track of when quoting is interpreted as Tcl parses the command vs. when a routine is interpreting the arguments it has received. This makes it possible to first inspect the value of the string before passing it to a routine:
% set var1 \{
{Many simple strings are lists:
% llength hello
1
% llength hello\ world
2
% llength {how I made a great mistake in quotation}
8Some strings, however, are not lists:
% llength \"
unmatched open quote in list
% llength \{
unmatched open brace in list
% llength "ab{ {x y"
unmatched open brace in list
% llength \{}a
list element in braces followed by "a" instead of space
% llength {{a b} {c}]}
list element in braces followed by "]" instead of space
% llength {{*}exactitude}
list element in braces followed by "exactitude" instead of spaceA single list may be represented by different combinations of double quotes, braces, and backslashes. For example, {[ value ]} is the same list as {{[} value \]}, but the latter is the canonical representation of the list.
To produce a canonical list:
list {*}$somelistOne quick and dirty way to check for list equality is to compare their canonical forms as strings:
expr {[list {*}$list1] eq [list {*}$list2]}This may be undesirable if the lists are large because the string values of both $list1 and $list2 are generated if they haven't been already.
list {*}$list1 also armors special characters against possible interpretation if evaluated as a Tcl script:
% list {*}{puts $hello; set b [list $one $two $three]}
puts {$hello;} set b {[list} {$one} {$two} {$three]}In other words, a list is formed such that the following is true (1 ):
expr {[eval list $list] eq [list {*}$list]}The canonical representation generated by Tcl uses braces to enclose words wherever possible. In cases where the canonical representation is generated on the fly as new data arrives this is problematic, since the choice to enclose a word in braces depends on knowing the entire content of the word up-front. A canonical streaming list uses only backslash escapes, except for the empty string, which is represented by a pair of adjacent braces {}. A streaming list is suitable for use in something like a critbit structure, where uniform representation is required.
In source code, braces are often used to write lists. One example is the arguments to proc:
proc move {item speed args} ...It would be a bit awkward, and a bit slower, to use list for that:
proc move [list item speed args] ...
Likewise, literal braced strings are used with switch (in the braced patterns-and-bodies form), and string map (the map is a list). These braced strings are usually not a problem, but you may need to think about list formatting when special characters (backslash, braces, whitespace, quotes) are involved. Sometimes, an extra layer of braces are required around a word in a list, but if it is unbalanced with respect to braces then you may need to backslash-escape all special characters in it instead.
Braces are also used to escape the body of a proc, but in that case, the body is not parsed as a list, but as a script (unescaped newlines take on special meaning):
proc myproc {} {
this
is not
parsed as a list,
but as a script
}One useful feature of list is that is produces a value that is properly quoted as a single word in a script. This feature is fundamental to the activity of generating scripts, and generating scripts for try, eval, apply, and friends is fundamental to the activity of writing programs in Tcl:
set value {lots of spaces}
set script [string map [list @value@ [list $value]] {
set list {}
lappend list @value@
llength list ;# -> 1
}]
try $scriptIn the previous example, if list had not been used to quote the whitespace in $value1, the length of the list would instead have been 3.
KBK writes on comp.lang.tcl (with some modifications):
If a string has been built up using list routines like list and lappend, then it is always well-formed as a command. The item in the list (word number zero) is the name of the routine and the remaining items are the arguments passed to that routine. This method, in fact, is one of only a very few ways to construct commands that have the desired parameters when one or more of the parameters contains user-supplied data, possibly including nasties like backslashes or unbalanced braces.
In particular, using double-quotes and the string routines such as append and subst is NOT safe for constructing commands from untrusted values.
Moreover, eval and its friends give you special support for the technique of using lists as commands. If a command being evaluated is a "pure" list, i.e. one that was constructed using the list routines, and has never acquired a string representation, then the evaluator is able to short-circuit the parsing process, knowing that all arguments have been substituted. It doesn't need to do the (fairly expensive) scan for $- - and \- substitution, nor balance "" and {}, but can go directly to looking up the routine by name and invoking it.
list can be used with {*} to the same effect as concat:
set a {a b c}; set b {d e f}
list $a $b ;# -> {a b c} {d e f}
concat $a $b ;# -> a b c d e f
list {*}$a {*}$b ;# -> a b c d e fSee Concatenating lists for a timing comparison of the various methods.
The following three methods for concatenating list are roughly equivalent in performance:
set list hello
concat $list $list
list {*}$list {*}$list
lappend list {*}$listThe difference is that concat does not make sure its arguments are valid lists, and lappend modifies $list
Before the advent of the {*} operator, the following syntax was used:
eval [list lappend baseList] $extraList
When writing a list of lists to a file, it's useful to represent it using the newline character to separate the words of the lists. Here's how to do that:
foreach list $tosave {
puts $chan \{
foreach word $list {
puts $chan [list $word]
}
puts $chan \}
puts $chan {}
}The result is a list of lists, having the same length as $tosave.
Internally, Tcl tracks the structure of the list, and the various list routines take advantage of this to improve performance to O(1) time (access time does not depend on the total list length or position within the list). A string representation of a list is not made until is is needed. Therefore, a string operation on a large list may incur a dramatic performance/storage penalty if it causes the string representation has to be generated. The rule of of thumb is to use list-aware routines for lists, and avoid string routines. One obvious exception to this rule is string is list, which is smart enough not to generate the string representation of a pure list.
concat is a string operation, but is smart enough not to incur the penalty if all its arguments are pure lists.
The internal representation of a list should be transparent at the script level, but for the curious:
In the C implementation of Tcl 8.x, a list is a type of Tcl_Obj, The items of a list are stored as C-style vectors of pointers to the individual Tcl_Obj item values, plus some extra data. The consequence of this is that llength and lindex are constant-time operations — they are as fast for large lists as they are for small lists.
concat also operates on lists, but does not require that its arguments be valid lists. foreach operates on lists. split creates lists, and join consumes them. Various other routines also make use of lists.
Since all values, including lists, are strings (but not all strings are lists!), it is possible to use string routines on lists, but performance can suffer and there are usually better ways to accomplish the task. In some common cases, Tcl is smart enough to do the right thing. For example, doing a string comparison between a list and the empty string is perfectly acceptable, and just as performant as the llength variant:
proc foo {bar args} {
if {$args eq {}} then { #string comparison might force internal Tcl gyrations
set args $::foo::default_for_args
}
# ...
}The llength variant:
proc foo {bar args} {
if {[llength $args] == 0} then { # $args is empty
set args $::foo::default_for_args
}
# ...
}In the following example, the value of a_single_backslash is a single backslash:
set a_single_backslash [lindex \\\\ 0]
prior to invoking lindex , Tcl performs backslash substitution on the four backslashes so that the first argument becomes two backslashes. To convert the first argument to a list, lindex then performs backslash substitution on the first argument (two backslashes), resulting in one backslash, which is then assigned to a_single_backslash.
split takes a string and returns a list, but the best choice depends on the task at hand. regexp is often handy:
set wordList [regexp -all -inline {\S+} $myGnarlyString]DKF proposed this pretty alias:
interp alias {} listify {} regexp -all -inline {\S+}Bill Paulson notes that this alias changes all whitespace to a single space, which might or might not be what you want.
Other than that, this listify is effectively a split that interprets adjacent delimiters as as a single delimiter rather than interpreting them as delimiting an empty string, like split does.
(2014-09-08): this can also be used to duplicate words in a list:
regexp -all -inline {\S+} {a b c}
# => a b c
regexp -all -inline {(\S+)} {a b c}
# => a a b b c c
regexp -all -inline {((\S+))} {a b c}
# => a a a b b b c c cetc.
Various ways to check whether a string is a list:
string is list $some_value
catch {lindex $some_value 0}
catch {llength $some_value}In later versions of Tcl, string is list is available.
escargo 2003-03-16: How can you tell if a value is a string of words or a list of strings of words?
The practical application that I had for this was an error-printing proc. It could be passed a value that might be a single error message or a list of error messages. If it were a single error message, then I could print it on one line; if it were multiple messages, then I wanted to print each on its own line.
So, how could I distinguish between the cases?
I think I eventually made all sources of errors provide a list of errors, even if was always a list of 1 (instead of just the error message string).
But the question always stuck with me? Was there a way I could have easily distinguished between the two? Could I look at the representation and see an opening curly brace if it were a list?
RS: The (outer) curlies are not part of the list - they are added, or parsed away, when needed.
Tcl lists are not fundamentally different from strings, rather, I'd say they are a "view" on strings. Just as 42 can be viewed as string, or integer, it can also be viewed as a one-word list. Except if you introduce your own tagging convention, there is no way of telling whether a list is in reality a string - in the other direction, only strings that cannot be parsed as lists (unbalanced braces, quotes..) cannot be viewed as lists. But for your concrete error-printing problem: if you simplify the interface to "a list of one or more error messages", you can have the desired effect with
puts [join $errormessages \n]
Just make sure that the "one message" case is properly nested, e.g.
errorprint {{This is a one-liner}}
errorprint "{This too, with a $variable reference}" ;# braces in quoted strings allow substitution
errorprint [list "another $variable reference"]
errorprint {{Two messages:} {This is the second}}PYK 2019-10-13: A deep list addresses this issue by encoding a single value as a list containing only one item. Everything else is interpreted as a sequence if it looks like one. To add a value that might be interpreted as a list, use list $myvalue. Routines that understand the format automatically unlist values as they are requested.
Many simple string values can also be interpreted as single-word lists. Programs should use additional data or rely on program logic to decide whether a value should be interpreted as a list or a string.
Here is another way of looking at the problem (lindex without an index returns the list argument unchanged):
% lindex a
a
% lindex a 0
a
% lindex [lindex a 0] 0
a
% lindex [lindex [lindex a 0] 0] 0
a
% lindex {a}
a
% lindex {a} 0
a
% lindex [lindex {a} 0] 0
a
% lindex [lindex [lindex {a} 0] 0] 0
a
% lindex {{a}}
{a}
% lindex {{a}} 0
a
% lindex [lindex {{a}} 0] 0
a
% lindex [lindex [lindex {{a}} 0] 0] 0
a
% lindex {{{a}}}
{{a}}
% lindex {{{a}}} 0
{a}
% lindex [lindex {{{a}}} 0] 0
a
% lindex [lindex [lindex {{{a}}} 0] 0] 0
aNo program can tell the difference between the string "a" and the one-word list "a", because the one-word list "a" is the string "a".
Dossy 2004-02-26: A co-worker yesterday who is new to Tcl discovered something that surprised me -- nested lists in Tcl don't work as I expected in a very specific case:
% list [list [list x]] x
Um, when I ask for a list of a list of a list with the single word 'x', I would expect '{{{x}}}' back. However, you just get 'x' back. Thinking about it, I understand why, but it means that Tcl lists alone cannot be used to represent ALL kinds of data structures, as Tcl lists magically collapse when it's a series of nested lists with the terminal list having only a single bare word that requires no escaping.
Lars H: It looks worse than it is. For one thing, it is only the string representation that collapses, not the internal representation, so the above nesting of list is not completely pointless. It is furthermore very uncommon (and this is not specific to Tcl) that nesting depth alone has a significance. Either you know the structure of the value, and thereby the intended nesting depth, or the list is some generic "thing", and in that case you anyway need a label specifying what kind of thing it is.
DBaylor: I think this is actually worse than it looks. I see lots of people trying to learn Tcl and the #1 point of confusion is Dossy's example. But what I really dislike about this behavior is that it hides bugs until specific input is encountered. If you ever mix up your data-types (string vs. list), your code will work fine 99% of the time - until special characters are involved. These bugs are inevitably found the hard way. Oh how I wish list x returned {x}.
PYK 2013-10-27: What would lindex x 0 return, then? An error that x is not a list? All commands are lists (after adjusting for script substitution), and x is potentially a valid command. Therefore, x must be a list. There is some subtlety to Tcl which can take a little time for beginners to wrap their heads around, but this subtlety is Tcl's strength, not its weakness. The gripe about hiding bugs until specific inputs are encountered is a gripe about dynamic languages in general, and not particular at all to Tcl.
LV Question: in Perl, Python, and a number of other languages, one has the ability to read and write to subsets of a list -- slices -- using an almost array like notation. Is this something that one could simulate without much grief in Tcl?
RS: But of course - our old friends (with wordier notation)
set slice [lrange $list $from $to] set newlist [eval lreplace [list $otherlist] $from $to $slice]
lset can only replace a single item, but possibly several layers deep in the nesting. For reading access to a slice of a list, check Salt and sugar for how the following is implemented:
set i [$ {a b c d e f g} 2 4] ==> {c d e}To flatten a list:
concat {*}$nestedIt can be applied multiple times:
proc flatten data {
concat {*}$data
}
set a {{a {b c}} {d {e f}}} ; # {a {b c}} {d {e f}}
flatten $a ; # a {b c} d {e f}
flatten [flatten $a] ; # a b c d e falternatively:
set flattened [join $mylist]
Another possibility:
foreach e $list {
foreach ee $e {
lappend flatList $ee
}
}eval is not a good option because it chokes on newlines:
% eval concat {a
b}
ambiguous command name "b": binary breakThe newline in the argument marks the end of the arguments to concat and the beginning of a new command. This is due neither to concat nor to the tclsh prompt loop, but to fact that of eval itself concatenates into a single script and then evaluates that script.
To get around that issue, use some list routine to convert the value into a list containing no newline characters:
eval concat [lrange {a
b
} 0 end]RS 2004-02-26: If you really want to flatten a list of any depth, i.e. remove all grouping, I think this way is simplest (and robust):
proc flatten list {
string map {\{ "" \} ""} $list
}
% flatten {a {b {c d {e f {g h}}}}}
a b c d e f g hLars H: No, that won't work. Consider
% flatten [list \{ \}]
\ \I'd give you that it isn't exactly clear what should happen to lists with such words, but the above doesn't get a single character right.
RS admits he was thinking of well-behaved lists (as built with list and lappend, where braces are only generated mark-up, not content :^) You're right to point out that flatten is not chimpanzee-proof, and robust, enough.
cyrilroux 2010-10-28: Maybe here is the solution to solve the escape issue? This always consist in replacing all { and } but NOT \{ and \} (escaped ones). That is to say 6 cases:
"^{" " {" "{{+" "}$" "} " "}}+"
proc lflatten list {
regsub -all {^\{| \{|\{\{+|\}$|\} |\}\}+} $list { } flatten
return $flatten
}
% set foo {{a {b c}} {d\{d\{ {e f} g}}
% lflatten $foo
a b c d{d{ e f g CMP: In general, string manipulation on lists all have the same problem; they do not consider the list structure (unless copying the complete implementation of the existing list routines). Hence, all list manipulations should be done using existing list routines.
Tcllib now contains a struct::list module. Its documentation can be found at http://tcllib.sourceforge.net/doc/struct_list.html .
dgp offers this example of making use of it:
% package require struct 1.3
1.3
% namespace eval my {
namespace import ::struct::list
set l [::list 1 2 3]
puts [list reverse $l]
}
3 2 1In the previous example, ::list refers to the standard list in the global namespace, and list refers to the routine in the current namespace that was imported from ::struct::list.
AMG: Is this an acceptable implementation of list?
proc list args {
return $args
}Looks right to me...
RS: To me too. That's because args is already a list, as by the Tcl parser... I'd just write
proc list args {set args}AMG: Chuckle.
proc list args [list set args]
LV: in response to a question, Jonathan Bromley, 2008-05-30, comp.lang.tcl, wrote the following Tcl proc for turning a long list into a list of lists:
proc split_list {L {n 50}} {
incr n 0; # thanks to RS for this cool "is it an int" check!
set result {}
set limit [expr {[llength $L] - $n}]
for {set p 0} {$p <= $limit} {incr p $n} {
lappend result [lrange $L $p [expr {$p+$n-1}]]
}
return $result
} arg: Just the code I wanted (needed to split results from SQLite). Thanks. But can I suggest a few tweaks:-
proc partitionlist {L {n 2}} {
incr n 0; # thanks to RS for this cool "is it an int" check!
set result {}
set limit [llength $L]
for {set p 0} {$p < $limit} {incr p $n} {
lappend result [lrange $L $p [expr {$p+$n-1}]]
}
return $result
} Lars H: If the partitioned list is then immediately going to be iterated over, it may of course be easier to take advantage of the fact that the variable arguments of foreach are really lists of variables. I.e., instead of
foreach row [partitionlist $data 3] {
lassign $row first second third
# Further processing...
}one can just do
foreach {first second third} $data {
# Further processing...
}Conversely, this can be used for the following braintwisters' implementation of partitionlist:
proc partitionlist {L {n 2}} {
set varlist {}
set body {lappend res [list}
for {} {$n>0} {incr n -1} {
lappend varlist $n
append body { $} $n
}
set res {}
foreach $varlist $L [append body \]]
return $res
}AMG PYK: When given no arguments, list returns empty string. I find this useful when entering Tcl commands interactively, e.g. into tkcon or tclsh. When I know that a routine will produce a lot of output, such as reading a whole file, and I don't want to have it all on my screen, I tack ; list onto the end of my command line.
$ tclsh % set chan [open bigfile] file12dcd00 % set data [read $chan]; list % close $chan
If not for the ; list, the second line would flood my terminal with lots and lots of garbage.
AMG: Another use for list is to pass it a single argument which it will then return. For an example, see SCT & RS's comments on the page for if. However, this works due to the "problem" noted above by Dossy 2004-02-26. Often the argument requires quoting to become a single-word list, in which case list will not return its argument verbatim. On the return page I discuss a few other, safer approaches. The simplest one is to instead use single-argument lindex.