split - Split a string into a proper Tcl list



Returns a list created by splitting ''string'' at each character that is in the ''splitChars'' argument. 
Each element of the result list will consist of the characters from ''string'' that lie between instances of the characters in ''splitChars''. 
Empty list elements will be generated if ''string'' contains adjacent characters in ''splitChars'', or if the first or last character of ''string'' is in ''splitChars''. 
If ''splitChars'' is an empty string then each character of ''string'' becomes a separate element of the result list. 

The result of `split` is a list of substrings of ''string'' that are delimitied
See the man page at http://www.tcl.tk/man/tcl8.5/TclCmd/split.htm


   [join]:   

   [list]:   

   [Additional string functions]:   
   
   [Arts and crafts of Tcl-Tk programming]:   

   [ycl]:   provides `string delimit`, which splits strings on `[string match]`-style patterns or `[regexp]`-style patterns
   [ycl]:   provides [[`string delimit`], which splits strings on [[`[string match]`]-style patterns or [[`[regexp]`]-style patterns
   [Splitting strings with embedded strings]:   
   [Splitting a string on arbitrary substrings]:   

*** Splitting by dots ***
======
split comp.unix.misc .
split "comp.unix.misc" .

======none
comp unix misc
======

*** Splitting into characters ***

split {Hello world} {}
split "Hello world" {}

======none
H e l l o { } w o r l d
======

Note that this is an optimised case; splitting into characters is an efficient operation.
Splitting on the empty string is an optimized case, and is an efficient

split { abc def  ghi}
split " abc def  ghi"
======none
{} abc def {} ghi
======

Usually, if you are splitting by whitespace and do not want those blank fields, you are better off doing:
regexp -all -inline {\S+} { abc def  ghi}
regexp -all {\S+} " abc def  ghi"

abc def ghi
======

** Questions **

   * [Splitting strings with embedded strings]

<<discussion>>
** Definition of White-Space Characters **

[DKF]: I believe there's a standard (ANSI? POSIX?) somewhere.  But the answer
[DKF]: I believe there's a standard (ANSI? POSIX?) somewhere.  
But the answer includes "space", "tab", and "newline".
[escargo]: By "tab" do you mean both horizontal tab (ASCII 9) and vertical tab
''[escargo]'' - By "tab" do you mean both horizontal tab (ASCII 9) and 
vertical tab (ASCII 11)?  (See http://www.asciitable.com/)  
Arguments could be made for most of the ASCII characters under 33.
[Strick]: Let's ask Tcl what it thinks are white:

$ env | grep en_
LANG=en_US.UTF-8
$ cat what-chars-does-split-think-are-white.tcl
for {set i 0} {$i<65536} {incr i} {
  if {[llength [format "/%c/" $i]] > 1} { puts -nonewline "$i " }
}
$ tclsh what-chars-does-split-think-are-white.tcl
9 10 11 12 13 32 $
======

[escargo] 2005-04-01 :
''[escargo] 4 Jan 2005'' -
9 = ASCII TAB, 10 = ASCII LF (line feed), 11 = ASCII VT (vertical tab), 
12 = ASCII FF (form feed), 13 = ASCII CR (carriage return), and of course 32 = ASCII Space.
9 = ASCII TAB, 10 = ASCII LF (line feed), 11 = ASCII VT (vertical tab), 12 =
I would have thought that the separator characters would count as white space (28-31, 
FS, GS, RS, US), but I guess they are regarded as "nonprinting" characters.
I would have thought that the separator characters would count as white space

[Strick]: Oops, i forgot to actually use split in my script above.  So now I
[Strick]: Oops, i forgot to actually use split in my script above.  
So now I test four different notions of white, and get three different answers.
I understand why Tcl's builtin list-splitting rules must be fixed, regardless of locale.
But it seems 'split' should use the list-splitting rule or the the 'string is space' rule, 
but it uses its own (pre-unicode?) rule:
$ cat what-chars-does-split-think-are-white.tcl
puts "tcl=[info patch] LANG=$env(LANG)"

puts -nonewline {according to llength: }
puts -nonewline "according to llength: "
  if {[llength [format "/%c/" $i]] > 1} { puts -nonewline "$i " }
}
puts {} 
puts ""
puts -nonewline {according to split: }
puts -nonewline "according to split: "
for {set i 0} {$i<65536} {incr i} {
  if {[llength [split [format "/%c/" $i]]] > 1} { puts -nonewline "$i " }
puts {} 
puts ""
puts -nonewline {according to 'string is space': }
puts -nonewline "according to 'string is space': "
for {set i 0} {$i<65536} {incr i} {
  if {[string is space [format "%c" $i]]} { puts -nonewline "$i " }
puts {} 
puts ""
puts -nonewline {according to regexp {\s}: }
puts -nonewline "according to regexp {\\s}: "
for {set i 0} {$i<65536} {incr i} {
}
puts {}
puts ""
$
$ tclsh what-chars-does-split-think-are-white.tcl
tcl=8.4.7 LANG=en_US.UTF-8
according to llength: 9 10 11 12 13 32
according to split: 9 10 13 32
according to 'string is space': 9 10 11 12 13 32 160 5760 8192 8193 8194 8195 8196 8197 8198 8199 8200 8201 8202 8203 8232 8233 8239 12288
according to regexp {\s}: 9 10 11 12 13 32 160 5760 8192 8193 8194 8195 8196 8197 8198 8199 8200 8201 8202 8203 8232 8233 8239 12288
$
======

[escargo] 2006-01-27: If split used chars 9 10 11 12 13 32 then there would be
''[escargo] 27 Jan 2006'' - If split used chars 9 10 11 12 13 32 then there would be only two sets, with the smaller set as a proper subset of the larger set.  
The two characters that would have to be added are the vertical tab and form feed.
----
Note that the argument named ''splitChars'' above is a series of 0 to n individual characters.  
However, if you want to split on a specific sequence of 2 or more characters together, 
or if you want to split on a regular expression, split will not work for you.  
See [Tcllib]'s [textutil]::[splitx] for that functionality.

[SS] 2004/01/31 - or you can use the following function:
proc wsplit {string sep} {
    set first [string first $sep $string]
    if {$first == -1} {
        return [list $string]
    } else {
        set l [string length $sep]
        set left [string range $string 0 [expr {$first-1}]]
        set right [string range $string [expr {$first+$l}] end]
        return [concat [list $left] [wsplit $right $sep]]
    }
}
======

This version is recursive, so it may be better to rewrite it if you plan to use
This version is recursive, so it may be better to rewrite it if you plan to use the function
against very long strings with many separators. The difference between wsplit and [splitx]
is that [splitx] uses [regexp], so it may create problems with unknown separators.
[IL] 2005-01-03: on the near anniversary of this proc, the iterative version,
[IL] 2005/01/03 - on the near anniversary of this proc, the iterative version, 
quick n dirty since I'm in a hurry to parse some html...
======
proc wsplit { str sepStr } {
   set strList   {}

   set strList   [list]

   while {[set index [string first $sepStr $str]] != -1} {
   while { [set index [string first $sepStr $str]] != "-1" } {

       set left [string range $str 0 [expr $index + $sepLength - 1]]
       set str  [string range $str [expr $index + $sepLength + 1] end]
   }
   return $strList

}
======

connected to the ones you want 

proc wsplit {str sepStr} {
proc wsplit { str sepStr } {

   if { ![regexp $sepStr $str] } { return $str }

   set strList   [list]

   set pattern "(.*?)$sepStr"
   while { [regexp $pattern $str match left] } {

       lappend strList $left
       regsub $pattern $str "" str
   }
    
   lappend strList $str

   return $strList
======

[RS] writes recently:

Note that the wsplit can be done simpler:

    2. split on that single char

proc wsplit {str sep} {
  split [string map [list $sep \0] $str] \0
}
% wsplit This<>is<>a<>test. <>
This is a test.
======

----

''2006-06-21'' [Sarnold] Here is my version of wsplit:
proc wsplit {str sep} {
    set out {} 
    set out ""
    if {$sepLen < 2} {
    if {$sepLen <2} {
    }
    while {[set idx [string first $sep $str]] >= 0} {
        # the left part : the current element
        if {$idx>=0} {
            # the left part : the current element
            lappend out [string range $str 0 [expr {$idx-1}]]
        }
        set str [string range $str [incr idx $sepLen] end]
    }
    # there is no separator anymore, but keep in mind the right part must be
    # there is no separator anymore, but keep in mind the right part must be appended
}
======

----

[escargo]: So what should you use when you don't care how many spaces were
So what should you use when you don't care how many spaces were between tokens,
you just want the nonblank tokens in the list and none of the separators?
''-- [escargo]''
[RS]: Easy, just use a filter:

proc filter {cond list} {
    set res {}
   set res {}
   foreach element $list {if [$cond $element] {lappend res $element}}
   set res
% filter llength [split "a   list   with many   spaces"]
a list with many spaces
======

... or use

% split [regsub -all {[ \t\n]+} "a   list   with many   spaces" { }] 
======


... or use 

% lreplace "a   list   with many   spaces" 0 -1
======



----
See [Counting characters in a string] where [split] was pretty good...
----
Kaitzschu mentions, on [comp.lang.tcl], this piece of info 
which is supposedly somewhere in the man pages:

"If splitChars is an empty string then each character of string 
becomes a separate element of the result list." 
======
Thus, 
proc hexdump str { 
set l [split {abcdefghijklmnopqrstuvwxyz} {}]
======
results in a list where each character is turned into a separate list entry.
----
[RS] 2006-07-04: When you split on "" on a byte array, it may be surprising that the result may contain un[scan]nable characters for \x00 bytes. I had to work around like this:
======
    set res {} 
  set res {} 
  foreach c [split $str ""] { 
     set i [scan $c %c] 
     if {$i eq ""} {set i 0} ;#<--------------------- here 
     lappend res [format %02x $i] 
  } 
  set res 
======

** Protecting Separators **

''[escargo] 18 Jun 2007'' - (We seem to have some edits getting lost; I'm putting this back in
since it seems to have disappeared.)  I was realizing that then I was writing a ''little language''
that there doesn't seem to be a Tcl command that returns a list made from breaking up a string as
the command parser would.  [split] doesn't do it.  It made me think how [subst] has arguments that
disable command substitution and variable substitution, while [eval] does not. There are commands
like [eval], [info complete], '''string is list''' that appear to do the tokenization, but the tokenized results are
not accessible to the script level. Is there some way (apparently not obvious to me) to get the
list form out of the string?
[PYK] 2014-03-02: There was previously a discussion by [escargo] here that made
[MG] Tcl 8.5's [{*}] expansion is probably the easiest way. In 8.4 you'd probably need to use [eval], which would mean a lot of work to escape special characters, make sure things that the parser would see as one word, such as
======
[string is int $foo]
======
become
====== 
"[string is int $foo]"
======
so the eval doesn't split it into separate words, etc. It would be possible, I imagine, but a bit of a pain to do, compared to something like
======
proc str2list {args} {
  return $args
}
set list [str2list {*}$string]
======
in 8.5.
<<categories>> Command | String Processing
''[escargo]'' - Reading the original TIP for {expand}, it appears that it was intended to expand
a list into separate items for processing.  So, using {*} as you did is working around the fact
that something is taking $string and tokenizing it, treating the result as a list, and passing it
to {*} to expand.  Again, it seems, the tokenizing is buried where it is not accessible to a script.

It really seems like there ought to be a way.  Right now split doesn't take any control arguments,
so that '''[[split -tokens $string]]''' wouldn't break any code.

<<discussion>>

<<categories>> Tcl syntax help | Command | String Processing