[Arjen Markus] (10 august 2006) In the chatroom yesterday [Richard Suchenwirth] came up 
with an elegant solution for the following problem:

Given a string (actually the contents of a file) that contains empty lines (so \n\n),
split it into parts on these empty lines.

You could use [regexp] to do this, but a much more elegant way (IMHO) is 
to replace the substring that you want to split on by a character that is not 
present in the original string and then use [split]. The choice of that character
is of course a bit delicate and the method is limited to ''fixed'' substrings.

Still, the code is simple:

   set list [split [string map [list $substring $splitchar] $string] $splitchar]

Now, what character could you choose for $splitchar? One fascinating choice is
'''\u0080''' - it is part of a region of the UNICODE character map that is 
more or less forbidden or reserved. It means that it is very unlikely to be
present in the original string (unless that is a binary string of course, 
in which case most if not all bets are off, but splitting binary strings is 
a rare and dangerous thing anyway).

If you need to split on a substring that may vary (for instance a sequence
of one or more empty lines), check out the [[split_re]] method in [Tcllib].

----
[WJP] (10 August 2006) \u0080 is fairly safe but you can't be quite sure since it
is a legal Unicode control character. A better choice is \uFFFE or \uFFFF. 
Both are guaranteed not to be characters and so are absolutely safe.
----

[JMN] 2006-11-02 <<br>>
My timings indicate the above method is about 10x faster than [textutil]::[splitx]

However.. Tcl's [split] alone on a single-char separator is 4x faster again.

I'd love to see a multi-character 'split' in the core. [string split] perhaps?

[Lars H], 2008-07-18:
Thinking about that same idea, I believe the following syntax may be appropriate for a [split] extended that way:

    :   '''split''' ''text'' ?''string'' ''list'' ...? ?''chars''?

The ''text'' is the string to split. Each ''string'' is a (possibly multicharacter) string at which it might be split, and when such a split occurs, the elements of the following ''list'' are appended to the result before processing resumes on the rest of the string. The ''chars'' are split-characters, as in current Tcl.

The idea of the ''list'' arguments is that sometimes you want to split on several substrings, but you also want to know which one it was at each position; current [split] doesn't provide that information. With the above you could do
  split {1+1-3+4-2} + plus - minus
to get
  1 plus 1 minus 3 plus 4 minus 2
or
  split {1+1-3+4-2} + {} - minus
to get
  1 1 minus 3 4 minus 2
However, one could probably use clever combinations of [regexp] -all, [regsub], and/or [string map] to get this effect as well, so it's no great leap in expressive power.


<<categories>> String Processing | Parsing