Version 1 of AlternativeToExec

Updated 2001-09-28 15:21:22

Tcl's "exec" command is ... I quote Cameron Laird:

 > One thing that is grossly underappreciated is how well
 > [exec] and [open] (and [bgexec]!) work across Unix and
 > Win*.  Unless you've lived through it, you can't imagine
 > how much pain people working with other languages endure
 > to
 >   exec $myprocess << $some_input
 > or
 >   open |$myprocess r+
 > In fact, the reality continues (in 2001!) to be that
 > several leading languages just give up on proper semantics
 > for Win*, because they believe the problems are insur-
 > mountable.

But still, exec has one bad shortcoming, that is nearly impossible to come by:

  • because the meta-arguments for redirection are mixed with normal arguments (for the external command), it is impossible(!) to pass a normal argument which looks like a redirection. This goes as far as that it is impossible to pass a string starting with a `<' or '>' character.
  • because exec wants the command line spread out argument by argument, it is quite a pain (involving eval and all lots of extra [list]-protection of the other arguments) to include a Tcl-list as part of the argument list.
 Example:
 % exec  /bin/echo "<hallo>"
 couldn't read file "hallo>": no such file or directory
 % 

To work towards a solution of this problem, I've proposed an alternative API to the awesome work that made exec possible.

!! I do NOT propose removal nor (incompatible) modification of "exec" !!

Now, there are generally a few possible ways to do it:

  • Use the same bourne-shell-like syntax, but add a way to keep apart the redirection-arguments from normal arguments.
  • Invite a new syntax for redirection that is more tcl'ish, by consisting more of readable words than character sequences.

These items are discussed in the next few sections below.

To me it looks pretty impossible to do it with compatible changes to exec, which means that for compatibility-reasons we won't get around making this a new command.

My original suggestion for the new command name was 'pipe', but in a mail someone expressed a dislike for this name and suggested 'run' / 'bgrun' instead. I could live with these, too :-)


Preserving bourne shell syntax:

 <file, >file, >>file , >&2, 2>&1, |, ...

The simplest way would be to define one marker-argument, e.g. a single or double dash "-", which would remove any meta-syntax meaning of the next argument:

 % pipe /bin/echo - "<hello>" - - >file

would then write the string "<hello> -" to the file named "file". (the doublequotes are actually redundant here, but improve readability)

Although it looks very close to current exec-syntax, it is not compatible; especially not design-bug compatible :-)

   Pro: easy to implement, and the result is safe.
   Con: the syntax is even obscurer than before.

More Tcl'ish syntax:

Like above section, but all the somewhat cryptic symbols are replaced by dash-option (some of them with arguments):

 % pipe /bin/echo "<hello>" - - -outtofile file

The "<Hello>"-strings needs no more protection, the single dash (the second one!) is protected by the preceding one, and -outtofile specifies where the program's stdout should go to.

   Pro: still easy to implement (& safe); clearer meaning, better readability
   Con: Tendence to over-verbosity; a whole large bunch of options to learn.

It should also be possible to mix them, allowing both "<file" and "-filetoin file" ...


Think about a bunch of new features that could be added, when we're at it.

  • embed command and arguments in sublists:
 % pipe {mail -s} - [list $Subject] - $Recipients 

This would save us a lot of dangerous eval's. All dash-protected arguments and all other non-redirection-arguments are concat'ed to form the external command line. Redirection-options and their arguments will not be treated as lists, which makes a big difference to the dangerous eval-constructions so far.

  • add new redirections:

for backgrounded jobs one could add redirection to tcl-channels, by which the functionality of '[open "|..."]' would be subsumed and extended, as stdout and stderr could be separately processed in the Tcl-scripts.

for synchronous jobs, one could redirect stdout and stderr to variables, which is what was most often done to the result of "exec", anyway. If stdout is not redirected, it would be returned as in "exec".

  • tee'ing:

why not duplicate the output of a command to both a variable or tcl-channel AND to the input of the next command in the pipeline ?

  • what else ?

Comment: even more verbose variant

Setok: I fully agree that there are problems with exec, and the proposed system solves those problems. However, I also proposed an alternative syntax:

  pipe {
    inputFromFile filename.txt
    command {wc -l}
  } {
    command {mail -s "wc -l" [email protected]}
  }

This avoids the readability problems you can get with heavy use of dashed options. It's not quite as straightforward as the original proposal but can make complex combinations much cleaner syntactically. The only real issue is that internally substitution would have to be performed on the blocks so you can do the following:

  pipe {
    command [list wc $wcOption]
  }

This can complicate matters a bit for the implementation, but it might be worth it for the cleaner syntax. Naturally I'm willing to hear about any other alternatives...

Andreas Leitgeb: Now that I read this suggestion again, I think it would indeed be nicest to have. Maybe this could be implemented as a tcl-prc wrapper around one of the first two solutions.

it would work like this: for each block, pseudo-commands "inputFromFile", "command", etc. are defined, then the block eval'ed, and the action of each of these pseudo-commands would be to build up a "newexec"-argument-list. the separate blocks are then joined with a "-pipe" or "|" argument.


Thank goodness exec will still exist - I can't imagine ever using this interface, but I have no objection to it existing for others. I just avoid the nasty characters myself...

AvL: does this refer to the whole alternativeexec discussion, or just the later (more verbose) suggestions ? Why do you think it is _posssible_ to avoid nasty chars (short of avoiding exec altogether) ?


You're welcome to add remarks and further suggestions. (Please above this line. Thanks)

One final statement (sorry for the Caps): THIS PROPOSAL IS DOOMED IF NO ONE VOLUNTEERS TO IMPLEMENT IT!


Category Suggestions