Version 21 of open

http://www.purl.org/tcl/home/man/tcl8.4/TclCmd/open.htm

Examples of open include:

 set fd1 [open "simpleinput" "r"]    ;# Read the input file
 set fd2 [open "reldirect/simpleoutput" "w"] ;# Write the output 
 set fd3 [open "/full/path/file" "r+"] ;# Read and write file
 set fd4 [open "/tmp/output" "wb"] ;# Windows only: don't do newline translation
 set fd5 [open "|simple1" "r"] ;# Read the stdout of simple1 as fd5
 set fd6 [open "|simple2" "w"] ;# Writes to fd6 are read by simple2 on its stdin
 set fd7 [open "|simple1 |& cat" "r"] ;# Results in stdout AND stderr being readable via fd7
 set fd8 [open "|simple2" "r+"] ;# Writes to fd8 are read by simple2 on its stdin, whereas reads will see the stdout of simple2

Are there other examples?

BR - Access mode "wb" doesn't work with any version of Tcl with which I tried it. It isn't really needed either as we have fconfigure for that.

One nice feature of Tcl is that one can not only open files, but at least on Unix and Windows, one can also open pipelines to other processes.

(does "pipeline" mean "I can write to this and the other process will read it", or does it also mean "I can read from this what the other process writes" ?) Both. The exec and open commands make a remarkable pair. No other common language has anything approaching their portability, maturity, and applicability.

Arjen Markus I have experimented a bit with plain Tcl driving another program. As this needs to work on Windows 95 (NT, ...) as well as UNIX in four or five flavours, I wanted to use plain Tcl, not Expect (however much I would appreciate the chance to do something really useful with Expect - apart from Android :-).

I think it is worth a page of its own, but here is a summary:

Open the pipeline and make sure buffering is minimal via

        set inout [open "|myprog" "r+"]
        fconfigure $inout -buffering line

Buffering might be out of your hands, though, for "real 16-bit commandline applications", which apparently don't have the ability to flush (reliably) except on close.

Set up fileevent handlers for reading from the process and reading from stdin.
Make sure the process ("myprog" above) does not buffer its output to stdout!

Neil Madden - here is a real-life example of interacting with a program through a pipe. The program in question is ispell - a UNIX spell-checking utility. I use it to spell-check the contents of a text widget containing LaTeX markup. There are a number of issues to deal with:

Keeping track of the position of the word in the widget.
Filtering out useless (blank) lines from ispell
Filtering out the version info that my version of ispell dumps out.
When passing in a word which is a TeX command (e.g. \maketitle), ispell returns nothing at all.
Careful handling of blocking.

The example does not use fileevent, as this would complicate this particular example. The options passed to ispell are -a (which makes it non-interactive) and -t (which makes it recognize TeX input).

 set contents [split [$text get 1.0 end] \n]
 set pipe [open "|ispell -a -t" r+]
 fconfigure $pipe -blocking 0 -buffering line
 set ver [gets $pipe] ;# Ignore the initial version line
 set linenum 1
 foreach line $contents {
     set wordnum 1
     foreach word [split $line] {
         puts $pipe $word   ;# Feed word to ispell
         while {1} {
             set len [gets $pipe res]
             if {$len > 0} {
                 # A valid result
                 # do stuff
                 break
             } else {
                 if {[fblocked $pipe]} {
                     # No output
                     break
                 } elseif {[eof $pipe]} {
                     # Pipe closed
                     catch {close $pipe}
                     return
                 }
                 # A blank line - skip
             }
         }
         incr wordnum
     }
     incr linenum
 }

Thanks to Kevin Kenny for helping me figure this out.

Note that specifying "a" for the open argument may NOT give you what you expect (a violation of Tcl's normal least surprise policy). The "a" has specialty code that does some sort of simulation of append by seeking to the end of file. This results in behavior quite different than one gets when doing appends in other languages.

Could you please explain what this difference which you find so important is? This is not clear from what you write. Also, is this something which is platform independent, or is it rather something which is only seen on e.g. POSIX?

So, instead of doing "a" or "a+", you should do

 set f [open $theLogFile {WRONLY CREAT APPEND}] ;# instead of "a"

 set f [open $theLogFile {RDWR CREAT APPEND}]   ;# instead of "a+"

to get the standard behavior of other languages.

Thanks to dkf for this tip.

DKF - Here's how to tell whether you've got a problem. Open a file in append mode (with "a" or "a+") several times and write a single line to the file each time (I'd suggest writing digits here; that's easy to understand!) Do not explicitly flush or close those file handles. Instead, exit the program; the language's shut-down code should handle the flushing and closing for you. Then look at the contents of the file produced

How to know if the right thing happened? Simple. Check if all the values you wrote actually made it to the file. Order does not matter here. If you got all the lines written, that is good. Your language is doing proper (thread- and process-safe) appending, and this is one of the things that must be supportable on platforms that do POSIX. It's also one of the things that tends to get supported early because it is useful for not losing data from system logs...

But what if not all the data makes it? Well, that's really bad because it means that if you had two programs writing to a log file, you might lose some of the log messages. It happens in Tcl because Tcl tries to do a seek() to the end of the file when it is first opened and then presumes that's good enough. Instead, we get a race condition. Using the APPEND flag (which translates into the low-level O_APPEND flag to the open() syscall) alters the semantics of writes so that a seek-to-end happens as an atomic part of the write itself, and that ensures that no data is ever lost (though of course apps still need to take care to only ever write complete records/lines to the file.)