Version 28 of open

http://www.purl.org/tcl/home/man/tcl8.4/TclCmd/open.htm

Examples of open include:

 set fd1 [open "simpleinput" "r"]    ;# Read the input file
 set fd2 [open "reldirect/simpleoutput" "w"] ;# Write the output 
 set fd3 [open "/full/path/file" "r+"] ;# Read and write file

 set fd4 [open "/tmp/output" "wb"] ;# Windows only: don't do newline translation
                        NOTE: This mode no longer appears in the documentation
                                and can't be found in the source code either.

 set fd5 [open "|simple1" "r"] ;# Read the stdout of simple1 as fd5
 set fd6 [open "|simple2" "w"] ;# Writes to fd6 are read by simple2 on its stdin
 set fd7 [open "|simple1 |& cat" "r"] ;# Results in stdout AND stderr being readable via fd7
 set fd8 [open "|simple2" "r+"] ;# Writes to fd8 are read by simple2 on its stdin, 
                                 # whereas reads will see the stdout of simple2

Are there other examples?

BR - Access mode "wb" doesn't work with any version of Tcl with which I tried it. It isn't really needed either as we have fconfigure for that.

Vince -- it would only take a few lines of code added to Tcl's core to automatically interpret "b" for binary.

kennykb reports in the chat room that he recalls it is from Tcl 7.3 days or earlier.

One nice feature of Tcl is that one can not only open files, but at least on Unix and Windows, one can also open pipelines to other processes.

(does "pipeline" mean "I can write to this and the other process will read it", or does it also mean "I can read from this what the other process writes" ?) Both. The exec and open commands make a remarkable pair. No other common language has anything approaching their portability, maturity, and applicability.

Arjen Markus I have experimented a bit with plain Tcl driving another program. As this needs to work on Windows 95 (NT, ...) as well as UNIX in four or five flavours, I wanted to use plain Tcl, not Expect (however much I would appreciate the chance to do something really useful with Expect - apart from Android :-).

I think it is worth a page of its own, but here is a summary:

Open the pipeline and make sure buffering is minimal via

        set inout [open "|myprog" "r+"]
        fconfigure $inout -buffering line

Buffering might be out of your hands, though, for "real 16-bit commandline applications", which apparently don't have the ability to flush (reliably) except on close.

Set up fileevent handlers for reading from the process and reading from stdin.
Make sure the process ("myprog" above) does not buffer its output to stdout!

Neil Madden - here is a real-life example of interacting with a program through a pipe. The program in question is ispell - a UNIX spell-checking utility. I use it to spell-check the contents of a text widget containing LaTeX markup. There are a number of issues to deal with:

Keeping track of the position of the word in the widget.
Filtering out useless (blank) lines from ispell
Filtering out the version info that my version of ispell dumps out.
When passing in a word which is a TeX command (e.g. \maketitle), ispell returns nothing at all.
Careful handling of blocking.

The example does not use fileevent, as this would complicate this particular example. The options passed to ispell are -a (which makes it non-interactive) and -t (which makes it recognize TeX input).

 set contents [split [$text get 1.0 end] \n]
 set pipe [open "|ispell -a -t" r+]
 fconfigure $pipe -blocking 0 -buffering line
 set ver [gets $pipe] ;# Ignore the initial version line
 set linenum 1
 foreach line $contents {
     set wordnum 1
     foreach word [split $line] {
         puts $pipe $word   ;# Feed word to ispell
         while {1} {
             set len [gets $pipe res]
             if {$len > 0} {
                 # A valid result
                 # do stuff
                 break
             } else {
                 if {[fblocked $pipe]} {
                     # No output
                     break
                 } elseif {[eof $pipe]} {
                     # Pipe closed
                     catch {close $pipe}
                     return
                 }
                 # A blank line - skip
             }
         }
         incr wordnum
     }
     incr linenum
 }

Thanks to Kevin Kenny for helping me figure this out.

Note that specifying "a" for the open argument need NOT be the optimal way of opening a file for appending (a violation of Tcl's normal least surprise policy). The "a" works by seeking to the end of file, whereas in some other languages the vanilla append utilizes a special service provided by the OS. The respecive behaviors can be quite different.

DKF - Here's how to tell whether you've got a problem. Open a file in append mode (with "a" or "a+") several times and write a single line to the file each time (I'd suggest writing digits here; that's easy to understand!) Do not explicitly flush or close those file handles. Instead, exit the program; the language's shut-down code should handle the flushing and closing for you. Then look at the contents of the file produced

How to know if the expected behavior happened? Simple. Check if all the values you wrote actually made it to the file. Order does not matter here. If you got all the lines written, that is good. Your language is doing proper (thread- and process-safe) appending, and this is one of the things that must be supportable on platforms that do POSIX. It's also one of the things that tends to get supported early because it is useful for not losing data from system logs...

But what if not all the data makes it? Well, that's really bad because it means that if you had two programs writing to a log file, you might lose some of the log messages. It happens in Tcl because Tcl tries to do a seek() to the end of the file when it is first opened and then presumes that's good enough. Instead, we get a race condition. Using the APPEND flag (which translates into the low-level O_APPEND flag to the open() syscall) alters the semantics of writes so that a seek-to-end happens as an atomic part of the write itself, and that ensures that no data is ever lost (though of course apps still need to take care to only ever write complete records/lines to the file.)

Thanks to dkf for this tip. So, instead of doing "a" or "a+", you currently need to

  set f [open $theLogFile {WRONLY CREAT APPEND}] ;# instead of "a"

  set f [open $theLogFile {RDWR CREAT APPEND}]   ;# instead of "a+"

to avoid race conditions.

Does Tcl's current (8.4.1.1 and earlier) behavior of "a" have some benefit over the behavior of APPEND? Or should this difference be considered a bug that is fixed?

Some additional discussion of the stderr gimmick above is probably appropriate.