'''`[http://www.tcl.tk/man/tcl/TclCmd/open.htm%|%open]`''', a [Tcl Commands%|%built-in] Tcl [command], opens an I/O [channel]. ** See Also ** [gets]: [read]: [file]: [puts]: [catch]: [stderr]: [pipeline]: [Example of reading and writing to a piped command]: `[chan pipe]`: [Execute in Parallel and Wait]: [http://groups.google.com/d/msg/comp.lang.tcl/kooDD1YlIIo/Po087UI0BNQJ%|%(Bug Tcl 8.1b1) console1 wasn't opened for writing under win32] [comp.lang.tcl], 1999-02-03: As true in 2013 as it was in 1999. ** Change Log ** `wb` appeared in Tcl 8.5. [kennykb] reports in the chat room that he recalls it was also available in Tcl 7.3 days or earlier. ** Synopsis ** : '''open''' ''fileName'' ?''mode''? ?''creationPermissions''? ** Documentation ** [http://www.tcl.tk/man/tcl/TclCmd/open.htm%|%official reference]: [http://www.tcl.tk/man/tcl8.5/tutorial/Tcl26.html%|%Running other programs from Tcl - exec, open]: [TIP] [http://www.tcl.tk/cgi-bin/tct/tip/183.html%|%183]: Add a Binary Flag to `open`: ** Description ** If `b` is appended to ''mode'' it has the same effect as `[fconfigure] ... -translation binary` ** Examples ** ====== set fd0 [open simpleinput] ;# open the file for reading set fd1 [open simpleinput r] ;# open the file for reading set fd2 [open reldirect/simpleoutput w] ;# open the file for writing set fd3 [open /full/path/file r+] ;# open the file to be Read and written set fd4 [open /tmp/output {WRONLY BINARY}] ;# Since Tcl 8.5 [http://tip.tcl.tk/183]: open in binary mode for writing set fd4b [open /tmp/output wb] ;# should be equivalent to the fd4 example. set fd5 [open |[list simple1] r] ;# Read the stdout of simple1 as fd5, stderr is discarded set fd6 [open |[list simple2] w] ;# Writes to fd6 are read by simple2 on its stdin set fd7 [open |[list simple1 |& cat] r] ;# Results in stdout AND stderr being readable via fd7 set fd7a [open |[list simple1 2>@1] r] ;# Same effect as fd7 set fd8 [open |[list simple2] r+] ;# Writes to fd8 are read by simple2 on its stdin, # whereas reads will see the stdout of simple2, # stderr is discarded set fd9 [open |[list simple1 2>@stderr] r] ;# stdout readable via fd9, stderr passed-thru to stderr of parent. ====== ---- One variation on the fd8 example above which can be useful for debugging is ====== set fd8 [open [list | tee tmp/in | simple2 | tee tmp/out 2>tmp/err] r+] ====== In addition to running '''simple2''' as an asynchronous child process, this also creates three files tmp/in, tmp/out, and tmp/err to which will be written everything that passes over simple2's stdin, stdout, and stderr respectively. `tee` here is a [Unix] utility that copies everything from stdin to stdout, but optionally also writes all of it to the files given as arguments. ---- [AMG]: Here's a one-liner for creating an empty file: ====== close [open $filename w] ====== [glennj]: see also "touch" from the tcllib package, `[fileutil]`. ** Command Pipelines ** Command pipelines executed by `[open]` generate errors exactly as described for `[exec]`, and `[close]` returns the error when it receives the channel returned by `[open]` as an argument. However, there is no `-ignorestderr` option. One way to work around that is to redirect stderr, perhaps straight back to stderr: `>@stderr`. ---- [AMG]: I prefer to construct the first argument to `open` thus: `|[[[list] progname arg1 arg2 arg3 ...]`. This protects against whitespace embedded in the arguments from futzing up the works. Even though there's no problem with your command line, one day you might change it, perhaps to use a parameter to your proc as an argument. So I just do it "right", right from the start, to prevent forgetting to make the change in the future. It's like the problem with optional { braces } in C: they're not needed for one-line `if`/`for`/`while`/etc. bodies, but when you add another line you might forget to add the braces. ** Capture [stdout], [stderr] and Child Exit Status ** The following procedure invokes an external command and returns the output, the diagnostic output, and the exit code of that command. ====== proc invoke command { lassign [chan pipe] chanout chanin lappend command 2>@$chanin set fh [open |$command] set stdout [read $fh] close $chanin set stderr [read $chanout] close $chanout if {[catch {close $fh} cres e]} { dict with e {} lassign [set -errorcode] sysmsg pid exit if {$sysmsg eq {NONE}} { #output to stderr caused [close] to fail. Do nothing } elseif {$sysmsg eq {CHILDSTATUS}} { return [list $stdout $stderr $exit] } else { return -options $e $stderr } } return [list $stdout $stderr 0] } ====== Example: ====== set command { puts stdout {hi to stdout} puts stderr {hi to stderr} exit 42 } lassign [invoke [list tclsh <<$command]] stdout stderr exit puts "stdout: $stdout" puts "stderr: $stderr" puts "exit: $exit" ====== Here is a similar procedure, which accomplishes almost the same thing as the first, except that in one case, it leaves an artifact in the stderr output. If the child process generates nothing to stderr but has an exit status other than `0`, `exec` writes the following message to stderr: "child process exited abnormally": ====== proc invoke command { set fh [open |$command] set stdout [read $fh] set stderr {} set status [catch {close $fh} stderr e] if {$status} { dict with e {} lassign [set -errorcode] sysmsg pid exit if {$sysmsg eq {NONE}} { #output to stderr caused [close] to fail. Do nothing } elseif {$sysmsg eq {CHILDSTATUS}} { return [list $stdout $stderr $exit] } else { return -options $e $stderr } } return [list $stdout $stderr 0] } ====== ---- Another method is to redirect `stderr` to `stdout`, so that it is only necessary to capture one stream: ====== set fd1 [open |[list somecmd 2>@stdout] r] ====== Here's a quick test sequence that takes advantage of `[close]` returning the standard error for ''blocking'' pipes: ====== set somecmd {sh -c { #this is sh, not Tcl syntax echo "to stdout" ; echo >&2 "to stderr" }} set errorCode {} puts {no redirection:} set f [open [list | {*}$somecmd] r] set std_out [read -nonewline $f] catch {close $f} std_err puts " std_out is {$std_out}" puts " std_err is {$std_err}" puts " errorCode is {$errorCode}" set errorCode {} puts redirected: set f [open [list | {*}$somecmd 2>@stdout] r] set std_out [read -nonewline $f] catch {close $f} std_err puts " std_out is {$std_out}" puts " std_err is {$std_err}" puts " errorCode is {$errorCode}" ====== The output should be: ====== no redirection: std_out is {to stdout} std_err is {to stderr} errorCode is {NONE} redirected: std_out is {to stdout to stderr} std_err is {} errorCode is {} ====== ---- ** Asynchronously Capture [stdout], [stderr] and Child Exit Status ** The following code is a template for executing an external program and collecting [stdout], [stderr], and the exit code, all asynchronously. This example is also available in [ycl]::exec. ====== #! /bin/env tclsh proc invoke {command {onexit {}} {onstdout puts} {onstderr {puts stderr}} {read {}}} { lassign [chan pipe] stderr stderrin lappend command 2>@$stderrin set stdout [open |$command] set handler1 [namespace current]::[info cmdcount] coroutine $handler1 handler $onstdout $onstderr $onexit fileevent $stderrin writable [list apply {{stdout stderrin} { if {[chan names $stdout] eq {} || [eof $stdout]} { close $stderrin } }} $stdout $stderrin] #warning: under the most recent Tcl release (8.6.1), any errors in handler #1 will not be reported via bgerror, but will silently disrupt the program. #For the status of this bug, see #http://core.tcl.tk/tcl/tktview/ef34dd2457472b08cf6a42a7c8c26329e2cae715 fileevent $stdout readable [list $handler1 [list stdout $stdout]] fileevent $stderr readable [list $handler1 [list stderr $stderr]] return $stdout } proc handler {onstdout onstderr onexit {read gets}} { set done {} lassign [yield [info coroutine]] mode chan while 1 { if {[set data [{*}$read $chan]] eq {}} { if {[eof $chan]} { lappend done $mode if {[catch {close $chan} cres e]} { dict with e {} lassign [set -errorcode] sysmsg pid exit if {$sysmsg ne "CHILDSTATUS"} { return -options $e $cres } } else { if {![info exists exit]} { set exit 0 } } if {[llength $done] == 2} { if {$onexit ne {}} { after 0 [list {*}$onexit $exit] } return } else { lassign [yield] mode chan } } else { lassign [yield] mode chan } } else { {*}[set on$mode] $data lassign [yield] mode chan } } } ====== Example: ====== set command { puts stdout {hi to stdout} puts stderr {hi to stderr} exit 42 } set chan [invoke [list tclsh <<$command] {apply {{code} { set ::exit $code }}}] puts "pid: [pid $chan]" vwait exit exit $exit ====== ** Capture [stdout] and [stderr] Using an External "Pump" ** [Alexandre Ferrieux]: In [TIP] [http://tip.tcl.tk/304%|%304], [Alexandre Ferrieux] writes: A popular workaround for script-only purists who want to get at the [stderr] of a command is to spawn an external "pump" like '''cat''' in an `open ... r+`, and redirect the wanted stderr to the write side of the pump. Its output can then be monitored through the read side: ====== set pump [open |cat r+] set f1 [open [list | cmd args 2>@ $pump] r] fileevent $f1 readable got_stdout fileevent $pump readable got_stderr ====== Now this is all but elegant of course, difficult to deploy on Windows (where you need an extra cat.exe), and not especially efficient since the "pump" consumes context switches and memory bandwidth only to emulate a '''single''' OS [pipe] when Tcl is forced to create '''two''' of them via `open ... r+`. [MSW]: Note that '''piping''' stderr and stdout simultaneously won't work for all operating systems. I remember having problems on solaris with lost data through the pipes (one filled up while the other wasn't drained?). I had to redirect stuff to files and watch the files to follow the output (invoked program in question was cvs). Furthermore buffering differences hurt when you try to simultaneously catch err and out. I did not try the exact proposed solution above. Someone with [Solaris] 8 and 9 might want to test this (don't have access to one atm). ** Caveat: mode `a` ** [DKF], et al: For mode `a`, Tcl seeks (seek()) to the end of the file immediately upon opening the file, and presumes that's good enough, but '''that can cause a [race condition].''' if another program then appends to the file before this one does. If the system does proper (thread-safe and process-safe) appending, as required by [POSIX], it's not a problem. Good append behaviour tends to get supported early in a system becuase it is useful for not losing data from the system logs. Here's how to detect a problematic system: Open a file in append mode (with `a` or `a+`) several times and write a single line to the file each time (I'd suggest writing digits here; that's easy to understand!) Do not explicitly flush or close those file handles. Instead, exit the program; the language's shut-down code should handle the flushing and closing for you. Then look at the contents of the file produced. Check if all the values you wrote actually made it to the file. Order does not matter here. If you got all the lines written, that is good. On a problematic system, using the `APPEND` flag, which translates into the low-level O_APPEND flag to the ''PrivoxyWindowOpen()'' syscall, alters the semantics of writes so that a seek-to-end happens as an atomic part of the write itself, and that ensures that no data is ever lost, though of course apps still need to take care to only ever write complete records/lines to the file. Example: ====== set f [open $theLogFile {WRONLY CREAT APPEND}] ;# instead of "a" ====== or ====== set f [open $theLogFile {RDWR CREAT APPEND}] ;# instead of "a+" ====== Does Tcl's current (8.4.1.1 and earlier) behavior of `a` have some benefit over the behavior of APPEND? Or should this difference be considered a bug that is fixed? [FW] seconds this question. [LES]: 2005-04-11: This page's last update was stamped "2003-09-01" when I came here now and changed it. Has that question been answered yet? [Lars H]: I'll hazard an answer. Perhaps the benefit of `a` over `APPEND` is that it is ''less surprising''?! Imagine a program that appends some data to e.g. a list of changes and then seeks to the start to write an updated timestamp. If someone went in and replaced that "a" with {WRONLY CREAT APPEND} then the surprising result would be that all timestamps ended up at the end of the file, because every puts would seek to the end, would it not? [DKF]: If you need random access to a file, you shouldn't use "a". You certainly wouldn't be doing this if you were using stdio. ** Roll Your Own Pipeline ** [PYK] 2015-04-27: As of [Changes in Tcl/Tk 8.6%|%8.6] it's possible to `[close]` only one side of a channel. Among other things, this can be use to harness another program as a data filter: ====== set data {one two four} set chan [open |md5sum w+b] puts $chan $data close $chan write set signature [read $chan] close $chan ====== ** Windows Alternative Data Streams ** Windows features the ability to store multiple data streams into one file: ====== foreach stream {one two} { set fh [open myfile:$stream w] puts $fh "data in stream $stream" close $fh } foreach stream {one two} { set fh [open myfile:$stream] puts [read $fh] close $fh } file stat myfile stats puts $stats(size) ====== output: ======none data in stream one data in stream two 0 ====== Since there is no data in the primary stream, the size of the file is reported to be `0`. In a [MS-DOS] shell, specifying a file name ending in a colon causes the primary stream to be used: ====== rem this is cmd.exe echo hello > one: type one ====== On Tcl, an attempt to open a file whose name ends with `:` is an error. ** Serial Ports ** `[fconfigure]` options that are specific for a certain channel type are documented on the man page for the command creating that type of channel. Hence the `''-mode''` option, for serial ports, is documented on the reference page for `open`. ** Examples: [Alpha] ** In [Alpha], there are many real-life examples of communicating with external processes using pipes. The examples include * aspell * tex, bibtex, dvips, gs (in fact it is altogether a very fancy Tcl frontend to tex, featuring many Tcl tricks like regexping log files to figure out what to do next, etc.) * maxima (cf. described on this Wiki at [Maxima through a pipe]) * coq as well as standard shells. There are several pages dedicated to these issues on the [http://alphatcl.sourceforge.net/wikit/%|%AlphaTcl Wiki]. see notably [http://alphatcl.sourceforge.net/wikit/180%|%Alpha and Unix], which is an overview page. The problems discussed include how to capture stderr, how to filter out prompts and transform an asynchronous interaction into a synchronous one with timeout mechanisms using vwait, how to link a process to a window, how to provide active cells in an Alpha document, a sort of worksheet interface to an external process, how to provide step functionality (stepping through a list of commands, at each step seeing the result in a console), and a check-as-you-type spellchecker (using aspell). Recall that [Alpha] is a text editor, just as powerful and configurable as [Emacs], but having Tcl as extension language instead of [lisp]! The AlphaTcl library (http://alphatcl.sf.net) is nearly 20000 lines of Tcl code, full of gems and useful tricks. AlphaX runs on OSX, AlphaTk (written entirely in TclTk) runs on any platform. ** [Microsoft Windows%|%Windows] Hidden Files ** Attempting to open a hidden file in `w` mode will fail: ======none set fid [open testhidden w] close $fid file attributes testhidden -hidden 1 set fid [open testhidden w] ====== Output: ====== couldn't open "testhidden": permission denied ====== However, opening such a file in `r+` mode will succeed. This has to do with what flags `open` passes to the [Microsoft Windows%|%Windows function, [http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx%|%CreateFile]. One relevent excerpt from that documentation: : `If CREATE_ALWAYS and FILE_ATTRIBUTE_NORMAL are specified, CreateFile fails and sets the last error to ERROR_ACCESS_DENIED if the file exists and has the FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM attribute. To avoid the error, specify the same attributes as the existing file.` See also: [http://msdn.microsoft.com/en-us/library/windows/desktop/aa363858%28v=vs.85%29.aspx%|%CreateFile]: Windows API function [http://stackoverflow.com/questions/13215716/ioerror-errno-13-permission-denied-when-trying-to-open-hidden-file-in-w-mod%|%IOError: [Errno 13] Permission denied when trying to open hidden file in “w” mode], stackoverflow, 2012-11-04: ** Unanswered Question: Plink ** [WangFan]: I use "catch {open "|plink -ssh $account@$host" w+} input}", in which "plink" is download from http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html. When pipe is readable, I read the pipe, but I always get NULL. Does anybody know why? The tcl script is shown below: ====== package require Tk frame .t text .t.log pack .t.log pack .t set account user set host 10.1.1.2 if [catch {open "|plink -ssh $account@$host" w+} input] { .t.log insert end $input\n return 0 } else { fconfigure $input -blocking 0 -buffering line -translation crlf -eofchar {} fileevent $input readable Log } proc Log {} { global input if [eof $input] { Stop } else { gets $input line .t.log insert end $line\n .t.log see end } return } proc Stop {} { global input catch {close $input} return } ====== <> Command | File | Channel | Interprocess Communication | Tcl syntax help | Arts and crafts of Tcl-Tk programming