Tcl Tutorial Lesson 26

Running other programs from Tcl - exec, open

So far the lessons have dealt with programming within the Tcl interpreter. However, Tcl is also useful as a scripting language to tie other packages or programs together. To accomplish this function, Tcl has two ways to start another program:

• open - run a new program and open a file-like connection to this program.
• exec - run a new program as a more or less independent subprocess

The open call is the same call that is used to open a file. If the first character in the file name argument is a "pipe" symbol (|), then open will treat the rest of the argument as a program name, and will run that program with the standard input or output connected to a file descriptor. This "pipe" connection can be used to read the output from that other program or to write fresh input data to it or both.

If the "pipe" is opened for both reading and writing, you must be aware that the pipes are buffered. The output from a puts command will be saved in an I/O buffer until the buffer is full, or until you execute a flush command to force it to be transmitted to the other program. The output of this other program will not be available to read or gets until its output buffer is filled up or flushed explicitly.

(Note: As this is internal to this other program, there is no way that your Tcl script can influence that. The other program simply must cooperate. Well, that is not entirely true: the Expect extension actually works around this limitation by exploiting deep system features.)

The exec call is similar to invoking a program (or a set of programs piped together) from the prompt in an interactive shell, a DOS batch file or a UNIX/Linux shell script. It supports several styles of output redirection, or it can return the output of the other program(s) as the return value of the exec call.

open |program ?access?

Returns a file descriptor for the pipe. The program argument must start with the pipe symbol. If program is enclosed in quotes or braces, it can include arguments to the subprocess.

exec ?switches? arg1 ?arg2? ... ?argN?

exec treats its arguments as the names and arguments for a set of programs to run. If the first args start with a "-", then they are treated as switches to the exec command, instead of being invoked as subprocesses or subprocess options. switches are:

• -keepnewline retains a trailing newline in the pipeline's output. Normally a trailing newline will be deleted.
• -- marks the end of the switches. The next string will be treated as arg1, even if it starts with a "-"

The arguments to the exec command, arg1 ... argN can be one of:

• the name of a program to execute
• a command line argument for the subprocess
• an I/O redirection instruction.
• an instruction to put the new program in the background:

   exec myprog &

will start the program myprog in the background, and return immediately. There is no connection between that program and the Tcl script, both can run on independently. The & must be the last argument - you can use all other types of arguments in front of it.

NOTE: Add information on how to wait for the program to finish?

There are many I/O redirection commands. The main subset of these commands is:

|

Pipes the standard output of the command preceding the pipe symbol into the standard input of the command following the pipe symbol.

< filename

The first program in the pipe will read input from filename.

<@ fileID

The first program in the pipe will read input from the Tcl descriptor fileID. fileID is the value returned from an open ... "r" command.

<< value

The first program in the pipe will read value as its input.

> filename

The output of the last program in the pipe will be sent to filename. Any previous contents of filename will be lost.

>> filename

The output of the last program in the pipe will be appended to filename.

2> filename

The standard error from all the programs in the pipe will be sent to filename. Any previous contents of filename will be lost.

2>> filename

The standard error from all the programs in the pipe will be appended to filename.

>@ fileID

The output from the last program in the pipe will be written to fileID. fileID is the value returned from an open ... "w" command.

If you are familiar with shell programming, there are a few differences to be aware of when you are writing Tcl scripts that use the exec and open calls.

• You don't need the quotes that you would put around arguments to escape them from the shell expanding them. In the example, the argument to the sed command is not put in quotes. If it were put in quotes, the quotes would be passed to sed, instead of being stripped off (as theshell does), and sed would report an error.
• If you use the open |cmd "r+" construct, you must follow each puts with a flush to force Tcl to send the command from its buffer to the program. The output from the program itself may be buffered in its output buffer. You can sometimes force the output from the external program to flush by sending an exit command to the process. You can also use the fconfigure command to make a connection (channel) unbuffered.
• This will fail - there is most probably no file with the literal name "*.tcl":

 exec ls *.tcl

• To pass a list of files, based on such a pattern use the {*} prefix, it forces the list to become individual arguments:

 exec ls {*}[glob *.tcl]

• If one of the commands in an exec call fails to execute, the exec will return an error, and the error output will include the last line describing the error. The exec treats any output to standard error to be an indication that the external program failed. This is simply a conservative assumption: many programs behave that way and they are sloppy in setting return codes. Some programs however write to standard error without intending this as an indication of an error. You can guard against this from upsetting your script by using the catchcommand:

 if { [catch { exec ls *.tcl } msg] } {
     puts "Something seems to have gone wrong but we will ignore it"
 }

As already mentioned, the Expect extension to Tcl provides a very powerful interface to other programs, which in particular handles the buffering problem. NOTE: add good reference to expect.

If one of the commands in an open |cmd fails, the open does not return an error. However, attempting to read input from the file descriptor with gets $file will return an empty string. Using the gets $file input construct will return a character count of -1.

To inspect the return code from a program and the possible reason for failure, you can use the global errorInfo variable:

 if { [catch { exec ls *.tcl } msg] } {
     puts "Something seems to have gone wrong:"
     puts "Information about it: $::errorInfo"
 }

Example

 #
 # Write a Tcl script to get a platform-independent program:
 #
 # Create a unique (mostly) file name for a Tcl program
 set TMPDIR "."
 if { [info exists ::env(TMP)] } {
      set TMPDIR $::env(TMP)
 }
 set tempFileName [file join $TMPDIR invert_[pid].tcl]

 # Open the output file, and # write the program to it
 set outfl [open $tempFileName w]
 puts $outfl {
     set len [gets stdin line]
     if {$len < 5} {exit -1}
     for {set i [expr {$len-1}]} {$i >= 0} {incr i -1} {
         append invertedLine [string range $line $i $i]
     }
     puts $invertedLine
     exit 0
 }

 # Close the file
 close $outfl

 #
 # Run the new Tcl script:
 #
 # Open a pipe to the program (for both reading and writing: r+)
 #

 set io [open "|[info nameofexecutable] $tempFileName" r+]

 #
 # Send a string to the new program
 #     *MUST FLUSH*

 puts $io "This will come back backwards."
 flush $io

 # Get the reply, and display it.

 set len [gets $io line]
 puts "To invert: 'This will come back backwards.'"
 puts "Inverted is: $line"
 puts "The line is $len characters long"

 # Run the program with input defined in an exec call

 set invert [exec [info nameofexecutable] $tempFileName << \
         "ABLE WAS I ERE I SAW ELBA"]

 # Display the results
 puts "The inversion of 'ABLE WAS I ERE I SAW ELBA' is \n $invert"

 # Clean up
 file delete $tempFileName

To invert: 'This will come back backwards.'
Inverted is: .sdrawkcab kcab emoc lliw sihT
The line is 30 characters long
The inversion of 'ABLE WAS I ERE I SAW ELBA' is
 ABLE WAS I ERE I SAW ELBA