Version 9 of FIFO

Event-driven I/O

In an interesting c.l.t posting, Colin Macleod describes what it takes to make a Unix FIFO (first in, first out) or named pipe, event-based:

I happen to be working on a project just now where I want to have an event-driven Tcl process read from a fifo. (This is on Solaris 2.6, in case it makes a difference.) My first attempts suggested that this could only be done by polling, but after digging around more I found that I could get fileevent to work as follows:

proc readpipe pipe {
    set data [read $pipe]
    ... process $data ...
}

### Open the fifo, set up event-driven input from it.
proc open_pipe pipename {
    exec /bin/sh -c "sleep 10 > $pipename" &
    set pipe [open $pipename r]
    fconfigure $pipe -blocking 0
    fileevent $pipe readable [list readpipe $pipe]
    set dummy [open $pipename w]
    after 20000 {exec /bin/true}; # to get zombie sleep reaped
}

The tricky bits are:

1. The process opening the fifo to read will block until another process opens it to write. I get past this by exec-ing a shell which opens the pipe, writes nothing, then exits after 10 seconds. Note - it is critical that the "> $pipename" redirection is done by the shell (after it is forked as a new process) and not by Tcl.
2. If the fifo is closed by the writing process, fileevent will fire continuously because it sees end-of-file on the fifo. We can avoid this by making sure that at least one process always has the fifo open for writing, so I also open the fifo for writing from the same Tcl process and never write to it or close it. The 10-second delay in step 1 is to give this time to happen.
3. Finally, I noticed that the exec-ed shell would hang around as a zombie after the sleep finished. The delayed and apparently pointless exec at the end is because running exec again has the side-effect of wait-ing for terminated background processes - documented under Tcl_DetachPids(3).

Lars H, 2008-08-10: My experience on the opening problem is slightly different:

1. Opening a pipe for reading is no problem, provided that you use the {RDONLY NONBLOCK} access mode; open returns immediately, and you can configure the channel as you wish.
2. Opening a pipe for writing is hazardous, because the open will block until the pipe has also been opened for reading; the access mode {WRONLY NONBLOCK} is rejected by the OS (throws error). The work-around is to first open the pipe for reading as above, second open it for writing (no NONBLOCK), and third close the read channel to the pipe.

DKF, 2011-09-30: The WRONLY NONBLOCK combo is correct, but you have to deal with a POSIX ENXIO error by rescheduling the attempt to open the pipe a bit later. I've seen reports that without NONBLOCK you can end up having problems in all threads in a process, but I don't know how much credence to give to those reports; after all, it might be a different bug...

Without suggesting that it is 100% bulletproof, this works fairly well:

## ********************************************************
##
## Name: fifo
##
## Description:
## Provide an anonymous fifo using cat.
##
## Parameters:
##
## Usage:
## trivial handler example:
##
## proc handle { fifo } {
##      puts -nonewline [ gets $fifo ]
## }
##
## set fifo [ fifo handle ]
## puts $fifo peep!
##
## Comments:
## The input side of the fifo should never be flushed!
## The fifo can be closed like an ordinary file.

proc fifo { { handler "" } } {
    if { [ catch {
        set fifo [ open |cat a+ ]
        fconfigure $fifo -blocking off
        fconfigure $fifo -buffering none
        if { [ string length $handler ] } {
            fileevent $fifo readable "$handler $fifo"
        }
    } err ] } {
        return -code error "[ myName ]: $err"
    }
    return $fifo
}
## ********************************************************

(Explain how general file append operations can be detected as events only through polling.)

An example of which can be seen in tailf

fifo module

dbohdan 2019-04-13: The following is a Tcl 8.6 module for working with named pipes. It creates named pipes and disposes of them automatically using the with pattern.

Use example

% fifo::with-fifos -template /tmp/pipe.io pipe1 pipe2 {
    puts [list fifos: $pipe1 $pipe2]
    exec tac $pipe1 &
    exec wc $pipe2 &
    exec echo hello\nworld | tee $pipe1 $pipe2 > /dev/null
}
fifos: /tmp/pipe_gwrbgf.io /tmp/pipe_7kG3E9.io
      2       2      12 /tmp/pipe_7kG3E9.io
world
hello

Source code

# Copyright (c) 2019 dbohdan
# License: MIT

package require Tcl 8.6

namespace eval fifo {
    variable version 0.1.0
}


proc fifo::mkfifo {{template {}}} {
    close [file tempfile path $template]
    file delete $path

    exec mkfifo $path

    return $path
}


proc fifo::with-fifos args {
    set template {}
    if {{-template} eq [lindex $args 0]} {
        set args [lassign $args _ template]
    }

    if {[llength $args] == 0} {
        error "wrong # args: should be\
               \"with-fifos ?-template template? ?varName ...? script\""
    }

    set varNames [lrange $args 0 end-1]
    set script [lindex $args end]

    try {
        foreach varName $varNames {
            upvar 1 $varName v
            set v [mkfifo $template]
        }

        uplevel 1 $script
    } finally {
        foreach varName $varNames {
            upvar 1 $varName v
            file delete $v
        }
    }
}

See also

• chan pipe for a built-in construct for working with unnamed pipes.