Version 2 of VFS, exec and command pipelines

Updated 2001-11-26 22:07:16

With Tcl 8.4's vfs, all filesystem access except TclpCreateProcess (and a few related helper functions) is hooked into a lookup table. This means that all filesystem operations visible at the Tcl or C level can operate on a vfs, with the exception of 'exec' and 'open |cmd'.

It would be nice to be able to provide a more general mechanism (*) to such command pipelines, especially given that stdout/stderr/stdin are already nicely abstracted as Tcl_Channels.

The stumbling block appears to be that all the 'pipe' related code doesn't actually use Tcl_Channels. Instead it uses a new 'TclFile' structure which is a lightweight wrapper around a native file-descriptor (and is slightly different on Windows vs Unix). Then there are a whole suite of new functions TclpOpenFile, TclpCloseFile, TclpMakeFile, TclpReleaseFile which are there just to operate on those structures. There is significant code-overlap between these TclFile-specific functions and the more general Tcl_Channel code.

Does anyone know any reason why all those 'TclFile' structures couldn't simply be removed and replaced with Tcl_Channels? If this were done, all of the 'exec' baggage and associated APIs could be made rather simpler and potentially exposed to the Tcl level through the tclvfs extension.

* Obvious benefits are:

  • easier to write things like BLT's 'bgexec' and generally interact with pipelines from within Tcl
  • allow implementation of pipeline components from within Tcl (e.g. pipe output from one application through a Tcl proc and into another application).
  • force all native filesystem access through the 'Tcl_FS' lookup tables, and thereby make it very easy to compile a version of Tcl which can't access the native filesystem (by removing the 'nativeFilesystem' lookup table).

AK: I don't know the true reason, but speculate that the implementation of command pipelines came before channels and there just left as they initially implemented. A problem might be how to implement the OS unnamed pipes between the processes in the pipeline without burdening tcl with doing this.