tclvfs is an excellent facility; however, there are some gotchas (as you'd expect for something that exposes the soft underbelly of tcl's file system).
This page is for folklore and here be dragons advice. Please track changes to any folklore you relate here, so it can be a useful resource for implementors and not a collection of old wives' tales.
Also please file bug reports or feature requests for problems you run into where you believe there is a better or more correct behaviour.
Also please make sure you develop with the very latest cvs version of tclvfs. It's a continuously improving system. Old versions can crash on unusual cases, you can waste a lot of time if you don't track the CVS for development.
SEH -- The fileattributes vfs handler is a potential performance land mine. If you do [file attributes $file] on a file in a virtual filesystem, the vfs extension fulfulls it by making a call to the fileattributes handler for a list of all attribute types, then a separate call to get each attribute value. On my Windows box that's seven separate calls.
If you've stacked two vfs's on top of each other, then the top vfs must resolve each of those 7 calls with a [file attributes] call of its own, which generates a new 7 calls from the extension to the bottom vfs for each of the 7 calls to the top vfs.
If you want to stack several vfs's on top of each other the result is a geometric explosion of calls to fileattributes handlers, which puts a serious practical limit on the stackability of virtual filesystems.
This problem can be ameliorated with caching of the results of [file attributes] calls within vfs namespaces, but in the future it would be nice if such complexity were dealt with within the extension itself, say with an addition to the fileattributes handler options to include an "allvalues" index that would return all file attribute values in one call, thus keeping performance of virtual filesystems roughly linear with stacking.
This of course would be very helpful for vfs's that employ network connections as well; getting all attribute info with one call would be much preferable performance-wise than having to deal with the network latency that comes with multiple separate calls.
Vince perhaps you should comment on the move tclvfs to tcl core TIP [L1 ] before it is voted on to try to get Andreas to improve the handling of 'file attributes'...
SEH [file exists $fileName] returns 0 if $fileName exists but is not executable. Use glob to test for existence.
Vince This bug is invalid. It is user error -- 'file exists' works perfectly if you implement your vfs 'access' handler correctly.
SEH I'll restate the nature of the gotcha: the docs for the Tclvfs package don't contain enough detail about the access handler method to allow certainty that you've implemented it correctly.
The vfs 'access' handler takes a mode argument in the form of an integer the meaning of which is not explained anywhere in the docs. A utility command (accessMode) is provided which resolves the integer into a string, but there is no explanation of the meanings of the possible strings either. Most of the meanings can be guessed, but if you guess wrong on one, your access handler won't work as you expect. The possible outputs of accessMode and their meanings are (I believe):
F file exists X file is executable W file is writable XW file is executable and writable R file is readable RX file is readable and executable RW file is readable and writable
I don't know if these designations are meant to check for exclusivity or simply sufficiency; i.e., if a file is readable and writable and the access mode is R, does access properly throw an error or not?
Vince feel free to file a bug report against the documentation. FYI, any unix man page for 'access()' will tell you how to interpret these fields (but of course we should still document it correctly in tclvfs).
SEH Extension has a deadly crashing bug when attempting to close a file opened for write. See [L2 ] (remove when fixed/closed with a note to use latest CVS)
Vince This has now been fixed. Use latest cvs.
SEH -- The vfs api layer will catch all errors in the close callback procedure, so if the procedure aborts due to an error, the close command that invoked the callback will still return OK, giving no indication that the close procedure failed and thus possibly lost data.
SEH -- Although the w channel open mode is supported in theory, in practice when the channel is passed to the close callback to be handled no capability is provided to read what's been written to the channel. Thus the written data is inaccessible to the vfs programmer.
So unless you want to limit yourself to read-only filesystems, you must kludge things somehow, like silently switching the mode to w+ in the open procedure and hoping nobody using the vfs assumes a file opened for writing only will have reads blocked.
CMcC I had problems with an old version of memchan which aped this. I see this problem, at base, is that the implementor of a vfs wants to create a channel in one mode (to give to a user), and subsequently escalate that mode to perform more powerful operations.
It would be possible to achieve this escalation in the tclvfs C code which calls the close callback - automatically escalating mode to the maximum possible. There is currently a TIP to expose those mode bits to extensions to try to solve this problem.
I note that the problem hasn't been obvious to date because most of the distributed filesystems use memchan, and memchans are always maximally permissive.
SEH -- You can't use fcopy in the close callback, or otherwise pass data directly from one file channel to another. You have to read channel data into a variable, then write it from the variable to the target channel.
CMcC Once a channel is closed, the only file facilities which the i/o core will permit are read and write. Asynchronous operations ought to be possible, but are probably risky.
SEH -- My understanding was that the channel is not closed in the close callback, merely restricted to seek and read operations. Fcopy illustrates the problem I'm talking about, but you can try the following in the callback procedure:
set newchannel [open newfile.txt w] seek $callbackchannel 0 puts $newchannel [read $callbackchannel]
and you will notice that no data is transmitted to newchannel, and no error is generated. You will only notice that there was a problem when you find that the data you thought you'd backed up is gone forever.
CMcC I just tried that (with the addition of [close $newchannel] :) and it worked fine. I had an old version of memchan, though, which didn't support seek properly (I think that was the problem) and which indeed failed silently to allow me to read its contents back ... it just always returned "". I found I had both a /usr/lib/Memchan2.2 and a /usr/lib/Memchan2.2.0 which was more current, but which was never being found by the package mechanism.
I think we need to get to the bottom of the problem you observe.
SEH -- I'm not using Memchan at all. The above callbackchannel was created by opening an existing real file.
CMcC -- Ok, well this is puzzling and disturbing ... I'm getting the output in the output file. Can you add tests to discover whether the problem is (a) empty read, (b) failed write, (c) some other error? Can you write to stderr from the close callback?
SEH -- I tried a simple scenario here at work (Win2000) and it seems to work OK as you say. At home (WinXP) is where I have the problems. The problem may be OS-dependent, or it may stem from the rather more complicated arrangements (daisy-chained vfs's) I'm using.
13dec04 jcw - I'm having some trouble throwing errors from VFS driver code. Am trying to report an error inside matchindirectory, when called with a bad path. There appears to be a "vfs::filesystem posixerror $N" call for this purpose, but I can't make it work. Looking into tclvfs's "vfs.c" code, something strange seems to be in there - the error is -1 i.s.o. TCL_OK. It leads to the following effect:
% vfs::filesystem posixerror 13
command returned bad code: -1
%