SEH 20080819 -- Version 1.0 of the tcl-fuse extension has been released [L1 ] as a product of the 2008 Google Summer of Code. This package makes many exciting new projects possible. Thanks to Alexandros Stergiakis for bringing it to us.
Linux (and now also FreeBSD) has a kernel module called "Filesystem in USErspace" [L2 ] which provides a user file system - basically it's the same as tclvfs but implemented at the linux kernel level. The linux fuse module and utilities can be found at http://fuse.sf.net and are required for this tcl package to function.
EG - NetBSD, since (the still unreleased) version 5.0, has a native implementation of filesystem in user space called puffs. It also provides librefuse, a reimplementation of the fuse API on top of puffs, which should allow NetBSD to run tcl-fuse. See http://www.netbsd.org/docs/puffs/ for further info.
I (CMcC) have written a tcl interface to fuse version 2.2. A preliminary version can be found here: [L3 ].
Note: the package is written to use tcl8.5 - it won't work with tcl8.4
The package provides a single file system called reflect, which reflects the file system tcl sees to the linux kernel. This is sufficient to export tcl vfs to any process in linux.
This has quite a few uses, giving linux all the vfs that tcl is capable of.
Currently (20 Mar 2005) I have only implemented read facilities. I'll add more capabilities over time.
SEH 22Mar05 -- In case it's not obvious, the combination of fuse and Tcl virtual filesystems is the critical path to total world domination by Tcl. It has the potential to increase ease of development, management and use of computers and software by an order of magnitude. All the complexity represented by acronyms like RPM, pkg, CVS etc. could be stuffed under the hood and rarely thought about by the average user or administrator, thus making Tcl the indispensible tool.
Nice work, Colin, I've been hoping to see a package like this for a while. I hope you continue to improve it.
CMcC Yes, you're right about total world domination. I've pointed out to the fuse dev list that while they're trying to develop filesystems, Tcl already has a bunch which can immediately be deployed. It should be obvious, too, that vfs can be developed more easily under Tcl than under C. Heute vfs, Morgens Welt!
escargo 7 Sep 2005 - This binding to FUSE is mentioned on a list of FUSE uses [L4 ]. FUSE is reportedly going to be added to the mainline Linux kernel.
escargo 14 Sep 2005 - Linus Tovalds wrote this on Sept. 13, 2005: "I've released a 2.6.14-rc1, and we're now all supposed to help just clean up and fix everything, and aim for a really solid 2.6.14 release...." "On the filesystem level, FUSE got merged, and ntfs and xfs got updated."
SEH -- Then now is the time to throw CMcC some love and get him to push development of this package forward.
escargo 7 Nov 2005 - The Tcl binding for FUSE is no longer mentioned on the [L5 ] page. It's not clear when it was removed (sometime before mid-October perhaps).
AMG: It's a wiki. Add it back! :^) Hmm, looks like you'll have to get an account.
SEH 20051107 -- There seems to be a relatively new FTP filesystem [L6 ](written in Perl) for FUSE. It might be possible to use the Tcl FTP server with this filesystem and thereby get access to Tcl virtual filesystems almost for free.
escargo 7 Nov 2005 - That sounds like a lot of layering to me, but maybe it doesn't matter.
SEH 20051129 -- I've been trying to understand the basic function of the FUSE package a little better, in hopes of using it or even contributing to the Tcl binding project in the future. This is not so easy since FUSE is sparsely documented. But after looking over the code and comparing different language binding packages (Perl, Python, Tcl), I think I have a little insight. I thought I would try to relate my understanding of how FUSE works in plain English here. I am not so much of a Unix systems programmer though, so I'm sure some things are wrong or poorly expressed. Please add corrections and clarifications if you can:
FUSE allows a Unix user with no special privileges to mount a virtual filesystem by using a custom executable and a standard program provided by the FUSE package (fusermount) to initiate a communication pathway to the FUSE kernel module.
The FUSE package provides boilerplate code for the custom executable which: sets up the connection with the kernel module, starts a while loop that continually checks if the connection has any data from the kernel module to be read, and provides a complete API interface which translates each request from the kernel module into a function call in the custom executable program. It is up to the virtual filesystem developer to write functions that handle each possible function call in the API and link those functions in somehow to the custom executable at compile time.
A virtual filesystem is mounted by running the custom executable with a mount point in real filespace as a command-line argument. The boilerplate code instantiates a socketpair, which creates two file descriptors (similar to a named pipe, but both file descriptors are fully read-write). One file descriptor value is stored in an environment variable , then fusermount is called with the mount point as a command-line argument. I gather that fusermount grabs the file descriptor value from the environment variable and stores it and the mount point in a table accessible to the kernel module. If fusermount returns successfully, the custom executable goes into a while loop and waits for service requests to come down its end of the socketpair from the kernel module.
The Perl and Python bindings make full use of the boilerplate code and provide only skeleton procedures in the given script language as needed by the API for the boilerplate to call. The Perl binding for example uses the Dynaloader package in order to make the API procedures callable as C functions by the custom executable boilerplate. It is then up to the VFS developer to flesh out the skeleton procedures with the specifics of the desired virtual filesystem.
The Tcl binding appears to be somewhat more clever. All the boilerplate code is replaced by Tcl script except for two functions needed for low-level Unix I/O, the mount function that creates the socketpair, and a receive function that uses rcvmsg to read from the socket. These are utilized by means of critcl within the Tcl script, which goes on to tie the file descriptor to a fileevent trigger, and then goes into a standard Tcl event loop, so that the custom executable is completely eliminated and replaced with Tcl scripts only. Thus the Tcl binding is able to take advantage of the existing Tcl event architecture, as well as get rid of lots of C code (with all the attendant benefits of installability, maintainability and portability one hopes).
Perl has a native socketpair call, so I suppose that the Perl binding could have taken a similar route. But then the authors would have had to come up with some kind of event-handling structure, and maybe they didn't think it was worth the bother. So the Tcl FUSE binding seems to be a nice showcase for Tcl's architectural advantages.
Before I or anyone else can extend the Tcl FUSE binding package, it will be necessary to decipher the kernel's over-the-socket request protocol (not documented anywhere as far as I can find) and figure out how those requests are transformed into procedure calls. If that can be done, and additions to the package made to handle all API services, then the Tcl binding should prove a superior solution for creating FUSE virtual filesystems (in short, every Tcl VFS will instantly become a FUSE VFS).
CMcC 20051130
I deciphered it, and the tclfuse package contains all the code necessary to en/decode requests and responses in the fuse version I used.
All tclfuse is missing is an implementation of the write side, IIRC.
Zarutian 13. may 2006 a link to how to write an Linux VFS module http://pages.cpsc.ucalgary.ca/~crwth/programming/VFS/VFS.php
"Tcl simplifies kernel programming" [L7 ]