Version 9 of package BoF: Impact of VFS

Updated 2012-07-11 10:59:12 by RLE

impact of new virtual file systems is very isolated. Wise design of package

package knows nothing about files at all. It works only with scripts.

package unknown is the hook you can use to give package a command that does know something about files and filesystems. By default that command is tclPkgUnknown.

Everything you think package knows about files and the file system (pkgIndex.tcl, auto_path, weird package names discovery of installed packages after first package require) are all really (mis-?)features of tclPkgUnknown.

So does the presence of virtual file systems imply any different useful default behavior for a default package unknown?

In particular, now that we can have multiple filesystems, should each filesystem itself have power to independently tell package how (index scripts of) packages are to be found within it?

It makes some sense, in that it gives the "owner" of a resource, offered to an interp as a virtual file system, to have some control over how that resource is accessed.

Also, if we don't give that cont

OTOH, to some extent it tears down the illusion of one single unified file system.

Up to now, there's been one filesystem - the native one, and source-ing init.tcl told the interp how to find packages in it.

Could extend that idea to a mount-time callback for a virtual filesystem to register info to be used by (a default?) package unknown command.

Problems: package manages per-interp resources, but virtual filesystems are per-process resources.

Any package unknown customization is problematic unless/until the package BoF: install-side of PACKAGE UNKNOWN is completed.

VFS can give us a new perspectives on installation.

Install means "do something to place files on my filesystem that have the effect of making a package available to Tcl programs I run".

Common model is retrieve some archive file from the network, or from removable media. Do something to that file, and have package(s) installed as a consequence.

VFS offers different models.

Starkit - don't install. Packages are already installed, but they are installed in a virtual file system that gets mounted at program startup.

Network loading - Network access can be presented to Tcl program as just another filesystem. Don't install. Just find a network resource that already has the package installed, and mount its VFS interface

Different style of traditional install -- one that does write files onto your local native filesystem, but could use network resource mounted as a VFS as its source; no need for explicit pack/unpack as an archive.

SEH 9/30/04 -- The achilles heel of package is that it loads all available packages at start; which is fine for a limited install, but doesn't scale. One could for example provide an ftp vfs into a massive archive and put the mount location into the library path variable, but it would take forever for the shell to start, and memory requirements would be huge. The solution is something like the Simple Tcl Archive Network, which overloads package unknown and makes a query via http to a central repository for a specific package only when it is required. The complexity of indexing and delivering available packages can be taken care of by the repository server.

I would like to see STAN extended so that it can reference a local directory for available packages; in a non-vfs situation it would relieve the burden of having to database all available packages at start time so that large installs with many packages would be practical -- on top of that, a custom vfs could be written that could query a number of archives using a range of connect methods (http, ftp, cvs, etc.), deliver the desired package if found and cache it locally. a collate/broadcast virtual filesystem could be configured to do much of this.

Also, imagine a combination of STAN and vfs that could handle not only specific package names but keywords or categories. If you want to demonstrate the power of Tcl and win over new users, imagine a newbie sitting down and typing %package require basic editor and having TkPad pop up, or typing %package require advanced editor and getting Alpha.