Version 8 of package BoF: Impact of VFS

Updated 2004-09-30 20:01:07 by SEH

impact of new virtual file systems is very isolated. Wise design of [package]

[package] knows nothing about files at all. It works only with scripts.

[package unknown] is the hook you can use to give [package] a command that does know something about files and filesystems. By default that command is [tclPkgUnknown].

Everything you think [package] knows about files and the file system (pkgIndex.tcl, auto_path, weird [package names] discovery of installed packages after first [package require]) are all really (mis-?)features of [tclPkgUnknown].

So does the presence of virtual file systems imply any different useful default behavior for a default [package unknown]?

In particular, now that we can have multiple filesystems, should each filesystem itself have power to independently tell [package] how (index scripts of) packages are to be found within it?

It makes some sense, in that it gives the "owner" of a resource, offered to an interp as a virtual file system, to have some control over how that resource is accessed.

Also, if we don't give that cont

OTOH, to some extent it tears down the illusion of one single unified file system.

Up to now, there's been one filesystem - the native one, and [source]-ing init.tcl told the interp how to find packages in it.

Could extend that idea to a mount-time callback for a virtual filesystem to register info to be used by (a default?) package unknown command.

Problems: [package] manages per-interp resources, but virtual filesystems are per-process resources.

Any [package unknown] customization is problematic unless/until the package BoF: install-side of PACKAGE UNKNOWN is completed.

VFS can give us a new perspectives on installation.

Install means "do something to place files on my filesystem that have the effect of making a package available to Tcl programs I run".

Common model is retrieve some archive file from the network, or from removable media. Do something to that file, and have package(s) installed as a consequence.

VFS offers different models.

Starkit - don't install. Packages are already installed, but they are installed in a virtual file system that gets mounted at program startup.

Network loading - Network access can be presented to Tcl program as just another filesystem. Don't install. Just find a network resource that already has the package installed, and mount its VFS interface

Different style of traditional install -- one that does write files onto your local native filesystem, but could use network resource mounted as a VFS as its source; no need for explicit pack/unpack as an archive.

SEH 9/30/04 -- The achilles heel of [package] is that it loads all available packages at start; which is fine for a limited install, but doesn't scale. One could for example provide an ftp vfs into a massive archive and put the mount location into the library path variable, but it would take forever for the shell to start, and memory requirements would be huge. The solution is something like the Simple Tcl Archive Network, which overloads [package unknown] and makes a query via http to a central repository for a specific package only when it is required. The complexity of indexing and delivering available packages can be taken care of by the repository server.

I would like to see STAN extended so that it can reference a local directory for available packages; in a non-vfs situation it would relieve the burden of having to database all available packages at start time so that large installs with many packages would be practical -- on top of that, a custom vfs could be written that could query a number of archives using a range of connect methods (http, ftp, cvs, etc.), deliver the desired package if found and cache it locally. a collate/broadcast virtual filesystem could be configured to do much of this.

Also, imagine a combination of STAN and vfs that could handle not only specific package names but keywords or categories. If you want to demonstrate the power of Tcl and win over new users, imagine a newbie sitting down and typing %package require basic editor and having TkPad pop up, or typing %package require advanced editor and getting Alpha.

[ Category VFS | Category Discussion | Category Community ]