A possible [tclvfs] - [CMcC] 20041029

Storage of metadata (data about data) is an important and undersupported facility of file systems.

tcl supports the [[[file] attributes] command to store some system-defined metadata, and [tclvfs] permits this to be redefined and extended.

What this allows is [vfs] which intercepts the [[file attributes]] command and loads/stores attribute (name,value) pairs in some parallel store.

The question is what form of parallel store would be best?

   * '''parallel hierarchy''' - metadata is stored in a file hierarchy which parallels the structure of the original, so there's a mapping from ${path} -> ${path}.metadata which contains an array dump of name->value pairs.  ''CONS'': expensive/slow, ''PROS'': persistence for free.

   * '''hidden directory''' - metadata stored in special files hidden in a per-directory ''.metadata'' directory.  ''CONS'': expensive/slow, invasive (less so than invasive-interpolation, below), ''PROS'': faster than parallel-hierarchy, metadata is joined with with data (less so than invasive-interpolation)

   * '''persistent array''' - metadata is stored in an array $metadata($path) as an [[array get]] form per file, to be loaded/stored once, then accessed from memory.  Could use [tie] for persistence.  ''CONS'': doesn't scale well, persistence needs work, ''PROS'': fast.

   * '''metakit''' - metadata stored in a metakit file, loaded/stored as required.  ''CONS'': not pure tcl, slower than persistent-array, ''PROS'': scales, faster than parallel-FS.

   * '''invasive interpolation''' - metadata stored at the head of each file in (say) RFC822-style name:value lines, some data (e.g. emails) are already of this form, some data (e.g. caches) can be coerced to this form ''CONS'': invasive, wrecks general files for general (non-tcl) use.

   * (add more here)

----
[[[Category VFS]]]