tclvfs

Difference between version 131 and 132 - Previous - Next
'''[https://core.tcl.tk/tclvfs%|%TclVFS]''', a Tcl [extension], exposes Tcl's [vfs%|%Virtual File System
(VFS)] subsystem to the script level.



** Attributes **

   repository:   https://core.tcl-lang.org/tclvfs

   historical repository:   http://sourceforge.net/projects/tclvfs

   version:   1.4.2



** Obtaining **

It is recommended to use the trunk branch at https://core.tcl-lang.org/tclvfs.

Note 2021-02-03: I was unable to clone that repository with fossil, and there is not TAR.GZ to download. Using the github mirror was painless.

[EMJ] 2021-03-03: What happened with your clone attempt? I had no problem cloning with
======
fossil clone https://core.tcl-lang.org/tclvfs tclvfs.fossil
======
And the quickest way to get a tar.gz is to click on the checkin id at the top of the timeline and
then click the Tarball link.
[aplsimple] 2021-03-04: The above note 2021-02-03 isn't totally irrelevant. The site (destined to download data!) has no direct "Download" link, which can perplex those unfamiliar to Fossil. It's common issue of most Fossil sites, though. The solution is either to insert the "Download" link into the starting page as in 
[http://chiselapp.com/user/dgroth/repository/tclcode/index%|%tclcode%|%] or (more radical) to remake the site's Fossil skin as in [http://chiselapp.com/user/aplsimple/repository/aloupe/index%|%aloupe%|%].
----

The last version published at Sourceforge was 1.3.0, under the filename
tclvfs20080503.tar.gz.  No additional versioned distributions were released.

    


The official homepage is https://core.tcl.tk/tclvfs. Bugs should be reported there. Ditto patches and enhancement requests.
Its previous location was http://sourceforge.net/projects/tclvfs which also hosts downloads. 

Take a look at the [One-line web browser in Tcl] using [TclVFS].

----
See also [VFS].  The C extension is now very robust and well tested.
But, distributed with tclvfs are a large number of virtual filesystem
implementations (in pure Tcl).  Many of these still need a lot of work
(but since it is pure Tcl, lots of people should be able to help).

tclvfs relies on [Memchan] or some equivalent to [open] (or [load] or [source]) any file. 

''Please help develop the tclvfs package further!''

Contributions of binary versions of tclvfs to be placed on sourceforge much appreciated.

TclVFS needs to be stress tested, QA'd and put into either the core, or a standard [Tcl Binlib] ... it's far too useful and fundamental for it to be kept as an optional extension.

----
This package is part of the [ActiveTcl] Batteries Included distribution.

----
**VFS links**
   * [tclVFS Usage]
   * [tclVFS Building]
   * There are some idiosyncrasies in writing a tclvfs, save some time by checking [tclVFS Gotchas]
   * [tclVFS Hints and Tricks]
   * [tclVFS Examples]
   * [tclVFS Needs]
   * Readonly - writable - translucent ... see [vfs filesystem configuration]

----
**VFSs distributed with the TclVFS extension**
   * package ifneeded [vfs::ftp] 1.0 
   * package ifneeded [vfs::http] 0.6
   * package ifneeded [vfs::mk4] 1.10.1 
   * package ifneeded [vfs::ns] 0.5.1 
   * package ifneeded [vfs::tar] 0.91
   * package ifneeded [vfs::test] 1.0
   * package ifneeded [vfs::urltype] 1.0
   * package ifneeded [vfs::webdav] 0.1
   * package ifneeded [vfs::zip] 1.0.3
   * package ifneeded [vfs::tk] 0.5
   * (this list will grow, incorporating elements from the other two)

----
**VFSs not (yet) distributed with the TclVFS extension**

   * [A collate/broadcast virtual filesystem]
   * [A template virtual filesystem]
   * [A quota-enforcing virtual filesystem]
   * [A versioning virtual filesystem]
   * [A delta virtual filesystem] (designed to be used with the versioning VFS)
   * [An LZW-compressing virtual filesystem]
   * [An SSH virtual filesystem]
   * [A chroot virtual filesystem]
   * [An SQL database backed VFS] using [SQLite]
   * [Caching VFS]
   * [Wikit VFS]
   * A vfs that accesses Microsoft structured storage files [http://www.patthoyts.tk/tclstorage/index.html]
   * A new davvfs is included in [davkit]
   * [vfs::inmem] - an in-memory virtual file system
   * [9P (Plan 9 Filesystem Protocol) VFS]
   * [CHMvfs]
   * [tclstorage] -- allows MS structured storage files to be mounted as virtual filesystems.
   * [cookfs]
   * [topcua] -- has an experimental VFS demonstrating read-only mapping of OPCUA objects and data variables 
   * (add more)



**Possible VFSs**

   * [metadata virtual filesystem]
   * [tclCVS virtual filesystem]
   * [pure tcl distributed filesystem]
   * (add more)
   * [SQLite] using the SQLite append VFS to easily allow an embedded filesystem in a single tcl script.

[AMG]: Would this SQLite append VFS be usable to construct a Starkit or Starpack?  Could the database be writable?  Could it contain custom tables besides the filesystem table?

See: [https://news.ycombinator.com/item?id=17266348] [rkeene]: "Expect an SQLite-backed Starkit soon !"



** Questions, Problems **
<<discussion>>
Can Tcl and Tclvfs together be used to poke around a starkit?

''Don't understand the question.  Yes, would be my answer, mount the starkit via vfs::mk4 ... -[jcw]''

[jcw] 2005-02-04: Would it be an idea to map an arbitrary X(HT)ML tree to VFS?

[LV]: So the results would be the ability to navigate an XML tree as if the nodes
were directories and the actual items would be files?  Sounds intriguing.  How would
the attributes be represented?

----

I think a very nice vfs would be an LDAP vfs using the tcllib ldap package to present a DIT as a filesystem. (and entries probably as LDIF formatted files)

2007-01-12 :There's a list of interesting VFS ideas and implementations at [http://fuse.sourceforge.net/wiki/index.php/FileSystems]
(based on FUSE [http://fuse.sourceforge.net/]).

----

'''[gpl] 2009-07-05 13:59:51:'''

A bug in the file templatevfs.tcl (see [A template virtual filesystem]) 1.5.2 (last lines) prevents the use of the versioning file system on my machine.

Instead of:

====== 
proc memchan {args} {
    if {$::tcl_platform(platform) == "windows"} {
        package require Memchan
        set chan [uplevel 1 ::memchan $args]
        return $chan
    } else {
        return ::vfs::memchan $args
    }
}
====== 

put:

====== 
proc memchan {args} {
    if {$::tcl_platform(platform) == "windows"} {
        package require Memchan
        set chan [uplevel 1 ::memchan $args]
        return $chan
    } else {
        return [::vfs::memchan $args]
    }
}
====== 

(the brackets around ::vfs::memchan).

I didn't even know that [return] could take several arguments, in which case it returns nothing...

'''[AK] 2009-07-06 13:03:41:'''

Actually the fix is not quite right, mishandling the $args argument. This is a list, and has to be expanded. So the proper replacement is

====== 
# Tcl 8.5: return [::vfs::memchan {*}$args]
return [eval [linsert $args 0 ::vfs::memchan]]
====== 

The similar construct in the other branch was semi-ok due to the use of uplevel.

----
[neb] 2010-05-25:

Be aware that using `file join` to build paths mungs the result somewhat; at least on Windows. `"file join http://wiki.tcl.tk/yadda yadda"` returns `"http:/wiki.tcl.tk/yadda/yadda"`. The missing forward slash was causing me problems in a script.

'''[LVwikignome] 2010-05-25 14:23:00'''

On SPARC Solaris using tclsh 8.6, I see

======none
$ tclsh8.6
% file join http://wiki.tcl.tk/yadda yadda
http:/wiki.tcl.tk/yadda/yadda
======

so it does not appear to be Windows specific

[bch]: Arguably the ` http://` shouldn't even be presented to anything requiring or working-with paths, though. The `"http://"` part is the "scheme" or transport mechanism. It's preamble to the path. See Wikipedia[http://en.wikipedia.org/wiki/Uniform_Resource_Locator] for more details.

[neb] 2010/09/03: perhaps, but it isn't restricted to that example:

======none
% file join \\\\server\\share\\yadda yadda
/server/share/yadda/yadda
% file nativename [file join \\server\\share\\yadda yadda]
\server\share\yadda\yadda
====== 

Changing the server\\share\folder convention to folder\folder\folder...
(edited, because my original addition was gibberish.

[neb] 2011-08-13: I'm answering my own question a year later, because I'm back in that same code again. The issue I was having was I wanted to be able to use a directory agnostically: I didn't want to have to worry about whether the dir was local, on a web site, whatever. What was dying was if I tried to join subdirectories or files to the path. Whenever I did, it stripped the slashes down to a single slash, and then following file accesses failed.

It turns out that it all works fine, *if* the protocol is mounted *before* you do path manipulations (I was passing a path joined to an url to a proc that mounted the appropriate protocol). eg:

====== 
puts [file join http://www.google.com dummy]
package require vfs::urltype
vfs::urltype::Mount http
puts [file join http://www.google.com dummy]
====== 

returns:

======none
http:/www.google.com/dummy
http://www.google.com/dummy
====== 

I started to override vfs::urltype Mount to let me specify a mountpoint, but when I did I realized that the paths were suddenly working. So I can get everything to work by just mounting the protocols before working with the paths.

----

2011-11-30: I just downloaded tclvfs-20080503 from Source Forge. README says I should do standard ./configure && make && make install. Let's see:

====== 
$ tclvfs-20080503> ./configure 
checking for correct TEA configuration... ok (TEA 3.5)
checking for Tcl configuration... configure: WARNING: Can't find Tcl configuration definitions
$ tclvfs-20080503> make
make: *** No targets specified and no makefile found.  Stop.
Exit 2
====== 


I am on Ubuntu 11.10 and I have no idea what that means and what I should do.

[SEH] :Do you have tclConfig.sh installed on your machine in a place where the configure script can find it?  A quick google of your error message finds this: [http://community.activestate.com/forum-topic/configure-warning-cant-fi]

[LES]: Thanks! It worked... to an extent.

======none
$ tclvfs-20080503> ./configure --with-tclconfig=/home/luc/bin/ActiveTcl-8.6/lib/ --with-tkconfig=/home/luc/bin/ActiveTcl-8.6/lib/
====== 

I'm not going to paste all the output here, it definitely looks like it worked. But then...

======none
$tclvfs-20080503> make
make: *** No rule to make target `vfs.o', needed by `libvfs1.3.so'.  Stop.
Exit 2
====== 

/me scratches head again. I wonder if the problem is my mixing of Tcl installations. I am using Ubuntu's Tcl/Tk standard packages, but you see I linked to an ActiveTcl installation I have in my home directory. I searched my entire disk and that was the only tclConfig.sh I could find. And all this trouble is just so I can use Mk4tcl...

[SEH]: Since tclvfs in in Ubuntu's package repository, is there a reason you need to compile it from scratch rather than just install it?

[LES]: Two reasons: 1) I would like to distribute my application as a complete, self-sufficient package (Starkit?) in the future; 2) I installed the Ubuntu package and it doesn't seem to work. Output in Tkcon:

======none
[luc]3759> source /home/luc/devel/tcl/Editor/metakit-2.4.9.7.kit
can't find package vfs
[luc]3760> package require vfs
can't find package vfs
====== 

----

[Kroc] 2016-02-02:

There is a bug in current tclvfs releases (tclvfs-20080503 1.3 from SourceForge or 1.4.2 from ActiveState) when mouting a zip with vfs::zip in [tkcon]:


======
% package require vfs 
1.3 
% package require vfs::zip 
1.0.1 
% set mnt_file [vfs::zip::Mount Archive.zip Archive.zip] 
invalid command name "vfs::RegisterMount"
======


This bug was described here: https://bugs.activestate.com/show_bug.cgi?id=81845 but it seems the solution doesn't work correctly when used in [tkcon].

I've found a workaround by turning vfsUtils.tcl into package vfs::utils   0.5. Here is the patch for tclvfs 1.3:


======none
diff -uNr tclvfs-20080503.orig/pkgIndex.tcl.in tclvfs-20080503/pkgIndex.tcl.in
--- tclvfs-20080503.orig/pkgIndex.tcl.in        2008-04-16 01:59:24.000000000 +0200
+++ tclvfs-20080503/pkgIndex.tcl.in        2016-01-29 15:01:44.000000000 +0100
@@ -49,6 +49,7 @@
 package ifneeded vfs::urltype 1.0 [list source [file join $dir vfsUrl.tcl]]
 package ifneeded vfs::webdav  0.1 [list source [file join $dir webdavvfs.tcl]]
 package ifneeded vfs::tk      0.5 [list source [file join $dir tkvfs.tcl]]
+package ifneeded vfs::utils   0.5 [list source [file join $dir vfsUtils.tcl]]
 #
 # Virtual filesystems based on the template vfs:
 #

diff -uNr tclvfs-20080503.orig/library/vfsUtils.tcl tclvfs-20080503/library/vfsUtils.tcl 
--- tclvfs-20080503.orig/library/vfsUtils.tcl        2004-07-05 01:42:04.000000000 +0200
+++ tclvfs-20080503/library/vfsUtils.tcl        2016-01-29 14:45:20.000000000 +0100
@@ -4,6 +4,7 @@

 package require Tcl 8.4
 package require vfs
+package provide vfs::utils 0.5

 namespace eval ::vfs {
     variable debug 0

diff -uNr tclvfs-20080503.orig/library/zipvfs.tcl tclvfs-20080503/library/zipvfs.tcl 
--- tclvfs-20080503.orig/library/zipvfs.tcl        2008-04-15 23:11:53.000000000 +0200
+++ tclvfs-20080503/library/zipvfs.tcl        2016-01-29 15:01:42.000000000 +0100
@@ -3,6 +3,7 @@
 package provide vfs::zip 1.0.1

 package require vfs
+package require vfs::utils 0.5

 # Using the vfs, memchan and Trf extensions, we ought to be able
 # to write a Tcl-only zip virtual filesystem.  What we have below

======

----
'''[aplsimple] - 2021-03-03 10:13:44'''

The fossil repo's problem is that the "Download" link is absent in the fossil skin.

To overcome this, try to hit "Files" link, then choose a current check-in link (72e30db4a7 as of 2021-03-03), then choose any download link, TAR or ZIP.

<<categories>> Package | VFS