CANTCL URL interface

CANTCL is a proposed Tcl package repository which will provide a URI based interface where each resource (package) has a unique URI and various kinds of package searches and browsing are available via URIs as well. Here's an evolving list, comments are welcome.

 cantcl/ -- front page 

The front page lists the various starting URIs below, it's HTML

 cantcl/package/<pkgname>  -- get a package

 cantcl/package/anagif.zip     -- get latest version of anagif
 cantcl/package/anagif.kit     -- get latest version of anagif as a starkit
 cantcl/package/anagif1.0.zip -- get version 1.0 

 cantcl/package/anagif/
  • returns a page of links to package contents which can then be browsed or traversed by a client, including at some point URIs of individual files inside the package, eg:
 cantcl/package/anagif/tcl/anagif.tcl  

Since we can look inside packages, we can get the descriptions, let's also support getting different versions:

 cantcl/package/<pkgname>/description.txt  
 cantcl/package/<pkgname>/description.html  
 cantcl/package/<pkgname>/description.rdf  

With the purl activated, this will give us a way of referencing the latest version of any package as

 http://purl.org/tcl/cantcl/package/anagif.zip


Browse Interface

 cantcl/browse

This page might eventually list the top level of some set of package categories (eg. Trove) from which a web user might browse.

 cantcl/browse?platform=Linux
 cantcl/browse?subject=CGI
 cantcl/browse?name=*image*   -- glob style search
  • Note the use of ? here to mark a query string so that normal CGI processing can be used on the arg value pairs.
  • each of these returns a package list with links into the /packages/ hierarchy, eg:
  <cantcl>
   <description>Results of query /browse/platform=Linux</description>
   <ul>
     <li><a href="/package/anagif/">Anagif</a> - ...description...</li>
     <li><a href="/package/bwidget/">BWidget</a> - ...description...</li>
   </ul>
  </cantcl>

perhaps with an appropriate stylesheet to allow it to render nicely in a web browser.

Upload

/cantcl/upload

According to REST, creation of new web resources should be done via POST or PUT. POST is for creating new sub-objects, PUT for completely new resources. PUT seems appropriate here, although supporting POST would allow for browser forms based upload as in the initial implementation.


It would be useful if the URL resolution was done in a manner that allowed authorized updating by authors or designated maintainers.

SC authorisation can be done via the usual HTTP methods if needed. Updates can be POSTed -- currently I've not concieved of updating already uploaded packages preferring to have new versions of packages uploaded (perhaps with the same version number overwriting the existing one). Upload authorisation would be a good idea to give some credibility to any repository.


Note: why not write a 'cantcl' implementation for tclvfs? Then you could mount a cantcl:/ new volume, and perhaps do things like glob -dir cantcl:/newpackages * -- this code could even transparently send information on the user's current system (win/mac/unix etc) and therefore only show things which apply. The newpackages directory could be constructed on the fly to represent only things the user doesn't yet have. Similarly there could be an upgrades directory -- all dynamic!!

Note: with tclvfs currently having a prototype webdav vfs, probably the best long term solution would be to a) provide support in cantcl for webdav and b) to update tclvfs so that its webdav support is better. Then we not only have CANTCL support, but also WebDAV.

SC since CANTCL works via http, the current http vfs should work via the browser interface. Having WebDAV support would depend on having an appropriately capable web server. It would be nice at some point to put together a customised tclhttpd (or the mini httpd implementation) to handle CANTCL and enable a starkit which includes everything needed to run a repository.

wcf3 I've been working on just such a beast, only I used secure WebDAV. It looks something like this:

# enable https support
package require tls
http::register https 443 ::tls::socket

# add remote library to auto_path
package require vfs::webdav
set ::webdavdir [vfs::webdav::Mount username:[email protected]/webdav/ mydav 1]
lappend ::auto_path mydav/lib
 
# unmount with the following
#vfs::webdav::Unmount $::webdavdir mydav

I did have to hack the http2.6 package a bit. First off, I patched it so that any FINDPROP method forced http 1.0 so that chunking was not used in the responce (yes, I know there is a version 2.6 with chunking support, see davkit, but it didn't work at all for me...let alone handle chunking). Secondly, I added caching of the results of the FINDPROP calls (webdav stuff) to speed things up...which helped a lot, but webdav seems painfully slow when Tcl starts another round of pkgIndex file searches. You may also notice I added an ugly hack to the vfs::webdav::Mount parameters, the trailing 1 which tells it to use a secure connection. I'm still playing around with it to see if I can speed things up, so please excuse the hacking :-) It works quite well, although slow, and can even load binary packages.

I've also been working on a way to handle platform specific extensions in a starkit which may be more directly related to CANTCL and is located on auto_path4platform