While discussing the need for a repository of tcl packages and code, and after SC had mentioned the availability of cantcl, LV said:
Is there a technical description of the functionality provided? Do people have ideas on what features is needed?
For instance, I'd like to have the ability to:
Some of the above wishes come from interacting with Perl's CPAN repository, some come from a partial understanding of ActiveState's Perl extension manager, and some come from listening to what the users have requested on clt.
MPJ: A nice example of a working repository is the plugin extensions for the text editor jEdit [L1 ]. The Plugin Manager lists the currently installed plugins. It has a button to updates all the installed plugin. Another button to install new plugins.
When you go to the install new plugin page it download the list from the server and displays them as a checkbox list. When a plugin is selected the following information is avaliable: Name, Author, Latest Version, Last Upadated, Description. It also has some options as where to put the plugin (user directory, system directory). Finally, the source code can be downloaded and stored in the plugin directory (if box is checked).
I think we could learn (copy/steal) a lot from how jEdit handles these extensions to the base package. Just my $.02 worth...
SC: there are other auto-plugin downloaders, xemacs [L2 ] can retrieve package lists and tell you what you have installed and what's new that's available. None of this is new or even difficult, it just needs designing and doing!
LV: And of course Firefox at http://mozilla.org/ has such capability for plugins, extensions, and themes. It also supposedly supports a check for me on some regular interval whether this is the latest, and if not, notify the user type functionality.
schlenk And especially for Firefox this feature was so buggy at first (they had XML/RDF descriptions of the updates but parsed them with some PHP regexps instead of a real XML parser), that we could easily do at least that good.
MG 2005-04-09: Is there any kind of repository for Tcl code, at present? If not, would there by any interest in me trying to set one up? It certainly wouldn't have all the features mentioned above at first, and might never get some of 'em, but it might at least help to stop some of the broken links, to code/packages that aren't stored on the Wiki...
RS: The closest I can think of at the moment is sdarchive.
LV: CANTCL is a nice start - but the creator might very well be wanting some help, or perhaps someone with whom he would share the code to get you started. Also take a look at the tcl modules tip.
MG: will read up on all the TIPs mentioned through those pages over the next couple of days.
SC: All of the CANTCL code is on Sourceforge, you're free to get it and work with it. I'll add anyone as a developer on that project. I don't have any time to spend on CANTCL at the moment but I'd still like to see a working repository.
AF: I am working on a repo but I have temporarily been pulled away by work and a project for my own business. here is some info on what i was working towards [L3 ]
SYStems: Okay, I never used CPAN, but if I understand corectly, it removes a perl package from the OS domain, meaning, if I want XOTcl on debian, I won't need a debian package, but a Tcl-CPAN package which as I can see will make life easier, as it can unify and formilize Tcl packages amongts linux distros, so even I am not competant enough to implement such a solution, I will shamelessly cheerlead it. For starter I think if the solution will be implemented as a web-service, web site admins a-la forums admins can be a good enough solution for spam. I imagine a solution like this
apt-get tcl-can apt-cache info search tcl tcl-can - Tcl comprehensive archive network # hint, tcl-cpan is the only thing I need to get tcl, tcl-can cannot depend on tcl tcl-can install XOTcl You will need to download 2 new packages tcl, xotcl Continue yes or no? # and so on, # tcl-can search ''pattern'' # tcl-can info ''name'' -- shows package info # tcl-can policy ''name'' -- same idea as debian, tell what I have and what I could have # tcl-can download ''name'' -- download package only, no deps necessary # tcl-can install ''name'' -- download and install with dependencies # tcl-can files ''name'' -- list files in package, whether the package is installed or not # tcl-can remove ''name'' # tcl-can add ''name.tp'' -- upload a tp (tcl package) to the rep, should be availble for anyone # of course we should find a way to deal with names conflicts, # I suggest that like webmails, first come, first take # popular packages can be added before this rep be made public # tcl-can delete ''name.tp'' -- remove package fro repository admins only, require pass # tcl-can login ''admin'' -- asks for a password, of course can be hacked, but other admin ca # recover the site from damage # etc ...
Of course, Tk and webmin front ends can be made availble later, a port to windows, etc ...
This will definitely be cool and I think it removes a huge load from linux distribution makers the package format (i.e. the file that contain the depencies list and much more) must be easily prepared and made, debian packaging manual is a ciptic 40 pages long monster
Also ideas to consider, running multiple versions concurently, also some obvious issues, how to deal with tcl extensions writen in other langs, mostly c, and what tcl packages who are just wrappers as this imagination seems to only work for pure tcl packages, and honestly I dont even have the knowledge to think about solving this.
PWQ 2005-05-15: One drawback I see in the central repository is just that. The Tcl community is very decentralised.
I would propose using a distributed archive. Each server maintains a server list that details how each server can be queried (URI) for information on available packages.
Setting the minimum standard of having urls for server and package lists, allows serving packages without having to have access to CGI and database backends.
Rationale: It is easy for a developer to maintain his own work and control the distribution. Every developer has at least access to a public web server serving static pages (A personal home page). Rather than having to conform to a dictated regeme and package format, each developer only needs to elicite the support of another developer to add in their server details (three url's) for their package/application to become part of the archive network.
Another advantage is that there can be multiple archives serving the same package, automatically building redundancy into the archive network.
escargo: I could see where packages might need to be cryptographically signed to ensure that they have not been tampered with (or just modified without changing the version number). It's one thing to have data distributed, but having executable code requires a signficant web of trust.
LES 2005-05-16: PWQ: But decentralization is what I most (if anything else) would like to see. The Tcl community is very decentralised, indeed, and I see that as a problem. As to the security aspect, ask the BSD Unix guys, or follow their example. They handle it well.
Lars H: A decentralised community is by itself no reason not to have a centralised repository -- a repository can even serve as the hub that holds the community together. My experience from the TeX community (which I'd say is in comparable dispersion) is that CTAN [L4 ] [L5 ] repository has done wonders for keeping it all together (and making it possible to find stuff long after the original authors have lost active interest). CTAN is however not an apt-get style repository, but rather like a large collection of package sources plus about a dozen "batteries included" distros.
The way I see it, the only reason to not aim at some kind of central repository would be that there are no resources to keep it up and running.
PWQ: Lars H, where is the repository now?. How many years does the community have to wait for the resources to become available. There was neosoft, then scriptics, now procplace. Why not make a Home for example of www.tcl.tk/RespoitoryList.txt (pick a name), that is the seeding point for a distributed repository. One only has to maintain one file with a list of top level archives (maybe only one). Clearly there is noone in the Tcl community currently that has the resources to maintain an archive long term.
LES: on May 17, 2005: GASP! Now I see I made a tiny but huge typing mistake. I actually meant the opposite: centralization is what I most (if anything else) would like to see. I totally agree with Lars. Really sorry about that...
Lars H: The only difference, as far as the required resources are concerned, between a central repository such as CTAN and the "distributed repository" PWQ proposes is disk space (and that is cheap nowadays). Both approaches equally require people who are willing to serve as moderators for the contributed material (to prevent spamming and the like). The big disadvantage with a decentralised repository is the constant erosion of contents due to links rotting away. Yes, if the central repository hardware breaks down then everything in it goes off-line, but e.g. CTAN is organised with three different central servers that all mirror each other, and such duplication is a normal precausion that I presume should be taken. Entrusting companies with central resources for a community is probably a bad idea (at least if one places all the eggs there), since companies tend to get restructured and change direction quite often; a better idea could be to focus on user groups, and perhaps academia.
CMcC: Leaving aside (for a second) the worthwhile discussion of contribution and authentication and such ... I most want the possibility of an interface to the package command, such that some variant of [package require fred 1] would:
Pragmatically, I think this just means that gutter (or similar) needs to be able to store multiple binaries by {version platform} and needs to deliver the metadata {version platform signature URL} by some well-known protocol, and needs to deliver the actual data by some well-known protocol too.
Taking a lead from Debian, there may need to be more complex version dependency metadata.
I think the most difficult and important problem is what metadata to store, so I'll leave that completely open.
I think it would be good if a known path relative to the repository root always returned a static file describing the complete repository's metadata - say e.g. Packages.xml or Packages.csv, and that optionally a non-static repository server could support a query of a form: Package.xml?name=foo&version=1&platform=linux386 and return a subset of the packages matching the query. Defaults should be something sensible, like version is latest, platform=linux :)
PWQ 20 May 05, I think Lars H comments sum up the problem. Adding support for a distributed repository is merely in the semantics, the fact that it is a devisive issue reflects on the fractured nature of the Tcl community. The fact that we already have had at least six attempts by various people to establish a central repository and none have attracted any support for adding applications let alone improving the code speaks volumes.
Unless the TCT takes a proactive role in providing management and official acknowledgment for such a facility it will never happen.
LV 2005-07-27: Well, here we go again. Over on comp.lang.tcl yet another thread concerning a desire for a code repository, in this case, at least for extensions, has arisen. Anyone have a list they could insert here of the various repository discussions that have arisen over the life of Tcl?
This time, someone has volunteered some funding for the first year of work. However, an evangelist / project leader role for the effort remains open. If people truly want this, it might be a good time to step up and help.
RLH: 2005-07-27: Just to be clear, the funding was for the hosting. : )
LV: Sorry - that's what I meant to type <blush>...
SDW 2005-07-27: Hosting I have. I run the network for a science museum. I will also be pleased to volunteer myself as punching bag...er... project leader.
schlenk 2005-07-27: Can someone add a link to the clt thread here?
LV: I don't know how to generate the URL of the thread, but my article that started the thread is [L6 ]
I wish I wish with all my might that Tcl would get a standard repository!