Towards a central catalogue of Tcl packages

Several attempts have been made to create a repository or rather catalogue of Tcl packages, see Category Repository for an overview. This page is intended to finally establish something that could serve this purpose without being left un-maintained after a short period of time.

Note: this catalogue is not supposed to be a central place where people can host their code, issue tracker and from which they can deploy their packages. It is rather a catalogue of metadata with pointers/links to the place where you can find the code and the downloads etc.

The motivation for this catalogue is the amount of packages scattered around and often being abandoned. Some have been adopted by new maintainers and then hosted in a different place. This makes it difficult for the average user to know where to find those packages in a recent version or even knowing whether a specific package is still being worked on. Kicked off by the effort to release Tcl 9, a current approach to keep a record of packages is here: Porting extensions to Tcl 9.

The primary of the catalogue here is to start simple, as simple as possible so it doesn't take much effort to have an initial catalogue up and running. We can use the Wiki itself as a starting point, making sure that the metadata are stored in a way here that enables us to later expand on the catalogue (both in terms of the metadata fields and the way to handle them) and e.g. build a website based on the Wiki page or even a stand-alone application (something like this: Tcl-Pkg). This especially means that the metadata cannot be stored as free-format Wiki text but rather in a structured way. For this, we will use the new (as of 2024-10-15) <!DOCTYPE TCLPART> of the Wiki. This allows us to write the metadata as Tcl code and thus have it in a future-safe and structured form. The visualization is then done by the Wiki engine.

The first step is the definition of proper metadata fields. Let's use something that is well-defined instead of inventing our own metadate scheme. So we will adopt the CodeMeta project and thus use metadata terms mainly from https://schema.org and supplemented with terms from the CodeMeta project.

Metadata fields

This is the list of metadata fields that can be supplied for a package. The metadata term comes first and in parenthesis is a still very short description on the content behind that term. Some of the terms can occur multiple times, e.g. "author", "version", "maintainer", :

  • name (package name)
  • version (package version)
  • description (package description)
  • applicationCategory (package category)
  • applicationSubCategory (package subcategory)
  • author (the original author of the package)
  • contributor (people not being the author(s) nor the maintainer(s) but having contributed to the code substantially
  • maintainer (currently being worked on by/claimed by)
  • codeRepository (URL homepage)
  • issueTracker (URL bug tracker)
  • datePublished (release date)
  • downloadURL
  • programmingLanguage (Tcl only, Tk, C, ...)
  • operatingSystem (where the package is running: macOS, Windows, Linux, ...)
  • developmentStatus (abdandoned, active, ...)
  • license
  • sameAs (e.g. to link to another package which is just a fork of this one or a re-implementation)
  • softwareHelp (URL of man pages etc)

Here are some metadata fields not in the CodeMeta schema but which could be useful:

  • source of this information (URL, pers. comm.)
  • release announcement URL (GUTTER has it)

Step 1: Settle which metadata to use

Realizing that there already is a TIP 55 on packages and metadata and an associated Wiki page Package MetaData Fields, the above list should be mapped to the TIP or even merged with it. We do not care too much whether the terms are clean in the sense that they come from one individual source. They should just be kind of standardized and TIP 55 operates based on DublinCore which is fine.

Step 2: Prepare some example metadata sets

Step 3: Prepare a Wiki page showing the metadata as a catalogue


RZ - 2024-10-15 10:14:20

Imho the metadata should be split in common informations (p.e. package name, purpose,..) and version specific informations (download, binaries,..). Some of the common could be defaults for version specific elements (author,..)

Torsten Berg Yes, this is is how it should be and this is the meaning of the usage. So, each version identified by the term "version" can have several metadata fields associated with it, practically producing a nested structure of the metadata. This will work just as in XML, only much simpler and expressed in Tcl (probably using a dict).


Siqsuruq - 2024-10-15 11:25:08

I'm currently working on this in my own project as well. My setup involves Naviserver running in Docker containers, with PostgreSQL as the database backend, and I'm utilizing the Next-Scripting framework Object Oriented. I'd be happy to collaborate with others on this if anyone is interested in helping or sharing ideas. I have a test development server dedicated to this project, though the code is currently spread across different repositories on Bitbucket and GitHub. So far, I’ve completed:

  • internal database design SQL DB Design
  • visual front-end https://tclrepo.msysc.org/
  • package search functionality GUI, RestAPI calls
  • package search functionality GUI, RestAPI calls
  • new user registration from GUI, RestAPI calls
  • ability to upload new or update existing PEM public key for package authenticity verification
  • verify package signature against user's PEM public key
  • helper classes for: license, category, author, user, login, package, filestorage, encryption, API calls, DB calls, UI
  • Dark mode GUI
  • Db is prepared to handle all metadata of the package mentioned here on this thread or link name here
  • standard error in JSON format, and user notifications

Still missing:

  • Show package metadata in GUI or RestAPI call
  • I broke download link :), but its a fast fix
  • translations to another languages
  • validate user email after register
  • package dependencies auto parsing
  • admin console

What I'm working right now:

  • refactoring package class

If anyone is interested, I’m happy to share credentials for collaboration, just drop me an email: siqsuruq(at)gm**l.com

Torsten Berg OK, this is a much more advanced goal. The catalogue on this page is supposed to be a simple registry while your system is much more sophisticated and also is supposed to host package tarballs/zips for download. Nonetheless, we should not duplicate the effort. So, I am glad that someone is willing to provide a website for hosting such a catalogue/repository. And thanks for the pointers to the TIP 55 and that other Wiki page! In the meantime, we can continue here with our own setups but should make sure there is a mapping from this scheme to your scheme in terms of metadata. Then, a simple Tcl script should suffice to translate the collected metadata from here to your tclrepo and then let the future happen there. At that point, our little catalogue here can be retired for good.