Towards a central catalogue of Tcl packages

Several attempts have been made to create a repository or rather catalogue of Tcl packages, see Category Repository for an overview. This page is intended to finally establish something that could serve this purpose without being left un-maintained after a short period of time.

Note: this catalogue is not supposed to be a central place where people can host their code, issue tracker and from which they can deploy their packages. It is rather a catalogue of metadata with pointers/links to the place where you can find the code and the downloads etc.

Design

The motivation for this catalogue is the amount of packages scattered around and often being abandoned. Some have been adopted by new maintainers and then hosted in a different place. This makes it difficult for the average user to know where to find those packages in a recent version or even knowing whether a specific package is still being worked on. Kicked off by the effort to release Tcl 9, a current approach to keep a record of packages is here: Porting extensions to Tcl 9.

The primary of the catalogue here is to start simple, as simple as possible so it doesn't take much effort to have an initial catalogue up and running. We can use the Wiki itself as a starting point, making sure that the metadata are stored in a way here that enables us to later expand on the catalogue (both in terms of the metadata fields and the way to handle them) and e.g. build a website based on the Wiki page or even a stand-alone application (something like this: Tcl-Pkg). This especially means that the metadata cannot be stored as free-format Wiki text but rather in a structured way. For this, we will use the new (as of 2024-10-15) <!DOCTYPE TCLPART> of the Wiki. This allows us to write the metadata as Tcl code and thus have it in a future-safe and structured form. The visualization is then done by the Wiki engine.

The first step is the definition of proper metadata fields. Let's use something that is well-defined instead of inventing our own metadate scheme. So we will adopt the CodeMeta project and thus use metadata terms mainly from https://schema.org and supplemented with terms from the CodeMeta project.

Metadata fields

This is the list of metadata fields that can be supplied for a package. The metadata term comes first and in parenthesis is a still very short description on the content behind that term. Some of the terms can occur multiple times, e.g. "author", "version", "maintainer", :

• name (package name)
• version (package version)
• description (package description)
• applicationCategory (package category)
• applicationSubCategory (package subcategory)
• author (the original author of the package)
• contributor (people not being the author(s) nor the maintainer(s) but having contributed to the code substantially
• maintainer (currently being worked on by/claimed by)
• codeRepository (URL homepage)
• issueTracker (URL bug tracker)
• datePublished (release date)
• downloadURL
• programmingLanguage (Tcl only, Tk, C, ...)
• operatingSystem (where the package is running: macOS, Windows, Linux, ...)
• developmentStatus (abdandoned, active, ...)
• license
• sameAs (e.g. to link to another package which is just a fork of this one or a re-implementation)
• softwareHelp (URL of man pages etc)

In addition to the above, the catalogue should include some metadata fields that are not in the CodeMeta schema:

• source of this information (URL, pers. comm.)
• release announcement URL (GUTTER has it)
• NRE enabled (whether the package uses commands based on NRE: the non-recursive engine in Tcl 8.6)

Storage format

The metadata are stored as YAML inside a Tcl array. This is also inspired by TIP 55 which proposed RFC 2822 (mail message encoding). As we want more depth than just a simple key-value-storage, YAML is the natural way to go as is can be regarded as a superset of RFC 2822, just being able to store a hierarchical structure instead. Each package has its own array element in the catalogue and the metadata fields are then encoded in YAML. Thus, the metadata is easy to read and write by humans who maintain the catalogue and the metadata can also easily be transferred to a database or whatever format that catalogue will end up with in future.

Implementation

Step 1: Settle which metadata to use

Realizing that there already is a TIP 55 on packages and metadata and an associated Wiki page Package MetaData Fields, the above list should be mapped to the TIP or even merged with it. We do not care too much whether the terms are clean in the sense that they come from one individual source. They should just be kind of standardized and TIP 55 operates based on DublinCore which is fine.

On the other hand, DublinCore is very generic and the names for the terms may be confusing when interpreting them in terms of software. This is why we start with CodeMeta for now.

Step 2: Prepare some example metadata sets

I have done this step for now together with step 3 and put the example metadata catalogue on the same page as the code producing the output from the catalogue. Later, it will probably be good to have only the metadata on one page, then include this on the other page doing the rendering.

Step 3: Prepare a Wiki page showing the metadata as a catalogue

The page Tcl package catalogue on the Wiki is soon have an example of how that catalogue can easily be built from the metadata, taking two random packages as an example.

Discussion