Version 114 of Batteries Included

Updated 2007-10-08 16:16:01 by LV

What would you, as a Tcl programmer, like to see included in a Tcl distribution such as these?

LV Well, for example, I'd like to see the following:

A wide variety of extensions such as:

and perhaps some lesser known, but useful extensions such as

Applications such as:


Where did the term "Batteries Included" come from?

The use of the term "Batteries Included" in the Tcl community began based on Frank Stajano's paper [L1 ] (first read by many through a magazine article [L2 ]) on why he chose Python over Tcl. His point was that Python (and this is largely true for such other languages as Perl) came out of the box with more than a minimal language implementation - instead, Python comes with a variety of add on components, sometimes referred to as modules, extensions, packages, etc. that could be compiled and installed during installation. These add-ons provide commands that people have found increasingly important in getting their jobs done.

LV NOTE: the paper and magazine article URLs above aren't working - anyone have working versions somewhere?


Unfortunately, though, this is just a gross misunderstanding. I don't know if anyone in the Tcl community has actually ever compiled a Python distribution from source. If you have, you know this "Batteries included" propaganda is very much bs. I'd say it's about the same functionality you see in our regular Tcl distribution. Modules for introspection, language transforms, interprocess communication and the like.

If you fall for the hype, you'd believe this "Batteries included" thing means Python (or Perl, or Ruby for that matter) comes out of the box with modules for database access (there's dbm), web scripting framework (well, there's the regular CGI handler and some html and xml parsers and generators) or anything like that.

Quite frankly, putting it all together would be a maintainer hell, because they're all separate projects building upon the same foundation: a good language and base libraries. That's why Tcl, Perl, Python and Ruby come like that...

rmalafaia -=@=- gmail_dot_com


LV No, I don't think there is a gross misunderstanding. I've built Python, Perl, and Tcl all from their source distributions, and the first two both come with more modules/functions, out of the core distribution, than does Tcl. The BI argument is that this results in someone getting a wider variety of work done faster - there is a lesser need to go running all over the internet looking for, downloading, building, and installing additional software pieces. If you think the 3 languages come with comperable functionality, then you have never sat down and actually taken a look at the facts.

If one compares ActiveTcl - a BatteriesIncluded binary distribution - against the results of building the other two languages, one certainly draws closer to equal functionality.


More modules/functions, out of the core distribution

RMSB: Last time i checked, there were like 4 modules to deal with html documents in the standard Python source distribution: some of them dealt with obsolete formats like HTML3.0 and earlier. There were also lots of tiny modules to deal with such abstract activities such as generating AST trees from source code, serializing objects, compiling, creating parsers, interfacing with the native platform and the likes. Some 4 different xml parsers, various modules to help Python "emulate" a user-defined behaviour for basic Python datatype classes, cgi assorted stuff etc. At least, it comes with a set of interesting multimedia-related stuff.

I ask: is this what you mean by a "Batteries Included" distribution? If yes, I have to admit that sounds like pretty low profile to me.

A "Batteries Included" distribution should come with robust modules for doing getting real work done: DBMS access drivers and higher-level access abstractions, good support for popular network protocols, libxml2 interfaces for dealing with a world set into xml craze, a good GUI-framework for end-user application development, and perhaps a servlet engine for web applications too.

Now that is a "Batteries Included" distribution, my friend.

Python certainly ain't that, nor Perl, Tcl or Ruby source distributions.

If you think the 3 languages come with comperable functionality, then you have never sat down and actually taken a look at the facts.

Sorry, just got into an installation fest over the weekend, which is why i found this topic so fascinating in the first place. Installed, from source: the latest Apache2, Python2.4, Ruby1.8, Tcl8.4, Pgdb, Pgtcl, ruby DBI::Pg and mod_python, mod_tcl and mod_ruby ( also Rails! ). And Postgres8.0, finally! :)

I'm not that much into Perl anymore, but i'm very anxious about Perl6 ( and Parrot as a general-purpose VM for dynamic languages ).

After setting all up, yes, i can say they are all pretty much equivalent. Similar functionality, some nice unique bright spots here and there, but pretty much all of the same. And not quite much of what i need for getting things done. I mentioned i had to install a Postgresql interface and Apache modules for each one of them, right?

So, please, don't come at me assuming i'm talking out of my ass and telling things I now for myself aren't true. I don't buy it.


LV Okay then, I guess that our definitions of equivalent vary quite a bit. And that's fine - conflict based on differing expectations is probably the most common one. And it's an irrelevant argument, since nothing said here matters anyways. The only thing that matters is people deciding to do something, or nothing. And given the current state of affairs, nothing major is going to be happening in this arena.

RMSB But really, is there a problem with that? A "Batteries Included" concept doesn't really get along very well with the open source model, kinda like IDEs. And, frankly, in this day and age of such sweet and painless extension building is it really so boresome to download the packages you need and build and install them? I just said i installed the whole "AMP(T|R)" part of LAMP in a freegin' weekend from source. What it took, like, other than "./configure; make; make install"?

painless...

LV Giggle - the comment about doesn't really get along well seems so disjoint from the state of perl, eclipse, python, java, emacs, etc. where massive amounts of functionality is frequently bundled into a simple distribution. But of course, nothing said here is going to convince someone who is determined to be opposed to the idea.


Later (over a year ago!), George Howlett submitted http://purl.org/tcl/tip/12.html as a description for a comprehensive Tcl/Tk distribution.


Keeping up with your batteries included Tcl distribution!

Vince: The problem with all of the efforts listed below is that they rely upon one person (or a v. small number of people) to create, compile, build, package and distribute the 'batteries included distribution'. This means they are always limited in scope, and evolve very slowly (of course this might make them more stable etc. which some people want, but I think the Tcl community realises it needs to evolve more quickly in at least some directions).

Why can't we create a web-based repository, perhaps using the semi-standard 'ARCH' directory structure (from tclkit etc) for locating platform/architecture dependent binaries, in which _anyone_ can upload extensions, whether they are script-only, compiled, or a combination of the two. One could provide a small .tcl script which tells the user exactly what platform/architecture combination they should use (if this catches on one could even write a small tk app which does all the uploading for you).

If there is then an automated interface for requesting particular 'batteries included' versions (e.g. "I want the latest 'stable' Windows distribution"), which triggers a script on the server to zip or tar up the appropriate files, then we would soon have something which could evolve as fast as people wanted it to.

Any thoughts?

(1) Helpful to be able to ask the server for an overview of what packages are available and what platform/architectures are missing certain extensions.

(2) Might be useful also to tag extensions by what Tcl version they work with (so I request "I want the latest Windows distn with Tcl 8.2 compatible extensions").


LV Have you seen the work on CANTCL? There seems to be some overlap with what you describe. I think that jcw has had a variety of suggestions as well.

The real meat of your question is "Why can't we ...". Obviously, there are no technology issues preventing anyone from trying to build such a thing. The issue is one of organization - to date, there have been a lot of people discussing / wishing for such a thing, and a significantly smaller number of people willing to work toward such a thing.

LV In 2007, ActiveState is working on teapot.


List of Tcl distributions

There have been a number of efforts that have been mounted over the years to create a Tcl distribution that would meet similar needs. I would like for our community to add to this list as they become aware of additional efforts.


Most Linux distributions come with Tcl/Tk

A different approach, which differs from the all in one file type approach, but which has existed for quite some time, is the centralized binary repository contribution of various Linux distributions such as RedHat, Suse, etc. where the Linux distribution comes with Tcl, Tk and a number of other extensions out of the box, with binary packages available from a central distribution point for a number of other add-on packages.

(Suse ships with tcl, tk, itcl, blt, bwidgets, expect, snack, tcllib, the plugin, tclx, tix and vtcl)

LV 2007 Oct 2 One of the downsides I've seen with these types of collections is that they appear to seldom have a Tcl regular actively maintaining the code that is made available. So you see Solaris shipping with TclX 8.3.3 and it's tcl command - no tclsh, no Tk!, but with expect. Or you see Redhat with 8.4.7 during 2007, and shipping with Tk and Tclx only. From what I can see, Redhat (I'm talking about the commercial version - not the community distribution) only makes a few additional extensions available through their commercial license distribution packaging mechanism. But then, perhaps it is only that, via their web site, I can only find a few vague references to tcl.

Other operating systems have similar things - for instance, Solaris comes with a CD of freely distributable binary builds, and that includes Tcl and Expect (but not Tk!).


Tom Wilkason's binary distribution

 What:        Batteries Included Distribution
 Where:       http://members.home.net/tfwilkason/tcl/Batteries/ - No longer provided due to space limitations
 Description: Binary distributions of Tcl with a variety of extensions
              included.
 Updated:     14 Feb 2003
 Contact:     mailto:[email protected] (Tom Wilkason)

LV Another approach is that taken by The Cool Store, where platform-specific applications are dynamically generated for download.


[ CL intends to write on Tcl as a gluing toolkit, and the administrative overhead of license issues, in regard to extensions.]


LES on Oct 30 2003: Freewrap bundles several extensions. Once it is renamed, it behaves just like the wish executable and can also run scripts. Doesn't it qualify as a "Batteries Included" distribution? And TixTclkit [L3 ], doesn't it too?


CMcC: I don't want to be contentious, but ... I wonder about the wisdom of a language which is distributed without batteries.

There are fundamentally and generally useful facilities which aren't in the Tcl system (e.g. some file and signal components of TclX).

There are incompatible implementations of some commands (e.g. fork in expect and TclX)

There are disparate interfaces to facilities (e.g. file and TclX's file commands.)

There are whole packages which one might not want to use, but which contain a single useful facilities - (e.g. expect's process and pty facilities)

There are useful facilities implemented in completely different ways (e.g. comm's implementation of OO, or ensemble, use of Snit vs Stooop) which has to be inefficient.

This all comes down to a couple of lacks in tcl, IMHO. Tcl has no uniform binary packaging/distribution technology ... rather it has several ... and that's a charitable interpretation: TEA, TEA2, autoconf - make - install, wish install.tcl, StarKit, critcl etc etc.


LV Well, TEA2 (and TEA3, etc.) are just updated versions of TEA. As for not having "uniform binary packaging" - is there a language or platform which does? I mean, I see on Windows people who distribute using msi, .exe, .zip, and source files. Basically, as long as humans are involved, someone is going to believe they have a better idea and will do things differently. However, the majority of Tcl extensions use packaging based on TEA.


[some anonymous writer]

It occurs to me that the relatively high (perceived) difficulty of newbie acquisition of tcl extensions is due to the disparate nature of tcl extension packaging technologies.

Another way to look at it is that newbies perceive Tcl as lacking facilities which PERL, et al, provide. This perception is not true, although it is strictly true - Tcl doesn't provide the facilities although there are many extensions to provide them, but it requires specialist skill and some dedication to acquire them. Core Tcl provides little or no effective support for that task, which is ironic for a language which was designed as glue.

This is not intended to be any kind of personal attack, but ... I feel that keeping this confusion may be, perversely, in the short-term interests of those who want to commercialize Tcl. I feel that it is to their long-term detriment.


LV Alas, it doesn't appear likely, in my lifetime at least, that this issue will ever be resolved. Again, as was mentioned earlier, to make the kinds of changes you suggest requires an increased level of effort from community members. Those increased levels of involvement occur in waves, and then return to a state of calm. The current functionality of Tcl is where the language has grown as a result of this cultural mindset. A change in that mindset seems unlikely - there have been 10+ years of passionate debate on the topic, and you can look at the Tcl and Tk source distributions to see how many people moved from talking to coding and coordinating changes. This is to in no way diminish the vast and wonderful work done to date in the core. This is only to point out that many, many, many more people write about wanting change than actually step up and begin doing anything about it. Many use the cop-out of oh, I'm not qualified to do anything - to that, I simply snort...

mh It's also indicative of the headaches people have trying to introduce these changes. After a while it just seems easier to put one's energies into some other effort.


LV Note that several activities are in motion which are going to impact the Tcl community in relationship to this topic. One positive action is the introduction of the teapot Tcl package repository at ActiveState. This appeared in the ActiveTcl 8.4.14.x release, and provides an interface to retrieving code from a server (conceptually similar to Perl's CPAN functionality).

One negative action, in my mind, is the transformation of ActiveTcl 8.5 beta from a "Batteries Included" distribution to a "Batteries freely available" distribution model. That is to say, ActiveTcl 8.5 beta comes with Tcl, Tk, and trofs (Tcl Read-Only Filesystem). Everything else is available via teacup.

The negative aspect of this, to me, has to do with creating a Batteries Included Tcl installation setup on a machine without internet access (or at least without ability to add extensions after the initial installation due to Windows Vista or XP security blocking, etc.) With the new setup, the person downloading activetcl's binary installation image also has to interact with teapot, downloading the entire archive, then figure out how to write it all out in a manner than will then be able to be used on the destination system. And that's no longer "Batteries Included", at least for me.


Category Distribution