Batteries Included

What would you, as a Tcl programmer, like to see included in a Tcl distribution such as these?

LV Well, for example, I'd like to see the following:

  • Tcl core distribution

A wide variety of extensions such as:

and perhaps some lesser known, but useful extensions such as

Applications such as:

Where did the term "Batteries Included" come from?

http://www.python.org/pics/batteries-included.jpg

The use of the term "Batteries Included" in the Tcl community began based on Frank Stajano's paper (formerly at http://www.uk.research.att.com/~fms/ipc7/tr-1998-9.html , now found at http://www.cl.cam.ac.uk/~fms27/papers/tr-1998-9.pdf ) (first read by many through a magazine article [L1 ]) on why he chose Python over Tcl. His point was that Python (and this is largely true for such other languages as Perl) came out of the box with more than a minimal language implementation - instead, Python comes with a variety of add on components, sometimes referred to as modules, extensions, packages, etc. that could be compiled and installed during installation. These add-ons provide commands that people have found increasingly important in getting their jobs done.

Unfortunately, though, this is just a gross misunderstanding. I don't know if anyone in the Tcl community has actually ever compiled a Python distribution from source. If you have, you know this "Batteries included" propaganda is very much bs. I'd say it's about the same functionality you see in our regular Tcl distribution. Modules for introspection, language transforms, interprocess communication and the like.

If you fall for the hype, you'd believe this "Batteries included" thing means Python (or Perl, or Ruby for that matter) comes out of the box with modules for database access (there's dbm), web scripting framework (well, there's the regular CGI handler and some html and xml parsers and generators) or anything like that.

Quite frankly, putting it all together would be a maintainer hell, because they're all separate projects building upon the same foundation: a good language and base libraries. That's why Tcl, Perl, Python and Ruby come like that...

rmalafaia -=@=- gmail_dot_com

LV No, I don't think there is a gross misunderstanding. I've built Python, Perl, and Tcl all from their source distributions, and the first two both come with more modules/functions, out of the core distribution, than does Tcl. The BI argument is that this results in someone getting a wider variety of work done faster - there is a lesser need to go running all over the internet looking for, downloading, building, and installing additional software pieces. If you think the 3 languages come with comperable functionality, then you have never sat down and actually taken a look at the facts.

If one compares ActiveTcl - a BatteriesIncluded binary distribution - against the results of building the other two languages, one certainly draws closer to equal functionality.

SG I'd say this appears to be the opposite from how I understand the term "batteries included". When I hand someone a floppy with a couple hundred kB of TCL to do some kind of job, I can throw a Tclkit onto the floppy as well (or if I want to get fancy I can wrap it into a starkit). That recipient can then run my code. Nothing else needed: Batteries included.

On the other hand, nobody has ever handed anybody a floppy with any kind of Python (Perl, Java, VB) code and it "just worked" -- all of these require that you download hundreds of MB of assorted junk, compile for hours or otherwise run obscure "installer packages" which install all kinds of crap that you don't know and can't opt out of - like java's "jusched" which will from now on run whenever your computer is on and you cannot turn it off. Cluttering your computer with directories (right up there in the root! You cannot have Python anywhere but C:\ unless you're willing to take a lot of pain!) that you often can't even recognize, frequently without even a proper uninstall option. And if you're unlucky, that's still not enough, because the developer used some clever little trinket that e got from who-knows-where...

Maybe I misunderstand the expression "batteries included" (I'm not an English native speaker) but when I hear it I don't think of ActiveTcl (which is just as monstrous as Python), I think of TclKits and Starkits which do, in fact, include absolutely everything you need to run the code you're distributing. Nothing else needed: Batteries included.

LV I understand SG's use of the term. However, that use of the term differs - not because of the English language, but within the Tcl community, based on the Stajano paper mentioned earlier on this page.

More modules/functions, out of the core distribution

RMSB: Last time i checked, there were like 4 modules to deal with html documents in the standard Python source distribution: some of them dealt with obsolete formats like HTML3.0 and earlier. There were also lots of tiny modules to deal with such abstract activities such as generating AST trees from source code, serializing objects, compiling, creating parsers, interfacing with the native platform and the likes. Some 4 different xml parsers, various modules to help Python "emulate" a user-defined behaviour for basic Python datatype classes, cgi assorted stuff etc. At least, it comes with a set of interesting multimedia-related stuff.

I ask: is this what you mean by a "Batteries Included" distribution? If yes, I have to admit that sounds like pretty low profile to me.

A "Batteries Included" distribution should come with robust modules for doing getting real work done: DBMS access drivers and higher-level access abstractions, good support for popular network protocols, libxml2 interfaces for dealing with a world set into xml craze, a good GUI-framework for end-user application development, and perhaps a servlet engine for web applications too.

Now that is a "Batteries Included" distribution, my friend.

Python certainly ain't that, nor Perl, Tcl or Ruby source distributions.

If you think the 3 languages come with comperable functionality, then you have never sat down and actually taken a look at the facts.

Sorry, just got into an installation fest over the weekend, which is why i found this topic so fascinating in the first place. Installed, from source: the latest Apache2, Python2.4, Ruby1.8, Tcl8.4, Pgdb, Pgtcl, ruby DBI::Pg and mod_python, mod_tcl and mod_ruby ( also Rails! ). And Postgres8.0, finally! :)

I'm not that much into Perl anymore, but i'm very anxious about Perl6 ( and Parrot as a general-purpose VM for dynamic languages ).

After setting all up, yes, i can say they are all pretty much equivalent. Similar functionality, some nice unique bright spots here and there, but pretty much all of the same. And not quite much of what i need for getting things done. I mentioned i had to install a Postgresql interface and Apache modules for each one of them, right?

So, please, don't come at me assuming i'm talking out of my ass and telling things I now for myself aren't true. I don't buy it.

LV Okay then, I guess that our definitions of equivalent vary quite a bit. And that's fine - conflict based on differing expectations is probably the most common one. And it's an irrelevant argument, since nothing said here matters anyways. The only thing that matters is people deciding to do something, or nothing. And given the current state of affairs, nothing major is going to be happening in this arena.

RMSB But really, is there a problem with that? A "Batteries Included" concept doesn't really get along very well with the open source model, kinda like IDEs. And, frankly, in this day and age of such sweet and painless extension building is it really so boresome to download the packages you need and build and install them? I just said i installed the whole "AMP(T|R)" part of LAMP in a freegin' weekend from source. What it took, like, other than "./configure; make; make install"?

painless...

LV Giggle - the comment about doesn't really get along well seems so disjoint from the state of perl, eclipse, python, java, emacs, etc. where massive amounts of functionality is frequently bundled into a simple distribution. But of course, nothing said here is going to convince someone who is determined to be opposed to the idea.

Some time after this page was started, George Howlett submitted http://purl.org/tcl/tip/12.html as a description for a comprehensive Tcl/Tk distribution. While at least one business has put together a comprehensive distribution for the 3 major platforms on the desktop (Windows, MacOS, and Linux), along with commercial versions for a few other platforms, there really hasn't been much done by the Tcl community as a whole.

Keeping up with your batteries included Tcl distribution!

Vince: The problem with all of the efforts listed below is that they rely upon one person (or a v. small number of people) to create, compile, build, package and distribute the 'batteries included distribution'. This means they are always limited in scope, and evolve very slowly (of course this might make them more stable etc. which some people want, but I think the Tcl community realises it needs to evolve more quickly in at least some directions).

Why can't we create a web-based repository, perhaps using the semi-standard 'ARCH' directory structure (from tclkit etc) for locating platform/architecture dependent binaries, in which _anyone_ can upload extensions, whether they are script-only, compiled, or a combination of the two. One could provide a small .tcl script which tells the user exactly what platform/architecture combination they should use (if this catches on one could even write a small tk app which does all the uploading for you).

If there is then an automated interface for requesting particular 'batteries included' versions (e.g. "I want the latest 'stable' Windows distribution"), which triggers a script on the server to zip or tar up the appropriate files, then we would soon have something which could evolve as fast as people wanted it to.

Any thoughts?

(1) Helpful to be able to ask the server for an overview of what packages are available and what platform/architectures are missing certain extensions.

(2) Might be useful also to tag extensions by what Tcl version they work with (so I request "I want the latest Windows distn with Tcl 8.2 compatible extensions").

LV Have you seen the work on CANTCL? There seems to be some overlap with what you describe. I think that jcw has had a variety of suggestions as well.

The real meat of your question is "Why can't we ...". Obviously, there are no technology issues preventing anyone from trying to build such a thing. The issue is one of organization - to date, there have been a lot of people discussing / wishing for such a thing, and a significantly smaller number of people willing to work toward such a thing.

LV In 2007, ActiveState began making access to the teapot repository freely available to the 3 major desktops (Windows, MacOS, and Linux).

List of Tcl distributions

There have been a number of efforts that have been mounted over the years to create a Tcl distribution that would meet similar needs. I would like for our community to add to this list as they become aware of additional efforts.

ActiveState's ActiveTcl Distribution
dqkit
eTcl
GarTcl
kbs (aka Tclkit Kitgen Build System)
Can build user defined batteries included interpreter
NeoTcl
Tclkit / Kitten
TclTkAquaBI
TixTclkit
WinTclTk
kbskit
Cygwin cygport scripts
Although binaries aren't currently available, the Cygwin cypgort scripts make it fairly painless to install a superset of the packages provided by ActiveTcl for use on a Windows machine.

Most Linux distributions come with Tcl/Tk

A different approach, which differs from the all in one file type approach, but which has existed for quite some time, is the centralized binary repository contribution of various Linux distributions such as RedHat, Suse, etc. where the Linux distribution comes with Tcl, Tk and a number of other extensions out of the box, with binary packages available from a central distribution point for a number of other add-on packages.

(Suse ships with tcl, tk, itcl, blt, bwidgets, expect, snack, tcllib, the plugin, tclx, tix and vtcl)

LV 2007 Oct 2 One of the downsides I've seen with these types of collections is that they appear to seldom have a Tcl regular actively maintaining the code that is made available. So you see Solaris shipping with TclX 8.3.3 and its tcl command - no tclsh, no Tk!, but with expect. Or you see Redhat with 8.4.7 during 2007, and shipping with Tk and Tclx only. From what I can see, Redhat (I'm talking about the commercial version - not the community distribution) only makes a few additional extensions available through their commercial license distribution packaging mechanism. But then, perhaps it is only that, via their web site, I can only find a few vague references to tcl.

Other operating systems have similar things - for instance, Solaris comes with a CD of freely distributable binary builds, and that includes Tcl and Expect (but not Tk!).

Tom Wilkason's binary distribution

 What:        Batteries Included Distribution
 Where:       http://members.home.net/tfwilkason/tcl/Batteries/ - No longer provided due to space limitations
 Description: Binary distributions of Tcl with a variety of extensions
              included.
 Updated:     14 Feb 2003
 Contact:     mailto:[email protected] (Tom Wilkason)

LV Another approach is that taken by The Cool Store, where platform-specific applications are dynamically generated for download.

[ CL intends to write on Tcl as a gluing toolkit, and the administrative overhead of license issues, in regard to extensions.]

LES on Oct 30 2003: Freewrap bundles several extensions. Once it is renamed, it behaves just like the wish executable and can also run scripts. Doesn't it qualify as a "Batteries Included" distribution? And TixTclkit [L2 ], doesn't it too?

CMcC: I don't want to be contentious, but ... I wonder about the wisdom of a language which is distributed without batteries.

There are fundamentally and generally useful facilities which aren't in the Tcl system (e.g. some file and signal components of TclX).

There are incompatible implementations of some commands (e.g. fork in expect and TclX)

There are disparate interfaces to facilities (e.g. file and TclX's file commands.)

There are whole packages which one might not want to use, but which contain a single useful facilities - (e.g. expect's process and pty facilities)

There are useful facilities implemented in completely different ways (e.g. comm's implementation of OO, or ensemble, use of Snit vs Stooop) which has to be inefficient.

This all comes down to a couple of lacks in tcl, IMHO. Tcl has no uniform binary packaging/distribution technology ... rather it has several ... and that's a charitable interpretation: TEA, TEA2, autoconf - make - install, wish install.tcl, StarKit, critcl etc etc.

LV Well, TEA2 (and TEA3, etc.) are just updated versions of TEA. As for not having "uniform binary packaging" - is there a language or platform which does? I mean, I see on Windows people who distribute using msi, .exe, .zip, and source files. Basically, as long as humans are involved, someone is going to believe they have a better idea and will do things differently. However, the majority of Tcl extensions use packaging based on TEA.

[some anonymous writer]

It occurs to me that the relatively high (perceived) difficulty of newbie acquisition of tcl extensions is due to the disparate nature of tcl extension packaging technologies.

Another way to look at it is that newbies perceive Tcl as lacking facilities which PERL, et al, provide. This perception is not true, although it is strictly true - Tcl doesn't provide the facilities although there are many extensions to provide them, but it requires specialist skill and some dedication to acquire them. Core Tcl provides little or no effective support for that task, which is ironic for a language which was designed as glue.

This is not intended to be any kind of personal attack, but ... I feel that keeping this confusion may be, perversely, in the short-term interests of those who want to commercialize Tcl. I feel that it is to their long-term detriment.

LV Alas, it doesn't appear likely, in my lifetime at least, that this issue will ever be resolved. Again, as was mentioned earlier, to make the kinds of changes you suggest requires an increased level of effort from community members. Those increased levels of involvement occur in waves, and then return to a state of calm. The current functionality of Tcl is where the language has grown as a result of this cultural mindset. A change in that mindset seems unlikely - there have been 10+ years of passionate debate on the topic, and you can look at the Tcl and Tk source distributions to see how many people moved from talking to coding and coordinating changes. This is to in no way diminish the vast and wonderful work done to date in the core. This is only to point out that many, many, many more people write about wanting change than actually step up and begin doing anything about it. Many use the self-defense of oh, I'm not qualified to do anything - to that, I simply snort...

mh It's also indicative of the headaches people have trying to introduce these changes. After a while it just seems easier to put one's energies into some other effort.

LV Note that several activities are in motion which are going to impact the Tcl community in relationship to this topic. One positive action is the introduction of the teapot Tcl package repository at ActiveState. This appeared in the ActiveTcl 8.4.14.x release, and provides an interface to retrieving code from a server (conceptually similar to Perl's CPAN functionality).

One negative action, in my mind, is the transformation of ActiveTcl 8.5 beta from a "Batteries Included" distribution to a "Batteries freely available" distribution model. That is to say, ActiveTcl 8.5 beta comes with Tcl, Tk, and trofs (Tcl Read-Only Filesystem). Everything else is available via teacup.

The negative aspect of this, to me, has to do with creating a Batteries Included Tcl installation setup on a machine without internet access (or at least without ability to add extensions after the initial installation due to Windows Vista or XP security blocking, etc.) With the new setup, the person downloading activetcl's binary installation image also has to interact with teapot, downloading the entire archive, then figure out how to write it all out in a manner than will then be able to be used on the destination system. And that's no longer "Batteries Included", at least for me.

MHo 2008-01-01: Yes - yes - yes. I totally agree with the previous statement. Finally having a repository does not neccessarily imply to leave out the extensions from the distribution. There are too many PCs without or without a fast connection. In the past, after just one or two clicks everyone, even a computer novice, had a complete, ready to use distribution. Now the installation has to be completed by some self written scripts. I think I'll need a long period of experimentation before I finally could switch every installation from 8.4 to 8.5....

EKB I also agree. I think batteries included is important. I distribute .exe's using freewrap, but when developing applications I sometimes ask my colleagues to download AT and run the app using wish. They're impressed with how smooth the process is and how easy it is to get up and running. But this only works because there's no learning on their end -- they have no intrinsic interest in Tcl. I'm hoping that one of these days it will switch from being "that weird language that Eric uses" to something that's used more widely, but if there are hurdles then it isn't going to happen.

I'm trying not to complain too much, since AS provides ActiveTcl for free and since I wasn't reviewing betas and so was out of a loop I could have tapped into. Anyway, other languages/platforms use a similar distribution system. But I can't help feeling a little bummed, because I was hopeful that via a batteries included distro I can eventually get more of my colleagues interested in Tcl. I think that the teacup approach to distribution works fine for Perl and Java (is it true that this is how Java works?) because they get so much airplay, so people are willing to work at it. It also works for Ubuntu, because Ubuntu's not aiming at power users (although power users can also use it). A regular user of an OS is wowed by an attractive, usable interface, and is overwhelmed (not wowed) by too much extra stuff. But a developer who regularly writes code is going to be wowed by the list of available libraries.

AW I also agree. The internet connection is less of an issue to me, but I distribute a small cross platform application among developers internally in my company. For now, I told everyone that needed it to download ActiveTcl. I limited myself to the package included with that, and everything worked. Now however, it seems that I will not only have to tell them to use some arcane command line tool to get a bunch of stuff they never heard about (and should not need to hear about), or write a script for every platform to do that. Worse still, every time I need to use a new package in a new version, I'm going to have to go around and update everyone. So while before, the problem was I limited myself to package installed with ActiveTCL, I'm now going to limit myself to using the packages I installed first time around.

And then teacup. I haven't used it, since I haven't installed 8.5. However, from what I read, it doesn't work out of the box in 8.5.0 (need to copy tclsh8.5.exe to tclsh.exe or something to make it go?), and is command line only. AFAIK ActiveTcl has provided a GUI installer for ages, so what on earth possessed anyone to think going back to command line stuff to complete the install was a good idea? (this is 2008, not 1978!)

I have looked at the list of other distributions, but there appear to be none that are anywhere near as complete as ActiveTcl 8.4.x was? I can only hope for some improvement, either with a BI 8.5.1, or a GUI for teacup (which is still a horrible user experience, as I know from being on the other end of things for installing cygwin, where a colleague has made something that needs a lot of 'stuff' you need to select in the cygwin installer (from in between an enormous amount of stuff you don't need to select)).

LV One thing that people with good internet connections could consider is writing a bit of code that did something like:

 See if package ABC exists
 If not, invoke teacup install ABC
 See if package ABC exists
 If not, then report that the application isn't currently compatible with the platform in question.

I'm guessing that is about a dozen lines of Tcl - maybe a bit less.

This is an issue of deployment. An alternative solution to asking users to download activetcl that is often used in these cases is that of tclkit/starkit distributions. In this concept, one takes their application as well as the extensions needed, puts them into a particular directory structure, test with a single file tcl interpreter, then creates a distribution image called a starkit. Then the instructions for the user is basically "download this .kit file and the tclkit that runs on your platform, put them both into a directory, and run the .kit" .

Roy Keene I've created a web interface to build Batteries Included versions of TclKits ("CustomKits", to avoid confusion) [L3 ]. It uses ActiveState's teapot repository as a source for extensions. It's not clear whether or not this is an acceptable use of the teapot repository.

Roy Keene This web interface has been removed since it has been determined that this is not an acceptable use of the teapot repository.

gasty About this topic, which can be an "acceptable use of the teapot repository"? Would it be fine if we can share a little app as an starpack (with tcl/tk interpreter for a given platform and pure tcl sources) and then make an "installer" that retrieve the necesary packages from the teapot repo for that platform, maybe after ask the user if agree with ActiveTcl license terms. This can be acceptable, legally speaking?

About the page main topic, I think THE great advantage of Tcl over others languages is the posibility of make very small programs with only the required modules. So, the batteries included distributions (say ActiveTcl) are useful for developers to try easily several packages; and the small, self-contained ones (say tclkit) are the primary way of share applications for end users, with only the needed packages.

LV I find myself back, again, looking around the community to see what Batteries Included alternatives people are trying. I've seen a few, but most involve gatheringn various extensions from various web sites, attempting to build, discovering that various tools or additional libraries and/or packages are needed, going to track down the appropriate versions of those, and cycling through this until one either gives up exhausted or perserveres.

Category Distribution
Updated 2015-11-22 02:18:46