Starkit

A Starkit , first devised by Jean-Claude Wippler, is a single file containing a directory in which are found Tcl scripts, platform-specific compiled code, and application data, and which can be appended to a Tclkit to form a starpack.

where "Star" is derived from STandAlone Runtime,

Description

A kitsh is a modiied Tcl interpreter capable of loading and executing a starkit. It automatically loads TclVFS and mounts the virtual filesystem in the starkit. This filesystem may be writable, allowing the program to save data back into the starpack at runtime. A starkit can be wrapped together with a Tclkit into a starpack that is executble on a particular platform.

There are three ways to run a starkit:

Invoke a starpack that includes both a Tclkit providing a kitsh and a starkit that the kitsh automatically finds and mounts:

mystarpack

Invoke a Tclkit, passing the starkit as an argument:

mytclkit mystarkit.kit

As detailed below, unpack the starkit, invoke an interpreter equipped with the prerequisites like tclvfs, and pass as an argument the main.tcl file found in the starkit:

what we have here is simply a Tcl version of Java ".jar" files, with the runtime environment optionally bound in. ... As for "simply jar files": not quite, SD's are r/w with transaction-safe commit/rollback because there's a database engine underneath. That means they can also store config info, update themselves, and store extensive datasets. -jcw

Documentation

Beyond TclKit - Starkits, Starpacks and other *stuff: Presented by Steve Landers at the Tcl/Tk 2002 conference in Vancouver.

Anatomia di uno StarKit ,by giorvio_v: A short summary of starkits, in Italian.

Community

official mailing list: For discussing tclkit/starkit/starpack ideas, development, use, etc.

Available Starkits

Starkits that have been published by various parties:

The Tcl Developer Exchange starkit repository

Building a Starkit

Building a Starkit is easy. Tclkit provides all the infrastructure, and sdx provides the tools.

Build Your First Starkit: How to convert a single-file application into a starkit.
A Simple Multi-File Starkit Example: How to create a multi-file starkit.
Complex Pure Tcl Starkit Example: Bundling a large application into a starkit.
Starkits with Binary Extensions: Create cross platform, compiled applications.
Starkits containing Tcl only extensions
So You Want to Use Starkits, Eh?: Contains another good example.

Run a Starkit with any tclsh

A Tclkit is not strictly necessary to execute a starkit. To execute a starkit with any tclsh:

Install tclvfs.
Obtain the kitgen source code.
Add path to kitgetn sources/8.x/tclvfs to TCLLIBPATH.
Unpack the .kit file using sdx.
Invoke tclsh <path to unpacked .kit file>/main.tcl.

Commands

Could someone list here the various starkit namespace procs that are available for use?

For instance, there is:

starkit::pload - use like this:

package ifneeded Mk4tcl 2.4.9.2 "package require starkit [list starkit::pload $dir Mk4tcl Mk4tcl]"

Variables

starkit::mode - The mode of the starkit. It can have the following values

      - starpack   when run from the binary/exe
      - tclhttpd   when run from a tclhttpd (proc Httpd_Server must exist)
      - plugin     when variable ::embed_args exists
      - service    when variable ::tcl_service exists (Windows only)
      - starkit    when called via tclsh myapp.kit
      - unwrapped  when called via tclsh myapp.vfs/lib/app-myapp/myapp.tcl
      - sourced    otherwise

starkit::topdir - The path of the "root" directory. For a starpack it is the full path of your binary/exe.

Adding Encodings

David Zolli mentioned on the starkit mailing list these steps for adding encodings (since tclkit only has a subset of the encodings that normal Tcl has - one of the (few) differences between Tcl and Tclkit...)

I've tried and tested this, and it's works:

Unpack your starkit if is not already done.
Create the directory .../appname.vfs/lib/tcl8.4/encoding
Copy the needed encoding(s) .enc file(s) in this directory (from an existing tcl installation or from tcl CVS repository here : http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/tcl/tcl/library/encoding/ if you using the last tclkit).
Rewrap your starkit with these new files.

Then, these encoding are available in you starkit/starpack and your starkit can be launched with a 'standard' tclkit.

HaO: I use the following way, which is more natural for me:

Add the following path to your application before wrapping and put the encodings in: (appname).vfs/lib/tcl8.5/encoding
Wrap with the standard starkit

History

Starkits were originally called Scripted Documents

Scripted Documents Are Obscure

Misc

When storing Starkits on a MacOS machine, to what are people generally setting the type and creator values?

Try type "TEXT" and creator "Tkd4" (sdx sets them that way too) -jcw

Recently while playing with kitten, I was seeing an error when I attemped to package require one of the extensions, but was getting an error. I mused to jcw that it would be nice if there were a way to cd into the directory where tclkit was reporting a problem to browse, and he replied that I should be able to do just that. After some experimenting (and a false trail due to my forgetfulness) I found this indeed works:

$ tclkit
% source /home/lwv26/kitten.kit
% cd /home/lwv26/kitten.kit/lib
% glob *
Class1.0 ClassyTk1.0 Extral2.0 Mpexpr10 Tktable2.7 app-kitten ascenc-0.11 autoproxy autoscroll blowfish-0.10 bwidget-1.3.0 cgi1.6 dyncall-0.11 efftcl-1.0 expect-5.31 gbutton gk5.1 hexdump-0.10 ihash-0.11 itcl-3.2 itk-3.2 itk-3.3 iwidgets3.0.1 iwidgets4.0.1 lzrw1-0.10 mathf-0.11 mclistbox-1.02 md5c-0.11 mentry2.4 mkWidgets1.3 mvec-0.12 narray-0.81 scratch-0.10 supertext-1.0.1 tablelist2.7 tcldom-2.0 tclexpat1.1 tcllib1.3 tclsoap1.6.5 tclx8.4 tclxml2.0 tdom0.7 tix8.2 tkhtml0.0 tls-1.4 wcb2.8 wikit
%

This may help you when attempting to debug strange errors.

Also, note that when I tried doing ls or exec ls, the ls command did not see the directories, etc. within the lib. This seems to me to be bad news for someone wanting to merge using exec, open |, or other external calls with VFS mounted filesystems.

(See VFS, exec and command pipelines for discussion on this issue; if someone wants to contribute the time/code, at least some sort of functionality ought to be possible and useful there).

LV: I discovered this week that ActiveTcl 8.4.1.0 ships with the necessary framework to allow one to say:

/path/to/activetcl8.4.1.0/bin/tclsh sdx.kit

or any other starkit, and the starkit executes. No requirement for a Tclkit. The Tclkit, of course, is still a superior distribution mechanism. Hopefully there will be tools one day that one can use to generate a starkit based on the Tcl environment available and the script(s) being wrapped.

Dossy: Hey, is there a standalone Starkit package out there? One that you could 'package require starkit' within a vanilla tclsh, and then go sourcing your Starkits? I was hoping to start using Starkits under AOLserver, so "use Tclkit" is not an acceptable solution, here. So, is there a starkit.tcl and corresponding pkgIndex.tcl that one can drop in their $TCL_LIBRARY and just "package require starkit" with?

LV: Hmm - let's think this one through. what is the difference between a tclsh and a tclkit? A package that resulted in the difference being applied to a tclsh is what we are talking about. I presume that one difference is the set of packages that need to be package required. What else?

jcw 2003-09-17: That's what ActiveTcl does. There is a package "starkit" and you need things like Metakit, trf-or-zlib, memchan-or-rechan, and of course the tclvfs package (C and scripts).

LV: JCW, I am trying to locate this package. I downloaded the latest Solaris Activetcl - 8.4.4.0. I install it. I then type this:

$ gfind . -name '*tarki*' -print
./lib/vfs1.2/starkit.tcl
lwv26awu (2044) $ pwd
/tmp/ActiveTcl
lwv26awu (2047) $ gfind . -type f -print | xargs grep -l -i starkit
./lib/ppm/log/activetcl/install.log
./lib/vfs1.2/starkit.tcl
./lib/vfs1.2/tclIndex
./lib/vfs1.2/pkgIndex.tcl
./MANIFEST

$ /tmp/ActiveTcl/bin/tclsh
% package require starkit
1.2
%

That starkit is the vfs 1.2 code. To end up with the same environment as a tclkit, I presume that I need to do some other package requires.

When I start a tclkit, and do a package names, I see:

rechan pwb zlib vfslib vfs::mk4 mk4vfs vfs Mk4tcl Tcl

When I start an ActiveTcl tclsh, and do a package names, I see: ActiveTcl Tcl

Inside the ActiveTcl tclsh, I see:

% package require rechan
can't find package rechan
% package require pwb
can't find package pwb
% package require zlib
can't find package zlib
% package require vfslib
1.3
% package require vfs::mk4
1.9
% package require mk4vfs
1.9
% package require vfs
1.2.1
% package require Mk4tcl
2.4.9.2
% package require Tcl
8.4

So, are rechan, pwb, and zlib not important?

Dossy: Is there an easy way to "reload" a Starkit at runtime? In other words, suppose I have a long-running (daemon) Starkit. I want to modify some code, re-wrap the Starkit, then signal the currently running process to reload itself (or at least, its code) from the Starkit. I suppose I could mount the VFS and just 'source' the necessary files (or 'package forget' and 'package require'), but I'm wondering if this functionality already exists. Thanks.

jcw 2003-09-17: There's no magic in starkits, it's simply a VFS-mounted area. You can umount, remount, etc. You can even - with care - leave mounted, close the MK db, and re-open the MK db (but there's some caching going on in the mk4vfs.tcl script). Personally, I'd not go through this trouble: hook to the running process, and have *it* write the modified files into the starkit it is holding open. Compare it to a Unix mount: to change a file on the disk, you don't unmount, change it some other way, and remount - you simply write while it's mounted. If more complex scenario's are needed - why not run unwrapped? I mean, what are you trying to accomplish here, in terms of a higher-level perspective?

LV: The impression I got from reading Dossy's description was a scenario like this. However, I'm certain Dossy can explain it better if I am off base.

I have a starkit that I launch on a regular basis. I discover it has need of some modifications. I modify the file that I launch - this way, next time the changes are in place. Now, I want the running process to reflect the new version as well. I'd rather not take the process down - if it is a web server, I'd possibly terminate a user's session. What I want to do is signal the running process in some way that it would update itself when convenient.

Dossy 2003-09-20: Jean-Claude, yes, I could run unwrapped. But, Starkits are convenient: after developing, I can 'sdx wrap' all the code into one file, and deploy it where it needs to run. One file to push around, instead of a tarball that needs to be untarred before it can be run. Larry, you had the right idea. Implementing some kind of service as a Starkit, such as a webserver, where restarting the process is a nuisance (in my case, the process has persistent TCP connections to other servers -- process restarts means re-establishing those network connections). So, it'd be nice to be able to just modify code, re-wrap a new .kit, copy the new .kit over the old .kit then send a HUP signal to the running Starkit to go and reload any changed files (using 'source', perhaps?) ... of course, it'd mean writing your code in such a way that it could tolerate a warm restart, but that's the price we'll have to pay to get this kind of functionality, I guess. Or, we'll just have to keep hard-restarting these Starkits to get changes to be picked up, which is annoying but acceptable. But, I think a dynamic update of a running Starkit is a neato feature. :-)

jcw: Fair enough... so why not unmount/remount to pick up the changes?

Dossy 2003-09-28: Is there an easy way? If I sdx wrap foo.vfs while tclkit is running with foo.kit, it causes a Bus Error. If I vfs::unmount foo.kit at runtime, then sdx wrap foo.vfs then try to starkit::remount, on the remount it gives an error along the lines of no such mount /path/to/foo.kit. I can manually package require vfs::mk4 then vfs::mk4::Mount foo.kit foo.kit ... but that seems klunky.

What would be nice is a starkit::rewrap that did the unmount, wrap, and remount all in one shot. Hmm ... in the meantime, is there a hook into sdx wrap from within tclkit or must I exec the 'sdx' Starkit? source sdx.kit expects args - other than clobbering $argv, is there a better way of doing this? vfs::mk4::Mount /path/to/sdx.kit sdx.kit -readonly then source sdx.kit/lib/app-sdx/wrap.tcl still requires clobbering $argv. Guess I'll just exec, for now.

Thanks for entertaining my foolish idea, though. Starkits are still immensely useful, even without a really easy on-the-fly code update mechanism. :-)

VL: As it is not mentioned on the sdarchive page I thought I would add that on windows, you can of course as usual "register" the extension .kit, as an alternative to writing bat files, hence enabling the windows-double click behaviour.

One would presume some a similar setup could be made on MacOS, or for that matter the X Window based platforms, in relation to the appropriate GUI file manager.

If someone is wondering about using starkits for tclblend applications - that's really going to take some work. Tclblend requires one to use a threaded tcl - and tclkit doesn't by default build with threads. Then, to make use of tclblend, there's a start up 'wrapper' that I seem to understand one should use to start up the threaded tclsh, that ensures that various environment variables, etc. are set appropriately.

One of the hardest things to wrap are applications which make use of Tcl extensions which are dependent on compiled libraries (regardless of whether they are written in C, C++, etc.). There are a variety of issues that need to be resolved by these things. Is this part of building starkits documented anywhere?

stevel: See http://www.equi4.com/papers/skpaper4_4.html

MDD: I've noticed a strange problem with tclkit 8.4.6 under QEMU.

I've got a Kanguru Zipper 4GB USB hard drive

that I've put a 2gb ext2 disk image onto. Using QEMU , I booted up an .iso image of Damn Small Linux (dsl083) and then installed it onto the ext2 disk image file. From any Windows machine, I can then launch into DSL, running in the QEMU emulator window. I've installed a bunch of apps, including Firefox and Open Office, with no problem. They all seem to work fine, even though the execution is slow due to the emulation overhead.

OK, so far everything's working great. I've got this really cool little Linux machine I can carry around in my pocket. I've even set up autorun.inf to automatically launch the whole thing whenever the drive is plugged into a usb port. Very cool. One small weirdness is that gzip errors out whenever I try to gunzip something, though.

My main problem is that, even though both tclkit and dqkit run, and load up Tk without a hitch, neither of them can run Starkits. They can't seem to find main.tcl. This happens with all Starkits, even the most simple ones.

Any idea what might be wrong? I'm guessing there must be something about the QEMU emulation that interferes with the decompression that must occur with both gzip and Tclkit's vfs code.

HJG: Would it make sense to 'advertise' Tcl and Starkits on http://portableapps.com ?

Category Glossary

Category Tclkit

Category Deployment