Version 117 of Jim

Updated 2011-12-01 13:49:14 by MHo
 What: Jim
 Where: http://jim.berlios.de/ and http://repo.or.cz/w/jimtcl.git
 Description: Small footprint re-implementation of Tcl.  Particularly suited for embedding. Currently at version 0.72
 Updated: 10/2011
 Contact: See web site

Recent Changes of this page and other news

  • 8 Nov 2011 Jim Tcl paper presented at Tcl/Tk 2011 is available at [L1 ]
  • 6 Oct 2011 Jim Tcl 0.72 has been released. See [L2 ]
  • 24 Jun 2011 Jim Tcl 0.71 has been released. See [L3 ]
  • 14 Apr 2011 Jim Tcl 0.70 has been released. See [L4 ]
  • 14 Apr 2011 Repository now mirrored at https://github.com/msteveb/jimtcl
  • 17 Nov 2010 Jim Tcl now supports UTF-8. See [L5 ]
  • 29 Oct 2010 Jim Tcl 0.63 has been released. See [L6 ]
  • The official source repository is now in GIT at http://repo.or.cz/w/jimtcl.git
  • View the most recent documentation at: Tcl_shipped.html

Index

Features

  • Jim is small. With no extensions, executable is around 84k compiled with -Os, and 138k with all extensions enabled.
  • Jim is modern. It supports {*}, dict, IPv6, TIP 288, live stack trace and error reporting with accurate line numbers
  • Jim is embeddable. Using autoconf, builds and runs on arm, mips, nios, x86 and more. Supporting Linux, FreeBSD, Mac OS X, eCos, cygwin, mingw32 and more.
  • Jim has lambda with garbage collection.
  • Every kind of Jim procedure is also a closure, thanks to support for static variables.

Applications using Jim

  • Open On-Chip Debugger (OpenOCD) [L7 ]

Jim Static Variables

  • Jim's static vars are trivial to use. Simply, proc can accept an optional statics list after the usual arguments list. Example:
 . proc foo {increment} {{value 0}} {incr value $increment}
 . foo 1
 1
 . foo 1
 2
  . foo 10
 12
  • If a static var is not initialized explicitly, the initial value is taken from the variable name with the same name in the local context:
 . set x 10  
 10
 . proc bar {} x {puts $x}    
 . bar
 10
  • Closure, Garbage collection, lambdas, makes Jim the first Tcl implementation where it's trivial to build the Paul Graham's well known accumulator function:
 # Write a function foo that takes a number n and returns a function that
 # takes a number i, and returns n incremented by i. Note: (a) that's number,
 # not integer, (b) that's incremented by, not plus.

 proc accumulator n {
    lambda increment n {
        set n [expr {$n+$increment}]
    }
 }

 set f [accumulator 100]
 puts [$f 10]
 puts [$f 20]
 puts [$f 1.25]

The output is

 110
 130
 131.25

RS: As Jim has prefix operators, I'd write it like this:

 proc accumulator n {
    lambda increment n {set n [+ $n $increment]}
 } 

Other Jim documentation in this Wiki

Commands already implemented

 . lsort [info commands]
 * + - / after alarm alias append array bio break case catch cd clock
 close collect concat continue curry debug dict env eof error errorInfo eval
 exec exit expr file {file copy} fileevent finalize flush for foreach format
 function getref gets glob global if incr info {info nameofexecutable} join
 kill lambda lambda.finalizer lappend lassign lindex linsert list llength
 lmap load local lrange lrepeat lreplace lreverse lsearch lset lsort open
 os.fork os.gethostname os.getids os.uptime os.wait package parray pid popen
 proc puts pwd rand range read readdir ref regexp regsub rename return scan
 seek set setref signal sleep socket source split stackdump stacktrace stderr
 stdin stdout string subst switch syslog tailcall tell throw time try unset
 update uplevel upvar vwait while
  • Jim has a simple to use references system to build arbitrary linked data structures, with garbage collection. The reference system can be also used to add automatic memory managment to every other kind of object. For example Jim's lambda uses the reference system for garbage collection, and so will do the Jim's OOP system (if it is ever built).
  • Jim has dynamic loadable libraries.

Benchmarks

Here is a comparison of a recent (Oct 2010) version of Jim. Note that Jim appears three times, compiled with -O3, -O2, -Os.

 Jim benchmarks - time in milliseconds
                     Jim-O2 Jim-O3 Jim-Os 8.4.19  8.5.8  7.6p2  6.8.1 
           PI digits    544    444    618    282    326      F      F 
     [for] busy loop    216    230    234    164    144   2725   2347 
   [while] busy loop    406    373    489    165    147   2709   2304 
                 ary    303    280    311    139    175    459    402 
          ary [dict]    294    264    313      F    171      F      F 
        dynamic code    194    180    225    227    422    262    251 
 dynamic code (list)     80     69     91    102    136    336    312 
              expand     74     65     98      F    103      F      F 
       fibonacci(25)    286    265    338    153    188   1023    944 
            heapsort    214    199    244     98     72      F      F 
              mandel    339    281    341    199    215   2007      F 
          mini loops    204    190    235    165    134   2413   2086 
        nested loops    246    231    292     99     89   1842   1665 
              repeat    201    184    228   1641    893   2289   2054 
              rotate     29     29     35      F      F      F      F 
               sieve    401    358    433    115    180    800    712 
        sieve [dict]    357    300    397      F    202      F      F 
               upvar    226    209    304    196    171    460    407 
    wiki.tcl.tk/8566    416    380    487    165    158      F 

Note that if you plan to write very dyamic code, with a lot of eval/upvar, Jim is probably currently the faster of the three. (See the repeat test).

The source for the benchmarks can be found in the repository as bench.tcl

License

Jim is under the "2 clause" FreeBSD Licence. This means that it's GPL compatible, but like Tcl itself can be used in commercial software without need to distribute the modified source code. In one word Jim is free for both the opensource and the commercial world!.

Commercial Support

Salvatore Sanfilippo, the main author of Jim, will be glad to support in the extension and integration of Jim in embedded systems, as scripting language of existing application, and any other need. For more information write an email to antirez at gmail dot com.

Steve Bennett, the current maintainer of Jim can also provide support [L8 ]

Interested in Jim?

If you want to stay informed about Jim you can subscribe to the jim-devel mailing list. There is a web form to subscribe at https://lists.berlios.de/mailman/listinfo/jim-devel . This is the primary forum for ongoing development of Jim. Also check the Jim development page.

Here is a place for your comments about Jim


02mar05 jcw - Jim is fascinating. Great engineering, IMO!

I've got an experimental reader for Metakit database files as well as a tiny Zlib decompressor based on the Vlerq project I'm working on. Should make it possible to support R/O starkits with very low footprint overhead (my guesstimate would be 25..35 Kb on top of Jim itself). The main issue is how to handle opens and globs and reads from things which are not files. As far as I can see, Jim has no I/O yet. Might be an idea to use a lightweight OO model ("$file read") so one can create objects that end up reading from a starkit. Could also be memory-mapped. Lots of ways to go from here...


SS 3-Mar-2005: Thanks for the good words! I followed the metakit discussion on the Feature Request thread but fortunately my lack of skills in metakits makes me unable to help from this point of view, with the exception that I can focus on the I/O extensions ASAP. For the OO, there will be Odys (with some change probably) integrated into the core (assuming I'll be able to implement it in max 1000 lines of code, otherwise I'll be forced to make it an extension, even if the Jim model is "little core, all extension" I really consider the OOP system to be a core language feature). Btw... I'm thinking that for starkits in unix-land another strategy that's possible to follow is to put POSIX I/O things into the jim-posix extension, and write an higher level interface directly in Tcl.

Yes, the Odys object system could work. No need for POSIX I/O if you *only* want to read from a starkit (which could make Jim secure, sort of). The code I have supports mem-mapped access on Windows, Linux, Mac OS X. -jcw


Steve Bennett Discussion on build systems moved to: Jim Tcl as a basis for configure and build system


Googie - How about speed? Are there any benchmarks available? I'm interested much in expr operations, but also others.

SS 2Mar2005: Hello Googie! Jim is currently slower than Tcl, but is faster than Tcl 7.6.


PT 3-Mar-2005: Jim supports dynamically loaded extensions. One of them supports a certain degree of COM automation under windows....

 load jim-win32com
 set ie [ole32 create InternetExplorer.Application]
 ole32.invoke $ie Navigate2 http://wiki.tcl.tk/jim
 ole32.invoke -put $ie Visible True

The above script results in Internet Explorer displaying this page.


SS 3-Mar-2005: That's cool! And is all merit of Pat. Also there is a start of win32 API binding. To hack with it:

  load jim-win32.dll
  info commands win32.*
  ...

SS 5Mar2005 - Update: in order to explore the proposal of SEH as well as the ability to run simple starkits with Jim, I'm working on a very simple Jim extension called AIO (Ansi I/O) that exports ANSI C file capabilities to Jim with a OOP Interface (and it is very simple to provide Tcl file API compatibility via some glue procedure). The initial version is on the CVS, people interested in this issue my follow the thread in the Feature Request at http://developer.berlios.de/feature/?group_id=3204 .

On the last days Jim reached a better shape, there are more core commands, it's more binary safe (now all the parsers are binary-safe), and robust. Also the exported C API is more complete.

That's how the AIO extension looks like:

 load jim-aio.so
 set f [aio.open /etc/passwd]
 set f2 [aio.open /tmp/FOOBAR w]
 while {[$f gets line] != -1} {
     $f2 puts "~~ $line ~~"
 }
 $f close
 $f2 close

Very similar to cfile.


LV Anyone have a table comparing features of Tcl and Jim, so that one can more easily understand the differences?

RS Jim is a rapidly moving target, but here's a quick sketch of what I can think of at the moment (see also Jim development for further differences):

  • Tcl, but not Jim: namespaces
  • Jim, but not Tcl: Jim closures - Jim references - lmap - prefix arithmetic operators - arrays are dicts are lists are strings - object-based I/O ...

On Jimulation I've started to "back-port" some useful Jim features to pure-Tcl, but most recently Jim seduces me to do even C fun projects... see below.


RS: See also Tiny OO with Jim - Jimulation / 2005-03-20: Here's this evening's fun project - adding an ** operator to jim.c (only added lines shown - "Doctor, somehow writing C doesn't hurt any more" :): ... snip... SS: ** added some time ago, thanks. For now it is supported only for integers because every mathlib related thing will probably go inside the math extension. This is so because the Jim core should not depend on the C math library.


RS was heard to aphorize, "Jim - because even simplicity can get simpler".



Anon Presented here without preamble is a patch to implement a backward compatible HEREDOC quoting syntax for Jim:

   int JimParseBrace(struct JimParserCtx *pc)
   {
         int level = 1;

         pc->tstart = ++pc->p;
         pc->tline = pc->linenr;
 + #if 1
 +       /* Check for HEREDOC quoting eg.
 +        *    set v {<<EOT}{
 +        *    ...
 +        *    EOT}
 +        */
 +       if(*pc->p == '<' && *(pc->p + 1) == '<') {
 + #define DELIM_SIZE 32 /* NB. a documented fixed limit here is probably
 +                      * not such a bad thing (Perl's is 256) */
 +         char delim[DELIM_SIZE], *delimp, *startp;
 +         int delim_len;
 +
 +         startp = pc->p; /* used to back out if failed syntax */
 +         pc->p += 2;
 +         delimp = delim;
 +         while (1) {
 +           if (*pc->p == '}') break;
 + #define VALID_HEREDOC_DELIM_CHAR(c) (isalnum((int)c) || c == '_')
 +           if(!*pc->p || !VALID_HEREDOC_DELIM_CHAR(*pc->p)) goto badheredoc;
 +           if(delimp - delim >= (DELIM_SIZE-1)) goto badheredoc;
 +           *delimp++ = *pc->p++;
 +         }
 +         if (*(pc->p + 1) != '{' && *(pc->p + 2) != '\n') {
 + badheredoc:
 +                 pc->p = startp;
 +                 goto retry;
 +         }
 +         pc->p += 3; /* skip "}{\n" */
 +         ++pc->linenr;
 +         pc->tstart = pc->p;
 +         *delimp = '\0';
 +         delim_len = delimp - delim;
 +         while (1) {
 +           /* At the start of a line. Check for delim */
 +           if ((!delim_len || (pc->p[0] == delim[0] && strncmp(pc->p, delim, delim_len) =
 = 0))
 +               && *(pc->p + delim_len) == '}')
 +           {
 +             pc->tt = JIM_TT_STR;
 +             pc->tend = pc->p - 1;
 +             pc->p += (delim_len+1);
 +             return JIM_OK;
 +           }
 +           /* Jump to end of line */
 +           while (1) {
 +             if(!*pc->p) {
 +               /* TODO Set a useful error message */
 +               goto retry; /* bail out */
 +             }
 +             if (*pc->p++ == '\n') {
 +               pc->linenr++;
 +               break;
 +             }
 +           }
 +         }
 +       }
 + retry:
 + #endif
 +
         while (1) {
                 if (*pc->p == '\\' && *(pc->p+1) != '\0') {
                         pc->p++;
                         if (*pc->p == '\n')
                                 pc->linenr++;
                 } else if (*pc->p == '{') {

So you can now write:

 set v {<<EOT}{
 blah blah...
 any old stuff: $$$ {{{ [[[ \\\ ]]] etc.
 EOT}

It simplifies inserting arbitrary text into a Tcl script variable definition as you no longer have to worry about backwacking.


Steve Bennett 26 Oct 2010 Remove long out of date discussion on downloading the source.


Felipe Voloch 5 Dec 2005 - I compiled Jim (binary here [L9 ]) for linux on the Jornada 680 handheld (from here [L10 ]). I wanted to compile Tcl but was having trouble with crosscompiling. Jim compiled and runs just fine. Nice stuff.


RJM 2010-10-16 I have put jim in the build tree of blackfin's uclinux-dist. It configures and builds right from the start correclty. I could for example successfully open and use a serial port (but opening pipes is apparently not supported yet).

Steve Bennett 26 Oct 2010 Great. pipes are supported. See: socket pipe, and the popen wrapper in tclcompat, which is used by open "|..."

Ask on the Jim mailing list if it isn't working.


MHo 2010-02-21: How to compile with MinGW???

Steve Bennett 26 Oct 2010 Now the same as all platforms:

  ./configure && make && make install

MHo Tried this, but only with partial success; I found a c:\Programme\MinGW\msys\1.0\local\bin\jimsh.exe after buildung...., about 930k in size...:

HOFFMANN@w000-428 /c/work/jim/jimtcl
$ ./configure && make && make install
Host System...i686-pc-mingw32
Build System...i686-pc-mingw32
C compiler... gcc -g -O2
C++ compiler... c++ -g -O2
Build C compiler...gcc
Checking for stdlib.h...ok
Checking for long long...ok
Checking for sys/socket.h...not found
Checking for netinet/in.h...not found
Checking for arpa/inet.h...not found
Checking for netdb.h...not found
Checking for sys/un.h...not found
Checking for dlfcn.h...not found
Checking for unistd.h...ok
Checking for crt_externs.h...not found
Checking libs for inet_ntop...no
Checking libs for socket...no
Checking for ualarm...not found
Checking for lstat...not found
Checking for fork...not found
Checking for vfork...not found
Checking for system...ok
Checking for select...not found
Checking for backtrace...not found
Checking for geteuid...not found
Checking for mkstemp...not found
Checking for realpath...not found
Checking for strptime...not found
Checking for gettimeofday...ok
Checking for regcomp...not found
Checking for waitpid...not found
Checking for sigaction...not found
Checking for sys_signame...not found
Checking for sys_siglist...not found
Checking for syslog...not found
Checking for opendir...ok
Checking for readlink...not found
Checking for sleep...not found
Checking for usleep...ok
Checking for pipe...not found
Checking for getaddrinfo...not found
Checking for utimes...not found
Checking for sysinfo...not found
Checking for _NSGetEnviron...not found
Checking environ declared in unistd.h...no
Checking for sys/types.h...ok
Checking for sys/stat.h...ok
Checking for mkdir with one arg...yes
Checking for termios.h...not found
Enabling line editing
Enabling references
Building static library
Extension aio...enabled (default)
Extension array...enabled (default)
Extension binary...not enabled
Extension clock...enabled (default)
Extension eventloop...enabled (default)
Extension exec...enabled (default)
Extension file...enabled (default)
Extension glob...enabled (default)
Checking libs for dlopen...no
Extension load...enabled (default)
Extension mk...not enabled
Extension nvp...not enabled
Extension oo...not enabled
Extension pack...not enabled
Extension package...enabled (default)
Extension posix...disabled (dependencies)
Extension readdir...enabled
Extension readline...not enabled
Extension regexp...enabled (default)
Extension rlprompt...not enabled
Extension sdl...not enabled
Extension signal...disabled (dependencies)
Extension sqlite...not enabled
Extension sqlite3...not enabled
Extension stdlib...enabled (default)
Extension syslog...disabled (dependencies)
Extension tclcompat...enabled (default)
Extension tree...not enabled
Extension win32...not enabled
Using built-in regexp
Jim static extensions: aio array clock eventloop exec file glob load package readdir regexp stdlib tclcompat
Created jim-config.h
Created jimautoconf.h
Created Makefile from Makefile.in
c:/Programme/Tcl/bin/tclsh.exe ./make-load-static-exts.tcl aio array clock eventloop exec file load package readdir re
xp glob stdlib tclcompat >_load-static-exts.c || ( rm _load-static-exts.c; exit 1)
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o _load-static-exts.o _load-static-exts.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-subcmd.o jim-subcmd.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-interactive.o jim-interactive.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-format.o jim-format.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim.o jim.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o utf8.o utf8.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jimregexp.o jimregexp.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o linenoise.o linenoise.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-win32compat.o jim-win32compat.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-aio.o jim-aio.c
jim-aio.c: In function 'JimMakeChannel':
jim-aio.c:1017:9: warning: variable 'OpenFlags' set but not used [-Wunused-but-set-variable]
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-array.o jim-array.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-clock.o jim-clock.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-eventloop.o jim-eventloop.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-exec.o jim-exec.c
jim-exec.c: In function 'JimCreatePipeline':
jim-exec.c:909:13: warning: variable 'pipe_dup_err' set but not used [-Wunused-but-set-variable]
jim-exec.c: In function 'JimStartWinProcess':
jim-exec.c:1470:11: warning: variable 'originalName' set but not used [-Wunused-but-set-variable]
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-file.o jim-file.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-load.o jim-load.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-package.o jim-package.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-readdir.o jim-readdir.c
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jim-regexp.o jim-regexp.c
c:/Programme/Tcl/bin/tclsh.exe ./make-c-ext.tcl glob.tcl >_glob.c || ( rm _glob.c; exit 1)
gcc -D_GNU_SOURCE -Wall  -I. -g -O2 -c -o glob.o _glob.c || ( rm _glob.c; exit 1)
c:/Programme/Tcl/bin/tclsh.exe ./make-c-ext.tcl stdlib.tcl >_stdlib.c || ( rm _stdlib.c; exit 1)
gcc -D_GNU_SOURCE -Wall  -I. -g -O2 -c -o stdlib.o _stdlib.c || ( rm _stdlib.c; exit 1)
c:/Programme/Tcl/bin/tclsh.exe ./make-c-ext.tcl tclcompat.tcl >_tclcompat.c || ( rm _tclcompat.c; exit 1)
gcc -D_GNU_SOURCE -Wall  -I. -g -O2 -c -o tclcompat.o _tclcompat.c || ( rm _tclcompat.c; exit 1)
ar cr libjim.a _load-static-exts.o jim-subcmd.o jim-interactive.o jim-format.o jim.o utf8.o jimregexp.o linenoise.o ji
win32compat.o jim-aio.o jim-array.o jim-clock.o jim-eventloop.o jim-exec.o jim-file.o jim-load.o jim-package.o jim-rea
ir.o jim-regexp.o glob.o stdlib.o tclcompat.o
ranlib libjim.a
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -c -o jimsh.o jimsh.c
c:/Programme/Tcl/bin/tclsh.exe ./make-c-ext.tcl initjimsh.tcl >_initjimsh.c || ( rm _initjimsh.c; exit 1)
gcc -D_GNU_SOURCE -Wall  -I. -g -O2 -c -o initjimsh.o _initjimsh.c || ( rm _initjimsh.c; exit 1)
gcc -D_GNU_SOURCE -Wall  -I. -g -O2   -o jimsh.exe jimsh.o initjimsh.o libjim.a
c:/Programme/Tcl/bin/tclsh.exe ./make-index  jim_tcl.txt | asciidoc -o Tcl.html -d manpage - || cp ./Tcl_shipped.html
l.html
/bin/sh: asciidoc: command not found
error writing "stdout": broken pipe
    while executing
"puts [string map $mapping $line]"
    ("foreach" body line 2)
    invoked from within
"foreach line $lines {
        puts [string map $mapping $line]
}"
    (file "./make-index" line 68)
mkdir -p /usr/local/bin
cp jimsh.exe /usr/local/bin
mkdir -p /usr/local/lib/jim
cp libjim.a /usr/local/lib
cp ./README.extensions   /usr/local/lib/jim
mkdir -p /usr/local/include
cp ./jim.h ./jim-eventloop.h ./jim-nvp.h ./jim-signal.h \
                ./jim-subcmd.h ./jim-win32compat.h /usr/local/include
cp jim-config.h /usr/local/include
mkdir -p /usr/local/doc/jim
cp Tcl.html /usr/local/doc/jim

HOFFMANN@w000-428 /c/work/jim/jimtcl
$

Best place for all these questions is the Jim mailing list. https://lists.berlios.de/mailman/listinfo/jim-devel


jSQLsh :

    jSQLsh is an SQLite shell based on the PostgreSQL command line shell.
    Implemented using the Jim Tcl interpreter. It is intended to be a simple
    light weight SQLite shell.