http://www.tcl.tk/community/tcl2007/proceedings/databases/speedtables-paper-2.pdf

Speed tables provides an interface for defining tables containing zero or more
rows, with each row containing one or more fields. The speed table compiler
reads a table definition (similar to a C struct definition) and generates C code to create and manage tables where the rows are such
structures, producing a set of C access routines and a C language extension
for Tcl to create, access and manipulate those tables. It then compiles the extension,
links it as a shared library, and makes it loadable on demand via Tcl's
"package require" mechanism.

Speed tables are well-suited for applications for which this table/row/field abstraction
is useful, with row counts from the dozens to the tens of millions, for
which the performance requirements for access, search and/or update frequency
exceed those of the available SQL database, and the application does not require
“no transaction loss” behavior in the event of a crash.

----
Project link (historic): http://sourceforge.net/projects/speedtables 

Current development: https://github.com/lehenbauer/speedtables
----

[jima] (2011-01-27)

I have been able to set up Speed Tables for my Linux box. As the authors stated in their documentation their main focus has been BSD OS. I have encountered some tricky points in my way and I wanted to reflect here my steps just in case this could be of value to other people interested in this project.

I hope this will also help authors and contributors to find a better way of packaging this so as to cover BDS and Linux in the same go. Unfortunately I don't know anything about autotools to help in a better way than simply pointing to the differences between both OS'es.

Thanks to all people in the Tcl chat that helped me in my way to success.

The original scheme of things:

   * In Speed Tables one can write a Tcl script that is the seed to create a set of (mainly) C files in a build directory. These files are compiled and built into a Tcl extension. This building process is done by a Tcl script which invokes the compiler and linker by means of '''exec''' command.

Main difficulties in my Linux (debian lenny) box:

   * (I) In general, some options (like using tclsh8.4) seem to be hardcoded in the files that guide the building process.
   * (II) The compiler seems to be writing warning to stderr. This causes '''exec''' command to stop execution.
   * (III) Speed Tables use C function '''qsort_r''' that has been developed originally in BSD. This function can be found in the C standard library for Linux but, alas, with a different order used for its parameters (as seen on the http://sourceware.org/ml/libc-alpha/2008-12/msg00003.html%|%web%|%).
   * (IV) Tcl would not find the entry point for the extension.

Workarounds:

'''For (I):'''

In file '''ctables/Makefile''' I had to set PREFIX to my install dir:

   * PREFIX=/mnt/tis/ins/src

In file '''ctables/config.tcl''' I tweaked the following directives:

   * set showCompilerCommands 1
   * At the end of the file: set withPgtcl 0

In file '''ctables/mkConfig.sh''' I did:

   * tcl_versions="8.6"
   * tcl_prefixes="/mnt/tis/ins/src/lib"

By the way, instead of letting the '''Makefile''' produce the '''sysconfig.tcl''' from '''config.tcl''' I was somehow forced to do it manually: '''mkConfig.sh > sysconfig.tcl'''.

After the generation of '''sysconfig.tcl''' I tweaked it to:

   * Add to the list of define directives: -DCTABLE_NO_SYS_LIMITS=1
   * Declare the linker in a proper way: set sysconfig(ld) {gcc -shared}

Last but not least:

In file '''ctables/tests/Makefile''' I just told the system to use my tclsh instead of a hardwired one:

   * TCLSH=tclsh

'''For (II)'''

In file '''ctables/gentable.tcl''' just tweak a proc called '''myexec''':

   * eval exec -ignorestderr -- $command

'''For (III)'''

First, I had to update my system to have a libc that included function '''qsort_r''' by grabbing latest version of it from debian unstable.

In file '''ctables/ctable.h''':

   * I had to load the ugly '''__USE_GNU''' directive ahead of stdlib to allow loading of '''qsort_r'''.

 #define __USE_GNU
 #include <stdlib.h>

   * I had to distinguish between the loading of the BDS version or the Linux one: 

 #ifdef JIMA_QSORT_R_BSD     
    int (*sort_compare) (void *clientData, const void *pointer1, const void *pointer2);
 #else
    int (*sort_compare) (const void *pointer1, const void *pointer2, void *clientData);
 #endif

In file '''ctables/ctable_search.c''':

This is where the actual call to qsort_r is done so I had to flag here as well:

 #ifdef JIMA_QSORT_R_BSD     
      qsort_r (search->tranTable, search->matchCount, sizeof (ctable_HashEntry *), &search->sortControl, creator->sort_compare);
 #else
      qsort_r (search->tranTable, search->matchCount, sizeof (ctable_HashEntry *), creator->sort_compare, &search->sortControl);
 #endif

Not only parameters for '''qsort_r''' have different orders for both OS'es. The comparison function being passed to '''qsort_r''' have different orders for its parameters too.

This I hacked in a very dirty way in file: '''ctables/gentable.tcl''' at the point of '''variable sortCompareHeaderSource''':

 // JIMA_NON_BSD
 int ${table}_sort_compare(const void *vRow1, const void *vRow2, void *clientData) 

'''For (IV)'''

In file '''ctables/init-exten.c-subst''':

   * Change mentions to '''EXTERN int''' by '''int DLLEXPORT'''.


'''TODO's'''

Obviously this is a very dirty set of workarounds... a better way to do it would require probably find the best point to insert C preprocessor directives and let all code be governed properly by them.

I haven't yet tested the install procedure and checked on shared memory goodies in a detailed way.

----

!!!!!!
%| [Category Package] |%
!!!!!!