Experimenting with Metakit

Arjen Markus (13 january 2003) I have been experimenting with Metakit as a convenient package to store measurement data (of course this was the original design goal of Metakit, as Jean-Claude Wippler has told me), but still getting some feeling for working with such a database package is worthwhile.

The script below (slightly adapted from the original) creates a database from a simple data file and does some basic statistical analysis.

The performance is quite acceptable to me:

The platform:

   Intel Pentium II, 350 MHz
   Windows 98
   tclkit, downloaded 3 july 2002 

The data file:

   Simple text file, 1.7 MB
   44000 lines each containing 9 integers

The results:

   11 seconds for reading the data file and creating the database
   11 seconds for retrieving 4 columns of 44000 data each and
      determining basic statistical properties
   resulting database is 0.9 MB
   about 80 lines of code

# testdata.tcl --
#
#    Create a Metakit database and perform some simple statistical
#    tests
#
# version 0.1: initial implementation, january 2003

package require Mk4tcl
source statistics.tcl

# fillDatabase
#    Fill the database with the contents of a tabular file
#
# Arguments:
#    db       Opened database
#    input    Name of the input file
#
# Result:
#    Name of table (view) that was created
#
proc fillDatabase { db input } {

    set table [mk::view layout $db.table \
       {date a:F b:F c:F d:F}]

    set infile [open $input "r"]

    # Skip header
    gets $infile line

    while { [gets $infile line] > -1 } {
       foreach {dummy yy mm dd hh a b c d} $line {break}
       set date [format "19%2.2d%2.2d%2.2d%2.2d" $yy $mm $dd $hh]

       mk::row append $table date $date a $a \
                             b $b c $c d $d
    }
    close $infile

    mk::file commit $db

    return $table
}

# getColumn --
#    Return a list consisting of the given column
#
# Arguments:
#    table    Table or view to retrieve
#    colname  Name of the column
#
# Results:
#    List of values
#
proc getColumn { table colname } {
    set result {}
    mk::loop c $table {
        lappend result [mk::get $c $colname]
    }

    return $result
}

# main --
#    Main code
#

console show

mk::file open db "mydata.tkd"
set table [fillDatabase db [file join .. "myfile.dat"]]
set table "db.table"
foreach col {a b c d} {
    set result [getColumn $table $col]
    puts "$col: [::math::statistics::basicStats $result]"
}
mk::file close db