Version 4 of manipulating data with Tcl

Updated 2009-01-13 16:37:42 by LV

Arjen Markus (13 january 2009) At the moment I am involved in a project where I frequently need to edit timeseries in an input file. For instance: to multiply the wind velocity with a factor 0.8 to see what the effect is on the result of my computations. A tedious task if done by hand. Many people would probably use a spreadsheet for this task: copy the raw data into the spreadsheet, put a formula like "=0.8*$A1" in cell B1, copy that to the cells below, and export the results to the input file.

That is all very well, but to me that is rather laborious - I am lousy with spreadsheets, I get confused easily and I find them rather tedious to use (lots of mouse clicks for instance).

One of the nice features of Tcl, in my opinion, is the very limited restrictions it poses on a program. So, when I have a timeseries of this form for instance:

2008/07/05-03:00:00 3.266219192649675
2008/07/06-03:00:00 5.680730549305496
2008/07/07-03:00:00 8.375445823799266
2008/07/08-03:00:00 6.92001047014345
2008/07/09-03:00:00 4.2700598446601665
2008/07/10-03:00:00 4.939019208757603
2008/07/11-03:00:00 5.869121431091313
2008/07/12-03:00:00 5.400178789428884
2008/07/13-03:00:00 2.49166379101902
2008/07/14-03:00:00 4.607648466722291
2008/07/15-03:00:00 5.109895484028547
...

then I can just surround it with { } to get a valid list:

set data {
2008/07/05-03:00:00 3.266219192649675
2008/07/06-03:00:00 5.680730549305496
2008/07/07-03:00:00 8.375445823799266
2008/07/08-03:00:00 6.92001047014345
2008/07/09-03:00:00 4.2700598446601665
2008/07/10-03:00:00 4.939019208757603
2008/07/11-03:00:00 5.869121431091313
2008/07/12-03:00:00 5.400178789428884
2008/07/13-03:00:00 2.49166379101902
2008/07/14-03:00:00 4.607648466722291
2008/07/15-03:00:00 5.109895484028547
...
}

Then, add a loop like this after the data:

foreach {time velocity} $data {
    puts "$time [expr {0.8*$velocity}]"
}

and run the program - the output is a timeseries with the velocity scaled to 80% of its original value. I can plug that output directly into the input file.

If I need a bit more sophisticated processing and even a picture to get a better feeling for the data, well, no problem:

#
# Get rid of the first column - no need for it now
#
set velocity {}
foreach {time vel} $data {
    lappend velocity $vel
}

package require math::statistics
package require Plotchart

pack [canvas .c -width 400 -height 300]
set p [::Plotchart::createXYPlot .c {0 50 10} {0 1 0.1}]

set i 0
foreach value [::math::statistics::autocorr $velocity] {
    puts "$i $value"
    $p plot data $i $value
    incr i
}

and the result is a picture of the autocorrelation function of the data. I would not know a (fast) way of squeezing that out of a spreadsheet. But of course, I am no expert wrt spreadsheets.


newp (13 january 2009) The above example throws an error in the Plotchart::create command:

invalid command name "console"

LV newp, are you trying the example on Windows? If so, then I expect the issue is that you are using a console only version of the tclsh interpreter, or at the very least you need to do a

package require Tk

before you try the example. If you are not using Windows, then one of those packages has a bug in that they are trying to use a Tk command (console) which unfortunately doesn't exist on your platform.