Indexed file reading

Richard Suchenwirth 2003-06-02 - If you have a very long text file, you may not want to read it all in memory, which is the easiest way in Tcl to process a file. Instead, you'd read it line by line with gets.

Now if you want random read access to a line in such a big file, given their line number, the following code might be helpful. It creates an index (basically a list of integers: tell positions of all lines) by reading through the whole file once. It only keeps one line's contents in always the same variable, so it may be slow, but not excessive on memory.

 proc lineindex filename {
    set res 0
    set fp [open $filename]
    while {[gets $fp line]>=0} {lappend res [tell $fp]}
    close $fp
    set res
 }

Once you have this index, you can randomly access a line by specifying its number (0 being the first line):

 proc lineread {fp index line} {
    seek $fp [lindex $index $line]
    gets $fp
 }

If you just append to the file, the index might be accordingly updated, by lappending before putsing. This appears to me like a poor man's ISAM...


See Also