Version 11 of Techniques for reading and writing application configuration files

Updated 2003-03-19 15:30:39

Purpose: to collect pointers and observations regarding techniques for reading and writing configuration files by an application


  • What is a configuration file?
  • What are the issues one needs to keep in mind?
  • What internet resources are available for exploring tcl techniques in writing and reading configuration files?

What is a configuration file?

A configuration file is a (usually small) file that contains information about a number of options that control the appearance or behavior of an application program. The end user can change the configuration by either editing the file or selecting options in the application program. See also the Tk option database.

Unix configuration files are often placed in the users' HOME directory with a leading "." in the name (making them hidden), like ".mozilla". XWindows also has APP_DEFAULT files which can define the appearance of a GUI. Windows configuration files are sometimes stored with the application program, and sometimes thrown into the system directory as initialization (.INI) files. MS Windows applications often store configuration information in the registry, which is a whole 'nother ball of wax.

Configuration File Issues

Configuration files need to be readable and writable by the user. They also need a relatively simple format if they are to be edited by hand, which often introduces errors. Parsing code should be able to handle errors, and general enough to accomodate feature growth of the application.


Arrays

bach: lvirden: what I like to do is using arrays. Writing: puts array get. Reading read eval array set

lvirden: That's a pretty common technique. That's why I was surprised not to find a page describing that as well as some of the other techniques (like option, etc.)

Mike Tuxford posted an example parser to comp.lang.tcl in Dec. 2002 which reads a configuration file and sets array elements. This code allows comments and blank lines, but malformed lists or other data errors could cause problems. This should be OK if the end user is never allowed to edit the configuration file.

    proc parseConfig {fname} {
        if {[file exists $fname]} {
            set fd [open $fname r]
        } else {
            puts "Can't find $fname or perms are bad"
            exit
        }
        while {![eof $fd]} {
            set data [gets $fd]
            if {[string index $data 0] == "#" || \
                    [string index $data 0] == " " || \
                    [string index $data 0] == ""} {
                continue
            }
            switch [lindex $data 0] {
                foo {
                    global [lindex $data 0]
                    set [lindex $data 0]([lindex $data 1]) [lindex $data 2]
                }
                bar {
                    global [lindex $data 0]
                    set [lindex $data 0]([lindex $data 1]) [lrange $data 2 end]
                }
                default {
                    global [lindex $data 0]
                    set [lindex $data 0] [lrange $data 1 end]
                }
            }
        }
        catch {close $fd}
        return
    }

RS: But, as I found out, source also works on pure data, so you can write:

 array set x [source x.dmp] 

Only you cannot have comments in the data then - or you can, if you accept an array element by the name of #, the , then you can dump

 # {Saved by ... on ...}

rmax: It's downside is, that this kind of configuration file isn't very readable or editable.

suchenwi: Oh, you could format it like this:

 foreach i [lsort [array names a]] {
        puts [list $i $a($i)]
 } 

That's actually what I do, so the file still is nice to browse.

aku AK: config files - There was a page about tcl patterns somewhere. I believe this contained something. also look for the pages by Koen Van Damme on the wiki.

aku: His homepage refers to a paper by him about parsing data files

rmax: Hmm, reading and writing configuration files of various flavour seems me worth spending a tcllib module for. I could add a package that parses windows-style .ini files, I once wrote.


Source

Another option for Tcl application configuration files is to just to code Tcl script, and [source] it into the interpreter. The application code is very simple, but malformed lists, misplaced [brackets], or malevolent code can wreak havoc on an application. (Or the user!) Bob Techentin posted code for evaluating a configuration file in a safe interpreter. This has the advantage of being very general, editable by an end user, and any errors will be caught by the standard Tcl error mechanism. The down side is that an error will stop the evaluation of the configuration file.

    proc parseConfig {fname} {
        if {[file exists $fname]} {
            set fd [open $fname r]
        } else {
            puts "Can't find $fname or perms are bad"
            exit
        }
        set configScript [read $fd]
        close $fd

        #  Create a safe interp to evaluate the
        #  code in the configuration script
        set si [interp create -safe]
        catch {$si eval $configScript}

        #  The only thing we expect from this
        #  configuration is setting some
        #  global variables, so we extract
        #  them thus
        global foo
        array set foo [$si eval array get foo]

        interp delete $si
    }

Andreas Kupries posted that he uses this procedure for Tcl DevKit, which is even more robust.

  • create an interpreter
  • delete _all_ commands in it, including _unknown_.
  • alias the commands I want to be able to process into it.
  • alias a replacement for 'unknown' into it. This replacement returns an empty string and records an error.
  • Eval the configuration script in the interpreter.

The special commands give him the data in the config file, and the replaced 'unknown' gives me the errors which occurred during the evaluation.


See Metakit


US: I always use the following code snippet as a configuration preamble [to ensure that one has a name for the current program running]

 if {[string length [info script]]} {
    # It's just a script
    set here [file dirname [info script]]
    } else {
    # Wrapped into an executable
    set here [file dirname [info nameofexecutable]]
    }
 set owd [pwd]
 cd $here
 cd ..
 set CFGDIR [file join [pwd] cfg]
 cd $owd

See Config File Parser too.


AM If you want to use the source statement to source configuration files (and why not? it is a very flexible way!) but are worried about safety (a malevolent or simply careless user might upset your application or your whole system), have a look at this script to safely source a file.


NEM Worth mentioning for configuration files, is that XML is being used more and more these days for this kind of thing. Tcl has two excellent XML extensions (TclXML and tDOM) both of which make parsing a configuration file into an in-memory representation very easy. The advantage of this is that the parsing is taken care of by the extension, and there are plenty of third-party applications for editing XML files, making it easier if the user evers wants to edit something.