Techniques for reading and writing application configuration files are presented on this page.
A configuration file is a (usually small) file that contains information about a number of options that control the appearance or behavior of an application program. The end user can change the configuration by either editing the file or selecting options in the application program. See also the Tk option database.
Unix configuration files are often placed in the users' HOME directory with a leading "." in the name (making them hidden), like ".mozilla". XWindows also has APP_DEFAULT files which can define the appearance of a GUI. Windows configuration files are sometimes stored with the application program, and sometimes thrown into the system directory as initialization (.INI) files. MS Windows applications often store configuration information in the registry, which is a whole 'nother ball of wax.
Configuration files need to be readable and writable by the user. They also need a relatively simple format if they are to be edited by hand, which often introduces errors. Parsing code should be able to handle errors, and general enough to accomodate feature growth of the application.
bach: lvirden: what I like to do is using arrays. Writing: puts array get. Reading read eval array set
lvirden: That's a pretty common technique. That's why I was surprised not to find a page describing that as well as some of the other techniques (like option, etc.)
RJ: It seems to me the only way to actually import the config efficiently (read it in) is using arrays whether you store it as delimited name/value pairs, source them in as raw data or tcl commands or use tdom/xpath to read them in XML format. They have to be variables eventually, and how else but arrayed? Is this wrongheaded? There is so much good information on this page by the way.
Mike Tuxford posted an example parser to comp.lang.tcl, 2002-12, which reads a configuration file and sets array elements. This code allows comments and blank lines, but malformed lists or other data errors could cause problems. This should be OK if the end user is never allowed to edit the configuration file.
proc parseConfig {fname} {
if {[file exists $fname]} {
set fd [open $fname r]
} else {
puts "Can't find $fname or perms are bad"
exit
}
while {![eof $fd]} {
set data [gets $fd]
if {[string index $data 0] == "#" || \
[string index $data 0] == " " || \
[string index $data 0] == ""} {
continue
}
switch [lindex $data 0] {
foo {
global [lindex $data 0]
set [lindex $data 0]([lindex $data 1]) [lindex $data 2]
}
bar {
global [lindex $data 0]
set [lindex $data 0]([lindex $data 1]) [lrange $data 2 end]
}
default {
global [lindex $data 0]
set [lindex $data 0] [lrange $data 1 end]
}
}
}
catch {close $fd}
return
}XB: So how do you do if you want to have an entry in your config like this:
SRC_FILES = file1.tcl\
file2.tcl\
file3.tcl (and so on) ???RS: But, as I found out, source also works on pure data, so you can write: array set x source x.dmp
Only you cannot have comments in the data then - or you can, if you accept an array element by the name of #, the , then you can dump
# {Saved by ... on ...}rmax: It's downside is, that this kind of configuration file isn't very readable or editable.
suchenwi: Oh, you could format it like this:
foreach i [lsort [array names a]] {
puts [list $i $a($i)]
} That's actually what I do, so the file still is nice to browse.
AK: config files - There was a page about tcl patterns somewhere. I believe this contained something. also look for the pages by Koen Van Damme on the wiki.
aku: His homepage refers to a paper by him about parsing data files
rmax: Hmm, reading and writing configuration files of various flavour seems me worth spending a tcllib module for. I could add a package that parses windows-style .ini files, I once wrote.
Scott Beasley Over the years I have used just about every mothod for config/property files. For many simple applications I use this array setup:
proc loadproperties {prop_file prop_array} {
upvar 1 $prop_array props
if {[file exists $prop_file]} {
set fd_prop [open $prop_file r]
set prop_data [string trim [read $fd_prop]]
set prop_lst [split $prop_data \n]
foreach prop_line $prop_lst {
if {[string index [string trim $prop_line] 0] eq {#}} {
continue
}
set prop_line_toks [split $prop_line {=}]
set props([string trim [lindex $prop_line_toks 0]]) \
[string trim [lindex $prop_line_toks 1]]
}
} else {
return 1
}
return 0
}
proc getproperty {prop_array property default} {
upvar 1 $prop_array props
if {[info exists props($property)]} {
set prop_val $props($property)
} else {
set prop_val $default
}
return $prop_val
}
# Test....
if {[loadproperties WebServer.properties Properties] == 1} {
puts {Need a property file.}
exit 1
}
parray Properties
puts [getproperty Properties port 8080]
puts [getproperty Properties webroot ./]
puts [getproperty Properties default index.html]
puts [getproperty Properties logging off]
# File name: WebServer.properties
# Test Property file....
port = 8080
webroot = ./
default = index.html
logging = onYou can also use ::fileutils::grep from tcllib.
set config [::fileutil::grep {^[^#]} $fname]
foreach line $config {
set var [lindex [split $line :] 3]
}Then you can save the vars in an array.
RJ: Why not:
::fileutil::foreachLine parmLine $fname {
set parm1 [split $parmLine {<delimiting char>}]
set var([lindex $parm1 0]) [lrange $parm1 1 end] ;#or [lindex $parm1 1]
}Another option for Tcl application configuration files is to just to code Tcl script, and source it into the interpreter. The application code is very simple, but malformed lists, misplaced [brackets], or malevolent code can wreak havoc on an application. (Or the user!) Bob Techentin posted code for evaluating a configuration file in a safe interpreter. This has the advantage of being very general, editable by an end user, and any errors will be caught by the standard Tcl error mechanism. The down side is that an error will stop the evaluation of the configuration file.
proc parseConfig {fname} {
if {[file exists $fname]} {
set fd [open $fname r]
} else {
puts "Can't find $fname or perms are bad"
exit
}
set configScript [read $fd]
close $fd
# Create a safe interp to evaluate the
# code in the configuration script
set si [interp create -safe]
catch {$si eval $configScript}
# The only thing we expect from this
# configuration is setting some
# global variables, so we extract
# them thus
global foo
array set foo [$si eval array get foo]
interp delete $si
}PWQ 2009-01-15: Whats wrong with $si eval source [list $fname] , saves the open/close.
Kevin Walzer: I just assign preferences to a specific variable, then write the data to a preferences file as a set of Tcl statements defining the variables:
set allprefs [open $prefsdir/myapp.rc w+] puts $allprefs "set textfont \"$textfont\"" puts $allprefs " set pointsize \"$pointsize\"" puts $allprefs "set manpath \"$manpath\"" puts $allprefs "set printpath \"$printpath\"" puts $allprefs "set statuspath \"$statuspath\"" puts $allprefs "set searchpath \"$searchpath\"" puts $allprefs "set apropospath \"$apropospath\"" puts $allprefs "set nomore \"$nomore\"" close $allprefs
This file is sourced as a regular Tcl file, and looks something like this when opened in a text editor:
set textfont Courier set pointsize 12 set printpath /usr/bin/lpr set statuspath /usr/bin/lpstat set manpath /usr/bin/man set searchpath /usr/local/man:/usr/share/man:/usr/X11R6/man:/opt/local/share/man:/sw/share/man set apropospath /usr/bin/apropos set nomore 0
There are probably more sophisticated ways to do this--an array comes to mind--but it's simple enough, especially if you are not managing a huge number of preference items, and works for me.
Lars H: To stay clear of quoting hell, those puts commands should rather be using list, like this:
puts $allprefs [list set printpath $printpath]
For a DOS-style path with blackslashes as separator, the differences are considerable! OTOH, the lack of quotes and non-tabular formatting of your example (despite the tabular formatting of the code which supposedly generated it) suggest that it in fact was generated by list already.
LV: It seems like, from a casual glance, that the code would be pretty much the same for DOS. The original code showed one one use of a slash. Change that open to
set allprefs [open [file normalize $prefsdir/myapp.rc] w+]
or some such thing and you should be all set, right?
Now, if the concern was the fact that the set commands were being written out with paths like /usr/bin/lpr in them - that's a different problem. That problem is how do I set configure variables appropriately for the platform and this is a problem not only for DOS, but different versions of Unix-like systems as well (lpr may be in /bin not /usr/bin, etc.).
Lars H: The problem is not with the open call or something like that, but how to ensure that the data written to the file are read back as the same thing. As long as one only uses characters not special to Tcl, such as /, then all is fine. Problems get significant when the data contains characters like backslash however, unless one uses list to build the commands.
AM: If you want to use source to source configuration files (and why not? it is a very flexible way!) but are worried about safety (a malevolent or simply careless user might upset your application or your whole system), have a look at this script to safely source a file.
See also the Slave interp section below.
US: I always use the following code snippet as a configuration preamble [to ensure that one has a name for the current program running]
if {[string length [info script]]} {
# It's just a script
set here [file dirname [info script]]
} else {
# Wrapped into an executable
set here [file dirname [info nameofexecutable]]
}
set owd [pwd]
cd $here
cd ..
set CFGDIR [file join [pwd] cfg]
cd $owdFabricio Rocha 2001-02-09: dicts are loveable. I have been using them a lot and one of the most useful features on them is that they can be directly saved to a file and then read again to a string buffer which can be used with all the dict commands. This makes dicts pretty useful for configuration files (and any kind of data files), specially those which are not intended to be directly edited by the user -- although some "pretty printing" routines may be used for generating quite human-readable files (I still have to learn and try about that). A dict can store a lot of information in many different ways -- one key may have a list of directories as value, another key may store a simple boolean flag, other key might contain a nested dict, and so on -- and, IMHO, when you don't need the portability of XML they are functionally similar and much easier to deal with in Tcl.
A preferences/configuration system for a big application can be easily created with a global dict -- or, as I have been using, a namespaced dict variable with a set of "accessor" commands which manipulate it -- in which every first-level key is a "section" and the second-level keys are the configuration options for each section. The whole dict can be loaded from a file in application startup, and saved back to file by a "quit" routine.
See Metakit
Remember the foo.ini files from Windows 3.1? The inifile package reads and writes these and is a good choice for handling any config file which doesn't need to store binary data. It manages to be easily readable by both humans and scripts.
Fabricio Rocha 2011-02-09: If you use dicts for storing configuration data but don't want to save them directly to files, maybe you will like a simple INI-files parser/writer, which is not fully INI-format compatible, but might serve to some purpose.
See also Config File Parser.
NEM: Worth mentioning for configuration files, is that XML is being used more and more these days for this kind of thing. Tcl has two excellent XML extensions (TclXML and tDOM) both of which make parsing a configuration file into an in-memory representation very easy. The advantage of this is that the parsing is taken care of by the extension, and there are plenty of third-party applications for editing XML files, making it easier if the user evers wants to edit something.
US: Do you really think XML files are easier to edit than a simple Tcl script, from a users point of view? If you have only a hammer, everything looks like a nail.
NEM: In many ways, yes. If you want to keep simple configuration of name/value pairs, then you can do a simple array get/array set combo for the config file. However, if you want to store more complicated configuration data, with sections and different types of parameters, then this becomes more difficult.
The simple array implementation is not going to work very well here. Moving to a full Tcl script which is sourced is a) dangerous, and b) offers a lot of scope for confusing the user, as they now may have to learn a full language to do a simple task (unless you limit the command set, and remove proc/rename/trace and probably others). The alternative middle-ground is to dream up your own configuration language, and parser to read it. You can manipulate Tcl's parser to do most of the hard work, but why bother when XML is ready designed with just these sorts of tasks in mind? I'd say XML is at least as easy to read as Tcl (bad tag structure ruins this, but then bad proc structure ruins Tcl code), plus if you think it's too much for the end user, you can just ship one of the many free or commercial XML editors with your application.
In my opinion, use Tcl arrays when you are storing a small amount of simple config data. Move to XML for larger more complicated configuration data. Don't use Tcl scripts for configuration, unless you need to do something really complicated (like allowing the user to define macros and such).
You could use Metakit instead of XML, and that has plenty of advantages (especially if you're already packaged as a starkit), although the file becomes binary, and there aren't so many editors for MK datafiles.
XML has the same problems as a Tcl script, in that a single error in the file will halt the parser with an error. However, if you are using a DOM parser (and not SAX) then there shouldn't have been any side-effects by then, IIRC.
WJR Here's how I use XML config files:
<?xml version="1.0" encoding="UTF-8"?>
<config>
<start_time>20</start_time>
<end_time>7</end_time>
<zip_exe>d:/zip2.3/zip.exe</zip_exe>
<zip_path>{some value}</zip_path>
<parts_path>{some value}</parts_path>
</config>Then I use tDOM to parse and assign variables, like this:
package require tdom # Open config file and read into a variable set config_file [open zip-service.xml r] set config [read $config_file] close $config_file # Parse the config and get all config elements set config_parse [dom parse $config] set params [$config_parse getElementsByTagName config] # Set the various config params set start_time [[$params selectNodes start_time/text()] data] set end_time [[$params selectNodes end_time/text()] data] set zip_exe [[$params selectNodes zip_exe/text()] data] set zip_path [[$params selectNodes zip_path/text()] data] set parts_path [[$params selectNodes parts_path/text()] data]
I find this technique works pretty well.
Lars H 2005-09-19:
In MacOS X, there is a command line utility defaults which can be used to read and write .plist (property list, an XML-based format) configuration files. It automatically looks in the ~/Library/Preferences/ directory. While not directly Tcl-related, it can be interesting to observe that it makes it fairly easy to edit XML format preferences from a command line.
% exec defaults read com.apple.loginwindow
{
"NSWindow Frame About This Mac" = "366 329 287 336 0 0 1024 746 ";
"NSWindow Frame ProcessPanel" = "644 110 346 284 0 0 1024 746 ";
"NSWindow Frame shutdown" = "282 420 455 154 0 0 1024 746 ";
"dv com.apple.soundmgr._DV Sound Output Settings" = <7d000000 >;
}
% exec defaults read com.apple.loginwindow {NSWindow Frame shutdown}
282 420 455 154 0 0 1024 746 Looking in com.apple.LaunchServices may sometimes expose interesting information about why a particular program was chosen for opening a file.
Another useful MacOS X tool in this area is the Property List Editor, which unfortunately is hidden away from ordinary users by being part of the developer tools (look in /Developer/Applications). If someone wants a project, then writing equivalent (but cross-platform) tools as tclkits could be quite useful.
escargo 2004-03-17: I thought I would stick my oar in the water and add my own take on this subject.
First, I want to draw a distinction between policy and mechanism. Configuration files are a mechanism for providing persistence for configurable options. The policy that the configuration files implement is that configuratable options do not need to be respecified every time an application runs. This makes using complex applications with lots of options much easier, because the user does not need to respecify the same options every time the application is run.
Other mechanisms that achieve the same goal are also possible. The primary alternative can been seen in Windows with its registry. Another alternative can be seen where an application updates itself, so that its own file(s) contain the current definition of its configurable options.
Sometimes options need to be specified at two (or more) different levels. There might be site-wide default values for some options, and user-specific values for other options.
One problem with configuration files is that in order to update them, the user must have write permission on them. Another problem is that if the application that writes them has bugs, the resulting configuration file might cause the application to crash the next time it starts up. (This could prevent the application from starting until the configuration file is manually repaired.)
If you are designing applications, please
Andreas Kupries posted that he uses this procedure for Tcl DevKit, which is even more robust.
The special commands give him the data in the config file, and the replaced 'unknown' gives me the errors which occurred during the evaluation.
MHo: And here's another solution, very similar to the above mentioned, but developed on my own: Matthias Hoffmann - Tcl-Code-Snippets - Misc - Readprof.
WJP Reading configuration files is a good use for slave interpreters. You create a safe interpreter and expose commands that set the things you want to let the user set. But what happens if the user makes a mistake, say getting the name of a command wrong or using the wrong arguments? By default, the error propagates up into the master interpreter, aborting executing of the init file, and spewing a stack trace all over the user.
Not only will the typical user have no idea what the error messages and stack trace mean, but the error messages will be doubly difficult to decipher because the names that they use for procedures will be those used in the master interpreter, which will often be different from those provided for the use of the user. They will be especially different if the program is heavily localized, with commands in the initialization file provided in the user's language.
For example, suppose that French-speaking user has forgotten the argument to the command "ActiverInfoBulles". We'll suppose too that we've suppressed the stack trace. Here's the error message that he or she will get:
Erreur en lisant le fichier d'initialisation .sbrc: wrong # args: should be "SetBalloonHelpShowP b"
Not very helpful if you don't understand English, and even then, what the heck is "SetBalloonHelpShowP"?
So, here are some ideas that I've developed for handling errors in slave interpreters.
First, we can suppress the stack trace by using a catch when we read the init file:
proc ReadInitFile {name} {
if {[catch {interp invoke init source $name} msg]} {
puts [format [_ "Error on reading init file %s: %s"] $name $msg]
} else {
ShowMessage [format [_ "Executed init file %s"] $name]
}
interp delete init
}Notice also that when we pass on the system error message to the user in this way, it may be cryptic, but it will use the command name that the user used, not the internal name.
We can trap unknown command errors within the slave interpreter by defining an unknown procedure for the slave, like this:
proc InitFileSetup {} {
interp create -safe -- init
#We use slaveMsgcat rather than SlaveMsgcat so that the name does not
#appear in the list that the user sees.
init alias slaveMsgcat SlaveMsgcat
init eval slaveMsgcat
init alias mc ::msgcat::mc
ExposeInitProcs init
#This takes care of errors in the name of commands while still in the
#slave interpreter, without aborting, and using the user-name in the
#error-message.
init eval {
proc unknown {args} {
set cmd [lindex $args 0]
Puts [format [mc {Unknown command %s in file %s.}] $cmd [info script]];
return ;
}
}
}These are the ancillary procedures that are called:
proc SlaveMsgcat {} {
package require msgcat
::msgcat::mcload [file join [file dirname [info script]] msgs];
}
proc SlavePuts {msg} {
puts $msg;
}
#This procedure takes the list of wrapped master
#procedures and exposes them in a specified, already
#created, slave interpreter.
proc ExposeInitProcs {Interpreter} {
foreach c $::WrappedMasterCommandList {
set sl [SlaveName $c];
$Interpreter alias $sl $c;
}
}What we haven't dealt with thus far are errors other than misnamed commands. We can't deal with all errors within the slave but we can trap certain large and important classes of errors. In particular, we can handle errors in the number of arguments supplied to procedures. The approach is to wrap every procedure that we expose in the slave so that it checks its argument count and handles any errors itself.
we first call createinitprocs to assemble a list of all of the procedures to be exposed in the slave and wrap them. in the case at hand, most of these procedures are constructed programatically from information about the font and color configuration. the procedures that do this are called, and the names of these procedures are merged with the various other procedures that we want to expose. we then run wrapmaster on all of them. the result is a set of procedures in the master that check their arguments.
#this procedure creates the master procedures that will be exposed
#in the slave interpreter. it takes care of creating the base procedures
#where, as in the case of color and font setting, these are created
#programmatically, then wraps these as well as the manually
#written procedures that are to be exposed.
proc createinitprocs {} {
set ::fontproclist {};
fontsel::definefontsettingprocs
definecolorsettingprocs;
#this is the complete list of commands that we want to expose
#in the slave init file interpreter. these are the raw, unwrapped, master
#level commands.
set ::mastercommandstobeexposedininitlist [
concat [
lsort $::colorproclist] [
lsort $::fontproclist] [
lsort $::miscellaneousslavecommandlist]];
#now we wrap them all so as to trap valence errors
set ::wrappedmastercommandlist [list]
foreach c $::mastercommandstobeexposedininitlist {
lappend ::wrappedmastercommandlist [wrapmaster $c]
}
}
#this procedure takes as its sole argument the name
#of an existing procedure in the same interpreter
#and creates a new master level procedure that
#wraps the other procedure with a test for the
#correct argument count.
proc wrapmaster {pn} {
set wrappedname wrapped${pn};
set sp [format "proc $wrappedname \{args\} \{\n"];
set argsneeded [llength [info args $pn]];
append sp [format "\tif \{\[llength \$args\] != %d\} \{\n" $argsneeded]
if {$argsneeded == 1} {
set emsg [format [_ "the command %s in the file %%s expects one argument"] \
[slavename $wrappedname]]
set cmd [format "\t\tputs \"$emsg\"\n" \$::initfile]
append sp $cmd
} else {
set emsg [format [_ "the command %s in the file %%s expects %d arguments\n"] \
[slavename $wrappedname] $argsneeded]
set cmd [format "\t\tputs \"$emsg\"\n" \$::initfile];
append sp $cmd;
}
append sp "\t\treturn;\n\t\}\n\t"
append sp $pn
for {set i 0} {$i < $argsneeded} {incr i} {
append sp " \[lindex \$args $i\]"
}
append sp "\n\}"
eval $sp;
return $wrappedname;
}notice that these procedures need to know the name that each procedure will have in the slave in order to get the error messages right. here is the procedure that controls the mapping between master names and slave names. this mapping is mostly regular, but we can set up irregular cases by putting them into the array mastertoslavenames.
#given the name of a wrapped command in the master interpreter,
#returns the name that the command is to have when
#exposed in the slave interpreter. in most cases the alias
#is computed from the master name, but in some cases
#the relationship is irregular.
proc slavename {command} {
regsub "^wrapped" $command "" unwrapped
if {[info exists ::mastertoslavename($unwrapped)]} {
set englishslavename $::mastertoslavename($unwrapped);
} else {
regsub "^set" $unwrapped "" englishslavename
}
return [_ $englishslavename]
}
set mastertoslavename(setdelta) windowmotionincrement;
set mastertoslavename(storecustomcharacterchartinplace) definecharacterchart
for example, the original procedure may be:
proc setdelta {s} {
set ::deltaseconds $s;
}the wrapped version created by wrapmaster will be:
proc wrappedsetdelta {s} {
if {[llength $args] != 1} {
puts "the command windowmotionincrement in the file $::initfile \
expects one argument"
return;
}
setdelta [lindex $args 0]
}by default, this would be alias in the init file as "delta", but since we have made an entry for it in mastertoslavenames, the alias is actually "windowmotionincrement".
it is possible to handle other types of errors by similar techniques. if you allow users to set colors via the init file, getting color specifications wrong is a likely source of errors. since i generate the color setting procedures programatically, on the basis of the colorspecs array, i build a check of color specification validity into each color setting procedure, like this:
proc definecolorsettingprocs {} {
global colorproclist;
foreach cs [array names ::colorspecs] {
set csl [split $cs ","]
set obj [lindex $csl 0]
set which [lindex $csl 1]
set procname [format "set%s%scolor" $obj $which]
lappend colorproclist $procname
eval [list "proc" $procname \
\
"\{cs\}" \
"if \{!\[::validcolor::iscolorspecq \$cs\]\} \{
puts \[format \[_ \"\\\"%s\\\" is not a valid color specification.\"\] \$cs\]
return;
\}
set ::colorspecs($obj,$which) \$cs"]
}
}the procedures that it generates look like this:
proc setmenubarbackgroundcolor {cs} {
if {![::validcolor::iscolorspecq $cs]} {
puts [format [_ "\"%s\" is not a valid color specification."] $cs]
return;
}
set ::colorspecs(menubar,background) $cs
}AMG: I just wrote a configuration file reader called grok which uses child (slave) interps. it also uses apply and functional composition. it's very flexible and might be work a look.
or on second thought, just leave it alone. it needs to be rethought in its entirety, probably scrapped altogether.
AMG: see config file using slave interp for a design which is vastly simpler and faster.
pwq 2009-01-15: Why do people make this such a complicated process. the title states the specific purpose, read and write config files. a simple solution minus error trapping (which again is simple):
#read set f [open cfg r] array set config [read $f] close $f # write set f [open cfg w] puts -nonewline $f [array get config] close $f
tk should have a built-in way of doing this like any other modern toolkit. if only the option command had write capability as well. of course this is the fault of the bizarre x-windows design decision to make the resources database read-only for applications.
jal 2011-02-17: I would really like a configuration language that allows for structure, is not xml (i prefer curly braces for scopes) and allows a simple schema to control parsing. the schema should specify constraints on attribute values, including references between objects. am i misguided? something like
(in the schema file or variable)
class food {
attr calories <number>
}
class sandwich {
base food
# a reference to an object of class meat...
reference meat @meat
list min 0 max 2 attr cheese choosefrom {provolone swiss none}
}
class megasandwich {
list min 0 max 3 reference meat @meat
list min 0 max 4 cheese
list extra @food
}
class extra {
base food
}
class meat {
base food
attr is_meat default=1
attr usda_grade {(a+|[bc])}
}
extra lettuce -calories 5
extra tomato -calories 30
extra horseradish -calories 30(in the actual config file)
meat ham -calories 200 -usda_grade aa meat turkey -calories 150 meat bacon -calories 100 -usda_grade b sandwich italian_bmt -meat ham -meat turkey -cheese provolone -extra tomato sandwich nuclear_sub -meat turkey -cheese provolone -extra horseradish
the result of parsing would be put into a tcllib graph or an array.
so, we have:
amg: see my code on the "config file using slave interp" page. i think it may give you the functionality you seek, without any need for defining a schema. it also provides a very powerful macro capability.