Version 38 of Where to store application configuration files

Updated 2011-04-18 11:12:35 by GeoffM

TR - Many applications need to remember things about themselves, like GUI appearance, default values, last opened files and so on. Since Tcl is cross-platform, once a program is deployed on different platforms, you want to handle this point in different ways. There are several places, where you can or should store data. A short and simple procedure is described in: Application-specific RC files.

I don't describe here, how you should store the data. This is dicussed in: Techniques for reading and writing application configuration files.


Unix

User specific settings normally go in the home directory in a dotted file (or directory) like ...

 file join $env(HOME) .configFile  

[And by convention, named something like .appnamerc RJ ]

BR: You can just write that as ~/.appnamerc or ~/.appnamerc.tcl.

GUI appearance is normally configured via the option database. The directory can be something like /usr/lib/X11/app-defaults/ but I am unsure, whether all Unix systems use this very directory.

Windows

Windows knows several places. First there is the same location as in Unix:

 file join $env(HOME) configFile.dat

The $env(HOME) part seems always to be there, also when the HOME variable is not set in the autoexec.bat file (see: HOMEPATH). But is this true for all incarnations of Windows (95,98,ME,XP,2000, ...)?

BR: Tcl insures that $env(HOME) is present, creating it from other environment vars if necessary. Again you can just write it as ~/.appnamerc.tcl.

Then there is the registry. You don't seem to need to know the location of the registry, just use the registry package:

 package require registry

Simple name-value pairs might appear in HKEY_CURRENT_USER/Software/Me/MyApp or HKEY_LOCAL_MACHINE/Software/Me/MyApp.

Many programs just store their data along with the installation directory, often in the same directory as the executable. So you would need to know this location from inside the script. This depends on using a simple script or a starkit or another kind of wrapped application like freewrap. Here are some references:

Good style would be to use the user profile to store such data on WinNT/2k/XP. It can be found with the help of TWAPI's get_user_account_info or get_shell_folder commands (the latter only if the program is running as that user).

MG notes that on his Win XP SP2 computer, ~ and $env(HOME) are both his user-folder, which for him is C:/Documents and Settings/Griffiths. So [file join ~ "Application Data"] is the place where many other (commerical) programs seem to store data.

GWM it is better to use the environment variable env(APPDATA) which is [file join ~ "Application Data"] on XP but \users\NAME\AppData\Roaming on Win7 and Vista (why did MS change the directory name?). This environment variable is reputedly set in all versions of Windows from Win2000, so Win95 still needs special processing.

EKB Yes, I use "Application Data" as well. Here's my usual code:

 set USERDIR [file dirname $argv0]
 if {$tcl_platform(os) == "Windows NT"} {
   if {[info exists env(USERPROFILE)]} {set USERDIR $env(USERPROFILE)}
 }
 if {$tcl_platform(os) == "Windows 95"} {
   if {[info exists env(windir)] && [info exists env(USERNAME)]} {
      set USERDIR [file join $env(windir) Profiles $env(USERNAME)]
   }
 }
 set USERDIR [file join $USERDIR "Application Data" "Your App Name"]
 set USERPREFS [file join $USERDIR prefs.tcl]

This defaults to the application directory in case none of the environment variables work. Then on either NT or Win95 it finds the user's directory. (Hm... and I don't use HOME... maybe that's better?) Then I append:

 [file join $USERDIR "Application Data" "Your App Name"]

It's polite to put your config files not only in "Application Data" but also in a subfolder just for your software. (The application name, or your company name, etc.)

ralfixx Microsoft recommends to use

  TCHAR szAppData[MAX_PATH];
  …
  hr = SHGetFolderPath(NULL, CSIDL_APPDATA, NULL, 0, szAppData);

to get the "Application Data" path since it might be named differently in different language versions of the OS (eg. it is called "Anwendungsdaten" in the german version).

kostix Yes, TWAPI wraps this function, and we also have an article on Getting Windows "special folders" with Ffidl.

HaO corresponding twapi command:

package require twapi
set path [twapi::get_shell_folder CSIDL_COMMON_APPDATA -create]

Macintosh

This might depend on the version of the Mac OS you have. Newer Tcl versions are Mac OS X only, but older applications may be for MacOS Classic.

I don't konw about these, because I don't have a Mac. Can someone fill the information here?

Zarutian: I think you use [file join $env(HOME) Library <your app's name>] but I am not sure.

BR: Mac OS X is a Unix. You can just use the Unix conventions. You can also use ~/Library/Preferences/appname.tcl, that is a convention that comes from the NextSTEP inheritance of Mac OS X.

slebetman On Mac OS, you should use /Preference/appname to store preferences. If there are multiple preferences then do what Adobe does: /Preference/appname_feature. The Preference folder was not only inherited from NeXT but from the old MacOS. Also note that the application should create preference file with default preferences if one is not already there. It is a common Mac debugging technique to delete corrupted preference files for misbehaving programs. Mac users will expect this. I think this is also a nice behavior for other platforms.

BR: I don't have a folder "/Preference" on my 10.3 system. I have "/Library/Preferences" but that is of course is a global folder. It is a bad place to put user-specific preferences by definition. Mac OS Classic was essentially single-user, while Mac OS X is a multi-user system, so there is a good reason to change to a user-specific directory. Of course if you want to have global defaults or settings, "/Library/Preferences" is a possibility. The directory might be write-protected on a secured system, so you might need admin authorization to write there.

An application should IMO only create a preference file if any option has been set to a non-default value. And of course an application should keep generated files (stuff that is generated from UI dialogs) separate from user-written files. Tcl was started as a language for configuration files. For me, having the ability to tweak an application with handwritten code in a configuration file is still a very strong point of the language.

anoved: The conventional Mac OS X preferences folder location is ~/Library/Preferences, where ~ represents the user's home folder. Supplemental files (resources needed by but external to the application, not user-created documents) may be stored in ~/Library/Application Support/appname. Apple provides some commentary and canonical guidelines for software installation at [L1 ]; the Which Should You Use? section provides a good overview. For what it's worth, the defaults command line program provides a simple mechanism for reading, writing, and manipulating the XML "property list" files commonly used to store preferences on Mac OS X. It is documented here: [L2 ]

Incidentally, if you use defaults to write preferences to the same domain specified by the CFBundleIdentifier property of a standalone Wish shell's Info.plist file [L3 ], the preferences you write and those managed by the operating system (open/save dialog locations, etc.) will be collected in a single preferences file. Of course, property name collisions may be possible in this case.


Storing data in the executable files

For the popular and wonderful approach of storing the configuration data in the executable itself using starkits and starpacks, you need to know this: only mechanisms designed for the usage of the virtual filesystem that make starkits/starpacks can be used. So you can use plain files that are handled by Tcl's core commands (open, read, gets, and so on) and files that build upon the vfs. You can not use extensions like SQLite, which is not vfs-aware.

Starkit

You need to make the starkit writable, if the data stored inside it must be changeable. This is done when the sdx command builds the starkit. Note that it currently is not possible to access a metakit db residing inside a starkit directory [L4 ]. So to have a metakit db in a starkit/starpack, the only way is to use the metakit db, which is the kit/pack itself. A starkit is a metakit db with a view named "dirs". You can just create new views as you like and use them along with the "dirs" view. Like this (taken from the Book Practical Programming in Tcl and Tk, Fourth Edition):

 set db [lindex [vfs::filesystem info $starkit::topdir] 1]
 mk::view layout $db.myView {what you need}

Starpack

It is not possible to change the contents of a starpack from inside it. This is a limitation of the operating system as it does not allow changes to the executable file while being executed. So you need to do that from outside the starpack, i.e. with another program. You can, of course, when the starpack is running, make a copy of yourself, modify that copy, and then delete the original somehow.


SEH 20080218 -- My home directory is littered with config files, some from Tcl applications, some not. Untidy! I wanted to reduce the proliferation of files in my home dir, and put the configuration information for the Tcl apps I write in a privileged place where it's easy to find and edit them... I figured Tclsh already has a config file, .tclshrc, so why create another? The code below tucks separate configuration sections for individual Tcl applications into the .tclshrc file, separated from the actual tclsh configuration contents by a Ctrl-z (Tcl logical end-of-file character). You can stick to using the config procs, or, once you've run the config_init proc, you can simply use a text editor to edit and save config info.

  • config_init project_name : creates the tclshrc file if necessary and formats an area for the named project's config info. The other procs call this one, so it need not ever be called directly. Returns pathname of tclshrc file.
  • config_get project_name : extracts config info from named project and returns it. Returns empty string on first use for project.
  • config_put project_name config_info : writes config info to tclshrc file.
  • config_source project_name : If the config info is executable Tcl code, use this proc to execute it via the eval command.
proc config_init project {
        set homeDir $::env(HOME)
        set tclshrc [file join $homeDir .tclshrc]
        if ![file isfile $tclshrc] {
                set tclshrc [file join $homeDir tclshrc.tcl]
                if ![file isfile $tclshrc] {
                        set tclshrc .tclshrc
                        if {$::tcl_platform(platform) == "windows"} {
                                set tclshrc tclshrc.tcl
                        }
                        set tclshrc [file join $homeDir $tclshrc]
                        close [open $tclshrc w]
                }
        }
        set f [open $tclshrc r]
        fconfigure $f -translation binary
        set tclshrc_conts [read $f]
        close $f
        set tclshrc_conts [split $tclshrc_conts \u001a]
        foreach cfg $tclshrc_conts {
                if ![string first "# Start $project.config --" [string trim $cfg]] {
                        return $tclshrc
                }
        }
        set f [open $tclshrc a]
        seek $f 0 end
        if {[llength $tclshrc_conts] < 2} {
                puts $f "\n# -- End [file tail $tclshrc]" 
        }
        puts $f "# Ctrl-Z (end-of-file):\u001a"
        puts $f "# Start $project.config --"
        puts $f "\n# -- End $project.config"
        close $f
        return $tclshrc
}

proc config_get project {
        set tclshrc [config_init $project]
        set f [open $tclshrc r]
        fconfigure $f -translation binary
        set tclshrc_conts [read $f]
        close $f
        set tclshrc_conts [split $tclshrc_conts \u001a]
        foreach cfg $tclshrc_conts {
                if ![string first "# Start $project.config --" [string trim $cfg]] {
                        set cfg [string map {\x0d\x0a \x0a} [string trim $cfg]]
                        set cfg [string map {\x0d {}} [string trim $cfg]]
                        set cfg [split $cfg \n]
                        set cfg [lrange $cfg 1 end-1]
                        if {[lindex $cfg end] == "# -- End $project.config"} {
                                set cfg [lrange $cfg 0 end-1]
                        }
                        set cfg [join $cfg \n]
                        return [string trim $cfg]
                }
        }
        return
}

proc config_put {project config} {
        set config "# Start $project.config --\n\n$config\n\n# -- End $project.config\n# Ctrl-Z (end-of-file):"
        set tclshrc [config_init $project]
        set f [open $tclshrc r]
        fconfigure $f -translation binary
        set tclshrc_conts [read $f]
        close $f
        set tclshrc_conts [string map {\x0d {}} $tclshrc_conts]
        set tclshrc_conts [split $tclshrc_conts \u001a]
        set cfg_count 0
        foreach cfg $tclshrc_conts {
                if ![string first "# Start $project.config --" [string trim $cfg]] {
                        break
                }
                incr cfg_count
        }
        set tclshrc_conts [lreplace $tclshrc_conts $cfg_count $cfg_count $config]
        set tclshrc_conts [string trim [join $tclshrc_conts "\u001a\n"]]
        set tclshrc_conts [string map {\x001a\n\n \x001a\n} $tclshrc_conts]
        if ![string first "# -- End " $tclshrc_conts] {
                set tclshrc_conts "\n$tclshrc_conts"
        }
        set tclshrc_conts [split $tclshrc_conts \n]
        set tclshrc_conts [lrange $tclshrc_conts 0 end-1]
        set tclshrc_conts [join $tclshrc_conts \n]
         set f [open $tclshrc w]
        puts $f $tclshrc_conts
        close $f
        return
}

proc config_source project {
        set config [config_get $project]
        eval $config
}