Getting Windows "special folders" with Ffidl

kostix 21-Jan-2007: Windows has its infamous history of playing with "standard locations" for various system and user data.

Probably the most interesting directory for a Windows developer is what called "application data" — the directory holding per-user configuration and data directories for various programs.

A well-behaved Windows program obeys this schema. For example, on a typical machine running Windows XP, a program "FrobozzMagic2000" should keep its settings in the "C:\Documents and Settings\<USERNAME>\Application Data\FrobozzMagic2000". (A useful side effect of keeping the program configuration in the standard place is that these settings will roam with the user's profile on a corporate network if the "roaming profile" feature is enabled.)

Tcl's mechanics for determining what "~" means on a particular Windows system are sane but insufficient since they rely on some environment variables which are not present on some systems (or have insane values on others). More specifically, in almost all cases it's impossible to reliably guess the "application data" directory pathname using only the value of "~".

There are three ways to know where the "application data" directory is located, but not all of them are applicable on each Windows flavor:

  • SHGetSpecialFolderPath [L1 ] of shell32.dll: works on any system from Win9x to Vista;
  • HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders\AppData registry variable: is present on any Windows flavor, but marked as deprecated in Vista;
  • The APPDATA environment variable: appeared in Windows 2000 and is present on any Windows including Vista.

Here's a simple reference table of what various Windows flavors offer for getting the "application data" directory pathname:

Win9x/ME

  • Has windir env var;
  • Does not have either HOMEDRIVE or HOMEPATH env vars (so the "~" on these systems expands to the name of the system disk drive);
  • Does not have either USERPROFILE or APPDATA env vars;
  • Has "application data" in %windir%\Application data;
  • Supports SHGetSpecialFolderPath;
  • Has the aforementioned registry value.

WinNT 4.0

  • Doesn't have APPDATA;
  • Has insane HOMEDRIVE and HOMEPATH values;
  • Has USERPROFILE which holds correct location of the user's "application data" directory;
  • Supports SHGetSpecialFolderPath;
  • Has the aforementioned registry value.

Windows 2000/XP

  • Has APPDATA;
  • Has sane HOMEDRIVE and HOMEPATH values;
  • Has USERPROFILE which usually equals to %HOMEDRIVE%\%HOMEPATH%;
  • Supports SHGetSpecialFolderPath;
  • Has the aforementioned registry value.

Windows Vista

  • Same as Windows 2000/XP, but
  • The registry key holding "standard folders" locations is marked as deprecated.

As can be seen, a well-behaved Windows program that wants to keep its configuration and local data in the standard manner has to deal with this mess and there are just two reliable ways to do this to date:

  • Reading the registry;
  • Calling SHGetSpecialFolderPath.

The first is much simpler and requires nothing but the core registry module. But, alas, that registry key may disappear in a future release of Windows. Which is much less likely for the SHGetSpecialFolderPath API function.

(neb A caveat: the above holds for standard profiles, but not all. Eg, the SYSTEM profile on XP is a limited profile, and does not have an APPDATA value. I have several thousand pcs littered with folders literally named %APPDATA% because installs assumed the value would be valid, but instead were run via SMS or something else that launched them via the SYSTEM account. When expanding environment variables on Windows, values that don't exist are left as-is, and %APPDATA% is a valid directory name.)

TWAPI has a very convenient wrapper around SHGetSpecialFolderPath [L2 ]. Unfortunately, TWAPI has two drawbacks compared to Ffidl in this respect:

  • TWAPI is much larger than Ffidl. This may be a crucial issue when considering what package to put into a starpack, especially if the program is not supposed to use Win32 API beyond the topic we're discussing here;
  • TWAPI doesn't work on Win9x/ME.

So here's a working example providing a Tcl wrapper around SHGetSpecialFolderPath using Ffidl:


 package require Ffidl

 ffidl::callout dll_SHGetSpecialFolderPath \
   {int pointer-utf16 int int} int \
   [ffidl::symbol shell32.dll SHGetSpecialFolderPathW]

 proc SHGetSpecialFolderPath {what create} {
   array set CSIDL {
     CSIDL_DESKTOP                 0
     CSIDL_INTERNET                1
     CSIDL_PROGRAMS                2
     CSIDL_CONTROLS                3
     CSIDL_PRINTERS                4
     CSIDL_PERSONAL                5
     CSIDL_FAVORITES               6
     CSIDL_STARTUP                 7
     CSIDL_RECENT                  8
     CSIDL_SENDTO                  9
     CSIDL_BITBUCKET               10
     CSIDL_STARTMENU               11
     CSIDL_DESKTOPDIRECTORY        16
     CSIDL_DRIVES                  17
     CSIDL_NETWORK                 18
     CSIDL_NETHOOD                 19
     CSIDL_FONTS                   20
     CSIDL_TEMPLATES               21
     CSIDL_COMMON_STARTMENU        22
     CSIDL_COMMON_PROGRAMS         23
     CSIDL_COMMON_STARTUP          24
     CSIDL_COMMON_DESKTOPDIRECTORY 25
     CSIDL_APPDATA                 26
     CSIDL_PRINTHOOD               27
     CSIDL_LOCAL_APPDATA           28
     CSIDL_ALTSTARTUP              29
     CSIDL_COMMON_ALTSTARTUP       30
     CSIDL_COMMON_FAVORITES        31
     CSIDL_INTERNET_CACHE          32
     CSIDL_COOKIES                 33
     !SIDL_HISTORY                 34
     CSIDL_COMMON_APPDATA          35
     CSIDL_WINDOWS                 36
     CSIDL_SYSTEM                  37
     CSIDL_PROGRAM_FILES           38
     !SIDL_MYPICTURES              39
     CSIDL_PROFILE                 40
     !SIDL_SYSTEMX86               41
     CSIDL_PROGRAM_FILESX86        42
     CSIDL_PROGRAM_FILES_COMMON    43
     !SIDL_PROGRAM_FILES_COMMONX86 44
     CSIDL_COMMON_TEMPLATES        45
     CSIDL_COMMON_DOCUMENTS        46
     CSIDL_COMMON_ADMINTOOLS       47
     CSIDL_ADMINTOOLS              48
     CSIDL_CONNECTIONS             49
     CSIDL_COMMON_MUSIC            53
     CSIDL_COMMON_PICTURES         54
     CSIDL_COMMON_VIDEO            55
     CSIDL_RESOURCES               56
     CSIDL_RESOURCES_LOCALIZED     57
     CSIDL_COMMON_OEM_LINKS        58
     CSIDL_CDBURN_AREA             59
     CSIDL_COMPUTERSNEARME         61
     # should not probably be here:
     CSIDL_FLAG_DONT_VERIFY           0x4000
     CSIDL_FLAG_CREATE                   0x8000
     CSIDL_FLAG_MASK                   0xFF00
   }

   set bCreat [expr {$create ? 1 : 0}]

   set path [string repeat \u0000 300] ;# MAX_PATH is actually 260

   set ok [dll_SHGetSpecialFolderPath 0 $path $CSIDL($what) $bCreat]

   if {$ok} {
     set ix [string first \u0000 $path]
     if {$ix > 0} {
       return [string range $path 0 [expr {$ix - 1}]]
     }
   } else {
     return {}
   }
 }

 puts "appdata: [SHGetSpecialFolderPath CSIDL_APPDATA false]"

I'm not sure, but Tk apps may probably want not to ignore the first argument of SHGetSpecialFolderPath (which is the handle of a window to associate any error dialog with (as I understand this)) and provide another argument (or an option) to the wrapper proc for specifying the Tk window. Then the handle of that window may be acquired using "winfo id".

Also note that a full-fledged implementation should check the result of creating the Ffidl callout and provide some fall-back way(s) for guessing the required "special folder", i.e. by reading the registry.

TODO: describe what's implemented in this respect in the ongoing release of Tkabber.
passer-by 2015-05-21 I beg your pardon, but are you spamming? I fail to see what the above line has to do with.
EMJ (2015-05-22) No need for accusations, that line was by the original page author!


Duoas 2008-01-08 Repaired some grammatical errors.


BC - 2009-08-19 02:57:39

ActiveState 8.4 and 8.5 (on WinXP) both seem to include a HOME element identical to USERPROFILE

% puts "$env(HOME) $env(USERPROFILE)"
C:\Documents and Settings\ben C:\Documents and Settings\ben