Version 3 of TIP #431 Discussion

Updated 2014-09-16 11:36:59 by aspect

discussion on tcl-core

Over several hours in the chat, this TIP was discussed. aspect tries to summarise the eventual consensus below. Any errors or misrepresentations are my own, so speak up :-).

While exposing mkdtemp(3) as file tempdir is desirable, the underlying issue can be addressed more usefully in a much simpler fashion.

file mkdir as implemented eats the EEXIST (or equivalent) error from the underlying syscall. This is desirable with mkdir -p style usage, where an arbitrary number of parent directories may or may not need to be created first, but makes it impossible for a script to safely generate a directory for its own use.

Presently, the best option a script writer has is to call file mkdir $dir after ensuring file exists $dir returns false. But here lies a race condition: in the short interval between file exists returning false and file mkdir being called, another process might have created the directory. file mkdir turns EEXIST into success, and the programmer is none the wiser.

In principle, it may be possible to recover security by checking the ownership and permissions of the directory after file mkdir, but this is nearly impossible to get correct. And there is no safe way to recover -- if we believe the directory has been tampered with, we cannot safely change it.

By contrast, the system call itself offers an "atomic create" which returns an error when the directory already exists. TclpObjCreateDirectory passes this error back to its caller, but TclFileMakeDirsCmd hides it.

So the solution seems to be a variant of file mkdir that respects EEXIST. Such a command could be used to implement file tempdir correctly at the script level.

Since file mkdir is N-ary, we can't simply add an option so a new command is needed: the best suggested name so far seems to be: file createdir.

Suggested behaviour for this command:

  file createdir $dir
    Attempt to create the directory specified.  Raises an error if:
      1 - The immediate parent of $dir (ie, [file dirname $dir]) does not exist (POSIX ENOENT)
      2 - A directory named $dir already exists (POSIX EEXIST)
    Otherwise, behaviour is equivalent to [file mkdir] called with one argument.

This corresponds to a simple call to Tcl_FSCreateDirectoryProc(), exposing the POSIX errors mentioned above. On Windows, these are translated from the native errors (ERROR_ALREADY_EXISTS, ERROR_PATH_NOT_FOUND) in tclWinError.c and this translation is tested in winFCmd-4.3 and 4.2.

A future enhancement might add platform-specific prefix options to file createdir, exposing permissions (on Unix) or security attributes (on Windows), but that's beyond scope for now.


Proof-of-concept implementation

This is a naive patch against trunk b5ecfdaff3:

Index: generic/tclCmdAH.c
--- generic/tclCmdAH.c
+++ generic/tclCmdAH.c
@@ -951,10 +951,11 @@
     static const EnsembleImplMap initMap[] = {
         {"atime",        FileAttrAccessTimeCmd,        TclCompileBasic1Or2ArgCmd, NULL, NULL, 0},
         {"attributes",        TclFileAttrsCmd,        NULL, NULL, NULL, 0},
         {"channels",        TclChannelNamesCmd,        TclCompileBasic0Or1ArgCmd, NULL, NULL, 0},
         {"copy",        TclFileCopyCmd,                NULL, NULL, NULL, 0},
+        {"createdir",        TclFileCreateDirCmd,        TclCompileBasic1ArgCmd, NULL, NULL, 0},
         {"delete",        TclFileDeleteCmd,        TclCompileBasicMin0ArgCmd, NULL, NULL, 0},
         {"dirname",        PathDirNameCmd,                TclCompileBasic1ArgCmd, NULL, NULL, 0},
         {"executable",        FileAttrIsExecutableCmd, TclCompileBasic1ArgCmd, NULL, NULL, 0},
         {"exists",        FileAttrIsExistingCmd,        TclCompileBasic1ArgCmd, NULL, NULL, 0},
         {"extension",        PathExtensionCmd,        TclCompileBasic1ArgCmd, NULL, NULL, 0},

Index: generic/tclFCmd.c
--- generic/tclFCmd.c
+++ generic/tclFCmd.c
@@ -209,10 +209,32 @@
  * Side effects:
  *        See the user documentation.
+    ClientData clientData,        /* Unused */
+    Tcl_Interp *interp,                /* Used for error reporting. */
+    int objc,                        /* Number of arguments */
+    Tcl_Obj *const objv[])        /* Argument strings passed to Tcl_FileCmd. */
+    if (objc != 2) {
+        Tcl_WrongNumArgs(interp, 1, objv, "target");
+        return TCL_ERROR;
+    }
+    if (Tcl_FSCreateDirectory(objv[1]) != TCL_OK) {
+        Tcl_SetObjResult(interp, Tcl_ObjPrintf(
+                    "can't create directory \"%s\": %s",
+                    TclGetString(objv[1]), Tcl_PosixError(interp)));
+        return TCL_ERROR;
+    }
+    return TCL_OK;
     ClientData clientData,        /* Unused */
     Tcl_Interp *interp,                /* Used for error reporting. */

Index: generic/tclInt.h
--- generic/tclInt.h
+++ generic/tclInt.h
@@ -2891,10 +2891,11 @@
 MODULE_SCOPE int        TclEvalEx(Tcl_Interp *interp, const char *script,
                             int numBytes, int flags, int line,
                             int *clNextOuter, const char *outerScript);
 MODULE_SCOPE Tcl_ObjCmdProc TclFileAttrsCmd;
 MODULE_SCOPE Tcl_ObjCmdProc TclFileCopyCmd;
+MODULE_SCOPE Tcl_ObjCmdProc TclFileCreateDirCmd;
 MODULE_SCOPE Tcl_ObjCmdProc TclFileDeleteCmd;
 MODULE_SCOPE Tcl_ObjCmdProc TclFileLinkCmd;
 MODULE_SCOPE Tcl_ObjCmdProc TclFileMakeDirsCmd;
 MODULE_SCOPE Tcl_ObjCmdProc TclFileReadLinkCmd;
 MODULE_SCOPE Tcl_ObjCmdProc TclFileRenameCmd;

Index: tests/fCmd.test
--- tests/fCmd.test
+++ tests/fCmd.test
@@ -390,10 +390,29 @@
 } -constraints {notRoot} -body {
     file mkdir tf1
     file exists tf1
 } -result {1}
+;# not sure how to number these ..
+test fCmd-4.20 {TclFileCreateDirCmd: TclpCreateDirectory succeeds} -setup {
+    cleanup
+} -constraints {notRoot} -body {
+    file createdir td1
+    file exists td1
+} -result {1}
+test fCmd-4.21 {TclFileCreateDirCmd: errno: EEXIST} -setup {
+    cleanup
+} -constraints {notRoot} -body {
+    file createdir td1
+    list [catch {file createdir td1} msg] $msg $errorCode
+} -result {1 {can't create directory "td1": file already exists} {POSIX EEXIST {file already exists}}}
+test fCmd-4.22 {TclFileCreateDirCmd: errno: ENOENT} -setup {
+    cleanup
+} -constraints {notRoot} -body {
+    list [catch {file createdir td1/td2} msg] $msg $errorCode
+} -result {1 {can't create directory "td1/td2": no such file or directory} {POSIX ENOENT {no such file or directory}}}
 test fCmd-5.1 {TclFileDeleteCmd: FileForceOption fails} -constraints {notRoot} -body {
     file delete -xyz
 } -returnCodes error -result {bad option "-xyz": must be -force or --}
 test fCmd-5.2 {TclFileDeleteCmd: accept 0 files (TIP 323)} -body {
     file delete -force -force