The '''`[exec]` quotes problem''' and the [exec ampersand problem] are due to the mismatch between the command execution paradigms of `[exec]` and [Microsoft Windows]. On that platform, a program is executed by passing ''[http://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx%|%the entire command line as a single value]'' to the system. Before it calls `main()`, the system calls `_setargv()` to parse the command line into the standard [C] `argc` and `argv[[]]` arguments that it passes to the `main()` function of the program. The program can provide its own `_setargv()`, giving it complete control over what syntax to impose on the command line. This creates an impedence mismatch with `[exec]`, which is provides an [API] compatible with that of the the `[Unix]` `[http://en.wikipedia.org/wiki/Exec_%28computing%29%|%execve()]` function, which accepts a sequence of values rather than a single command line. For Windows, `[exec]` must convert its arguments into a single command line. Since the syntax of the command line for any given program is a moving target, and individual programs can get wild and woolly about parsing the command line, the only sane thing for `[exec]` to do is to convert according to the rules of the default `_setargv()` function, which are documented in in [http://msdn.microsoft.com/en-us/library/a1y7w461.aspx%|%Parsing C Command-Line Arguments], Visual Studio 2013. Another `_setargv()` that is a part of the standard Windows development environment expands wildcard characters, and is described in [http://msdn.microsoft.com/en-us/library/8bch7bkk.aspx%|%Expanding Wildcard Arguments], Microsoft Visual Studio 2013. It can be selected via the appropriate command-line switch at compile time. Still other programs roll their own `_setargv()`. Since there is no way to tell `[exec]` to skip the step of translating its arguments according to the default `_setargv()`, other workarounds must be improvised. For example, the Windows `FIND` command has its own peculiar syntax: ======none FIND [/V] [/C] [/N] [/I] [/OFF[LINE]] "string" [[drive:][path]filename[ ...]] ====== At first blush the syntax looks fairly normal, but in contrast to other programs, which don't see the double quotes because the default `_setargv()` stripped them off prior to invoking the program, `FIND` does see them, and uses them to determine string to search for. Even though its documentation doesn't say it, `"string"` can be the second argument instead of the first as long as the first argument doesn't contain any double quotes ======none FIND c:/myfile.txt "somestring" ====== In fact, the arguments don't even have to be delimited by whitespace. `"string"` is simply the first two double quotes in the command line and whatever is between them, although if the second double quote is not followed by whitespace, `FIND` returns an error. According to `FIND`, the following syntax is perfectly legal, causing it to search for `somestring` in `c:/myfile`: ======none FIND c:/myfile.txt"somestring" ====== In the following example, `FIND` searches in `c:/myfile.txt` and `c:/myotherfile.txt` for `somestring`: ======none FIND c:/myfile.txt"somestring" c:/myotherfile.txt ====== Using `[exec]` to execute `FIND` can be tricky. The following example won't work because Tcl strips the double quotes even before `[exec]` is called. ====== exec FIND "somestring" c:/myfile.txt ====== The following example won't work because `[exec]`, in accordance with the default `_setargv()` rules, adds a backslash character to each double quote: ====== exec FIND {"somestring"} c:/myfile.txt Access denied - \ ====== Find, of course, doesn't follow the default `_setargv()` rules, and the backslash in the `Access denied` error is the first backslash in the command line that `[exec]` generated, which actually looked liked this: ======none FIND \"somestring\" c:/myfile.txt ====== which lead `FIND` to conclude that it should search in the file `\` and in the file `c:/myfile.txt` for `somestring`. Since `[exec]` is always going to translate its arguments into a single command ine, the workaround is to have `[exec]` call [http://technet.microsoft.com/en-us/library/bb490880.aspx%|%CMD] with an argument that is properly formatted for the target command: ====== exec $::env(ComSpec) /c find {"somestring"} c:/myfile.txt ====== Another option is to pipe the script into the standard input of `CMD`. The trailing newline is necessary: ====== exec $::env(ComSpec) << {find "somestring" c:/myfile.txt; } ====== The `cmd <<` variant behaves more like an interactive command shell, displaying an initial message, and requiring a newline to terminate the last command, so it may be considered cleaner to use `cmd /c` instead. The script that is passed to `CMD` can of course use environment variables and `CMD` syntax: ====== set ::env(QT) \"; exec $::env(ComSpec) /c find %QT%somestring%QT% *.txt ====== `FIND.exe` interacts with the console directly, and its output will not be captured by `[exec]`. It also won't work in [Wish] on Windows for the same reason. ** Proposal: `-noquoting` ** [MJ]: A `-noquoting` option to `[exec]` could tell it to skip the step where it attempts to quote the command line according to the rules of the default `_setargv()`: ====== exec -noquoting {test.bat "a,b"} ====== This will also make use of `[auto_execok]` without eval possible (although in 8.5 [{*}] solves this in different way): ====== exec -noquoting "[auto_execok start] http://www.tcl.tk" ; # 8.4.13 exec {*}[auto_execok start] "http://www.tcl.tk" ; # 8.5 ====== This new flag will only lead to problems with older scripts if a command named -noquoting exists which seems unlikely. [RS]: Another point that `exec -noquoting` should handle is to take < and > literal, not as redirections, e.g. ====== exec echo "" ====== [MJ]: Ideally, -noquoting would refrain from interpreting any special characters such as `" < << 2>` etc. So the tag case should be handled correctly eg: ====== % exec echo "" couldn't read file "starttag>": no such file or directory % exec -noquoting echo "" % ====== [MJ]: You can download a patch on 8.5a3 from [http://marknet.tk/downloads/exec.patch] that adds a `-noquote` option to `[exec]`. The command must still be expanded so: ====== exec [auto_execok start] ====== will still not work. All other quoting of `" | & <` etc. will be disabled. Note however that `| &` and others will still have a meaning for the shell. To apply the patch go to the root of the Tcl source dir and type: ====== patch -p1 < exec.patch ====== After running all the test cases for exec only one test case fails (understandably), so the patch shouldn't break any existing functionality: ======none ==== exec-14.3 unknown switch FAILED ==== Contents of test case: list [catch {exec -gorp} msg] $msg ---- Result was: 1 {bad switch "-gorp": must be -keepnewline, -noquote, or --} ---- Result should have been (exact matching): 1 {bad switch "-gorp": must be -keepnewline or --} ==== exec-14.3 FAILED ====== '''Examples''' ======none % exec cmd /c echo \"test\" \"test\" % exec -noquote cmd /c echo \"test\" "test" % # example of disabling interpretation of < % exec cmd /c echo <> ; # error from Tcl couldn't read file ">": no such file or directory % exec cmd /c echo \"<>\" ; # quoting doesn't help \"<>\" % exec -noquote cmd /c echo <> ; # error from shell > was unexpected at this time. % exec -noquote cmd /c echo \"<>\" ; # quoting helps "<>" ====== [MJ]: After using the `exec -noquote` version for a while I noticed that [TWAPI] already offers much of the requested functionality in `twapi::createprocess` ====== package require twapi interp alias {} twexec {} ::twapi::create_process {} -showwindow hidden -cmdline twexec "echo <>" ====== I don't see a clear way to get the output of the command into Tcl yet so it might not be usable in cases where you need the output of the command. [LV] 2007-06-07: to move forward it would be worthwhile to submit a [TIP] if you haven't already, detailing the `-noquote` option, and pointing to your patch. That way, the [TCT] can debate what you've proposed, and perhaps help tweak things so that it works even better. [PYK] 2014-08-29: `+1` for `-noquote`. ** Batch Files ** [MJ]: One example I have encountered is when passing parameters to a batch file that contain comma's: ======none test.bat a,b ====== will call test.bat with two parameters a and b. This can be very useful, but leads to problems when you want to send a,b as one parameter. The solution is to quote the a,b ======none test.bat "a,b" ====== If we want to call this batchfile from Tcl, the first idea would be to use ======none exec test.bat {"a,b"} ====== For the reasons describe above, exec will transform this to: ======none test.bat \"a,b\" ====== One of the techniques described above can be used instead: ======none exec cmd /c "test.bat \"a,b\"\n" ====== ** Misc ** [HaO] Example opening files in windows explorer with a file selected. Possible invocations on a windows command prompt: ======none C:\Windows\explorer.exe /select,C:\Program Files\tcl8.5\doc\tcl85.hlp ====== or ======none C:\Windows\explorer.exe /select,"C:\Program Files\tcl8.5\doc\tcl85.hlp" ====== but not ======none C:\Windows\explorer.exe "/select,C:\Program Files\tcl8.5\doc\tcl85.hlp" ====== which is issued by the exec quoting rules when using: ====== set filename [file nativename {C:/Program Files/tcl8.5/doc/tcl85.hlp}] eval exec [auto_execok explorer] [list /select,$filename] ====== Found solutions: ====== eval exec [auto_execok explorer] [string map {\\ \\\\} /select,$filename] eval exec [auto_execok cmd] [list << "explorer.exe /select,$filename\n"] eval exec [auto_execok cmd] [list << "explorer.exe /select,\"$filename\"\n"] ====== The first passes the path as multiple arguments and thus avoids exec adding quotes. The second and third use the previously-mentioned cmd.exe magic to get the required data one and two. I was not able to get controled quotes using solution 1. [PYK] 2014-08-29: `explorer.exe` is another example of a program which does not use the standard `_setargv()` command-line parsing. It parses the command line on its own, and does not require any kind of quoting of whitespace in the filename portion of the `/select,` flag. `cmd /c` is a little cleaner than `cmd <<`, so the following is probably the optimal syntax: ====== exec $env(ComSpec) /c "[auto_execok explorer] /select,$nativename" ====== ** Reference ** [http://msdn.microsoft.com/en-us/library/a1y7w461.aspx%|%Parsing C Command-Line Arguments], Microsoft Visual Studio 2013: describes the default `_setarv()` behaviour. [http://msdn.microsoft.com/en-us/library/windows/desktop/ms682425%28v=vs.85%29.aspx%|%CreateProcess function], Microsoft Visual Studio 2013: [http://msdn.microsoft.com/en-us/library/zay8tzh6.aspx%|%Customizing C++ Command-Line Processing], Microsoft Visual Studio 2013: [http://msdn.microsoft.com/en-us/library/8bch7bkk.aspx%|%Expanding Wildcard Arguments], Microsoft Visual Studio 2013: describes an alternate `_setargv()` [http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/batch.mspx?mfr=true%|%Using batch parameters], Microsoft Windows XP: <> Command | Discussion | Windows