clonerun

Starpacks: Why and how to copy your own executable and launching the copied instance instead

The Problem: Executables, like starpacks, are often called from single network shares from many users in parallel. So the chance is good, that at least one user has an instance of that specific program running all the time. Now the administration wants to update that program. But this is not possible (at least) under Microsoft Windows because each executable on disk that has running instances active is locked (such lock is invisible via net files).

The Solution: Each user has to launch his own private copy of the program. But this must not lead to an distribution overhead: the program is still distributed only once to the network share, then each instance copies itself to a temporary position, runs from there and terminates quickly. The main executable is locked only for short load periods, so the program file on disk remains in a state where it can be overwritten (with retries, if required) for update purposes. Ideally, the per-user copy of the program is stored in an temporary location that is periodically freed, so the hard disk will not be cluttered (for the above mentioned reasons, the 'user-copy' cannot delete itself while terminating...).

Note: A starpack cannot copy its own executable with Tcl commands like file copy or a open/read/put/close-sequence (because as a result always the whole directory structure gets unpacked in the destination). This has something to do with the underlaying VFS Implementation; one must use an external copy/xcopy etc. instead that treats the .exe like any other disk file. Or the VFS needs to be unmounted before the file copy, as described under the following link:

http://permalink.gmane.org/gmane.comp.lang.tcl.starkit/2537

Here's the latest standardized version of the above mentioned snipped:

################################################################################
#
# Modul    : autoclone.tcl
# Date     : 29.02.2008, 14.07.2010
# Purpose  : copy the own program to a temporary folder and continue loading
#            the long-running program from there, to avoid locked and non-
#            updatable executables (starpacks under MS-Windows)
# Author   : M.Hoffmann
# Remarks  : A starkit/pack cannot copy itself with 'file copy' etc. without
#            first unmounting its own virtual file system
#            Programs locating profiles only in their own load path may fail;
#            they can use autoCloneOrgPath.
# History  :
# 13.11.05 v1.0: first version derived from individual code in RECEIVE program
# 11.05.07 v1.0: minor internal changes and updated wiki page
# 28.06.07 v2.0: using suspendVFS instead of XCOPY; quit if no starpack; always
#            copy to temp folder to avoid security risc; translated source;
#            autoCloneOrgPath
# 29.02.08 v2.1: bugfix: missing catch in suspendVFS possibly left VFS unmounted
# 29.02.08 v2.2: bugfix: missing quotes caused return to crash w/blanks in dirs
################################################################################

package provide autoclone 2.2

################################################################################
# see http://permalink.gmane.org/gmane.comp.lang.tcl.starkit/2537
#
proc suspendVFS {code} {
     set me [info nameofexe]
     lassign [vfs::filesystem info $me] drv dbh
     vfs::filesystem unmount $me
     # unmounted, now run the code
     catch {uplevel $code} rc
     # Remount
     vfs::filesystem mount $me [list vfs::mk4::handler $dbh]
}

################################################################################
# use proc to avoid cluttering the global namespace
#
proc autoClone {} {

     set myself [file normalize [info nameofexecutable]]
     proc autoCloneOrgPath {} "
          return \"[list $myself]\"
     "

     if {[string equal -nocase [lindex $::argv end] "--noautoclone"]} {

        # user don't want to use autoclone now or recursive call. Remove the
        # switch to keep things transparent for the later command line parsing.
        set ::argv [lreplace $::argv end end]
        set ::argc [llength $::argv]
        if {[string equal -nocase [lindex $::argv end-1] "--orgPath"]} {
           # remember the original load path, perhaps for loading configs from there
           proc autoCloneOrgPath {} "
                return \"[lindex $::argv end]\"
           "
           set ::argv [lreplace $::argv end-1 end]
           set ::argc [llength $::argv]
        }

     } else {

        set mytempP $::env(temp); # hm... no fallback yet for missing temp-var
        set mytemp  [file normalize [file join $mytempP [file tail $myself]]]

        # v2.0: if we are not a starpack we can skip
        if {[string first $myself [file normalize [info script]]] != 0} {
           return
        }

        # no further action if called directly out of temp folder by the user
        if {[string compare $myself $mytemp]} {
           # alternative, old copy-method (piping 'copy' to 'cmd.exe' does not work for everyone):
           # catch {exec -- [auto_execok xcopy] [file nativename $myself] [file nativename $mytempP] /Y}
           # copy will also fail if dest is locked (because another instance already active. iow: the 2nd
           # instance alway launches from the original position).
           if {![catch {suspendVFS [list file copy -force -- $myself $mytempP]}]} {
              # only ever continue if copy succeeds to avoid calling a trojan horse
              set ::argv [linsert $::argv 0 $mytemp]
              # attempt to use START not successfull yet (EXEC acts a bit strange...):
              # set ::argv [linsert $::argv 0 [auto_execok cmd] /c start \"[file nativename $mytemp]\"]
              # avoid recursion early
              if {[catch {eval exec -- [linsert $::argv end --orgPath $myself --noautoclone] &} rc]} {
                 # our copied prog does not start; silently ignore the error and continue with this instance...
              } else {
                 exit 0; # termine immediately as the launched copy takes control now!
              }
           }
        }
     }
}

# Attention: 'package require' automatically starts this code, so do it very early!
autoClone;

It should be mentioned that the code is intended for use by starpacks only, especially for GUI-Starpacks (problems with tclkitSH-version are likely, since the EXECd process eventually inherits some resources belonging to the console, so the EXECing process doesn't end...).

EF I was trying to point out that the locking mechanism that you describe here is the reason why my updater library [L1 ] does not work on Windows. However, the techniques that you are describing could be combined to the library in order to provide for binaries that update themselves when a new version appears at a known Internet location

SRIV I routinely use a similar technique on Linux with starkits:

run starkit
start a background proc that checks the apps mtime every 10 seconds
the at proc detects the mtime change, unmounts the vfs, re-sources itself.
repeat

LV Also, note that your starkit should never expect to save state within the executing starkit itself. In the olden days, people used to save state within the starkit so that only the 2 files (tclkit and starkit) needed to be copied around. If the application had an internal database or other data structure that would be updated (like a wiki, notepad, or even default configuration), the starkit would be installed writable and the code would just update the executable file. If, however, one is copying the code to a temporary location, then executing the code at that temporary location, then the program has to make certain that what it is updating the original file and not the one in the temporary location. Also, in the scenario described here, you have multiple users using the application. That multiple use will require careful locking so that two users don't attempt to write to the original executable at the same time.

Category Tclkit