Slow Starpack Start

MHo, 2010-07-13: I notice the following symptom with starpacks on MS windows:

  • Starting them from a local drive (like c:\) is very fast ;-)
  • Starting them from a mapped network drives is relatively slow
  • Starting other programs from the same network drive is not very slow

I don't know the reason yet, as I have only started to analyse what's going on a few hours ago... It seems that the phenomen has nothing or little to do with the size of the executable. Sure, an antivirus engine is active and accessing the network drive for the first time always gives a little delay but... it seems that something's going on inside the starpack which slows down the whole "bootstrap" under certain circumstances. Well, maybe the reason is my ugly coding and not the tclkit itself...

Maybe someone can confirm the behaviour in general?

I have to take a deeper look at the output of FileMon and ProcMon.

AMG: I once encountered a program that ran super-slow when run off a network share, yet was fine when running off a local hard disk. It was slow because it did many thousands (tens? hundreds?) of open/read all/close cycles on configuration files stored in the same directory as the program. The tricky part was that these reads were being done deep inside a Windows API call, so it wasn't obvious that they were taking place. Replacing the Windows API call with my own caching implementation made the program fast again.

Perhaps something similar is happening with Starpacks.

AK: IIRC the metakit system tries to access its database (our container for the virtual filesystem) via memory mapping for speed, and falls back on plain read/write calls when the access is over a network, i.e. memory mapping not possible.

AMG: Yeah, that could be it. Maybe metakit needs some caching.

MHo It seems that the starpack reads itself several times (because of the contained VFS) in thousands of separate IO-chunks, each of which are sometimes only 4k or 512 byte in size.

uz - 2010-07-15 20:26:31

I observed the same behaviour with Freewrap: starting from a local network drive is very slow (in the order of 10 to 30 sec) compared to launching from a local drive.

Martyn Smith - 2010-07-16 I use Freewrap as well and have slow startup problems, but not from a network drive, for some reason Freewrap is accessing a non existing /net or /users/martyn directory, which do not exist. The users/martyn is the internal virtual ZIP directory where my code is but for some reason freewrap tries to access this on the external filesystem causing a long timeout.

MHo - 2010-11-11: Unfortunally, again I trapped into the above mentioned quirks. It do makes a difference if a user waits 10 seconds for a program to start or just 1 second. Definitely, starting starpacks from network locations on MS Windows is super slow. And again unfortunally, I can't help an other way than recoding in another language (which is the last thing on my mind). I wonder if this behaviour will change with tclkit 8.6 (which is, on the other hand, much bigger then 8.5 which is much bigger than 8.4 which...). I had to test it, but my time is limited... (one reason for using Tcl ;-).

Maybe those plain Metakit read/writes mentioned above could be optimized by using a greater buf size or so.

Meanwhile I opened an issue [L1 ] at google code.

Ro Did you try using the lite-style tclkits that use a non-C++ extension to access the vfs (vqtcl) ?

A tclkit that uses a non-C++ extension to access the vfs (vqtcl). The -cli version have no-Tk. Also missing Incr-tcl. NOTE: NO C++ LIBRARY REQUIRED.
As above but with Tk included in the vfs.
As above but Tk is linked into the executable.

MHo I'm using those kits, too, but there are no such kits for 8.4.19, I think (the tcl version which I still prefere because of its small size). I will test if those lite-kits behave different.

Ro Try freewrap 6.32: Changes for version 6.32

1. FreeWrap 6.32 is based on TCL/TK 8.4.19

2. Improved startup time for wrapped WISH applications under Windows. Slow startups were especially noticable when running wrapped applications from network directories, remote shares or from a Web server.

MHo I'm aware of freewrap, too, but all of my accompanying tools are made around tclkit...

Martyn Smith - 2010-11-22 I have rebuilt Freewrap and have slow startup on a couple of machines (see this thread on comp.lang.tcl). I have not had time to investigate any further. This is directly linked to freewrap accessing the /net directory which causes a DNS timeout on these machines.

MHo Meanwhile I've tested behaviour with tclkit-lite. I don't think they run much faster...