'''[Philip Quaife] 27 Mar 05'''

'''Purpose'''

This page exists as an historic document for reference from other sites.

'''Content'''

This page contains a definition of the change requirements to the Tcl core 
to eliminate the problems that developers have in distributing and deploying their code.


'''Introduction'''

In the present time, end users have no clue about installing software. 
The analogy I use is that of people driving cars. People expect a car to just work 
without having to understand what goes on under the hood.

The past and present view of core developers is that end users should understand how to operate Tcl in order to use the applications that have been developed.
''Note that the word application is used here in its wider meaning''.

The end user has a '''Click and Run''' mentality, 
if the process is more complicated than that they don't want to know.

'''The Present Situation'''.

The end user locates and downloads a Tcl application. They then find it does not run. 
Either Tcl is not installed or it is the wrong version.

The end users are then required to locate the tcl interpreter, download, compile, and install it.
They then locate, download, compile, and install any extensions that are needed by the application they wish to run.

Most users have given up well before the end of that process 
because they either lack the skill or inclination to do the required work.

'''Attempts to address the above'''.

Skiping alot of pain and suffering by developers (at the hands of the cruel TCT overlords), 
we now have two solutions, ''starkits'' and ''freewrap''.

The deficiency in both of these is illustrated in the following example.

  1. DeveloperA tclkit and makes it available for common platforms.
  2. DeveloperB uses tckit and makes an application , 
     puts it on the internet and anounces 'look at me, look at me'.
  3. UserC  downloads the kit and it does nothing. So they email
      developerB for help.
  4. DeveloperB directs the user to DeveloperA for the appropriate
       tclkit to run the application.
  5. DeveloperA informs UserC that there currently is no tclkit for
       their platform (platformX) and to see DeveloperB to make one.
  6. UserC goes looking for another application and now has the view
       that Tcl and DeveloperB suck.

We can shortcut the above general procedure by creating ''starpacks'' or using ''freewrap'' 
in the above example, but the end problem still remains (no runtime for platformX).

'''Whos task is it to create a tclkit/freewrap for platformX ?'''

DeveloperA does not have access to the hardware platform, 
UserC has the hardware but can't program, 
DeveloperD has the hardware but does not understand the build process for tclkits 
and can't resolve any build failures.

No matter which way you try and solve the problem we always end up in a ''Catch 22'' or ''Mexican Standoff'' position.


'''The Solution'''

The failure in the process is that in order to release binary files for the users convenience 
we must create a custom tcl interpreter. A task which is beyond most developers.

The question then is , what changes to the core are required to allow the above to happen without creating a custom tcl interpreter that will operate as a stand alone executable.

These are the minimum changes required.

   1. Make Tcl boot process detect that data has been attached to the end of the tcl interpreter executable file.
   1. Make the binary data accessable at the script level.
   1. Source the binary data.

Lets see an example application built with the existing and mythical method proposed here:

      STARPACK                MYTHICAL
   |-----------|           |------------| 
   |  custom   |           | standard   |
   |   tcl     |           |   tcl      |
   |  interp   |           |  interp    |
   |    +      |           |------------|
   |  vfs code |           |marker|
   |   +       |           |------------|
   |extensions |           |boot script |
   |   +       |           |^Z|
   |init code  |           |------------|
   |-----------|           |   binary   |
   |   binary  |           |   data     | 
   |   data    |           |------------|
   |-----------|           |marker|
   |marker|


'''A simple example'''

To demonstrate the boot process lets take a simple example
to say play a sound file using snack:

   # Boot code
   set f [open tmpfile w O_BINARY]
   puts -nonewline $ [embed payload 0 187645]
   close $f
   load tmpfile snack
   snack::play $argv
   exit
   ^Z

To understand the above, we assume that the standard tcl interpreter includes an ''embed'' command 
that performs as follows:

    embed payload ?start? ?end? 
''allows access to the attached data, starting with the first character after the ^Z.''

Other sub commands to ''embed' will be outlined later in this document. 
The above however is the minimal new command necessary.


'''Practical issues'''

The binary data is likely to be quite large and so we would like to compress it 
to reduce transfer time and such.

We can do this without any change to the core if we make our file as:

  | tcl interpreter |
  | marker|
  |boot script ^Z   |
  |inflate dll      |
  |rest of data     |
  | marker|


The boot script would become:


   # Boot code
   set f [open tmpfile w O_BINARY]
   puts -nonewline $ [embed payload 0 15345]
   close $f
   load tmpfile inflate
   set f [open tmpfile w O_BINARY]
   puts -nonewline $ [embed inflate [embed payload 15346 234567]]
   close $f
   load tmpfile snack
   snack::play $argv
   exit
   ^Z

Any modern decompression algorithm is likely to be less than 10K of binary code so it is not a necessity to include such code inside the core itself. By not including one we can also take advantage of newer more efficient compression algorithms as they are developed without having to recompile Tcl or requiring the end users obtain a newer version of Tcl.

'''Cross platform compatibilty and trade offs'''

While any decompression routine would be small enough to attach to a payload we may still have the problem of getting a compiled decompression library for every platform that the Tcl interpreter can be compiled for.

It then makes considerable practical sense to include an inflate type command inside the core as an exension library so that it will be built as part of the normal Tcl build process and thus will automatically be available wherever Tcl is. By building it as an extension the developer is free or include something better but always has the Tcl extension as a fall back position. 

Another issue is the loading of shared dlls. Some platforms do not support this, 
while others can load an extension directly from memory.

This makes it a practical necessity to provide a boot process extension loading mechanism outside of the normal Tcl load command. On most platforms it is likely that the normal code will be used, but for those platforms that don't we can implement a platform specific solution.
We could call this command:
	embed load ?start? ?end? #''This loads an extension straight out of the binary data''.

Another requirement is to free the developer from having to maintain binary versions of their applications for end users to download. There are currently projects under development that are designed to handle this deployment issue so the solutions will not be outlined here.

'''Planned Core changes'''

It is planned that the core will include a number of extensions such as '''libZ''', which are aimed at addressing the issues raised here. Unfortunately the problem is more basic than the issue of core features and libraries.  The  issue is in initialization and startup of the interpreter which defeats the present day developer.

'''In Summary'''

The minimal changes required to the core are:

  1. Detect an attached payload deferr initialisation and source the binary payload.
  2. Add a command to retrieve the binary data from script level.
  
Practical extensions to the core could be:

  3. A decompress command.
  4. Load an extension directly from the binary command.


----
[Category Distribution] | [Category Deployment] | [Category Concept]