***Tcl and Network/Process Control***

Tcl is very good for network/process-control systems because it has an event system built in, because it's strong on string manipulation, and most recently because it has [coroutines].  This is not surprising, really, because of its pedigree.  Tcl, as an embedded control language, has this kind of thing in its DNA (as the cliche goes.)

The purpose of this page it to explore, in abstract, the kinds of facilities Tcl provides across the whole range of network/process-control systems, and which language cliches are useful when Tcl facilities must be blended together, and where Tcl could be improved to suit these applications.

***Ideal Client/Server***

Most (if not all) network/control systems involve message passing of some kind. Request-response, indication-confirmation: a chunk of data arrives, something is done with it, and a chunk of data is returned to the caller.

'''Ideal Server''':  

   1. `session` starts,
   2. `request` is received,
   3. `processing` occurs,
   4. `response` is sent
   5. repeat 2-4
   6. `session` terminates.

''Lexicon'': `session` above can be synonymous with ''transaction'', ''connection'', ''pipeline''.

A simple example is a time-of-day server.  (1,2) A connection opens (implying a request,) (3) the current time of day is calculated and formatted, and (4) returned. (5,6) The connection is closed by the recipient.

This simple case is amply served by event-driven code which runs to completion.  Any time taken to perform `processing` is considered negligible, such that there is no requirement for multiple threads of control to intervene, provide input to, or suspend `processing`.

The '''Ideal Client''' looks very similar to the server:

   1. establish `session`,
   2. send `request`,
   3. wait for `response`,
   4. receive `response`,
   5. repeat 2-4,
   6. `session` terminates.

Each client state has 1:1 mapping to a presumed/hypothetical ideal server state.  And this is how we like it.  It makes life simpler to reason about.

An ideal client can be written [[Connect(); foreach request $requests {Send(); Receive();} Disconnect()]], and the current execution state '''is''' the current communication state ... the client can put its grubby index finger on one place in the code, and say "we have Connected, Sent, Received or Disconnected.  This is a considerable benefit in writing and reasoning about client code.

The idealised model is also useful in that it expresses (in plain English) the passivity of the server.  All of the verbs describing server processing are in the passive voice, all of the verbs describing client processing are in the active voice - consistent with how we think about clients and servers.

These two models serve well in many cases, but not in all. In particular, the implicit control flow in the ideal client is mostly linear (or clearly nested loops) and single-threaded.

***Complexity 1 - the multi-client server***

Naive run-to-completion won't work if you expect to be able to serve an arbitrary number of clients with an arbitrary number of requests per client (apparently) simultaneously, because while you're serving the current client's requests you can't serve any other client requests.

Examples of real protocols where server naive run-to-completion will possibly work include strict original HTTP 1.0 ... (others?) because although HTTP 1.0 servers expect to serve multiple clients, each client's requests per connection are limited to 1 per connection.  Of course, in practice this never happened, because the extreme expense of setting up and tearing down a TCP connection per request was prohibitive.

A run-to-completion model for servers, where each request is processed completely and replied to before any other request is considered or acted upon can work, but only if the time to process is short and in any case bounded [[Aassumption 1``]].  In practice (outside of contrived examples,) this seldom happens, so in practice the strict run-to-completion server is limited to a server with a single client, and the interactions could more properly be regarded as remote procedure calls.

A more realistic example is a server providing HTTP 1.0 + 100 Continue messages to multiple clients ... these are expected to provide an arbitrary number of clients the ability to request multiple entities, and must use Tcl events to provide this service.

The good news is that each 'session' is adequately identified by a single instance of Tcl connection / [chan]nel, and the Tcl [fileevent] facility is sufficient (in theory) to contain/represent all state relevant to a given client instance in the server (all state could be associated with the script prefix in a pending [fileevent], and could therefore be inspected/mutated in that locus.)

In practice, then, a slight variant on the ideal server is often adopted:

'''Quasi-Ideal Server''':  

   1. `session` starts on $chan, [[[fileevent] readable State2 $chan; return]
   2. `request` is received, 
   3. `processing` occurs,
   4. `response` is sent
   5. repeat 2-4
   6. `session` terminates.

The [[[fileevent] readable State2 $chan; return] cliche has the effect of freeing the server to accept more connections on new channels.  Reception, processing, response are all able to be implemented in run-to-completion form within the readable event-handler.

Note, all of this processing, 2-6, *could* be performed in a Tcl [thread], and written linearly.  However, the overhead of creating/maintaining/destroying a thread per client instance is prohibitive, complex, doesn't scale (insert your favourite scare-words here) and since Tcl provides for a lighter-weight alternative, why not use it?

[coroutine]s provide an even cleaner approach to this problem, in that they permit states 2-6 to be encapsulated in a [proc] or [apply] wrapper, and written as if they expressed a single thread of control (using green threads.)  This is wonderful, in that it makes the server again look as simple as the client.

But ... all too predictably, it isn't sufficient ... if State (3) `processing` entails (a) access to shared resources, or (b) external systems, or (c) subsystems which take considerable time to complete, run-to-completion of processing will inevitably degrade the client-illusion of having a dedicated server.  ``Assumption 1`` is often invalid.

***Complexity 2 - the multi-client server with unbounded processing ***

 (todo: intervening network/control/sensor activity.)