I/O paradigms in a multi-processing environment - event/datagram v. stream

Tcl and Network/Process Control

Tcl is very good for network/process-control systems because it has an event system built in, because it's strong on string manipulation, and most recently because it has coroutines. This is not surprising, really, because of its pedigree. Tcl, as an embedded control language, has this kind of thing in its DNA (as the cliche goes.)

The purpose of this page it to explore, in abstract, the kinds of facilities Tcl provides across the whole range of network/process-control systems, and which language cliches are useful when Tcl facilities must be blended together, and where Tcl could be improved to suit these applications.

Ideal Client/Server

Most (if not all) network/control systems involve message passing of some kind. Request-response, indication-confirmation: a chunk of data arrives, something is done with it, and a chunk of data is returned to the caller.

Ideal Server:

  1. session starts,
  2. request is received,
  3. processing occurs,
  4. response is sent
  5. repeat 2-4
  6. session terminates.

Lexicon: session above can be synonymous with transaction, connection, pipeline.

A simple example is a time-of-day server. (1,2) A connection opens (implying a request,) (3) the current time of day is calculated and formatted, and (4) returned. (5,6) The connection is closed by the recipient.

This simple case is amply served by event-driven code which runs to completion. Any time taken to perform processing is considered negligible, such that there is no requirement for multiple threads of control to intervene, provide input to, or suspend processing.

The Ideal Client looks very similar to the server:

  1. establish session,
  2. send request,
  3. wait for response,
  4. receive response,
  5. repeat 2-4,
  6. session terminates.

Each client state has 1:1 mapping to a presumed/hypothetical ideal server state. And this is how we like it. It makes life simpler to reason about.

An ideal client can be written [Connect(); foreach request $requests {Send(); Receive();} Disconnect()], and the current execution state is the current communication state ... the client can put its grubby index finger on one place in the code, and say "we have Connected, Sent, Received or Disconnected. This is a considerable benefit in writing and reasoning about client code.

The idealised model is also useful in that it expresses (in plain English) the passivity of the server. All of the verbs describing server processing are in the passive voice, all of the verbs describing client processing are in the active voice - consistent with how we think about clients and servers.

These two models serve well in many cases, but not in all. In particular, the implicit control flow in the ideal client is mostly linear (or clearly nested loops) and single-threaded.

Complexity 1 - the multi-client server

Naive run-to-completion won't work if you expect to be able to serve an arbitrary number of clients with an arbitrary number of requests per client (apparently) simultaneously, because while you're serving the current client's requests you can't serve any other client requests.

Examples of real protocols where server naive run-to-completion will possibly work include strict original HTTP 1.0 ... (others?) because although HTTP 1.0 servers expect to serve multiple clients, each client's requests per connection are limited to 1 per connection. Of course, in practice this never happened, because the extreme expense of setting up and tearing down a TCP connection per request was prohibitive.

A run-to-completion model for servers, where each request is processed completely and replied to before any other request is considered or acted upon can work, but only if the time to process is short and in any case bounded [Aassumption 1``]. In practice (outside of contrived examples,) this seldom happens, so in practice the strict run-to-completion server is limited to a server with a single client, and the interactions could more properly be regarded as remote procedure calls.

A more realistic example is a server providing HTTP 1.0 + 100 Continue messages to multiple clients ... these are expected to provide an arbitrary number of clients the ability to request multiple entities, and must use Tcl events to provide this service.

The good news is that each 'session' is adequately identified by a single instance of Tcl connection / channel, and the Tcl fileevent facility is sufficient (in theory) to contain/represent all state relevant to a given client instance in the server (all state could be associated with the script prefix in a pending fileevent, and could therefore be inspected/mutated in that locus.)

In practice, then, a slight variant on the ideal server is often adopted:

Quasi-Ideal Server:

  1. session starts on $chan, [fileevent readable State2 $chan; return]
  2. request is received,
  3. processing occurs,
  4. response is sent
  5. repeat 2-4
  6. session terminates.

The [fileevent readable State2 $chan; return] cliche has the effect of freeing the server to accept more connections on new channels. Reception, processing, response are all able to be implemented in run-to-completion form within the readable event-handler.

Note, all of this processing, 2-6, *could* be performed in a Tcl thread, and written linearly. However, the overhead of creating/maintaining/destroying a thread per client instance is prohibitive, complex, doesn't scale (insert your favourite scare-words here) and since Tcl provides for a lighter-weight alternative, why not use it?

coroutines provide an even cleaner approach to this problem, in that they permit states 2-6 to be encapsulated in a proc or apply wrapper, and written as if they expressed a single thread of control (using green threads.) This is wonderful, in that it makes the server again look as simple as the client.

But ... all too predictably, it isn't sufficient ... if State (3) processing entails (a) access to shared resources, or (b) external systems, or (c) subsystems which take considerable time to complete, run-to-completion of processing will inevitably degrade the client-illusion of having a dedicated server. Assumption 1 is often invalid, but not often enough that Tcl has evolved any particular strengths in facilities to handle it, so here's where the fun starts.

Complexity 2 - the multi-client server with unbounded processing

A well-known example of this is database access.

Quasi-Ideal Server Split in two:

  1. session starts on $chan, [fileevent readable State2 $chan; return]
  2. request is received,
  3. processing starts
  4. repeat 2-4
  5. session terminates

and its accompanying half

  1. processing occurs, then completes
  2. response is sent

Here, the completion of processing is an event which runs in a distinct thread of control, maybe a thread or maybe as a consequence of an external event. Processing could be querying a database, fetching a web page from a different server, getting some readings from an external device that is going to take a while, invoking an external program under unix, or it could just be complex processing which is going to take a long time and is delegated to a thread. Often, processing entails the server acting as a client to a different server. Notably, the Ideal Client is often chosen as the appropriate model to express inter-server processing.

Complexity 3 - multi-client server with scatter-gather

If the server's processing phase requires more than a single interaction with external asynchronous data sources, the usual choice is to make these requests synchronously and await their completion, effectively to enact the Ideal Client with multiple Ideal Servers.

Sometimes this just won't work as well as it might. Examples: A server which functions as an SMTP client, in which an email can be sent, simultaneously, to a large number of recipient machines, and resultant delivery success/failure is to be reported to the server's client. In this case, processing is complete when all client-relationships terminate. The appropriate paradigm is scatter-gather.

The problem here is that Tcl doesn't provide any obvious way to aggregate responses from asynchronous processes, but it does provide one: variable traces on arrays and on array elements. Using this mechanism, it is efficient to wait on both individual channels *and* on the set of all channels, orthogonally giving events per-external resource, and events on the entire collection of resources.

Criticism of Ideal Client

Humans are notoriously bad at dividing their attention (and not just people who are a bit autistic, like me.) We can't talk on the phone and drive, some of us can't walk and chew gum at the same time.

Computers aren't bad at multi-focal attention at all, but (I posit) because we're bad at it, we build our faults into our code.

Tcl coroutines are able to provide a perfect illusion of single thread of control while accessing chans, by means of Coronet (a package which hides the event system in a nice coroutine-enabled wrapper.) This advances the green threads for Tcl project, and also raises the question: if threads are now green, and we can use them anywhere, even to the extent of shoehorning code into green threads without it having to be modified ... where are the control mechanisms and language cliches which make that simple?