Erlang Idioms for Tcl

Erlang is very cool. It's functional (declarative too), supports massive concurrency and promotes a number of programming idioms I find myself applying to Tcl. You don't have to emulate it via syntactical FP sugar (Lambda in Tcl, List Comprehension, etc). The idioms are rich and Tcl is up to the task! Tcl (with Threads) can do COP too ;-)

My network servers (coded in Tcl) are starting to model the way Erlang/OTP does things.

(Some of the items below are FP idioms, not just Erlang)

Some of the simple idioms:

  • Minimize side effects -- Try writing Tcl apps without global or namespace variables. Pass state as proc parameters (localize your errors). E.g. Rewrite fileevent/after handlers to accumulate state...
  • Concurrency interaction through message passing -- The Tcl thread package does this beautifully.
  • Controllers/Monitors -- Layer your app with processes/threads that do logic, monitor the threads that do the logic, manage the threads that monitor the threads that do the logic ... etc
  • Handle errors in a higher layer -- Interesting... Don't waste valuable processing time wrapping catch { } around everything. Handle it in a separate (controlling) thread that is signaled with the error.
  • Threads/processes should do only one thing, etc.
  • Message Receivers in Erlang: Queue messages received and use pattern matching to select the ones you are interested in.

Some things you can't really do well (or at all) in Tcl:

  • Guarded Function Clauses

Could you elaborate upon this, please?

lexfiend: To illustrate, in:

 factorial(0) -> 1;
 factorial(N) when N > 0 ->
   N * factorial(N - 1).

the expression N > 0 guards the function clause factorial(N) -> N * factorial(N-1) (the reserved word when introduces the guard). It essentially says "Don't evaluate this function clause if the guard expression fails". It basically translates to an if test, so I assume Todd's commenting about it being difficult to similarly express such things in Tcl.

  • Tail Recursion instead of iteration (KBK 2003-05-23 Tail call optimization has come up as a topic before.)
  • lexfiend Massive concurrency: Erlang spawns VM threads rather than OS threads, so applications using literally thousands of threads (actually processes, since they don't share data) are both amazingly performant and not unheard of.

If one had a copious amount of time, one could adapt Erlang/OTP to Tcl. That would be very nice...

-- Todd Coram

Although I have seen variations upon this elsewhere (Tail call optimization), this continues to amuse me: Factorial Using Event Loop.

Erlang makes extensive use of dictionaries (like Tcl arrays). It has a fast and scalable implementation but Scott Lystig Fritchie discovered a way to incorporate something called Judy Arrays into Erlang. His paper compares different implementations of hashes for Erlang.

-- Todd Coram

NEM Erlang is a nice language. Guarded functions aren't too difficult to do (I'm sure this is probably on the wiki somewhere already):

 proc func {name args} {
    set params [lrange $args 0 end-1]
    set body   "when [list [lindex $args end]]"
    proc $name $params $body
 proc when cases {
    foreach {cond -> action} $cases {
        if {$cond eq "otherwise"} {
            return [uplevel 1 [list expr $action]]
        } elseif {[uplevel 1 [list expr $cond]]} {
            return [uplevel 1 [list expr $action]]
    error "undefined case"
 func factorial n {
    {$n == 0}   -> { 1 }
    {$n >  0}   -> { $n * [factorial [expr {$n - 1}]] }
 puts [factorial 0]
 puts [factorial 5]
 puts [factorial -1] ;# Error case

It's not exactly like the Erlang (no pattern matching, etc) but you could make it nearer if you really want. You can also do more to "compile" the proc down to something sensible at definition time -- the version above delays processing to runtime, for neatness mainly.

Emulating Erlang's processes would be a neat trick; perhaps something useable could be built over the event loop (i.e., a lightweight process==proc type solution, rather than something more heavyweight). lexfiend: I believe Factorial Using Event Loop describes the basic methodology for that. NEM Well, not exactly. Are Erlang's processes cooperative or preemptive? lexfiend: Well, Erlang's scheduler is preemptive from the programmer's perspective (full story here: [L1 ]), but I think there should be little perceptive difference using the Factorial Using Event Loop methodology provided you're careful about not writing huge linear procs.

NEM I'm not sure what the Factorial Using Event Loop methodology is, in terms of comparison to Erlang's process model. Do you mean manually converting code to continuation-passing style (i.e., passing a result callback to each function) and then using after to schedule calls? lexfiend: Yes, though I'll confess to not thinking too deeply about this topic. 8-)

DKF: Tcl can't really do massively (when I say "massive" I mean hundreds-of-thousands) threaded stuff as it uses OS threads for its threading model. On the other hand, Tcl's threading works with OS calls quite nicely (depending on the OS itself) without great gymnastics, and its event handling model is really great (I'm not aware offhand of any other language that is quite as event-oriented in practice.) lexfiend: In fact, I'd avoid threads altogether (as far as possible) for the 100,000 "thread/event" scenario. The event queue seems to be limited only by memory, and looks to be very robust.

Cooperative multi-tasking depends on, well, on all of the threads being cooperative. This can be a pain to program. Perl tries to do the event-queue (callback) approach for multi-tasking in POE (a nice attempt at doing what Tcl already has built in).

Erlang's (the language, not the math guy) philosophy is to do everything thru light-weight user level threading (processes). Rather than factor your code out into procedures/functions, an Erlanger can factor code out into processes. This is an interesting idea, but not one Tcl can easily emulate. Instead, a Tcler (with the thread extension) could factor computationally complex (or state-heavy) things into threads and factor cooperative stuff out into (continuable) events.

For example, Erlang could do a webserver as: 100 processes for 100 concurrent user connections plus 1 process for the database and N processes for backend computationally intensive apps (let's say N is 5).

In Tcl, I would do this as: 4 threads (each handling 25 concurrent user connections) plus 1 thread for the database and N processes for backend apps (where N is 5).

-- Todd Coram

I keep gushing on Tcl's event loop, but I can't help it. Erlang does concurrency better than most languages I've seen (Haskell, Mozart and Concurrent ML do concurrency very well too, but they lack the industrial support of Erlang -- Erlang/OTP lives and breathes real world concurrency while most other languages are more academic about it). Tcl's event loop support is stronger than any other mainstream language. And, it was brilliant how the Tcl threading library took this into account! -- Todd Coram