NRE: Coroutines in 8.6

The second talk in Miguel Sofer's NRE series at Tcl2008, following up on NRE: the non-recursive engine in Tcl 8.6:

Coroutines--what?

• A computation that can be suspended and resumed on demand or killed while suspended
• A tool to create generators
• A tool to simplify the coding of event-based programs
• A tool to enable cooperative multitasking within a single interp, without OS support

TIP #328 [L1 ] specifies three new commands: coroutine, yield, and info coroutine. [coroutine] is evaluated equivalently to uplevel #0 [list cmd ?arg ...?]. [yield] suspends operation. Walked through an example from the tip.

[yield] returns twice: when yielding (caller reeives a return value from the yielding coroutine); when resuming coroutine sees the [yield] command returning. [cmdName] is "garbage collected". Example at http://msofer.com:8080/wiki?name=Coroutines

Went through the wiki page Coroutines for event-based programming showing converting a console application to a GUI.

Coroutines: why?

• GUI programming "made easier" (NEM)
• nestable vwaits?
• Generators
• Servers (wub)
• Cooperative multitasking (event loop)
• Cooperative multitasking (dispatcher)

Wil need to learn from experience which is better/faster (event loop vs dispatcher)

Coroutines: tech?

• coroutine: creats a command and a new execEnv for it; callback in the old execEnv to swap back to it; swaps in the new execEnv, callback to clean up; launches script
• yield: freezes TEBC; swaps out coro's execEnv; returns value

The byte code engine doesn't know much about coroutines; things happen behind its back (that's mostly true).

Deleting a suspended coroutine winds-down the suspended execution (as if interp's limit had been reached).

Special considerations:

• Commands run at [uplevel #0]
• You can yield from a nested call. These are asymmetric stackful coroutines in terms of Lua
• If the command pushes a CallFrame (proc or lambda or TclOO method): local variables retain their state when suspended, local variables are at level #1

Tailcalls: what?

(...Due to time constraints a couple slides were on screen for only a few seconds each...)

A pseudo-implementation (may be useful for debugging purposes: stack trace!):

 proc tailcall args {
     set cmd [[uplevel 1 [[\
         list namespace which [[lindex $args 0]]]]]]
     lset args 0 $cmd
     uplevel 2 $args
 }

Tailcalls: why?

• Infinite recursion depth possible
• Functional programming constructs
• Psuedo-macros (without the speed gain)
• continuation passing style
• State machines
• "Delegation"

Tailcalls: tech: tailcall arranges for the command to be run right after all callbacks that were scheduled by the callee's implementation.

Question from DKF: What's next Miguel? Answer: coffee?

Question from DGP about whether coroutine resume command name could be pre-resolved with [namespace origin]. Answer: great feature request. DGP I thought more about this, and on deeper reflection, it's not obviously true this is a good idea. It comes down to subtle distinctions about exactly what timing of command resolution is desired, and whether we want to foreclose any of the possible choices.

Question from the chat about heap memory usage. (Some KB; didn't catch how many if coroutines used at all today; will be less before 8.6.0 is released).
DKF: I double-checked this. It's 8kB, which is far too hungry. But to reiterate, this is untuned.
MS: I just thinned them down 10x, they are below 1kB now. Further tuning may yet occur ...