Version 8 of ycl coro relay

ycl coro relay is a small but complete system that provides order/receive facilities among asynchronous coroutines. Features include timeouts, cancellation, error propagation, and scatter/gather capabilities.

Commands

accept ?spec? ?payload?: yield until called, and return a list in which the first value is the sender's information, and any subsequent values available the purposes of the current coroutine. spec is a list of names to assign from the list returned by yieldto to local variables, in the same manner as the second argument of proc, to local variables. If the last name is args, a list of remaining values is assigned to it. payload is a value to yield.

call ?spec? args: Like order, but the evaluated command does not have to call accept or use deliver, or even be a coroutine, although it can be. It's just a normal command that returns a result. Returns an order id that can passed to cancel. delay is the number of milliseconds to delay before sending evaluating the command. args is the command to evaluate.

cancel id: Cancel an order that hasn't been received. If the order has not been started, it won't be. Otherwise, any results it delivers are discarded.

deliver delay ?arg ...?: Deliver an order. Typically only used internally by order . delay is the number of milliseconds to delay before mmaking the delivery. The first arg is a command prefix for the sender. It's typically acquired as the first value in the list returned by accept. Remaining args are passed as additional in the command. In other words, items in the first value are expanded and replace the first value, and the resulting list is the command .

iter names coroutine script: Iteratively place orders to $coroutine and evaluate $script for each received order. $names is a list containing the name of a variable in which to place the received order and, optionally, the name of a variable containing a list of items to send as arguments of the next order. Within $script break and continue can be used as might be supposed.

last: Returns the order id of the last delivered task.

order delay ?arg ...`?: Place a new order and return an order id that can be passed to cancel. Number of milliseconds to wait before placing the order. If $delay contains more than one item, the second item is the maximum number of milliseconds alloted for delivery, order times out, after which the order is not be initiated if it hasn't been started, and if it has, any subsequent deliveries are discarded. If there is a third item, the second item is the maximum number of milliseconds alllotted for the order to be started, and the third item is the maximum number of milliseonds alloted for delivery. If the completion timeout occurs before the order is complete, any subsequent delivery is discarded.

receive: Yield until a previously-placed order comes in, and then return the value and the options dictionary of that order.

switch ?option value ...?: Yield until an order was received, and respond to it according to the first word of the order. routes is a dictionary of words and corresponding scripts. These are merged into the default routes.

wrap iter wrapper ?name? name: Returns the name of a new coroutine named name that acts as a proxy for iter, internally placing orders to iter, passing each delivery and the options dictionary to wrapper (a command prefix), and delivering that result to the caller. Useful to create filters, monitors, and multiplexers. If name is not provided, a name is automatically selected.

Examples

ycl coro relay test: The test suite for the package.

ycl comm http: An http client that makes use of ycl coro relay.

Example

#! /bin/env tclsh

package require {ycl coro relay}

namespace import [yclprefix]::coro::relay

proc adder {} {
    while 1 {
        relay accept {deliver args}
        {*}$deliver [::tcl::mathop::+ {*}$args]
    }
}

proc multiplier {} {
    while 1 {
        relay accept {deliver args}
        {*}$deliver [::tcl::mathop::* {*}$args]
    }
}

after 0 [list coroutine main apply [list {argv0 argv} {
    coroutine a1 adder
    coroutine m1 multiplier
    set count 10
    set tasks {}
    for {set num1 0; set num2 100} {$num1 < $count} {incr num1; incr num2} {
        dict set tasks add [relay order 0 a1 $num1 $num2] {}
        dict set tasks mult [relay order 0 m1 $num1 $num2] {}
    }
    while {[llength [dict keys [dict get $tasks add]]]
        || [llength [dict keys [dict get $tasks mult]]]} {
        set received [relay receive]
        foreach type {add mult} {
            if {[dict exists $tasks $type [relay last]]} {
                dict unset tasks $type [relay last]
                lappend ${type}_results $received
            }
        }
    }
    rename a1 {}
    rename m1 {}
    puts [list {add results} $add_results]
    puts [list {mult results} $mult_results]
    exit 0

} [namespace current]] $argv0 $argv]


vwait forever

result:

{add results} {100 102 104 106 108 110 112 114 116 118}
{mult results} {0 101 204 309 416 525 636 749 864 981}

APN: How compatible is this system with the coroutine package in tcllib ? In particular, can one of these cooperating coroutines call commands from that package such as coroutine::util gets ? Or would there be a conflict between "competing" yields?

PYK: coroutine::util::gets is blocking, and ycl coro relay receive is also blocking. Both commands yield, and then expect their coroutine context to be entered again when a response is prepared. Both commands can be used in the same coroutine context, but after calling ycl coro relay order or ycl coro relay call, the script should consider itself a callback script for the answer, and should only yield for that purpose. The same is true for coroutine::util::gets. It dominates the coroutine context, looping around yield until it gets what it wants. If something else happens to be calling the coroutine command for the same context in the meantime, that's probably a bug.

APN I don't see it as a bug. I see it as an incompatibility between the above framework and coroutine packages like the one in tcllib. Consider a "task" written in terms of your framework above which receives and responds to orders. This task makes use of a brand new coroutine based http client library that happens to make use of the tcllib coro gets to read from the socket. When the gets yields, some other task happens to send an order to the coroutine which now "wakes up" expecting the return value of gets but gets the accept arguments instead. Does chaos not ensue? I would not think it would be feasible to require knowing which library or package calls that a "task" makes might yield in this fashion. One possible solution, which I currently use in my fiber package, is to use an explicit queue which includes a flag that is set when the "task" makes a receive call. A message (order in your case) is not dispatched to the task unless this flag is set so the case I describe above is taken care of. Even then, I'm unsure it covers all cases of such conflict.

PYK 2016-04-26: That wouldn't really happen with this system unless the programmer "crossed the streams." since there's a one-to-one corespondance between a coro relay send and a coro relay receive, all a programmer needs to do is remain aware of what code ends up between those two statements. As long as a call to coroutine::util::gets doesn't get in the middle, it will work as expected, and both can be used in the same coroutine context. But ycl coro relay switch, or something similar can be used for the situations where the coroutine expects to be entered by various third parties. It's up to the programmer to device some mechanism for identifying the caller, but ycl coro relay switch strongly hints at how one might go about doing that. One of the design goals of ycl coro relay was to not have any explicit queues, and so far, I'm proud to have succeeded on that point. I'm hoping nothing comes up that requires their introduction.

coroutine