ycl coro relay is a small but complete system that provides order/receive facilities among asynchronous coroutines. Features include timeouts, cancellation, error propagation, and scatter/gather capabilities.
#! /bin/env tclsh package require {ycl coro relay} namespace import [yclprefix]::coro::relay proc adder {} { while 1 { relay accept {deliver args} {*}$deliver [::tcl::mathop::+ {*}$args] } } proc multiplier {} { while 1 { relay accept {deliver args} {*}$deliver [::tcl::mathop::* {*}$args] } } after 0 [list coroutine main apply [list {argv0 argv} { coroutine a1 adder coroutine m1 multiplier set count 10 set tasks {} for {set num1 0; set num2 100} {$num1 < $count} {incr num1; incr num2} { dict set tasks add [relay order 0 a1 $num1 $num2] {} dict set tasks mult [relay order 0 m1 $num1 $num2] {} } while {[llength [dict keys [dict get $tasks add]]] || [llength [dict keys [dict get $tasks mult]]]} { set received [relay receive] foreach type {add mult} { if {[dict exists $tasks $type [relay last]]} { dict unset tasks $type [relay last] lappend ${type}_results $received } } } rename a1 {} rename m1 {} puts [list {add results} $add_results] puts [list {mult results} $mult_results] exit 0 } [namespace current]] $argv0 $argv] vwait forever
result:
{add results} {100 102 104 106 108 110 112 114 116 118} {mult results} {0 101 204 309 416 525 636 749 864 981}
APN: How compatible is this system with the coroutine package in tcllib ? In particular, can one of these cooperating coroutines call commands from that package such as coroutine::util gets ? Or would there be a conflict between "competing" yields?
PYK: coroutine::util::gets is blocking, and ycl coro relay receive is also blocking. Both commands yield, and then expect their coroutine context to be entered again when a response is prepared. Both commands can be used in the same coroutine context, but after calling ycl coro relay order or ycl coro relay call, the script should consider itself a callback script for the answer, and should only yield for that purpose. The same is true for coroutine::util::gets. It dominates the coroutine context, looping around yield until it gets what it wants. If something else happens to be calling the coroutine command for the same context in the meantime, that's probably a bug.
APN I don't see it as a bug. I see it as an incompatibility between the above framework and coroutine packages like the one in tcllib. Consider a "task" written in terms of your framework above which receives and responds to orders. This task makes use of a brand new coroutine based http client library that happens to make use of the tcllib coro gets to read from the socket. When the gets yields, some other task happens to send an order to the coroutine which now "wakes up" expecting the return value of gets but gets the accept arguments instead. Does chaos not ensue? I would not think it would be feasible to require knowing which library or package calls that a "task" makes might yield in this fashion. One possible solution, which I currently use in my fiber package, is to use an explicit queue which includes a flag that is set when the "task" makes a receive call. A message (order in your case) is not dispatched to the task unless this flag is set so the case I describe above is taken care of. Even then, I'm unsure it covers all cases of such conflict.
PYK 2016-04-26: That wouldn't really happen with this system unless the programmer "crossed the streams." since there's a one-to-one corespondance between a coro relay send and a coro relay receive, all a programmer needs to do is remain aware of what code ends up between those two statements. As long as a call to coroutine::util::gets doesn't get in the middle, it will work as expected, and both can be used in the same coroutine context. But ycl coro relay switch, or something similar can be used for the situations where the coroutine expects to be entered by various third parties. It's up to the programmer to device some mechanism for identifying the caller, but ycl coro relay switch strongly hints at how one might go about doing that. One of the design goals of ycl coro relay was to not have any explicit queues, and so far, I'm proud to have succeeded on that point. I'm hoping nothing comes up that requires their introduction.