Error processing request

Parameters

CONTENT_LENGTH0
REQUEST_METHODGET
REQUEST_URI/revision/Coroutines+for+event-based+programming?V=15
QUERY_STRINGV=15
CONTENT_TYPE
DOCUMENT_URI/revision/Coroutines+for+event-based+programming
DOCUMENT_ROOT/var/www/nikit/nikit/nginx/../docroot
SCGI1
SERVER_PROTOCOLHTTP/1.1
HTTPSon
REMOTE_ADDR172.70.134.247
REMOTE_PORT51360
SERVER_PORT4443
SERVER_NAMEwiki.tcl-lang.org
HTTP_HOSTwiki.tcl-lang.org
HTTP_CONNECTIONKeep-Alive
HTTP_ACCEPT_ENCODINGgzip
HTTP_CF_IPCOUNTRYUS
HTTP_X_FORWARDED_FOR52.23.219.12
HTTP_CF_RAY6ba2473eeede7fa6-IAD
HTTP_X_FORWARDED_PROTOhttps
HTTP_CF_VISITOR{"scheme":"https"}
HTTP_USER_AGENTCCBot/2.0 (https://commoncrawl.org/faq/)
HTTP_ACCEPTtext/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
HTTP_ACCEPT_LANGUAGEen-US,en;q=0.5
HTTP_CF_CONNECTING_IP52.23.219.12
HTTP_CDN_LOOPcloudflare

Body


Error

Unknow state transition: LINE -> END

-code

1

-level

0

-errorstack

INNER {returnImm {Unknow state transition: LINE -> END} {}} CALL {my render_wikit {Coroutines for event-based programming} {[NEM] 2008-08-26: In [A brief introduction to closures and continuations for event-based programming] I described
a simple programming task which is very simple to write when interacting with a console or other synchronous device,
but which becomes trickier to implement in an event-oriented manner. The difficulty is that we not only have to
change the particular implementation of our software, but we also have to restructure the flow of control of the entire
program to accommodate an event-based approach. I then discussed how various programming features, such as
[closures] (which we (can) have), and [continuations] (which we don't) can help to improve this situation. In this article,
I want to show how the experimental [coroutine] facilities provided in the 8.6 alphas can be used to address the same
task. Hopefully this demonstrates a powerful example of what these facilities can achieve, whilst focussing attention on
those areas that might need more thought.

As a recap, the original problem is to write a program that asks the user for two numbers and then displays their sum.
The "spec" for the program is represented by this simple main routine, written in ''direct style'':
======
proc main {} {
    set x [ask "First number:"]
    set y [ask "Second number:"]
    tell "Sum = [expr {$x + $y}]"
}
======
The task then is to implement suitable ''tell'' and ''ask'' procedures, ideally without having to alter the main routine at all.
For a simple console application, this is straight-forward:
======
proc ask question {
    puts -nonewline "$question "
    flush stdout
    gets stdin
}
proc tell message { puts $message }
main ;# Works as expected
======
When we move to a [Tk] [GUI] implementation, however, things start to fall apart. The trouble is that Tk GUIs just don't
work in this direct style; they work in an event-based style. In the above linked page, I show how to adapt the program
to this new style, and then how to slowly move back to something resembling the simple direct program. I won't recap
that here. I did however, hint at how [continuations] can be used to completely recapture the direct style program. In this
page, I want to show how a similar result can be achieved using the coroutines that [MS] has provided for us. Indeed,
the kind of coroutines implemented are roughly equivalent to a kind of one-shot continuation. Indeed, with the facilities
in the latest CVS HEAD version of Tcl, our main can be ''exactly'' the same as the simple version above (see note below for making it run in Tcl8.6a2). All that needs to
change is our implementation of ''tell'' and ''ask'' and a simple change to the way main is called. This is something of
a holy grail of event-oriented programming, and says a lot for the power of the coroutine mechanism. Here's the new
procedure implementations:
======
namespace path ::tcl::unsupported
proc ask q {
    toplevel .ask
    pack [label .ask.l -text $q] [entry .ask.e]
    raise .ask; focus .ask.e
    bind .ask.e <Return> [format {
	set ans [.ask.e get]
	destroy .ask
	%s $ans
    } [infoCoroutine]]
    yield
}
proc tell msg { tk_messageBox -message $msg }
======
These are essentially the same as the callback versions from the previous page, but with a mysterious [yield] at the end of
the ask procedure. To tie this all together, we implement a ''spawn'' command to launch the initial ''main'' coroutine:
======
proc spawn cmd {
    set k [gensym]
    coroutine $k $cmd
}
proc gensym {{prefix "::coroutine"}} {
    variable gensymid
    return $prefix[incr gensymid]
}
======
Finally, we can call our main:
======
spawn main
======
If you run this, you'll see that it does in fact work.

So what's actually going on here? Well, the trick is to make the main routine itself into a co-routine, so that it can be
interrupted and resumed as events occur. The spawn procedure simply creates a unique coroutine that calls the main
proc. The ask procedure
then creates the GUI and uses this coroutine argument as the callback to be called when the user enters data. It then
calls [yield] cause the ''main'' computation to suspend (and thus yielding back to the event loop). The event GUI then
carries on until the user enters a number and hits ''Return''. At this point, the coroutine is invoked, resuming the main
computation and returning the result as the result of the yield. This then becomes the result of the ''ask'' procedure as
if it had worked exactly like our original synchronous version. Voila!

Hopefully this is comprehensible. So, what conclusions/issues can we draw from this?
   * The proposed [coroutine] mechanism is very powerful, and meshes well with existing Tcl/Tk idioms.
   * In particular, this is a promising approach to taming [event-based programming].
   * There is the need to generate unique names for the coroutines (not a massive burden).
   * We need a spawn to convert the main into a coroutine (again, not a massive burden).

Overall, an extremely interesting capability.
----
[MS] (2008-08-28) notes that [[infoCoroutine]] did not make to Tcl8.6a2, this code requires cvs HEAD to run. With a small modification to the helper procs it can be run in Tcl8.6a2 too:

======
 proc infoCoroutine {} {uplevel #1 {set name}}
 proc spawn cmd {
    set k [gensym]
    coroutine $k ::apply [list name $cmd] $k
 }
======
This exploits the fact that within a running coroutine #0 is as usual, #1 gives access to the coroutine's first level: in this case, to the local scope of the coroutine's defining lambda.
----
[NEM] Discussion welcome.

[MS] your wish is my command: HEAD now has '''tcl::unsupported::infoCoroutine''' that returns the fully qualified name of the currently executing coroutine. With this I think that the pesky $k can go. Too bad it didn't make it in time for Tcl8.6a2.

[NEM] Updated above to make use of the new facilities. Here is also a little [ensemble] wrapper around the unsupported
facilities:
======
# coroutine.tcl --
#
#       Ensemble wrapper around the coroutine facilities in Tcl 8.6
#
package require Tcl	 8.6a2
package provide coroutine   1.0

namespace eval ::coroutine {
    namespace export create spawn yield current
    namespace ensemble create -map {
	create      ::tcl::unsupported::coroutine
	spawn       ::coroutine::spawn
	yield       ::tcl::unsupported::yield
	current     ::tcl::unsupported::infoCoroutine
    }
    variable gensymid 0
    
    # wrapper around [coroutine create] that generates a unique name.
    proc spawn {cmd args} {
	set id [gensym]
	coroutine create $id $cmd {*}$args
    }
    proc gensym {{prefix "coroutine"}} {
	variable gensymid
	return [namespace current]::$prefix[incr gensymid]
    }    
}
======
[CMcC] 2008Aug27 - I just had a cool idea for uniquely naming coroutines for [fileevent] handling - name the coroutines after the file handle.  Those should be unique.

[NEM] Indeed. The really good thing about these coroutines is that they automatically clean up once the coroutine is "exhausted". Something like
the ''coroutine spawn'' I provide above is really pretty much good enough already: the commands are created in a separate namespace, so should be
free of clashes, and they are automatically deleted when no longer needed, so there is no garbage collection issues. It really works out very well.

[CMcC] one [fileevent]s problem is that a [socket] server can generate file handles faster than you can clean up after a file has closed.  This means that all your handlers can be reading/writing a socket which has the same file handle name as that they were created for, but which is actually a different connection.

If one uses the coroutine abstraction, and names [fileevent] handling coroutines after the file they are dedicated to, one will get an error at the point of coro creation, indicating that there's still a handler which could conceivably be trying to manipulate the socket which has now disappeared.  The timely, simple detection of this, at a time when one has the new socket's name, and hence the handler's name, may make it possible to neatly avoid this problem.

Further reflection leads me to conclude that there's nothing inherent in tcl causing this race condition, that good socket hygeine would dictate that you fully process the closure of a socket before actually closing the socket handler.  However, the naming convention suggested defensively detects the race condition however caused.

[ferrieux] Yes, the EOF condition doesn't instantly destroy the socket handle. Instead, the script has to detect the condition, and possibly decide to take action like [close], at which point it is natural to get rid of any associated persistent things (like coroutines or array keys). So  there is no race condition to begin with ;-)

[NEM] Example of [CMcC]'s idea:
======
proc chaniter chan {     
  coroutine $chan ::apply {chan {
    yield ;# start-up
    while {[gets $chan line] >= 0} { yield $line }
    close $chan
    return -code break
  }} $chan
  return $chan
}
set in [chaniter [open $somefile]]
while 1 { puts [$in] }
======
[Jaf] (20080903) Am I being overly thick here, or why am I still not "getting it"? 

I Just can't help thinking that a trace on the first variable (x) and one on the second (y), with the trace calling a proc  yielding the result would do the job, just as effectively (??).     
======
----
!!!!!!
%| [Category Discussion] | [Category Example] |%
!!!!!!} regexp2} CALL {my render {Coroutines for event-based programming} {[NEM] 2008-08-26: In [A brief introduction to closures and continuations for event-based programming] I described
a simple programming task which is very simple to write when interacting with a console or other synchronous device,
but which becomes trickier to implement in an event-oriented manner. The difficulty is that we not only have to
change the particular implementation of our software, but we also have to restructure the flow of control of the entire
program to accommodate an event-based approach. I then discussed how various programming features, such as
[closures] (which we (can) have), and [continuations] (which we don't) can help to improve this situation. In this article,
I want to show how the experimental [coroutine] facilities provided in the 8.6 alphas can be used to address the same
task. Hopefully this demonstrates a powerful example of what these facilities can achieve, whilst focussing attention on
those areas that might need more thought.

As a recap, the original problem is to write a program that asks the user for two numbers and then displays their sum.
The "spec" for the program is represented by this simple main routine, written in ''direct style'':
======
proc main {} {
    set x [ask "First number:"]
    set y [ask "Second number:"]
    tell "Sum = [expr {$x + $y}]"
}
======
The task then is to implement suitable ''tell'' and ''ask'' procedures, ideally without having to alter the main routine at all.
For a simple console application, this is straight-forward:
======
proc ask question {
    puts -nonewline "$question "
    flush stdout
    gets stdin
}
proc tell message { puts $message }
main ;# Works as expected
======
When we move to a [Tk] [GUI] implementation, however, things start to fall apart. The trouble is that Tk GUIs just don't
work in this direct style; they work in an event-based style. In the above linked page, I show how to adapt the program
to this new style, and then how to slowly move back to something resembling the simple direct program. I won't recap
that here. I did however, hint at how [continuations] can be used to completely recapture the direct style program. In this
page, I want to show how a similar result can be achieved using the coroutines that [MS] has provided for us. Indeed,
the kind of coroutines implemented are roughly equivalent to a kind of one-shot continuation. Indeed, with the facilities
in the latest CVS HEAD version of Tcl, our main can be ''exactly'' the same as the simple version above (see note below for making it run in Tcl8.6a2). All that needs to
change is our implementation of ''tell'' and ''ask'' and a simple change to the way main is called. This is something of
a holy grail of event-oriented programming, and says a lot for the power of the coroutine mechanism. Here's the new
procedure implementations:
======
namespace path ::tcl::unsupported
proc ask q {
    toplevel .ask
    pack [label .ask.l -text $q] [entry .ask.e]
    raise .ask; focus .ask.e
    bind .ask.e <Return> [format {
	set ans [.ask.e get]
	destroy .ask
	%s $ans
    } [infoCoroutine]]
    yield
}
proc tell msg { tk_messageBox -message $msg }
======
These are essentially the same as the callback versions from the previous page, but with a mysterious [yield] at the end of
the ask procedure. To tie this all together, we implement a ''spawn'' command to launch the initial ''main'' coroutine:
======
proc spawn cmd {
    set k [gensym]
    coroutine $k $cmd
}
proc gensym {{prefix "::coroutine"}} {
    variable gensymid
    return $prefix[incr gensymid]
}
======
Finally, we can call our main:
======
spawn main
======
If you run this, you'll see that it does in fact work.

So what's actually going on here? Well, the trick is to make the main routine itself into a co-routine, so that it can be
interrupted and resumed as events occur. The spawn procedure simply creates a unique coroutine that calls the main
proc. The ask procedure
then creates the GUI and uses this coroutine argument as the callback to be called when the user enters data. It then
calls [yield] cause the ''main'' computation to suspend (and thus yielding back to the event loop). The event GUI then
carries on until the user enters a number and hits ''Return''. At this point, the coroutine is invoked, resuming the main
computation and returning the result as the result of the yield. This then becomes the result of the ''ask'' procedure as
if it had worked exactly like our original synchronous version. Voila!

Hopefully this is comprehensible. So, what conclusions/issues can we draw from this?
   * The proposed [coroutine] mechanism is very powerful, and meshes well with existing Tcl/Tk idioms.
   * In particular, this is a promising approach to taming [event-based programming].
   * There is the need to generate unique names for the coroutines (not a massive burden).
   * We need a spawn to convert the main into a coroutine (again, not a massive burden).

Overall, an extremely interesting capability.
----
[MS] (2008-08-28) notes that [[infoCoroutine]] did not make to Tcl8.6a2, this code requires cvs HEAD to run. With a small modification to the helper procs it can be run in Tcl8.6a2 too:

======
 proc infoCoroutine {} {uplevel #1 {set name}}
 proc spawn cmd {
    set k [gensym]
    coroutine $k ::apply [list name $cmd] $k
 }
======
This exploits the fact that within a running coroutine #0 is as usual, #1 gives access to the coroutine's first level: in this case, to the local scope of the coroutine's defining lambda.
----
[NEM] Discussion welcome.

[MS] your wish is my command: HEAD now has '''tcl::unsupported::infoCoroutine''' that returns the fully qualified name of the currently executing coroutine. With this I think that the pesky $k can go. Too bad it didn't make it in time for Tcl8.6a2.

[NEM] Updated above to make use of the new facilities. Here is also a little [ensemble] wrapper around the unsupported
facilities:
======
# coroutine.tcl --
#
#       Ensemble wrapper around the coroutine facilities in Tcl 8.6
#
package require Tcl	 8.6a2
package provide coroutine   1.0

namespace eval ::coroutine {
    namespace export create spawn yield current
    namespace ensemble create -map {
	create      ::tcl::unsupported::coroutine
	spawn       ::coroutine::spawn
	yield       ::tcl::unsupported::yield
	current     ::tcl::unsupported::infoCoroutine
    }
    variable gensymid 0
    
    # wrapper around [coroutine create] that generates a unique name.
    proc spawn {cmd args} {
	set id [gensym]
	coroutine create $id $cmd {*}$args
    }
    proc gensym {{prefix "coroutine"}} {
	variable gensymid
	return [namespace current]::$prefix[incr gensymid]
    }    
}
======
[CMcC] 2008Aug27 - I just had a cool idea for uniquely naming coroutines for [fileevent] handling - name the coroutines after the file handle.  Those should be unique.

[NEM] Indeed. The really good thing about these coroutines is that they automatically clean up once the coroutine is "exhausted". Something like
the ''coroutine spawn'' I provide above is really pretty much good enough already: the commands are created in a separate namespace, so should be
free of clashes, and they are automatically deleted when no longer needed, so there is no garbage collection issues. It really works out very well.

[CMcC] one [fileevent]s problem is that a [socket] server can generate file handles faster than you can clean up after a file has closed.  This means that all your handlers can be reading/writing a socket which has the same file handle name as that they were created for, but which is actually a different connection.

If one uses the coroutine abstraction, and names [fileevent] handling coroutines after the file they are dedicated to, one will get an error at the point of coro creation, indicating that there's still a handler which could conceivably be trying to manipulate the socket which has now disappeared.  The timely, simple detection of this, at a time when one has the new socket's name, and hence the handler's name, may make it possible to neatly avoid this problem.

Further reflection leads me to conclude that there's nothing inherent in tcl causing this race condition, that good socket hygeine would dictate that you fully process the closure of a socket before actually closing the socket handler.  However, the naming convention suggested defensively detects the race condition however caused.

[ferrieux] Yes, the EOF condition doesn't instantly destroy the socket handle. Instead, the script has to detect the condition, and possibly decide to take action like [close], at which point it is natural to get rid of any associated persistent things (like coroutines or array keys). So  there is no race condition to begin with ;-)

[NEM] Example of [CMcC]'s idea:
======
proc chaniter chan {     
  coroutine $chan ::apply {chan {
    yield ;# start-up
    while {[gets $chan line] >= 0} { yield $line }
    close $chan
    return -code break
  }} $chan
  return $chan
}
set in [chaniter [open $somefile]]
while 1 { puts [$in] }
======
[Jaf] (20080903) Am I being overly thick here, or why am I still not "getting it"? 

I Just can't help thinking that a trace on the first variable (x) and one on the second (y), with the trace calling a proc  yielding the result would do the job, just as effectively (??).     
======
----
!!!!!!
%| [Category Discussion] | [Category Example] |%
!!!!!!}} CALL {my revision {Coroutines for event-based programming}} CALL {::oo::Obj38281987 process revision/Coroutines+for+event-based+programming} CALL {::oo::Obj38281985 process}

-errorcode

NONE

-errorinfo

Unknow state transition: LINE -> END
    while executing
"error $msg"
    (class "::Wiki" method "render_wikit" line 6)
    invoked from within
"my render_$default_markup $N $C $mkup_rendering_engine"
    (class "::Wiki" method "render" line 8)
    invoked from within
"my render $name $C"
    (class "::Wiki" method "revision" line 31)
    invoked from within
"my revision $page"
    (class "::Wiki" method "process" line 51)
    invoked from within
"$server process [string trim $uri /]"

-errorline

4