Abstractly, a continuation is a representation of the state of an execution, and
especially its "execution stack" 
[http://en.wikipedia.org/wiki/Continuation].

There's a delightful article explaining continuations at [http://www.intertwingly.net/blog/2005/04/13/Continuations-for-Curmudgeons].

''[DKF] 17-Apr-2005:'' Interesting. I just wish it was easy to apply in [Tcl], but our openness to extension, especially by [C] libraries, makes this very tricky indeed.

''[WHD] 17-Apr-2005:'' Damn.  I want this.  The idea of User Interface Continuations makes me drool.

''[jcw] - Yeah, me too, same reaction.
DKF, yes the C stack can cause trouble, but only when C calls back into Tcl.  There are probably many cases when this need not be a show stopper.
I think there are ways to get there.  Might be something to discuss at the tcl2005e conf - are you going to be there?''

''[NEM] 18-Apr-2005:'' You could sort of get this to work, if we went a similar route to Stackless [Python] (from what little I know of that project [http://stackless.com]). You'd have to change all the C-API so that instead of re-calling into the interpreter with Tcl_Eval* APIs, they instead return a TCL_CALL or similar code with some code of where to go next in the interpreter result (i.e. [trampoline]). This is very similar to the recent proposals for tail-call optimisation (which is also very connected to continuations, through continuation-passing style, CPS). Obviously, it'd be an enormous amount of work to restructure the core and many many C extensions to do things this way. Continuations rock. (I should be at tcl2005e, and would love to discuss these ideas with people).

''The core is not that far from doing it.  At one point I had an experimental patch working, which copied the C stack away and back.  Not sure it covered all the cases, but it was definitely very close to full contins across all of Tcl and C. -[jcw]''

''[WHD]:'' I don't see why the C stack is an issue, based on what the article says.  A [closure] consists of a stack frame and its parent stack frame, and so on recursively, it says; but a continuation, so far as I can tell, is a single stack frame.  Suppose we made the rule that you could "yield" only from the body of a normal proc....  '''Oh.'''  If you do it from within a while loop, say, you've got the C stack involved.  '''Never mind.'''  (Or is there something else I'm missing?)

''Will, the stack becomes a tree, because processing (and further nesting) can proceed at any continuation point.  Yes, C loops and C re-entering Tcl (as [foreach] and [proc] do now when evaluating their bodies) is where things get tricky. -[jcw]''

'''[RHS]''' ''17Apr2005'' One of the most interesting (and informative) descriptions I've seen of the use of continuations was in the area of web programming. The programmers used them to use web pages as if they were stateful. The "process" that the web pages are involved with work linearly. Each time it needs input from the user it saves its current continuation (which contains everything about where it is in the process) in a hashtable. It then returns a web page with a form to the user, and one of the elements of the form (a hidden one, I presume) contains the key into the hash table for the current continuation. Once the user fills in the form and hits submit, the process can look up the continuation in the hash table and continue right where it left off... Only, now, it has the data it needs to continue, as provided by the form.

'''JR''' ''18Apr2005'' What I think would be a nifty use for continuations would be to unnest [vwait].  As everyone knows, if you call [vwait] from inside an event handler then the outer [vwait] cannot return until the inner one does.  This gets messy when things that you may not expect call [vwait] internally, with the general conclusion of '''vwait considered harmful'''.  However, it is also very useful in some circumstances which makes this all a pain.  If we had continuations, then vwait could simply be reimplemented as something like

   proc vwait {var} {
      uplevel trace variable $var w [info continuation 1]
      return -code return
   }

Where '''info continuation''' ''level'' returns a command that executes the continuation.  Then make the stack frames into Tcl_Objs and increment the reference counts to keep them around and it all works naturally.  ''(if only it were that simple)''

'''[SS]''' ''18Apr2005''' Using setcontext and getcontext calls that are POSIX and AFAIK also available in some form on every platform Tcl supports allows for
continuations without a stackless interpreter (basically you save the C stack too). This is how it's in plan to implement continuations into [Jim].
(this is also how [Ruby] implements continuations IIRC).

[jcw] - Setcontext/getcontext (SC/GC) work by switching stacks, which is not 100% as general as being able to resume any stack frame in what can become a tree of stack frames in the general case.  There's also the issue of whether lower stack frames are treated as copy-on-write or not, i.e. whether changes are isolated, when these frames are shared by multiple continuations.

'''[SS]''' Sure [jcw], what SC/GC does is to make you able to implement continuations without to write a stackless interpreter. Of course the Tcl stackframes
are to be copied (in the trivial implementation at least) but that's not a too big problem as it is as simple as to copy some hashtable and linked data
structure. The real problem is to write a stackless interpreter and given that it's a major work and that there are other systems to reach the goal why don't
use them.

'''[chi] 19Apr2005:''' I mean to remember, that Ruby does its continuation and thread thingy by simply copying the whole C stack from some point remembered during initialization of the interpreter up to the context of the calling C function. Switching back to that continuation simply copies back that remembered stack. So it works within C functions. Unfortunately setcontext und getcontext seem not to be widely available :-( Perhaps the same implementation as in Lua could be facilitated? In Lua, C context will not be remembered, AFAIK. So a C function has to come back. But a Lua function could be suspended. This restriction seems not to be too hurting, would it?

'''[SS] 19Apr2005:''' Indeed the Ruby trick may work well on many platforms where there is no support for SC/GC, about [Lua], maybe the ability to suspend
a function allows for coroutine and generators? Btw given that there are ways to copy the whole C stack why not :) In Tcl the ability to reactivate a single
procedure may be more complex than what happens in other languages because there is [upvar]/[uplevel] that may notice the difference. Not that this is a big problem
after all... the user can just avoid them. Well an interesting alternative btw. Oh.. well also there is another problem with the Lua approach, in Tcl the
interpreter calls itself recursively even in the context of a single procedure! While in languages like Lua it only happens when a function calls another one
I guess.

'''[chi] 19Apr2005:''' That Tcl interpreter calls itself recursively should not matter at all. We would being restoring the whole C stack. So the interpreter would itself find back in the C function it was called to save the context. I had once written an extension for another language that worked pretty well. Then I had simplified the extension to make a C module out of it so that it could be used even in plain C programms. But to use such an extension in Tcl or Jim, we would need a mechanism to store the Tcl/Jim stack as well to protect it against freeing by refcount dropping to zero, though. And we would have to restore the Tcl/Jim stack as well. Further I would not like to keep secret, that there is some performance penalty taking this approach for implementing continuations, as the whole C stack has to be copied over and over again. But it should be doable.

'''[SS]''' Agreed. What I mean is that "without to copy the C stack" some language is still able to do generators (ala Python), just activating the same function
every time it's needed. This is sometimes possible in languages that have recursive calls to the interpreter only when functions are called, and not for every
thing like [if], [while] and so on like Tcl. Btw this is poor anyway, you can't even write recursive generators. Copying the C stack, there is no problem at all
of course. Also to copy the stack frames should not be hard, for instance in Jim there is just a linked list of stackframes containing an hashtable
of local variables. There is just to duplicate the list and hash tables just incr-ref-counting every object representing a variable, this should be pretty
easy but as you said is very slow, you don't only need to copy the C stack but the whole Tcl stack. Still I think it's nice enough as a first try just
to experiment with it. After all in Jim at least I create an hash table with local vars at every function call... and still it's not as slow as you may
think, so maybe even to copy the C stack and the stackframes is viable. A bit slow... bust still useful, and requires very little work. So maybe we
should try ;) In Jim there are tailcalls, anonymous functions, closures. It seems like that the other big missing abstraction is continuations.

[DKF]: I get very uneasy about tinkering around with the C stack. I know that it is definitely asking for trouble in Tcl's implementation as there are a number of places where we use pointers into the stack, which means that copying will not work. In theory, we could arrange for the byte-compiling of all core commands (we're actually pretty close) but that would massively raise the bar for anyone wanting to produce a third-party block-structured command in C, and there are lots of those about. Possible? Perhaps. Easy? No. Cheap to execute? Very unlikely (the stack can be multiple megabytes long). Worth it? You be the judge...

[SS]: It always works even if you have pointers on the stack. You ''save'' it, don't ''use'' it while it's saved. When you restore it, it gets copied at the
same place it used to be. About complexity, I think I can implement this for [Jim] as an extension without to touch the core at all.

'''[chi] 19Apr2005:''' If you do, perhaps you should add refcounting to the Jim stack as well? Then it would be unnecessary to copy it. Simply IncrRefCount'ing every interesting element and then the Jim stack as well. The continuation could store a pointer to that shared stack then. Should be a bit cheaper then, no?

[SS]: Yes, I think my first try will be very simple, copying everything, just to avoid complexity that may make debugging of the C-level continuations harder,
for the next step I was thinking about to add the refcount to individual stackframes instead to add it to the whole stack, this way a continuation is cheap to
create and even if there is some copy-on-write occurring it's likely to interest only a minor part of the stack (few stackframes) in many uses.
Btw if to copy the whole stack is reasonably fast I'll be more happy to have slow continuations as an extension that does not make the core more complex,
so after I've a working "vanilla" version there will be to do some benchmarking to check if it's acceptable or too slow to be used.

----
[Category Concept]