[iu2] 2007-02-01 Inspired by [What kinds of variable names can be used in Tcl] and [Looking at LISP's SERIES extension] I tried to write a generator with Python's syntax.

The generator creates values '''upon demand'''. For example, given (in python)

 >>> def inc1():
	c = 0
	while 1:
		c += 1
		yield c

		
 >>> gen = inc1()
 >>> gen.next()
 1
 >>> gen.next()
 2
 >>> gen.next()
 3
 >>> gen.next()
 4
 >>> 
 >>> 

More information can be found in [generators] and Python's doc at [http://docs.python.org/tut/node11.html#SECTION00111000000000000000000]
and the itertools library [http://docs.python.org/lib/module-itertools.html]

The syntax is simple and powerful. The mechanism involves saving the context after the '''yield''' command.
Saving context in tcl? --- using threads....

So it occured to me that if I use threads I can immitate Python's generator. This is the method: I write a generator proc just like in Python, using '''yield'''. Then I call a special function '''generate''' that converts my proc to a functioning generator by launching a thread, and establishing communication with the calling thread upon '''yield''' occurances. 

This is the '''generate''' proc
  proc generate {var proc args} {
    # creating a thread
    uplevel #0 set thread$var [thread::create {thread::wait}]
    upvar #0 thread$var th
    thread::send $th "set th [thread::id]" ;# notifying about caller thread
    tsv::set gen $var 0
    uplevel #0 set wait$var 0
    upvar #0 wait$var wait
    
    # change the desired proc so that 'yield' causes a generation of a value and a pause
    set b [info body $proc]
    regsub -all -linestop {\yyield\y\s+(.+)} [info body $proc] [format {tsv::set gen %s [subst \1]; thread::send -async $::th {set ::%s 1}; vwait sleep} $var wait$var] b
    append b "\nthread::send -async \$::th {set ::wait$var 2}"
    thread::send $th [list proc $proc [info args $proc] $b]
    
    # create a proc for making the generator advance - communication with the thread
    proc $var {} [format {
      if {$::%s == 0} {thread::send -async %s "%s %s"} else {
      thread::send -async %s {set sleep 1}}
      vwait ::%s
      if {$::%s == 2} {error "Generator exausted"}
      return [tsv::get gen %s]} wait$var $th $proc $args $th wait$var wait$var $var]
  }

An example of a generator function:
  proc list3 {a b c} {
    for {set x 0} {$x < $a} {incr x} {
      for {set y 0} {$y < $b} {incr y} {
        for {set z 0} {$z < $c} {incr z} {
          yield [list $x $y $z]
        }
      }
    }
  }

Now I convert list3 to a generator
 generate next list3 2 2 3

And here is the usage and result for '''next'''
  for {set i 0} {$i < 20} {incr i} {
    if {[catch {
      puts [next]
    }]} break
  }

result
 0 0 0
 0 0 1
 0 0 2
 0 1 0
 0 1 1
 0 1 2
 1 0 0
 1 0 1
 1 0 2
 1 1 0
 1 1 1
 1 1 2

Another example
  proc plus1 {} {
    set i 0
    while 1 {
      yield [incr i]
    }
  }

And the usage:
  generate p1 plus1
  while 1 {
    gets stdin s
    catch {eval $s}
    puts [p1]
  }
The ''catch {eval $s}'' is to be able to write ''exit'' and leave the program.

One more:
  proc manyyields {} {
    yield 1
    yield {Hello world!}
    yield {1 2 3 4 5}
  }

  generate my manyyields
  while 1 {
    puts [my]
  }
And the result
 1
 Hello world!
 1 2 3 4 5 
 Generator exausted
    while executing
 "error "Generator exausted""
    (procedure "my" line 5)
    invoked from within
 "my"
    ("while" body line 2)
    invoked from within
 "while 1 {
   puts [my]
 }"

----

[schlenk] Why not use simple slave interpreters to save the context (even namespaces or dynamically rewritten procs with default arguments might be enough)? No need to involve threads.