MJ - YATS (Yet Another Tuplespace) that supports:
Source code at chiselapp
You can download a package containing the server and the client from [L1 ].
Missing functionality
Possible improvements
tuplespace contains tuid
extra table contains:
tuid field value id 0 <number of elements N> id 1 <value of 1st elelemt> . . id N <value of Nth element>
MJ - This is not really useful because selecting matching tuples is non-trivial in a DB design like this.
MJ - Thanks to Aku for suggesting for {* * * 4}:
select id where field==0 and value==4 intersect select where field==4 and value like 'a';
Tupleserver commands
The following commands are supported by the server:
The server will respond with a 2 element list. The first element will contain the status of the command (ok or error). The second element will be the actual result.
Code for the Tupleserver
# server.tcl -
#
# Copyright (c) 2008 Mark Janssen
#
# A Tcl implementation of a Linda Tuplespace
package require Tcl 8.5
package require snit 2.2
package require sqlite3 3.3
package provide tuplespace::server 0.1
namespace eval tuplespace {
snit::type server {
option -port 4040
option -db {}
variable dbcmd
variable commands "ain aread cb dir exit help in out ping purge read"
constructor {args} {
$self configurelist $args
socket -server [list $self Connection] $options(-port)
set dbcmd ${selfns}::db
sqlite3 $dbcmd $options(-db)
$dbcmd eval {CREATE TABLE IF NOT EXISTS tuples(id INTEGER PRIMARY KEY,space,value)}
$dbcmd eval {DROP TABLE IF EXISTS callbacks}
$dbcmd eval {CREATE TABLE callbacks(id INTEGER PRIMARY KEY,direction,space,pattern,socket)}
$dbcmd function tuple_match [list $self TupleMatch]
}
method Connection {socket host port} {
fconfigure $socket -buffering line -blocking no -encoding utf-8
fileevent $socket readable [list $self Request $socket {}]
}
method Result {socket status value} {
puts $socket [list $status $value]
}
method RemoveCallback {socket} {
$dbcmd eval {delete from callbacks where socket=$socket}
}
method AddCallback {socket direction tuplespace pattern} {
$dbcmd eval {INSERT INTO callbacks VALUES(NULL,$direction,$tuplespace,$pattern,$socket)}
}
method Request {socket data} {
if {[catch {set rc [gets $socket line]}] } return
if {[chan eof $socket]} {
chan close $socket
$self RemoveCallback $socket
return
}
if {$rc == -1} {
return
}
lappend data $line
if {![info complete [join $data \n]]} {
fileevent $socket readable [list $self Request $socket $data]
return
} else {
fileevent $socket readable [list $self Request $socket {}]
}
set data [join $data \n]
if {![string is list $data]} {
$self Result $socket error "request should be a valid Tcl list"
return
}
set cmd [lindex $data 0]
if {[lsearch $commands $cmd] < 0} {
$self Result $socket error "unknown server command \"$cmd\""
return
}
set internal_cmd [string totitle $cmd]
set params [lrange $data 1 end]
lassign [$self $internal_cmd $socket {*}$params] status result
if {$status eq {}} {
return
} else {
$self Result $socket $status $result
}
}
method TupleMatch {tuple pattern} {
if {[llength $tuple] ne [llength $pattern]} {
return 0
}
foreach tup $tuple pat $pattern {
if {![string match $pat $tup] && ![string match $tup $pat]} {
return 0
}
}
return 1
}
# server commands
method Ping {_ args } {
return [list ok $args]
}
method Exit {socket} {
chan close $socket
$self RemoveCallback $socket
return {}
}
method Help {_ args} {
variable commands
return [list ok $commands]
}
method Dir {_ args } {
set tuplespace [lindex $args 0]
set pattern [lindex $args 1]
switch -- [llength $args] {
0 {
return [list ok [$dbcmd eval {SELECT DISTINCT space FROM tuples}]]
}
1 {
set tuples [$dbcmd eval {SELECT value FROM tuples WHERE space GLOB $tuplespace}]
return [list ok $tuples]
}
2 {
if {![string is list $pattern]} {
return [list error "pattern should be a valid Tcl list"]
}
return [list ok [$dbcmd eval {SELECT value FROM tuples WHERE space GLOB $tuplespace AND tuple_match(value,$pattern)}]]
}
default {
return [list error "wrong # of args, should be \"dir ?tuplespace? ?pattern?\""]
}
}
}
method Purge {_ args } {
if {[llength $args] != 1} {
return [list error "wrong # of args, should be \"purge tuplespace\""]
}
set tuplespace [lindex $args 0]
$dbcmd eval {DELETE FROM tuples where space=$tuplespace}
return [list ok {}]
}
method Cb {_ args} {
set callbacks {}
$dbcmd eval {select * from callbacks} {
lappend callbacks [list $direction $space $pattern]
}
return [list ok $callbacks]
}
# server IO commands
method Out {_ args} {
if {[llength $args] != 2} {
return [list error "wrong # of args, should be \"out tuplespace tuple\""]
}
set space [lindex $args 0]
set tuple [lindex $args 1]
if {![string is list $tuple]} {
return [list error "tuple should be a valid list"]
}
# check for pending callbacks
$dbcmd eval {select id,direction,socket from callbacks where space=$space and tuple_match($tuple,pattern)} {
set err [catch {puts $socket [list ok $tuple]}]
$dbcmd eval {delete from callbacks where id=$id}
if {$direction eq "in" && !$err} {
# do not add the tuple when a in callback is registered and the tuple was succefully delivered
return [list ok {}]
}
}
$dbcmd eval {INSERT INTO tuples VALUES(NULL,$space,$tuple)}
return [list ok {}]
}
method Aread {socket args} {
$self Areq read $socket {*}$args
}
method Ain {socket args} {
$self Areq in $socket {*}$args
}
method Read {socket args} {
$self Req read $socket {*}$args
}
method In {socket args} {
$self Req in $socket {*}$args
}
method Areq {direction socket args} {
if {[llength $args] != 2} {
return [list error "wrong # of args, should be \"a$direction tuplespace pattern\""]
}
set tuplespace [lindex $args 0]
set pattern [lindex $args 1]
set id {}
set value {}
$dbcmd eval {SELECT id,value FROM tuples where space=$tuplespace AND tuple_match(value,$pattern) LIMIT 1} {
if {$direction eq "in"} {
$dbcmd eval {DELETE FROM tuples WHERE id=$id}
}
}
if {$id eq {}} {
return [list nomatch {}]
} else {
return [list ok $value]
}
}
method Req {direction socket args} {
if {[llength $args] != 2} {
return [list error "wrong # of args, should be \"$direction tuplespace pattern\""]
}
set tuplespace [lindex $args 0]
set pattern [lindex $args 1]
set resp [$self A$direction $socket $tuplespace $pattern]
if {[lindex $resp 0] eq "nomatch"} {
$self AddCallback $socket $direction $tuplespace $pattern
return {}
} {
return $resp
}
}
}
}And a client using the Tupleserver:
proc connect_tuplespace {name namespace} {
set s [socket localhost 4040]
fconfigure $s -buffering line
fileevent $s readable [list handle_command $s $name $namespace]
puts $s "in $name {* * *}"
}
# connect to server
connect_tuplespace LOGsrv logsrv
puts "Client connected to server"
# define commands understood by logsrv
namespace eval logsrv {
proc log {severity description} {
puts "logging $severity: $description"
}
}
proc handle_command {s ts ns} {
gets $s resp
# get next lines if response has embedded new lines
while {![info complete $resp]} {
append resp \n[gets $s]
}
set status [lindex $resp 0]
set resp [lindex $resp end]
if {$status eq "error"} {
puts stderr $resp
return
}
set cmd [lindex $resp 0]
set args [lrange $resp 1 end]
if {[catch {::${ns}::$cmd {*}$args} error]} {
puts stderr $error
}
puts $s "in $ts {* * *}"
}
vwait foreverIf you now connect to the server running the tupleserver on port 4040 the following command:
out LOGsrv {log INFO {testing info message}}will cause one (of possibly many clients listening to the LOGsrv tuplespace) to pick up the message and display the log line.
MJ - Below an initial version (using Snit) to simplify writing of TS clients.
package require snit
snit::type client {
variable connection {}
variable active_tuplespace {}
method connect {server port} {
if {$connection ne {}} {
return -code error "already connected close current connection first"
}
set connection [socket $server $port]
fconfigure $connection -buffering line -encoding utf-8
return
}
method close {} {
if {$connection eq {}} {
return -code error "not connected"
}
close $connection
set connection {}
}
method send {cmd args} {
puts $connection [list $cmd {*}$args]
return [$self response]
}
method response {} {
gets $connection resp
while {![info complete $resp]} {
append resp \n[gets $connection]
}
return $resp
}
method open {tuplespace} {
set active_tuplespace $tuplespace
}
method out {args} {
if {[llength $args]==2} {
$self send out {*}$args
} elseif {[llength $args]==1} {
$self send out $active_tuplespace {*}$args
} else {
return -code error "wrong # of args: should be \"$self out ?tuplespace? tuple\""
}
}
method in {args} {
if {[llength $args]==2} {
$self send in {*}$args
} elseif {[llength $args]==1} {
$self send in $active_tuplespace {*}$args
} else {
return -code error "wrong # of args: should be \"$self in ?tuplespace? tuple\""
}
}
}
# testing
set c [client ts%AUTO%]
$c connect localhost 4040
$c open Test
$c out {1 2 3 4} ; # takes the open TS
$c out Test {4 5 6 7} ; # explicitly specify it
puts [$c in {* * * *}]
puts [$c in {* * * *}]
console showLV 2007 Jan 19 - Across this wiki, various styles of managing the page are practiced. In some cases, one authority code is edited and improved. In other cases, people add corrections and improvements as follow on comments. There are likely other methods in use as well.
I just wonder if, for this page for instance, the original author is monitoring the page and updating some authority copy of the code, so that someone could come along and not have to make an attempt to merge changes and proc replacements into a copy of the code and then try to get it to work....
escargo 19 Jan 2007 - This is also an issue for programs such as wish-reaper that extract code from the pages. Where there are multiple versions, or even code intended for separate files on the same page, grabbing all the code in one (somewhat undifferentiated) block doesn't work very well. It's a problem without a consensus solution.
LV Good point - if I had remembered wish-reaper, I would have mentioned that problem. Thanks!
MJ 05 Feb 2008: Good point, which is why I moved this implementation to a page of its own.
AM I am interested in distributed computing - that is: let different (instances of) programs (or threads) cooperate with each other to get the job done. I do not see, however, from the examples how you could create such a system using tuplespaces (more the practical implementation, I guess, than the principle). Anything that is easier than mutexes and their fiendish friends is welcome :).
MJ - Without knowing the details of your exact problem, it is difficult to be precise. Maybe a small example will help. Consider a simulation you want to run with a large number of parameters a possible use of tuplespaces could be:
This will allow you to start additional simu instances on different nodes without any changes to the code (the Tuplespace will handle the coordination). If analyzing takes a long time, multiple instances could be started as well. To easily monitor progress one could image a Log process (as in the example above) that checks for tuples in the form {log src severity msg} and writes them to a central log file. All other processes can then do logging using the tuplespace. For instance the simu processes could post for instance {log simu info {simulation finished}}.
Does this help?
AM I think it does. I wll try and see how this fits into the sort of systems I am thinking about (domain decomposition for the solution of PDEs for instance where values on the boundaries of the various domains are exchanged and the processes need to wait for the results of their neighbours to become available), but this may be a small modification of the above.
AM (Mostly as a reminder for myself) Alternative applications of tuplespaces:
(Well, these are examples I have been thinking about from time to time, nothing very original, but they could be well implemented using tuplespaces instead of via the "observer" pattern)
Oh, I did find a nice tutorial on the subject, with a rather lame elaborated example (lame, because the technique used for solving the number crunching is awkward to say the least, but perhaps shows the principle better): http://www2.java.no/web/files/moter/tuplespace.pdf
MJ - The original paper "Generative communication in Linda" by David Gelernter contains some more examples and provides some higher abstractions(like P2P communication) based on the basic primitives. Unfortunately I cannot find a free download, but the article is available from the ACM http://portal.acm.org/citation.cfm?id=2433 . (NEM: At least one free version available from [L2 ]).
MJ - 2007-11-16 Added updated version that:
MJ - Feb 2008 Added abillity to use multiline tuples (as long as it is a valid Tcl list). Added a return code for ain and aread if there was no match.
AM (5 february 2008) I have started to develop a practical application of tuplespaces. The idea is that you can use the tuple server to manage data (both input and output) so that you can easily do computations on various machines.
More specifically:
Right now it is getting in the stage where it actually works (I had to cope with a few oddities in my code that led to the loss of files, but solving them was actually easy). More to come - like the actual source code, but that will be on a separate page with a better description. For the moment: have a look at A poor man's grid computing
DKF: Something I've always wondered about tuplespaces; what do they do for security? Especially access control.
AM (6 february 2008) That is a matter orthogonal to the idea of tuplespaces (I like that word, orthogonal :)). In Mark's implementation access control could be added in at least three ways:
As for general aspects of security, the server and the client could agree in a standard way on encrypting the tuples that are sent. But I have no experience at all with such matters, just these suggestions that came to mind.
DKF: There's three levels that need security. Firstly, you need encryption (tls::socket does well at this) of the communication channel, secondly you need authentication of the client to the server (username/password works for this, or you could use client-authenticated SSL) and thirdly you need access control on the tuplespace itself. It's the third step that needs most consideration within the tuplespace framework itself.
The key access control steps are:
Whether the access control information should be in the tuple or as metadata, that's a good question. (Should the tupleserver ever look inside the tuples it holds for itself? Should it constrain the tuples that way?)
Encryption of (some of) the contents of a tuple is entirely up to the clients working with that tuple; the server does not need to know anything about it.
jima (2008-02-13)
Could this we used in the implementation of a distributed computation scheme like [L3 ] ?
There are plenty of blogs commenting on MapReduce lately... I remember having read (don't know where, otherwise I would post the link) that the whole data passing scheme was based on... yes, tuples. Or so I gathered.
MJ - From a quick glance at that wikipedia link it seems that this would definitely be possible. It remains to be seen if you get a real performance boost though, this is largely dependent on the amount of overhead the passing around of the tuples adds. If large chunks can be passed around that require considerable effort to process, the overhead of the tupleserver shouldn't be too bad. Note that the tupleserver at the moment is geared towards ease of distribution not raw speed.