Ants

Let's connect the Tcl computers of the world!

What is the Ants project?

It is effectively an open, grass-root effort to build a grid computing network around services built in Tcl.

 What: Ants
 Where: http://www.fishpool.com/~setok/ants/
 Description: Ants is a generic system for running any kind of distributed
        code, intended to replace application-specific systems like the one
        used by SETI@home. The idea is to be able to write scripts with areas
        defined to be run on the Ants network. These script fragments can then
        communicate with the original script by passing messages. The server
        admin can specify which libraries scripts are allowed to load. In
        addition to that, by configuring the system and attaching more scripts
        to it, it can be a valuable system administration framework for both
        small and large networks.
        Currently at version 1.0 .
 Updated: 02/2003
 Contact: See web site

Ants has a server component (acting as an application) and a client library that can be used to enable use of the distribution process.

Talk

Lars H: Interesting! I have a scientific problem (basically solving a huge equation system by breadth-first search) that could have use for this, although it might be using a bit more memory (a couple of 10MB) than servers find comfortable.

Setok: At the moment there is no limitation on how much memory scripts can use as I don't think you can control this in pure Tcl? I didn't want to make the first version contain any platform dependencies. I personally don't have any problem allowing 10MB of any server I connect to that to be used for something cool. What really remains is marketing: convincing people to connect their machines to that as resources for others to use. Note that the scripts are run in safe Tcl environments so there shouldn't be any security risks. So start connecting ;-)

Lars H: My concern was rather that it would be a pity if the first real utilization of Ants should turn out to be such a resource hog that the Ants themselves got a bad reputation. :-( I wouldn't want to be responsible for that. On the other hand I've come to think of a few changes in my procs that could reduce memory consumption considerably. :-) It'll take some rewrite, though.

Setok: Maybe I could add a preliminary solution that would tell clients how much memory a script is expected to use, at most. While currently that limit would not be enforced then at least well-behaving scripts wont cause havoc. One friend suggested that there should be some statistics available: that always gets people interested when they see their machine in the stats ;-) Note also that the client lib is still fairly simple: care needs to be taken to handle exceptions and connections properly. I'd be glad to help here. PS. A new version with some bug fixes should be coming out soon.

Lars H: Well, I don't think I'll have much time to work on this for another week or so. As for statistics, another factor that might interest people is getting some kind of output on what the program is doing. The (non-distributed) script I current has writes lines such as the following to stderr, to show what is going on:

  Eq+CP+mS+aVars = 580+  72+72+174   Examined 393. Split at c(2,1,18). Gained 4:4. Bits =    4.503 :   4.360
  Eq+CP+mS+aVars = 580+  72+72+175   Examined 392. Split at c(2,1,12). Gained 3:4. Bits =    3.849 :   3.786
  Eq+CP+mS+aVars = 580+  72+72+175   Examined 392. Split at a(3,1,4). Gained 2:6. Bits =    2.066 :   6.035
  Eq+CP+mS+aVars = 580+  72+72+175   Examined 392. Split at c(2,1,12). Gained 3:4. Bits =    3.723 :   3.734
  Eq+CP+mS+aVars = 580+  72+72+175   Examined 392. Split at a(3,1,4). Gained 2:6. Bits =    2.053 :   6.017
  Eq+CP+mS+aVars = 600+ 163+72+150   Forced by tautology at a(1,3,5): a(1,2,5) = 1.
  Eq+CP+mS+aVars = 600+ 163+72+151   Forced by contradiction: c(1,2,1) = 0, c(1,3,1) = 0, c(1,2,3) = 0, c(1,3,3) = 0, c(1,2,2) = 0, c(1,3,2) = 0.
  Eq+CP+mS+aVars = 594+  79+72+157   Forced by contradiction: c(2,2,1) = 0, c(3,2,1) = 0, c(2,3,1) = 0, c(3,3,1) = 0, c(3,1,3) = 0, c(3,1,2) = 1, c(2,2,3) = 0, c(2,3,3) = 0, a(1,3,5) = 1, c(3,2,3) = 0, c(3,3,3) = 0, c(2,2,2) = 0, c(3,2,2) = 0, c(2,3,2) = 0, c(3,3,2) = 0.
  Eq+CP+mS+aVars = 579+  37+72+172   Examined 395. Split at a(1,2,4). Gained 6:4. Bits =    6.498 :   4.032

I'm too involved to tell whether this is interesting to the average user, though.


AK: Another topic, which came up on the IIRC is 'security' in general. The server authenticating that a client is a true client, the client that it speaks to a true server, both checking the integrity of the code snippets send around so that no interloper can modify them (or the results) while they are in flight between the client and server, and possibly encryption for privacy.

Setok: Wouldn't TLS pretty much take care of that? While the servers form a network, the client (at least for the moment) always talks directly to the end server. Also verification that the sent result is actually valid is up to the application (no real way to generalise that).


escargo: I don't know if this is taking the Ants analogy too far, but perhaps there needs to be tunnels between clients and servers (of the ant farm). This is related to the security issue. There are at least three issues, authentication, authorization, and integrity (are you who you say you, do you have permission, and is what you say you sent really what I received), that need to considered. One solution might be to have a virtual private network (VPN) established between the clients and the servers. Once the VPN is set up, then data and programs should be able to flow securely. This VPN might be built on top of Tcl, or it might use something like ssh tunnels.


Setok: Agreed, the security with the current version is not perfect at all and would need encryption on the channels between the server and client. Any ideas and help on this is appreciated. For the next version I'm thinking about a way for the client to search for servers from the whole network (or limited to an area) which provide access to certain libs. Ie. that way the client could search for servers providing a "prime number" package, or whatever, and thus get more optimal results.

How about TLS, i.e. the Tcl extension which supports SSL, the Secure Socket Layer? -jcw

Setok: That's pretty much exactly what I had in mind ;-)

You might want to try Zebedee (http://www.winton.org.uk/zebedee/ ), which is the tunneling tool used in the Tcl-based Privaria system. - MDD


escargo: You might also be able to build on existing Tcl software (like Tuplespace except with persistance and secure networking support), where there might be tuples that each server uses to advertise the packages (and versions) that it has. If Ant were set up to produce and consume tuples, then a lot of the communication issues can be handled at the lower level. There is also probably a need for transfer of code and data outside of tuples, but maybe not.

Setok: Ah, I remember reading about Tuplespaces somewhere a while ago. That is worth looking into.