Let's connect the Tcl computers of the world!
What is the Ants project?
It is effectively an open, grass-root effort to build a grid computing network around services built in Tcl.
What: Ants
Where: http://www.fishpool.com/~setok/ants/
Description: Ants is a generic system for running any kind of distributed
code, intended to replace application-specific systems like the one
used by SETI@home. The idea is to be able to write scripts with areas
defined to be run on the Ants network. These script fragments can then
communicate with the original script by passing messages. The server
admin can specify which libraries scripts are allowed to load. In
addition to that, by configuring the system and attaching more scripts
to it, it can be a valuable system administration framework for both
small and large networks.
Currently at version 1.0 .
Updated: 02/2003
Contact: See web siteAnts has a server component (acting as an application) and a client library that can be used to enable use of the distribution process.
Talk
Lars H: Interesting! I have a scientific problem (basically solving a huge equation system by breadth-first search) that could have use for this, although it might be using a bit more memory (a couple of 10MB) than servers find comfortable.
Setok: At the moment there is no limitation on how much memory scripts can use as I don't think you can control this in pure Tcl? I didn't want to make the first version contain any platform dependencies. I personally don't have any problem allowing 10MB of any server I connect to that to be used for something cool. What really remains is marketing: convincing people to connect their machines to that as resources for others to use. Note that the scripts are run in safe Tcl environments so there shouldn't be any security risks. So start connecting ;-)
Lars H: My concern was rather that it would be a pity if the first real utilization of Ants should turn out to be such a resource hog that the Ants themselves got a bad reputation. :-( I wouldn't want to be responsible for that. On the other hand I've come to think of a few changes in my procs that could reduce memory consumption considerably. :-) It'll take some rewrite, though.
Setok: Maybe I could add a preliminary solution that would tell clients how much memory a script is expected to use, at most. While currently that limit would not be enforced then at least well-behaving scripts wont cause havoc. One friend suggested that there should be some statistics available: that always gets people interested when they see their machine in the stats ;-) Note also that the client lib is still fairly simple: care needs to be taken to handle exceptions and connections properly. I'd be glad to help here. PS. A new version with some bug fixes should be coming out soon.
Lars H: Well, I don't think I'll have much time to work on this for another week or so. As for statistics, another factor that might interest people is getting some kind of output on what the program is doing. The (non-distributed) script I current has writes lines such as the following to stderr, to show what is going on:
Eq+CP+mS+aVars = 580+ 72+72+174 Examined 393. Split at c(2,1,18). Gained 4:4. Bits = 4.503 : 4.360 Eq+CP+mS+aVars = 580+ 72+72+175 Examined 392. Split at c(2,1,12). Gained 3:4. Bits = 3.849 : 3.786 Eq+CP+mS+aVars = 580+ 72+72+175 Examined 392. Split at a(3,1,4). Gained 2:6. Bits = 2.066 : 6.035 Eq+CP+mS+aVars = 580+ 72+72+175 Examined 392. Split at c(2,1,12). Gained 3:4. Bits = 3.723 : 3.734 Eq+CP+mS+aVars = 580+ 72+72+175 Examined 392. Split at a(3,1,4). Gained 2:6. Bits = 2.053 : 6.017 Eq+CP+mS+aVars = 600+ 163+72+150 Forced by tautology at a(1,3,5): a(1,2,5) = 1. Eq+CP+mS+aVars = 600+ 163+72+151 Forced by contradiction: c(1,2,1) = 0, c(1,3,1) = 0, c(1,2,3) = 0, c(1,3,3) = 0, c(1,2,2) = 0, c(1,3,2) = 0. Eq+CP+mS+aVars = 594+ 79+72+157 Forced by contradiction: c(2,2,1) = 0, c(3,2,1) = 0, c(2,3,1) = 0, c(3,3,1) = 0, c(3,1,3) = 0, c(3,1,2) = 1, c(2,2,3) = 0, c(2,3,3) = 0, a(1,3,5) = 1, c(3,2,3) = 0, c(3,3,3) = 0, c(2,2,2) = 0, c(3,2,2) = 0, c(2,3,2) = 0, c(3,3,2) = 0. Eq+CP+mS+aVars = 579+ 37+72+172 Examined 395. Split at a(1,2,4). Gained 6:4. Bits = 6.498 : 4.032
I'm too involved to tell whether this is interesting to the average user, though.
AK: Another topic, which came up on the IIRC is 'security' in general. The server authenticating that a client is a true client, the client that it speaks to a true server, both checking the integrity of the code snippets send around so that no interloper can modify them (or the results) while they are in flight between the client and server, and possibly encryption for privacy.
Setok: Wouldn't TLS pretty much take care of that? While the servers form a network, the client (at least for the moment) always talks directly to the end server. Also verification that the sent result is actually valid is up to the application (no real way to generalise that).
escargo: I don't know if this is taking the Ants analogy too far, but perhaps there needs to be tunnels between clients and servers (of the ant farm). This is related to the security issue. There are at least three issues, authentication, authorization, and integrity (are you who you say you, do you have permission, and is what you say you sent really what I received), that need to considered. One solution might be to have a virtual private network (VPN) established between the clients and the servers. Once the VPN is set up, then data and programs should be able to flow securely. This VPN might be built on top of Tcl, or it might use something like ssh tunnels.
Setok: Agreed, the security with the current version is not perfect at all and would need encryption on the channels between the server and client. Any ideas and help on this is appreciated. For the next version I'm thinking about a way for the client to search for servers from the whole network (or limited to an area) which provide access to certain libs. Ie. that way the client could search for servers providing a "prime number" package, or whatever, and thus get more optimal results.
How about TLS, i.e. the Tcl extension which supports SSL, the Secure Socket Layer? -jcw
Setok: That's pretty much exactly what I had in mind ;-)
You might want to try Zebedee (http://www.winton.org.uk/zebedee/ ), which is the tunneling tool used in the Tcl-based Privaria system. - MDD
escargo: You might also be able to build on existing Tcl software (like Tuplespace except with persistance and secure networking support), where there might be tuples that each server uses to advertise the packages (and versions) that it has. If Ant were set up to produce and consume tuples, then a lot of the communication issues can be handled at the lower level. There is also probably a need for transfer of code and data outside of tuples, but maybe not.
Setok: Ah, I remember reading about Tuplespaces somewhere a while ago. That is worth looking into.