Purpose: to inventory the various options one has, from within Tcl, to perform Inter-Process Communication ([IPC]). **Basic mechanisms** The following list collects items by the underlying mechanism that is used. ***Communication through channels*** The most common forms of IPC in Tcl work through a [channel]. They differ quite a lot in how that channel is created, and sometimes also in what properties it may have, but once it's established one uses the same kind of [read], [gets], [puts], [fconfigure], [fileevent], etc. commands for all of them. Pipes (unnamed [pipeline]s): [[exec]], [[open |...]], [[chan pipe]], [[ [bgexec] ]] from ''[BLT]'', ... See also [http://www.sunworld.com/unixinsideronline/swol-01-2001/swol-0119-regex.html] TCP sockets: The "usual" kind of [internet] socket, opened using the core Tcl command [socket]. Most IPC packages use these in some way. [UDP] sockets: See [UDP for Tcl]; requires extensions. Unix-domain sockets: [ceptcl]. Like TCP sockets, but exist as objects in the file system, rather than as [IP]address:port combinations on the internet. Files: Open with [open]. Not particularly efficient, but sometimes sufficient. Also, can be used on different machines, even of different operating systems, as long as they share a file system ([RS]) [FIFO] or [named pipe]: FIFOs are also known as [named pipe]s on Unix. (The '''fifo''' of [memchan] is however something different.) See [http://sun.systemnews.com/articles/53/5/opt-dev/7227] for a pointer to a technical article about this method. [dislocate] is an [Expect] program which uses FIFOs. The extension [WNPComm] implements a reliable method of communication based on this. For Windows, [TWAPI] provides commands [http://twapi.magicsplat.com/namedpipe.html] for communicating over named pipes. ***Shared memory*** Quite a few extensions have been written for [shared-memory] IPC. semaphores: http://www.equi4.com/pub/pp/sorted/net/svipc-2.2.0/Index.html (or perhaps ftp://ftp.tcl.tk/pub/tcl/mirror/ftp.procplace.com/alcatel/extensions/svipc-2.1.1.tar.gz ) which covers semaphores, shared memory, and message queues. [lexfiend] ''2007-12-30'': It's definitely not building with modern Tcl's & *nixes, but I'm working on a patch for that. More when it's done. message queues: POSIX Message Queues [http://sourceforge.net/projects/tcl-mq] mmap'd files: tcl-mmap [https://sourceforge.net/projects/tcl-mmap/] ***Application-level message passing*** The big command in this area is Tk's [send], which uses [X] server resources to communicate scripts and their results between processes (or more precisely, between Tcl interpreters which have Tk loaded). IPC packages such as [comm] emulate send's API. The [selection] and [clipboard] are technically also IPC mechanisms, at least on [X] platforms. [DDE] and [COM] are Windows only. Also, [sh8] is Windows only. Macintosh, of course has [Applescript] (see also [http://wiki.tcl.tk/1221.html]) and [TclAE]. Both work by sending AppleEvents (AEs) to target applications. The difference is that TclAE lets you construct the AE directly using Tcl commands, whereas Applescript is a separate scripting language whose '''tell''' command constructs an AE and sends it. A system of [Linux] origin is [DBUS%|%D-Bus%|%] [http://www.freedesktop.org/wiki/Software/dbus] [http://www.linuxjournal.com/article/7744], where the bus is operated by a daemon process that several parties may connect to. Underlying mechanisms include (but are not limited to) unix sockets and TCP/IP sockets, but that level is usually seen only by the standard D-BUS library, which provides a message passing interface. ***Other mechanisms*** pseudoterminals (ptys): [Expect] (Q: Should this be with the channels?) Database: A database server provides a state that potentially is shared between several clients, and can thus be used for IPC. (This is often a wrapper around a lower-level mechanism, however.) [MPI]: Origin in parallelisation, for use when "the same program" is distributed over several machines. [Spread]: [http://www.spread.org] [[Provide info. State of Tcl binding unclear. [Perl], [Ruby], [PHP], ... all connect to it ... 'Seems like the kind of thing [davidw] would have encountered ...]] [YAMI]: Yet Another Messaging Infrastructure [signal]s: Very low level mechanism, primarily available in Unixy environments. Similar to an interrupt in assembly language programming: can be used to make a process do something, but provides no information (not even source of signal) other than the signal number. [m2]: network message bus Tcl package [ZeroMQ]: also has a Tcl interface, albeit for a older version **IPC packages** [[List probably incomplete. Some of what is listed above may also provide high-level interfaces comparable to those of these packages.]] ***[send]-like*** [Tk] provides the original [send]. [comm] (Previously at [http://www.schooner.com/~loverso/tcl-tk/#comm] - now a part of [tcllib]) is a [socket]-based send. Another alternative is ftp://ftp.procplace.com/pub/tcl/alcatel/extensions/tclipc1-0.tar.gz Yet another is [winsend] which uses Windows [COM] to implement send. Tk[pvm] claims to do something along this line. ***Remote Procedure Call*** Unlike [send], which presumes Tcl, these often seek to be language-independent. (internet) sockets: [socket], [TCL-DP] (and [dp_RPC] for a pure-Tcl substitute), [CORBA], [Scotty]'s [UDP], RPC, [SIMPL], [XML-RPC] server/client code [http://tclsoap.sourceforge.net/], [tclSOAP]/[SOAP] server/client code [http://tclsoap.sourceforge.net/] [http://phaseit.net/claird/comp.infosystems.www.misc/SOAP.html] ***Service-oriented*** A process declares a set of "services" that other processes may call or send messages to. Could also be viewed as '''object-oriented''' IPC: a process exposes one or several objects, letting other processes send them messages. Service-oriented interfaces tend to be closer to the user than remote procedure call interfaces (e.g. only doing things for which there is also a UI), but there is no clear boundary. ''The packages listed under "Application-level message passing" above usually provide this kind of interface.'' [apptalk] builds on Tk's [send], and provides for starting the target process if is isn't already running. ***Shared state*** [Tequila]: Uses traces, fileevents and sockets to create distributed [array]s, i.e. arrays whose content is shared between several applications and where changes are automatically distributed to them all. It makes use of a central server. It does have the advantage that applications can be built using arrays for data storage, and separated into client/server components with only a few additional lines of code. The "msg" [http://cfa-www.harvard.edu/~john/msg] interface that is used to operate F5 instrumentation at the MMT Observatory [http://cfa-www.harvard.edu/mmt/]. Very similar to [Tequila] above but operates on global variables. It supports publish, subscribe of values and commands. It includes synchronous, asynchronous and unacknowledged messages. Clients persistently reconnect to servers and reestablish thier subscription states automatically. It works very well and is used to tie dozens of hardware and software control functions at the MMT Observatory together. -- [JBR] (updated by [escargo]). [Linda] (note that [Todd Coram] cobbled together a tuple space ([Tuplespace]) in a few lines of [incr Tcl]), ... ***Message-passing*** An implementation of a [POSIX] Message Queues interface for Tcl can be found here [http://sourceforge.net/projects/tcl-mq]. [XPA] is a messaging system for communication between processes. [tmpi] provides bindings to the MPI library. ---- Have an interest in a protocol or technology you don't see mentioned here? As [LV] once wrote, "Note that this page is a growing list however. Tcl programmers are a versatile bunch, and new methods of communicating between processes are being implemented all the times. So if you have an idea of doing this sort of thing, but do not see it listed on the page, don't take the fact that it is missing as being a limitation of Tcl, but as a limitation of the minds/memories of the people writing the page ." So, just add a wiki page regarding the protocol or technology, and what it has to do with Tcl, and then add a link to that page on this page. ---- **Other stuff** <> How about ZeroC's ICE [http://zeroc.com/]? This ICE ''(Internet Communications Engine)'' is distinct from X11's Inter-Client Exchange (ICE) protocol, which is used for IPC by Tk's [send]. ''Appears to be fairly high level and very object-oriented, perhaps as much an OO framework as an IPC framework. There does not appear to be a Tcl binding.'' Does any part of [PVM] fit in anywhere? Sun used to have an IPC '''bus''' called Tooltalk. Over on the [writing extensions] page I see a reference to ''Toocl'', which is one developer's binding between Tooltalk and Tcl. I don't know if this is the same thing as I am semi-recalling or not. On the [tcl bibliography] page I see a reference to a paper by Michael Jipping, Hope College , (1993) ````Using Tcl as a Tool Talk Encapsulation'', in the Sun User Group Eleventh Annual Conference and Exhibition PROCEEDINGS. That seems quite likely to be what I am remembering. ---- [RS]: A simple file signalling between two processes, that both have write access to the same directory (but could run on different boxes and OSes, like Unix and WinNT), can look like this: proc fsignal {cmd name {msg ""}} { switch $cmd { send { set fp [open $name.sig w] if [string length $msg] {puts $fp $msg} close $fp } wait { while 1 { if [file exists $name.sig] { set fp [open $name.sig] set msg [read $fp] close $fp file delete $name.sig break } else {after 100} } } } set msg ;# return the message from sender in both cases } # process 1: (when conditions are ripe) fsignal send Go 42 # process 2: (before the real action) set Magic [fsignal wait Go] See also [filewait] for a related scriptlet. ---- [TV] Bear in mind, for those who don't already do, that there are a few basic mechanisms on the os-es and machines I'm aware of, of which signals, sockets (of the local and inet kind), and shared memory are the main ones. Most of the others, including many packages, don't add anything at all or much to the fundamental capacities of these mechanisms in essence, so many limitations and shortcomings of a lot of the parallel programming aids or simulations simply run into the problems and limitations of these facilities, which are on unix, linux, windows, and probably (though there I didn't program them myself), and probably on some of the less well known os-es, too. To begin with, there is hardly formatting involved in the basics, except essential flow control, there is always overhead for copying data around except in few extreme cases, already in the basic library use, there is no or not much support for actual parallel machine concepts, except that ethernet and maybe some others can be made to broadcast over standard enough socket interface, and generally there is bad or absent exact definition of the operation of the basic library functions, for instance with the important aspect of flow control. Which is the direct and only reason a lot of things fuck up or don't work right over various versions, brands and programmers on the internet. In java for instance. In many printer spoolers, for instance. Not to mention what this does to performance, a concept a modern university informaticist couldn't even spell out let alone specify, measure right and interpret with some engineering sense to begin with, let alone be able to incorporate in a design, let along in an important language definition. A lot of interfaces and languages serve no purpose one can optimize much or say something positively discriminating about because its just a style someone likes, or something their mothers don't recognize, or maybe a certain concept applied consistently. Some things in tcl, such as the copying of data over sockets are quite optimizable and well designed and for a scripting language highly optimal in certain sense. ''(oct 14 03)'' I did [pcom] years ago, which can be used for for instance remote command execution, see examples down the page, and [remote execution using tcl and Pcom]. ---- M&M's [BOOK Effective Tcl - Writing Better Programs in Tcl and Tk] examples no longer appear to be available on-line at the publishers. There is a version available at http://sf.net/projects/efftcl . ---- [AM] Here is another shot at interacting processes: [Distributing a series of tasks] ---- [vkvalli] A simple chat server and chat client library can be used for inter process communication. There are quite a few chat server and client implementations in this wiki. It seems to be suitable candidate for simple coordination among processes with messages. But generally people miss it when they look for IPC mechanisms. ---- [TV] '''(jan 7 ' 07)''' In [Tcl on Cuda] there is another mechanism at stake: interprocssing communication between the Parallel Processing Elements via shared registers or (fast) memory, and PCI express based passing of data (through pointers and mallocs on both sides) with CopyCputoGpu and CopyGputoCpu functions. Advantages: very high speed, in my case 1000 Megabytes/sec measured also with low latency! When the OpenGL processor is considered as taking part in inter-process communication, like when it reaceives data from the cpu or even from the Cuda processors/thread engines (for recent mainstream NVidia cards), see for instance the gaussianblur example from the sdk. This can be linked with tcl, and via tcl3d for instance (se e.g. [bwise 3D graphics viewer block]) with Tk. [Lars H]: That sounds more like inter-''processor''-communication than inter-''process''-communication. [TV] Right. The processors all run at least one process (or a kernel and a process) and the host runs processes, the openGL is probably viewable as a process per pipeline and a global processor, and shader processes. What a grammar! [Lars H]: Linguistic contortions aside, it's still not inter-process communication (but rather some form of [threads]), and therefore not relevant on this page. [TV] Sorry lad, the thing on the hostis a process, period. Apearently you aren't aware of what I'm doing but I inted to port things in tcl to cuda, which runs processes on threads on processors. If you find that linguistically a problem, fine. ---- <> **See also** * [Concepts of Architectural Design for Tcl Applications] * [How can Tcl programs on two different machines communicate] * [Spread] * [Tcl implementations of publish-subscribe mechanisms] * [How do I manage lock files in a cross platform manner in Tcl] * ''Beej's Guide to Unix Interprocess Communication'' [http://www.ecst.csuchico.edu/~beej/guide/ipc/] ---- !!!!!! %| [Category Survey] | [Category Concept] | [Category Interprocess Communication] |% !!!!!!