memchan

Memchan, by Andreas Kupries, is currently hosted on SourceForge . Its homepage is http://memchan.sourceforge.net/ This package is part of the ActiveTcl Batteries Included distribution. It provides several new channel types whose instances store the transferred information in memory and do not go to the OS at all.

See Also

Extensions for Tcl and Tk

ycl string chan

An alternative implementation of an in-memory channel. Allows multiple channels to attach to the same buffer.

Channel types

• memchan Read and write a memory buffer via a channel. Deprecated by tcllib's tcl::chan::memchan, which is available for Tcl >= 8.5.
• fifo A fifo buffer presented as a channel
• fifo2 Creates a pair of channels with a fifo connecting them
• null Like the unix null device, discards everything and returns eof on read.
• zero Like the unix zero device. Discards all writes and reads return all zeros.
• random Like the unix random device. Writing seeds an ISAAC pseudo-random number generator. Reading returns random bytes from the generator.

 What: memchan
 Where: http://memchan.sourceforge.net/ >
        http://www.purl.org/NET/akupries/soft/memchan/
 Description: A new channel type for Tcl 8's channel system.  Memory channels
        conform to the same interface as files and sockets, but the data
        is stored in memory rather than in files.  They are good for
        long dynamic strings and passing large quantities of data.
        Supports Window and Unix.  See the web page for pointers to the
        source and binary downloads.
        Currently at version 2.2.1
 Updated: 03-Dec-2004
 Contact: mailto:[email protected] (Andreas Kupries)
          mailto:[email protected] (Pat Thoyts)

Bug: Memchan: [fileevent readable]

PYK 2013-11-21: This same issue has been observed and diagnosed in virtchannel_base. Does this package need a similar fix?

JMN 2003-07-15: This isn't directly referring to the eof command - but is an EOF issue that I wasn't expecting so here looks as good a place as any to mention it.

Using the Memchan package, I was trying to fcopy from a memchan to a file roughly like so:

package require Tcl
package require Memchan

set fd [open /tmp/output.txt w]
set m [memchan]
puts $m someStringData
seek $m 0 start
fcopy $m $fd -command CopyFinished

proc CopyFinished args {
        puts $args
}

puts {Application has completed}

This performed the copy to file just fine, but the CopyFinished proc never got called. To fix this, I called

fconfigure $m -eofchar {}

before the puts someStringData line and

fconfigure $m -eofchar \x1a

before the fcopy line. Then I changed the puts line to:

puts $m someStringData\x1a

This seems like a bit of hoop-jumping to get the expected fcopy behaviour.. am I missing something simple?

Misc

LES 2005-09-12: Would it be possible for memchan or a similar hypothetical extension to store data in memory in such a way that it survives the exit command and thus remains available to other Tcl programs or subsequent calls of the same program?

TP 2005-09-12: Maybe not with memchan, but look into shared-memory and Inventory of IPC methods for SysV IPC shared memory extensions, which can survive after process exit (but not reboot). Also, Tequila and Tuplespace for Tcl solutions that would require another running Tcl process to hold the data.

On Linux, Solaris, and probably other POSIX OS's, it is as simple as using /dev/shm as a directory to read/write files into a memory-backed filesystem:

# first process
set fd [open /dev/shm/yourfile w]
puts $fd $yourstuff
close $fd
exit

# next process
set fd [open /dev/shm/yourfile]
set yourstuff [read $fd]
close $fd

Check that you have proper permissions to create files in /dev/shm.

I'm not quite sure what Windows equivalents there are.

ZB 2009-04-15: Memchan seems to be not maintained anymore. Is it about the same, as reflected channel, but of older implementation?

JMN: I hope it's not abandoned.. As far as I'm aware - there's no other equivalent of Memchan's 'fifo2' command for example.

EG: I've never used the 'fifo2' command, but from reading the manual page it looks equivalent to 8.6's chan pipe.

AMG: chan pipe has the significant advantage of producing actual operating system file descriptors which can be used in exec redirection.

AK: fifo2 is purely in memory. Useful to talk between threads without going though the OS. chan pipe on the other is a regular pipe under OS control.

EG: After reading carefully the 'fifo2' man page, I realized it is quite different from chan pipe. Since we have now (8.5+) the possibility to implement channel drivers in pure Tcl (thanks to refchan), I've been trying to write a pure Tcl implementation. See fifo2 in pure Tcl.

LV: I would expect that it is being maintained, as needed, given that the author is also a maintainer of ActiveTcl and memchan is a part of ActiveTcl.

If specific bugs are found in the code, be certain to file a bug report at the very least at the memchan sf.net site, and perhaps additionally with ActiveTcl (if you have verified the bug exists in the version of memchan that is available in ActiveTcl).

One of the things I have noticed is that the Tcl community has moved towards fixing code in source code repositories without necessarily creating specific new releases. In this specific case, the CVS head snapshots provided by ActiveState each time a change to a CVS repository has some sort of an update are a reflection of the current version of many Tcl packages. For memchan, the latest CVS snaphot is dated March 16, 2009.

AK: That, and memchan is stable. It doesn't need change, only very occasional a bugfix. So, it is maintained, but there is no change for changes sake. And I have to update the website part to reflect the most uptodate release etc.

FPX: Hint for those who are trying to build Memchan from the latest stable package on SourceForge: There is a bug in Memchan-2.2.1 that messes up the VPATH setting in the Makefile during configuration. To fix, edit Makefile after running configure, and on line 348 add $(srcdir) at the beginning of VPATH so that it reads:

 VPATH = $(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win:$(srcdir)/isaac:$(srcdir)/doc

Also, attempting to cross-compile Memchan-2.2.1 fails at configuration time with:

 checking for 64-bit integer type... configure: error: cannot run test program while cross compiling

To work around this issue, set

 export tcl_cv_type_64bit="long long"

before running configure.

LV: Remember to open a new bug report at the SF.net site mentioned at the top of this page, so that the maintainer can fix the distribution.

chw (2021-04-26) https://www.androwish.org/home/dir?ci=tip&name=jni/Memchan has an unofficial updated version of memchan which supports new Tcl_Channel methods for truncation and channel transfer between threads. It also dynamically grows and shrinks its memory buffer. IMO it should not be deprecated w.r.t. tcllib's tcl::chan::memchan since its performance is comparable to native file system operations whereas the tcllib implementation in pure Tcl is about three times slower.