memchan

Difference between version 32 and 34 - Previous - Next

&|[Memchan Logo 100]|Memchan, by [Andreas Kupries], is currently hosted on [http://sf.net%|%SourceForge].  <<br>>Its homepage is http://memchan.sourceforge.net/ <<br>>This package is part of the [ActiveTcl] Batteries Included distribution.<<br>>It provides several new [channel] types whose instances store the transferred information in memory and do not go to the OS at all.|&



** See Also **

   [Extensions for Tcl and Tk]:   

   [ycl%|%ycl string chan]:   An alternative implementation of an in-memory channel.  Allows multiple channels to attach to the same buffer.



'''Channel types'''
   * '''memchan''' Read and write a memory buffer via a channel. Deprecated by [tcllib]'s [virtchannel_base%|%tcl::chan::memchan], which is available for Tcl >= 8.5.
   * '''fifo''' A fifo buffer presented as a channel
   * '''fifo2''' Creates a pair of channels with a fifo connecting them
   * '''null''' Like the unix null device, discards everything and returns eof on read.
   * '''zero''' Like the unix zero device. Discards all writes and reads return all zeros.
   * '''random''' Like the unix random device. Writing seeds an [ISAAC] pseudo-random number generator. Reading returns random bytes from the generator.

----
 What: memchan
 Where: http://memchan.sourceforge.net/ >
        http://www.purl.org/NET/akupries/soft/memchan/
 Description: A new channel type for Tcl 8's channel system.  Memory channels
        conform to the same interface as files and sockets, but the data
        is stored in memory rather than in files.  They are good for
        long dynamic strings and passing large quantities of data.
        Supports Window and Unix.  See the web page for pointers to the
        source and binary downloads.
        Currently at version 2.2.1
 Updated: 03-Dec-2004
 Contact: mailto:[email protected] (Andreas Kupries)
          mailto:[email protected] (Pat Thoyts)




** Bug: Memchan: [[`[fileevent] readable`] **

[PYK] 2013-11-21: This same issue has been observed and diagnosed in
[virtchannel_base].  Does this package need a similar fix? 

[JMN] 2003-07-15: This isn't directly referring to the eof command - but is an
EOF issue that I wasn't expecting so here looks as good a place as any to
mention it.

Using the [Memchan] package, I was trying to [fcopy] from a memchan to a file
roughly like so:

======
package require Tcl
package require Memchan

set fd [open /tmp/output.txt w]
set m [memchan]
puts $m someStringData
seek $m 0 start
fcopy $m $fd -command CopyFinished

proc CopyFinished args {
        puts $args
}

puts {Application has completed}
======

This performed the copy to file just fine, but the CopyFinished proc never got
called. To fix this, I called 

======
fconfigure $m -eofchar {}
======

before the puts someStringData line and

======
fconfigure $m -eofchar \x1a
======

before the fcopy line. Then I changed the puts line to:

======
puts $m someStringData\x1a
======

This seems like a bit of hoop-jumping to get the expected fcopy behaviour.. am
I missing something simple?



** Misc **

----

[LES] 2005-09-12: Would it be possible for memchan or a similar hypothetical extension to store data in memory in such a way that it survives the [exit] command and thus remains available to other Tcl programs or subsequent calls of the same program?  

[TP] 2005-09-12: Maybe not with memchan, but look into [shared-memory] and [Inventory of IPC methods] 
for SysV IPC shared memory extensions, which can survive after process exit (but not reboot).  Also,
[Tequila] and [Tuplespace] for Tcl solutions that would require another running Tcl process to hold the data.  

On Linux, Solaris, and probably other POSIX OS's, it is as simple as using /dev/shm 
as a directory to read/write files into a memory-backed filesystem:

======
# first process
set fd [open /dev/shm/yourfile w]
puts $fd $yourstuff
close $fd
exit

# next process
set fd [open /dev/shm/yourfile]
set yourstuff [read $fd]
close $fd
======

Check that you have proper permissions to create files in /dev/shm.  

I'm not quite sure what Windows equivalents there are.

[ZB] 2009-04-15:  Memchan seems to be not maintained anymore. Is it about the same, as [reflected channel], but of older implementation?

[JMN]: I hope it's not abandoned..  As far as I'm aware - there's no other equivalent of Memchan's 'fifo2' command for example.

[EG]: I've never used the 'fifo2' command, but from reading the manual page it looks equivalent to 8.6's [chan pipe].

[AMG]: [chan pipe] has the significant advantage of producing actual operating system file descriptors which can be used in [exec] redirection.

[AK]: fifo2 is purely in memory. Useful to talk between threads without going though the OS. [chan pipe] on the other is a regular pipe under OS control.

[EG]: After reading carefully the 'fifo2' man page, I realized it is quite different from [chan pipe]. Since we have now (8.5+) the possibility to implement channel drivers in pure Tcl (thanks to [refchan]), I've been trying to write a pure Tcl implementation. See [fifo2 in pure Tcl].

----

[LV]: I would expect that it is being maintained, as needed, given that the author is also a maintainer of [ActiveTcl] and memchan is a part of ActiveTcl.

If specific bugs are found in the code, be certain to file a bug report at the very least at the memchan sf.net site, and perhaps additionally with ActiveTcl (if you have verified the bug exists in the version of memchan that is available in ActiveTcl).

One of the things I have noticed is that the Tcl community has moved towards fixing code in source code repositories without necessarily creating specific new releases. In this specific case, the CVS head snapshots provided by ActiveState each time a change to a CVS repository has some sort of an update are a reflection of the '''current''' version of many Tcl packages. For memchan, the latest CVS snaphot is dated March 16, 2009.

[AK]: That, and memchan is '''stable'''. It doesn't need change, only very occasional a bugfix. So, it is maintained, but there is no change for changes sake. And I have to update the website part to reflect the most uptodate release etc.

----

[FPX]: Hint for those who are trying to build Memchan from the latest stable package on SourceForge: There is a bug in Memchan-2.2.1 that messes up the VPATH setting in the Makefile during configuration.  To fix, edit Makefile after running configure, and on line 348 add $(srcdir) at the beginning of VPATH so that it reads:

 VPATH = $(srcdir)/generic:$(srcdir)/unix:$(srcdir)/win:$(srcdir)/isaac:$(srcdir)/doc

Also, attempting to cross-compile Memchan-2.2.1 fails at configuration time with:

 checking for 64-bit integer type... configure: error: cannot run test program while cross compiling

To work around this issue, set

 export tcl_cv_type_64bit="long long"

before running configure.

[LV]: Remember to open a new bug report at the SF.net site mentioned at the top of this page, so that the maintainer can fix the distribution.

----
[chw] (2021-04-26) https://www.androwish.org/home/dir?ci=tip&name=jni/Memchan has an unofficial updated
version of [memchan] which supports new Tcl_Channel methods for truncation and channel transfer between
threads. It also dynamically grows and shrinks its memory buffer. IMO it should not be deprecated w.r.t.
[tcllib]'s [virtchannel_base%|%tcl::chan::memchan] since its performance is comparable to native file
system operations whereas the tcllib implementation in pure Tcl is about three times slower.
----

<<categories>> Package | Channel