memory file system

George Peter Staplin - It's often useful for developers to run a memory file system that is contained in RAM. This is usually much faster, and easier on busy disks. With my Fed Builder and ProcMeUp projects which save to the disk automatically I've found it helps the applications stay responsive when the disk is busy.

Here are some techniques I use for creating a memory file system to work on my Tcl/C projects.

In NetBSD and OpenBSD you can use the mount_mfs command to associate memory with a directory. First create the /work directory and then add this to your /etc/rc.local (or edit your /etc/fstab):

 mount_mfs -s 20m swap /work

Caveat: It doesn't actually use the swap space when you use that command. You may find that using some sizes results in strange error messages upon creation. This is due to the ulimit being reached/surpassed. If you need a bigger file system you can increase the ulimit. /etc/login.conf may need to be added/changed in some cases too.

What if the power goes out? Well, you will of course lose the files, unless you have a battery backup/UPS. Here's a script I created that automatically makes backups of my /work directory every 4 minutes:

 $ cat sync_work.sh                                                             
 #!/bin/sh 

 while true
 do
  day=`date +%A`
  p=/home/gps/src/automated_backups/$day
  mkdir -p $p
  cd $p
  d=`date +%s`
  tar -cPf work_$d.tar /work 
  #60 * 4 = 240
  sleep 240 
 done

George Peter Staplin Apr 17, 2006 - Here's a more Tclish script also by me:

 #!/usr/bin/env tclkit8.5

 cd ~/src/backups

 proc find.latest.mtime dir {
  set peak 0

  foreach f [glob -nocomplain [file join $dir *]] {
   if {[file mtime $f] > $peak} {
    set peak [file mtime $f]
   }

   if {[file isdirectory $f]} {
    set mtime [find.latest.mtime $f]

    if {$mtime > $peak} {
     set peak $mtime
    }
   }
  }

  return $peak
 }

 proc main {} {

  set lastpeak 0
  # 5 minutes
  set delay [expr {1000 * 60 * 5}]

  while 1 {
   set curpeak [find.latest.mtime /work]
   if {$curpeak == $lastpeak} {
    puts NOSAVE
    after $delay
    continue
   }

   catch {exec tar -zcvf [clock format [clock seconds] -format "%a_%b-%d_%H-%M"].tar.gz /work} err
   puts $err
   after $delay
   set lastpeak $curpeak
  } 
 }
 main

George Peter Staplin- A memory file system that uses Tcl's VFS would be interesting. Please add info here if you have done such a thing. :)

George Peter Staplin- Are there free/low-cost tools that you would recommend for a memory file system in Windows?

YES. The problem is that mfs is the least common name for what you are describing. Try using google with "windows ramdisk".

jcw - You can create and mount an in-memory FS with the mk4vfs extension used for starkits by giving it an empty string as path:

    % vfs::mk4::mount "" mountpath
    mk4vfs1
    %

The returned value of this command is a Metakit in-memory db name. It can be used to save the in-memory fs to file (or send it over a socket), after the fact:

    set fd [open snapshot w]
    mk::file save mk4vfs1 $fd
    close $fd

Look inside the "starsync" package in the SDX utility for more tricks.

CMcC - I've written dictvfs over snitvfs. It's an in-memory vfs built entirely over dict, tcl8.5 required of course. It's got to be faster than mk4vfs, as it's a significant reduction in strength - extremely minimal. It's to be found here: [L1 ] in a set of reimplemented vfs (including a significantly improved metakit vfs).

SEH 20060414 -- The Tclvfs package has a namespace virtual filesystem which uses Tcl namespaces to store its hierarchical filesystem information; this is essentially an in-memory vfs.