Version 8 of filter

Updated 2006-08-26 15:23:00

The filter paradigm is a concept that takes this form: the application takes its input, massages it, and outputs the data. Under Unix, this concept, paired with pipelines, allows one to build bigger tools by combining filters to create new tools.

For instance, one might build a sequence like this:

 grep MyName logfile | sort -u -f | wc -l

which reads logfile, producing on stdout only those lines containing the string MyName. This output is passed, in parallel (on Unix anyways), to the stdin of a command to sort the resulting lines, writing to its stdout lines which differ from one another (all lines identical without regard to upper or lower case are collapsed into one copy of the line). The stdout of the sort is then passed, in parallel, to the stdin of a filter that counts the number of lines and outputs the total.


Tcl's simple syntax for reading and writing to stdin and stdout makes it fairly easy to write these types of programs - see the filter idiom.


Sometimes you will read references to Tcl being used as glue; this is, in my opinion, another way of referring to the filter paradigm.


RS 2005-03-08 - filter is also a classic operator in functional programming, and it makes nice simple code:

 proc filter {list script} {
   set res {}
   foreach e $list {if {[uplevel 1 $script $e]} {lappend res $e}}
   set res
 }

Testing, with a simple way to do intersection:

 proc in {list e} {expr {[lsearch -exact $list $e]>=0}}

 % filter {a b c} {in {b c d}}
 b c

smh 2006-08-26 Here is a proc based on this to find the intersection of many lists:

 proc intersection { items } {
   set res [lindex $items 0]             ;    # start with the first sub-list
   foreach item [lrange $items 1 end] {  ;    # loop through the rest
      set res [filter $res {in $item}]   ;    # and filter each new subset against the next
   }
   set res
 }

Category Glossary