JBR - 20101022 I have been using swish-e and swish++ for a while but they are too slow and don't have very nice incremental index features. Here is a full text search system written in tcl using sqlite3's fts3 virtual tables. I'm a rank novice at sql and implemented the searchrank function as simply as possible so any suggestions are welcome.
The search system in configured with a configuration file similar to swish++.
fts.conf:
set tmp /var/tmp stopwords stopwords filter *.bz2 bunzip2 -c %f > @%F filter *.gz gunzip -c %f > @%F filter *.Z uncompress -c %f > @%F filter *.doc catdoc %f filter *.xls xls2csv %f filter *.ppt catppt %f filter *.prn pdftotext %f filter *.pdf pdftotext %f filter *.PS ps2ascii %f filter *.ps ps2ascii %f filter *.eps ps2ascii %f filter *.txt cat %f filter *.htm lynx -dump %f filter *.html lynx -dump %f filter */page/* cat %f exclude *.dxf exclude *.exe exclude *.gif *.jpg *.png .jpeg exclude *.mpeg *.mov .avi exclude *.mf1 *.mf2 exclude *.sym exclude *.zip exclude *~ exclude *.files/ index-path /data/cgijohn/paige/file index-path /data/cgijohn/paige/page index-path /data/wdocs/john/idoc-db index-path /data/wdocs/dfabricant/www-docs