wiki database for offline use

Difference between version 153 and 154 - Previous - Next
2017-11-22 -- Any hope of somebody giving me access to a recent database backup?

Zip files containing all [Wub], [wikitcl] and [tcllib] code you need to run a local wiki can be found http://code.google.com/p/wubwikit/downloads/list%|%here%|%. From 30-oct-2009 on, [wubwikit] is working with the new [tdbc::sqlite] backend.

This wiki's data is stored in a single datafile, the path to which is then provided to the wiki executable code on the command line.
[LES] on 2022-10-28 That link and many related ones I could find all over the wiki are gone. Most are just gone, equi4 has pages but no binaries. I can't seem to find wikit.kit or wubwiki anywhere.

**News**

[jdc] 2-may-2012: Added support for HTML 5.

[jdc] 24-apr-2012: Added support for [sqlite] FTS. The copy of the wiki database available at [http://sourceforge.net/project/showfiles.php?group_id=211498] now also contains the required table for FTS. FTS replaces the (Google) search in titles/pages. 

[jdc] 18-mar-2011: Fixed Windows bugs. No longer providing starkits.

[jdc] 17-feb-2011: Possibility to use and generate your own [Wub] config file, option to enable SQL queries.

[jdc] 13-jan-2011: Fixed possible SQL-injection bug.

[jdc] 03-jan-2011: Added options to disable ReCaptcha's.

[jdc] 10-sep-2010: Updated to latest version of [Wub].

[jdc] 11-jun-2010: Bug fixes for utils and release based on [Wub] `REL-2.0.1`.

[jdc] 31-may-2010: Better support for using WubWikit as simple [CMS], support for dumping images stored in the database, support for ADMIN:MOTD, ADMIN:TOC and ADMIN:Welcome pages, making it easier to restyle your Wiki.

[jdc] 12-mar-2010: Support for dumping a wiki as a static site.

[jdc] 12-dec-2009: Added [doctools] support.

[jdc] 27-nov-2009: New [sqlite] Wiki database with support for in-db images. New archive to run your own wiki. No longer uses command line arguments to set the wiki parameter but a configuration file. A default configuration can be generated. This release also supports in-db images and has more reports.

[jdc] 04-nov-2009: Rework of utils.

[jdc] 30-oct-2009: Archive to run your own wiki with [tdbc] backend. To convert your [Metakit] database to [sqlite] check [http://wikitcl.googlecode.com/svn/trunk/wubwikit/tools/mk2tdbc.tcl]. This release no longer supports the [Tk] GUI.

[jdc] 28-oct-2009: New [sqlite] Wiki database. The latest [Metakit] database is `wikit-20091026.tkd.zip`, from `wikit-20091028.tkd.zip` on you'll get [sqlite] databases.

[jdc] 7-aug-2009: Added GUI to stop server and start browser (contributed by Gaston Rodriguez). Reworked utility to get pages from a wiki database. Files can be found at http://code.google.com/p/wubwikit/downloads/list.

[jdc] 5-aug-2009: Added utilities to get headers and pages from a wiki database. Files can be found at http://code.google.com/p/wubwikit/downloads/list.

[jdc] 23-jun-2009: Release including `<<categories>>` markup and adding comments. Files can be found at http://code.google.com/p/wubwikit/downloads/list or https://sourceforge.net/project/showfiles.php?group_id=211498&package_id=265649

[jdc] 9-jun-2009: Bugfixes issued by Gaston Rodriguez for `yscroll` and `registry` problem. Files can be found at http://code.google.com/p/wubwikit/downloads/list.

[jdc] 19-mar-2009: More control over url as displayed on wiki pages. Released as kit and as zip. Zip can be used without needing a tclkit. Files can be found at http://code.google.com/p/wubwikit/downloads/list .

[jdc] 18-feb-2009: Release based on co-routine powered [Wub]. The project to provide a wiki for offline use is now hosted on http://code.google.com/p/wubwikit . The database stays on http://sourceforge.net/project/showfiles.php?group_id=211498

[jdc] 12-dec-2008: Release with latest wiki modifications. A new kit is available for download [https://sourceforge.net/project/showfiles.php?group_id=211498&package_id=265649&release_id=646757].

[jdc] 7-nov-2008: Fixed problems with passing font settings to Tk mode, fixed gui problem in Tk mode where a gap appeared between TOC and data fram when resizing the main window, added options to set port and command port in wub mode. A new kit is available for download [http://sourceforge.net/project/showfiles.php?group_id=211498].

[jdc] 3-jul-2008: Fixed problem if database was placed in same directory as the wukwikit.kit, a new kit is available for download [http://sourceforge.net/project/showfiles.php?group_id=211498].

[jdc] 26-mar-2008: A new wubwikit starkit is released [http://sourceforge.net/project/showfiles.php?group_id=211498] adding command line arguments to set font families and sizes used when used as [Tk] application.

[jdc] 13-mar-2008: A new wubwikit starkit is released [http://sourceforge.net/project/showfiles.php?group_id=211498] adding new command line argument `toc` to set table-of-contents in the specified database.

[jdc] 12-mar-2008: A new wubwikit starkit is released [http://sourceforge.net/project/showfiles.php?group_id=211498] solving the following problems:

   * `couldn't read file local.tcl` (reported by [MHo] 3-mar-2008)
   * `db.tcl: no such file or directory` (reported by [LV] on 12-mar-2008)

[jdc] 3-mar-2008: I packed [Wub] and wikit in one [Starkit].  It can downloaded from http://sourceforge.net/project/showfiles.php?group_id=211498%|%here%|%. This kit needs a 8.5 [Tclkit].

[jdc] 4-feb-2008: From wikit-20080204.tkd on, I'm running `sdx mkpack` on the database before uploading it.

[jdc] 7-dec-2007: The wiki database can be downloaded for offline use from http://sourceforge.net/project/showfiles.php?group_id=211498%|%here%|%.

**Usage of unzippped archive:**

 % tclsh wubwikit<version>/main.tcl <options>

**Requirements:**

   * tclsh based on Tcl/Tk 8.6
   * package [tdbc], part of Tcl/Tk 8.6
   * package tdbc::sqlite3 from http://tdbc.tcl.tk, version >= 1.0b16
   * package [sqlite]3, version >= 3.7.3

**Typical usage:**

***Create a new wiki database:***

 % tclsh wubwikit<version>/main.tcl mkdb mywiki.tkd title "My Wiki"

This will create a new wiki database and use the specified title as wiki title.

***Start a wiki:***

 % tclsh wubwikit<version>/main.tcl wikidb mywiki.tkd

***Start a wiki with a copy of the Tcler's wiki database:***

 % tclsh wubwikit<version>/main.tcl wikidb wikit.tkd

**Command line options**

***Basic options:***
 
+++
help  Show this message
wikidb ''<file>''  Set path to Wiki database. Mandatory!
mkdb ''<file>'' title ''<title>''  Create empty Wiki database.
mklocal ''<path>''  Create default Wiki `local.tcl` to configure your Wiki.
mkconfig ''<path>''  Create default Wiki config file to configure the Wub webserver.
mkfts ''<boolean>''  Add full-text-search table to specified wiki-database.
+++

***Configuration options:***

+++
port ''<port>''  Set port used by [Wub], default is 8080.
cmdport ''<port>''  Set command port used by [Wub], you can [telnet] to this port to interface with the webserver, default is 8082.
logfile ''<file>''  Set name of log file
local ''<file>''  Wiki config file to be used for your wiki. The output of the `mklocal` command is used as default.
config ''<file>''  Wub config file to be used for your wiki. The output of the `mkconfig` command is used as default.
+++

***Utilities:***

+++
util ids  Print one line per page with page-id, indication if ok or empty and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util html|markup page ''<page-id>'' ''?opath <path>?'' ''?html_ext <html-extension>?''  Print html or markup for specified page to file `<opath>/<page-id><html-extension>|.txt`.
util html|markup pages ''<file>'' ''?opath <path>?'' ''?html_ext <html-extension>?''  Print html or markup for each page specified with its `<page-id>` in `<file>` to file `<opath>/<page-id><html-extension>|.txt`. Put each page-id as first item on a separate line in `<file>`. The output of other util commands can be used as input for this command.
util stats  Print some statistics about the wiki database.
util binary_ids  Print list of image text pages. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util count_empty_text_pages  Count all empty text pages (content size <= 1).
util count_image_pages  Count all image pages.
util count_image_pages_unreferenced_by_others  Count image pages not referenced by any other pages.
util count_non_empty_text_pages  Count all non empty text pages (content size > 1).
util count_non_empty_text_pages_unreferenced_by_others  Count non-empty text pages not referenced by any other pages.
util count_non_empty_text_pages_without_references_to_others  Count non-empty text pages without any references to other pages.
util count_pages  Count all pages.
util count_pages_without_content  Count without content. This are pages which have a title but haven't got content yet, not even content with size < 1.
util count_text_pages  Count all image pages.
util empty_text_pages  Print list of empty text pages (content size <= 1). Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util image_pages_unreferenced_by_others  Print list of image pages not referenced by any other pages. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util non_empty_text_pages  Print list of non-empty text pages (content size > 1). Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util non_empty_text_pages_unreferenced_by_others  Print list of non-empty text pages not referenced by any other pages. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util non_empty_text_pages_without_references_to_others  Print list of non-empty text pages without any references to other pages. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util pages_without_content  Print list of pages without content. This are pages which have a title but haven't got content yet, not even content with size < 1. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util references_from_other_pages ''?page <page-id>? ?pages <file>?''  Print references from other pages to specified pages. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util references_to_other_pages ''?page <page-id>? ?pages <file>?''  Print references found in specified pages to other pages. Print one line per page with id and page title to standard output. The resulting output can be used as input for the `util html|markup` commands.
util sql sqlstmt ''<sql statement>''  Execute the SQL statement and print the results
+++

<<discussion>>

**Questions/remarks**

[MHo] 28-mar-2008: Sorry. Just downloaded the latest wubwikit.kit, and tried tclkit and tclkitsh. Only the first page looks correct after navigating to http://localhost:8080. Clickering every other link leads to an nearly empty page.... Tried local mode, too: clicking on the scrollbar again gives an error... [jdc] 04-apr-2008 I'm also seeing empty pages when I put the database and the kit in the same directory. I need to check what's going wrong there. Can you try with db and kit in different directories? About the local mode, did you add the `toc wub` command line options? [jdc] 03-jul-2008 Fixed to file location problem.

----

[LV] So, I download the wubwikit.kit from sf.net. I run it as listed above
to start it as a Tk application. What I see is the error:

======
couldn't read file
"./wubwikit-20080303.kit/lib/wikitcl/wikit/db.tcl": no such file or directory
   while executing
 "source ./wubwikit-20080303.kit/lib/wikitcl/wikit/db.tcl"
        ("package ifneeded" script)
        invoked from within
   "package require Wikit::Db"
        (file "gui.tcl" line 17)
        invoked from within
    "source gui.tcl"
        invoked from within
      "if { $wub } {
        lappend auto_path [file join [file dirname [info script]] lib] [file join [file dirname [info script]] lib wikitcl]
        lappend arg..."
        (file "./wubwikit-20080303.kit/main.tcl" line 28)
           invoked from within
        source "./wubwikit-20080303.kit/main.tcl"
           ("uplevel" body line 1)
                invoked from within
                "uplevel [list source [file join $self main.tcl]]"
======

After poking at it a bit, it looks like the problem is that the application
is changing directories and no longer seeing the virtual file system.
So, I invoke things as:

======
tclkit /tmp/.lwv/wubwikit-20080303.kit wikidb /tmp/.lwv/wikit.tkd wub 0
======

and now the wikit comes up.

----
[MHo] 2008-03-03: WinXP. Local mode works, but clicking on the vertical scrollbar gives an error. WebServer mode: port 8082 offers some internas. Tk-window is empty. Closing Tk-windows leavs process running. Starting with tclkitsh:
======
error @@View: can't rename to "_Vvfield": command already exists (-code 1 -level 0 -errorcode NONE -
errorinfo {can't rename to "_Vvfield": command already exists\n    while executing\n"rename [mk::vie
w open $db.$view] $vcmd "\n    (procedure "::View::init" line 9)\n    invoked from within\n"View ini
t v$view db.$view"} -errorline 9)
error @@View: can't rename to "_Vvview": command already exists (-code 1 -level 0 -errorcode NONE -e
rrorinfo {can't rename to "_Vvview": command already exists\n    while executing\n"rename [mk::view
open $db.$view] $vcmd   "\n    (procedure "::View::init" line 9)\n    invoked from within\n"View ini
t v$view db.$view"} -errorline 9)
log @@RESTART: Mon Mar 03 23:13:32 +0100 2008 'couldn't read file "D:/Home/Arbeit1/pgm/tcl/usr/bin/t
clkit8.5/wubwikit-20080303.kit/lib/wikitcl/wubwikit/local.tcl": no such file or directory' (-code 1
-level 0 -errorcode {POSIX ENOENT {no such file or directory}} -errorinfo {couldn't read file "D:/Ho
me/Arbeit1/pgm/tcl/usr/bin/tclkit8.5/wubwikit-20080303.kit/lib/wikitcl/wubwikit/local.tcl": no such
file or directory\n    while executing\n"source [file join [file dirname [info script]] local.tcl]"}
 -errorline 1)
log @@Listening on http://matthias:8080/ using docroot /tmp/wiki/docroot
======

[jdc] 4-mar-2008: I tested [Wub] and wikitcl on windows, but not this kit. When started with tclkitsh in webserver mode, it works? You can ignore the errors listed above. I'll fix the `couldn't read file` error, the others are related to the `dub` mode of [Wub].

About local-mode and the scroll-bar, that's a difference in TOC handling between the webserver and local mode. To have a TOC in local mode, you need to copy the contents of the TOC file (you can find that in the kit) into page 8 of the data base.

[LV] 2008 March 12: jos, I'm seeing the scrollbar problem in local mode on Solaris as well. Is there a way that, in local mode, the wiki local code could just do the copy itself? Otherwise, we have to make this change each week. [jdc] I'll check how i can automate this. [jdc] 2008-mar-13 Added possibility to set TOC.



----
[LV] 2008 March 12
Just wondering - is the .tkd update process automated or manual? That web page that you point to above for wubwikit shows a March 3rd update.
But there used to be a page somewhere on sf.net where the tkd file was being updated once a week.
I didn't know if that weekly update was a cronjob, or just a manual process. [jdc] it's manual and I don't find the time every week to copy the db over to SF. 
----
[LV] jos, what is the ramification of [sdx] mkpack for someone using the tkd file?
[jdc] `sdx mkpack` removes free space from a Metakit file, a starkit or a starpack. As such, the wiki database gets 40M smaller.
----
[CMcC] is thinking about a special purpose application which updates a local wikit.db by reference to the current wiki.  It would be a fairly simple modification of the Recent Pages algorithm.  I don't think I'd be bothered updating History.  Thoughts anyone?
----
[LV] Does local mode use of wubwikit maintain a history? Display a history?
[jdc] Using wubwikit with [Tk] interface does maintain the history but can't show it. Using wubwikit as webserver has the same features as on-line one.
----
[KJN] Many thanks for this - it is very useful to have an offline copy of the Wiki as a reference when away from an internet connection.  For that purpose, 95% of the value comes from having just the latest version of each page, and I would not care if the full history was not supplied.
----
[WJP] 2008-04-05: A crucial piece of information is missing from this page: how to use the data file.
[apw] describes on the [Wub] pages how to use the database offline.

[LV] The .tkd file is read by the wiki software itself, by [wikitool],
and possibly some other tools.

In the old days, you would just say something like:
 wikit.kit wikit.tkd

and a desktop version of the wiki would come up on your desktop.
----
[MHo] 2008-11-07: Just downloaded the latest wubwikit and database from the above link, done a few tests on WinXP:
   * In local tk mode, clicking on the vertical scrollbar still gives an error... that's not the case when '''toc wub''' is specified
   ** [jdc] 7-nov-2008, Wub and Tk mode handle the TOc differently. When running in Tk mode, you have to add your TOC to page 8 or use the '''toc wub''' option.
   ** [gasty] After unwrap the kit and edit a little the proc yscroll in file wubwikit20090319.vfs\lib\wikitcl\wikit\gui.tcl, I can run the wiki without '''toc wub''' and the scroll ploblem dissapear. [jdc] 9-jul-2009 Fix is part of latest release. Thanks!
======
        proc yscroll {args} {
            variable D
            variable toc
            eval $D yview $args
            if {$toc ne ""} {
              eval $toc yview $args
            }
        }

======
  

   * tk load lasts several (!) seconds, main window does not come to foreground (I use ''focus -force .'' one windows regulary)
   ** [jdc] 7-nov-2008, also when you just start-n-wait?
   * seems that font_ options are ignored
   ** [jdc] 7-nov-2008, could trace this bug, will be solved in next release.
   *** [MHo] 7-nov-2008: solved with new version
   * window resizing leads to ugly results, a purple block appears between toc and content
   ** [jdc] 7-nov-2008, could trace this bug, will be solved in next release.
   *** [MHo] 7-nov-2008: happens no more with new version
   * window width is flipping depending on the page
   ** [jdc] 7-nov-2008, can you give some examples?
   * Webserver mode: seems I had to request the starting page twice before it is served
   ** [jdc] 7-nov-2008, couldn't see this problem ,but I've only been testing on Linux yet
   *** [MHo] 7-nov-2008: tested with new version, again had to reload (only the ''first page'')
   **** [jdc] 7-nov-2008 odd; I'll try to test on XP.
   * Q: Where can I specify the server-''port''?
   ** [jdc] 7-nov-2008, you can't with the current version; I'll add this option.
   * Q: How can I gracefully terminate the server?
   ** [jdc] 7-nov-2008, use these commands (I will also make the telnet port configurable):
======
      % telnet localhost 8082
      ::exit
======
   * Webserver says on console:
======
error @@View: can't rename to "_Vvfield": command already exists (-code 1 -level 0 -errorcode NONE -
errorinfo {can't rename to "_Vvfield": command already exists\n    while executing\n"rename [mk::vie
w open $db.$view] $vcmd "\n    (procedure "::View::init" line 9)\n    invoked from within\n"View ini
t v$view db.$view"} -errorline 9)
error @@View: can't rename to "_Vvview": command already exists (-code 1 -level 0 -errorcode NONE -e
rrorinfo {can't rename to "_Vvview": command already exists\n    while executing\n"rename [mk::view
open $db.$view] $vcmd   "\n    (procedure "::View::init" line 9)\n    invoked from within\n"View ini
t v$view db.$view"} -errorline 9)
log @@RESTART: Fri Nov 07 01:27:33 +0100 2008 '' (-code 0 -level 0)
log @@Listening on http://matthias:8080/ using docroot /tmp/wiki/docroot
log @@Site LOCAL: '' (-code 0 -level 0)
======
   ** [jdc] 7-nov-2008, you can safely ignore these error messages.

([jdc] 7-nov-2008 I'll look into these issue in the next couple of days, thanks for reporting)

[MHo] 7-nov-2008: Another question: the whole *.tkd is copied to the temp directory where wub additionally creates some other files (*.css, etc.). What *.tkd is altered afterwords - the original one, or the copy? Why is the whole file copied?
   ** [jdc] 7-nov-2008, The copy will be modified, I'll check if it's possible to work with the specified file.
   ** [jdc] 19-feb-2009 The latest kit no longer makes copies and the database must be specified at the command line.
----
[RZ] Is there a reason why there are now two different locations, one for the database and one for the kit?
Having both pieces in one place is at least for me more convenient. [jdc] 19-feb-2009 I find the repository at google code more convenient than the one at source forge. I'll provide the kits at both places. This first release on google code needs more testing and needs Tcl 8.6 (including co-routines and TclOO) which is not widespread yet at this moment.
----
'''[gasty] - 2009-09-29 00:04:58'''

Hi. In my work we start using wubwikit to make an internal wiki. It's amazing!
We wanted a way to reference images and other files stored in the server's filesystem within wiki pages. 

After read some wub documentation and make a little of reverse engineering of wikit, I add the following (little) patches to a couple of files and works very well:
   * In "wubwikit20090806.vfs/lib/wikitcl/wubwikit/wikit.nub" I add this line to create a new domain:
======
 domain /_mydomain/ {File mydomain} root <path-to-files> expires "next week"
======
   * In "wubwikit20090806.vfs/lib/wikitcl/wikit/format.tcl" I add some code starting from line 1590:
======
  # link of the form [file:*] are interpreted as local files
  lassign [split $text ":"] type newText
  if {$type eq "file"} {
    set text _mydomain/$newText
  }
======
between
======
x {
======
and
======
  lassign [split_url_link_text $text] link text
======

Now we can put links to files using [[file:<filename>]]. If the file has extension gif, png, jpeg or jpg, then the image are showed inlined. 

The directory <path-to-files> can be a "shared folder", so we can copy files to it without the need to access physically to the server.

Files within sub-directories of <path-to-files> can be accessed as [[file:subdir/<filename>]], wich are useful to organize the <path-to-files> content.

I hope this can be useful to other wubwikit users!

----

[zz] When I use kit file, searching a word will error, and I use wub2012.vfs/main.tcl, it is normal when I search. Why?

======

f:\wikit>tclkit sdx.kit wrap wub2012.kit
f:\wikit>tclkit wub2012.kit wikidb 999.tkd
....
....
....
....
Error from thread tid0000012C
can't find package Http
    while executing
"package require Http"
1342704805530-179637mS log @@{Bogus watchdog over -1}

======

[jdc] Must be something wrong with the [auto_path] to look up packages in the kit version. I always run without wrapping up in a kit.

----

[How to create and clean up a Wikit .tkd file]



<<categories>> Tcler's Wiki | Wikit