** Introduction ** Wub is a web-server written in pure-Tcl. It requires Tcl 8.6. It should help in creating highly-dynamic (and portable) web applications. It is the successor of [Tclhttpd], which is (2008-07-05) not maintained anymore. [RLH] Why does wub require 8.6 when it isn't even out yet? [CMcC] because it depends on facilities which are only in 8.6, of course. [RLH] That is obvious. However, I would think it would be a very few that are going to run 8.6 at this point and so...can't use Wub. Of course, it is yours to do with what you will I just think that a shame. ** News ** 13Jan09: Version 4.0 of the Httpd server engine released. Now it's purely and only a [coroutine]-driven engine. Application Changes: Responder::post and Responder::do become ::Httpd::post and ::Httpd::do. ** Quick Start ** ---- svn checkout http://wub.googlecode.com/svn/trunk/ wub-read-only cd wub/Wub/ tclsh8.5 Application.tcl (there may, of course, be transient svn server outages.) See http://code.google.com/p/wub/source/checkout for details. Default comes up at port 8080. There's a control port at 8082 - tcl commands into running app from localhost only. ** External Dependencies ** Optional: [tls] for SSL ** Configuration ** Wub/Application.tcl is a demo application showing all available [Wub Domains]. It may be configured by reference to Site.tcl in the same directory. Wub/Minimal.tcl is an extremely minimal demo application which provides only a file system based domain. ** Wub Architecture ** Wub is a framework for writing web servers. It provides a collection of utilities enabling a programmer to assemble a range of useful dedicated web servers. While the programmer is free to assemble the components in any way they wish, some approaches are better supported than others, and this could be called Wub's architecture. Wub's Httpd module converts HTTP protocol interactions into tcl [dict]s containing all the request's protocol fields. These 'request [dict]s' are transformed into 'response [dict]s' by the user-supplied application using [Wub Utilities] and [Wub Domains] (Utilities/ and Domain/ respectively) and are returned via HTTP to the client. A request/response dict resembles a whiteboard upon which Domains and Utilities operate, reading fields left by earlier processing, creating and modifying fields as a result of their own operation. Most modules act as functions, taking a request dict argument and returning a response dict result. One convention which is rigorously enforced is that non-protocol fields in request/response dicts have names of the form '-*', distinguishing them from HTTP fields which will eventually be sent to the client. ** Dataflow ** Httpd package receives protocol transactions and generates a request dict. After performing some analytic transformations, such as Url and Query processing, User Agent processing and malicious client Blocking, Httpd (optionally) consults one of two Cache packages. If the Cache can satisfy the request, it does so. Otherwise, a configurable Incoming proc is called with the request. A sample Incoming is provided in the supplied example Application.tcl, which acts through the Response module to dispatch incoming requests to an appropriate Domain by means of a switch over some fields in the request dict - specifically and primarily the -url field which contains the incoming request URL. After Domain package handling, the returned response dict is queued up to be returned to the client via Httpd. In addition to the Cache module, there are two other filtering modules: Convert (which handles mime type content negotiation and conversion) and Session (which manages sessions) which can apply to the output from any other Domain. Domain packages provide application-level semantics to a web server. A range of example domains are provided. ---- Wub documentation is proceeding slowly. http:/_wub/ contains what will eventually be a discursive description with examples. http:/_doc/ contains a class/namespace level live comment-scrape from the source actually running what you're reading now. ---- ** Related ** [Wub Suspend] - how to suspend under Wub [Wub Consumer] - how to interact with HTTP clients via Wub - the consumer API *** Archeology *** Wub's complete rewrite is complete, and is running [this wikit]. (25Apr07) Wub's available here: http://code.google.com/p/wub/ under subversion control. Try grabbing the wikit code from http://code.google.com/p/wikitcl/ into a sibling directory, and running tclsh Site.tcl from that (wikit) directory. More later. [CMcC] ---- Wub was underperforming relative to Apache et al, so I've rewritten it all to be multithreaded and lighter-weight. The pursuit of a Lego-like plugging [little language] has been abandoned, under the following reasoning: (a) the audience for lego-plugability is probably not an audience that should be building websites anyway, (b) building nice things out of Lego is hard, (c) the overhead of maintaining a plug-together API was considerable although I did learn a lot about the new [return] functionality, (d) tcl is amply competent at parsing and dispatching URLs - which is all the plugin/domain model really had to offer, (e) multithreading made site writing easier (giving a thread-level execution environment), (f) multithreading made the simple domain plugging model non-viable (caching, for example, has to be in the top level dispatcher thread.) Preliminary benchmarks suggest a rough parity with Apache out-of-the-box. New caching architecture will make it easier to interact with a decent caching reverse-proxy if one's ever written. ---- If you consider [tclhttpd], take a look at Wub too. Wub will probably see more active development than tclhttpd in the future (as the wub author is/was one of the main contributors to the tclhttpd code...). Wub is a complete redesign of a http (nearly-1.1) webserver using dicts for state instead of all the global vars used in tclhttpd. Wub owes a kind of filial respect to tclhttp, as a few core techniques are derived from it, but I hope Wub will be more convivial for developers. The Wub architecture is designed to be highly modular. All protocol operations occur in two modules ''Listener'' and ''Httpd''. Once an [http] request has been parsed to a dictionary (and after possible further processing by ''Cookie'', ''Entity'' and ''Query'' modules) the request dict is transformed by a series of '''Domain'''s. I would like Wub to resemble a [lego] set of modules out of which one can assemble a pachinko or a pinball machine for handling http requests (the requests and responses are like the balls ... that's the metaphor). The following Domains (and more) are available: Session: handles cookie-based sessions Cache: provides caching of content File: reflects a filesystem to the internet Convert: converts content to an arbitrary [mime]-type using a [graph] of individual conversions, so xyz->html->pdf type conversions are possible. Mason: emulates a quite good [PERL] CMS-like system called Mason (http://www.masonhq.com/). Db and [Sql]: reflect a [metakit] and [sqlite] database (respectively) to the [internet]. [Http] query language is used to construct [SQL]-ish queries, and the results returned. Coupling this with the Convert module, one can generate [CSV] or [HTML] or [Sylk] or whatever form seems most convenient. (Not part of the kit distro - I'm still working on authentication of access - ask [CMcC] if you're interested.) [RLH] 2006-09-07: Is the Mason portion done? I would have thought that Template-Toolkit for Tcl would have been a better fit, but you probably have you reasons for the Mason version as well. Where did Wub come from? And lastly...if you need help give a shout out (docs, testing as I am on 3 different operating systems, etc.). [CMcC] RLH, Mason's done, yeah. '''What is Template-Toolkit for Tcl?''' Templating in tcl's pretty easy really, [subst] does the job very nicely. Wub the name? I just thought it was a lame-sounding variant on Web ... and there's the PK Dick story 'beyond lies the Wub,' which sounds good. [RLH] Is there any documentation on the Mason part? I am very interested in seeing the Tcl version (as there is a Python version as well). [CMcC] The parts of Mason I thought worth lifting, and acknowledging as lifted by the use of the name, are the wrapper and the notfound scripts, and the idea of searching up the file system hierarchy for candidates. I didn't see anything else worth emulating, but if there's something worthwhile, I'm happy to consider them. In answer to the question about documentation: what docco there is (and there is some) is to be found within the Wub distribution - start it running and look in http://localhost:8080/docs/. The source itself it (I hope) reasonably well internally narrated. [RLH] Just a note that 8.5 is required to run it. At least that is the error message I got when running it under 8.4. [GP-L] I downloaded wub.kit but the source of the pages is served. It seems that no content-type is returned... Lynx -head displays: HTTP/1.0 200 OK Date: Tue, 06 Feb 2007 22:16:58 GMT Server: Wub 1.0 connection: close set-cookie2: session="4 C6A76862EBC01B6542D3332C7A8B09AB 0";version=1 etag: 6D084DA7CBBF3F51674F9CC444FC7BC0 last-modified: 1170800126 vary: cookies set-cookie: session="4 C6A76862EBC01B6542D3332C7A8B09AB 0";version=1 and the log in the terminal says: [Tue Feb 06 23:17:53 CET 2007] [8080] [info] 'Send code: 200 reply: -code 200 accept {text/html, text/plain, text/sgml, */*;q=0.01} etag 6D084DA7CBBF3F51674F9CC444FC7BC0 -session 5 accept-language fr -ipaddr 127.0.0.1 -dynamic 0 -port 8080 -scheme http -Query {} -host xxxxx.xxxxxx -version 1.0 set-cookie {session="5 291F3BE767501A5C875EF7BCE19FFA58 0";version=1} accept-encoding {gzip, compress} -uri http://xxxxx.xxxxxx:8080/ -shandler ::session host 127.0.0.1:8080 -fd file14 -entry http://xxxxx.xxxxxx:8080/ -times {reply 158737238453437 read 158737233680374 dispatch 158737233919356 start 158737232717051 seconds 1170800273} -depends {/private/tmp/docroot /private/tmp/docroot/index.tml} -url http://xxxxx.xxxxxx:8080/ -rtype CachedContent -generation 4 content-type text/html content-length 3535 -path / -cookies {session {-expires {Mon, 06 Aug 2007 21:17:53 GMT} -value {5 291F3BE767501A5C875EF7BCE19FFA58 0} -path / -changed 1}} last-modified 1170800126 -continuations {} user-agent {Lynx/2.8.5rel.2 libwww-FM/2.14} -transaction 1 -listener ::Listener1 -rport 61612 -http ::Pool::Httpd1 -sock sock12 -dispatch_count 0 set-cookie2 {session="5 291F3BE767501A5C875EF7BCE19FFA58 0";version=1} -modified 1170800126 -method HEAD -vary {cookies 1}' ---- [RLH] 2007-02-19: Can you publish a tar.gz or zip with the source along with the kit? The source is already contained in the kit-file, simply UNWRAP the kit-file using http://www.equi4.com/pub/sk/sdx.kit, see [sdx] for further doco. [RLH] Yes, I know but I really don't like messing with kit files. [CMcC] Yes. Now it's under subversion (as detailed above) ---- [JAH] - 2007-02-20: Downloaded wub.kit and am running on OS X 10.4 with swisskit, which has the tcl8.5 engine. wub.kit grashes on startup with: Well, lots of error messages ... Basically swisskit has the 8.5 version of tcl, it does not have the mk4tcl database which wub needs. So, don't use swisskit to run wub. I finally found a compatible tclkit at http://www.equi4.com/pub/tk/downloads.html which does have the required version and mk4tcl database. End of round one. ---- [JAH] - 2007-02-20: Round 2: Problems running 'out of the box' with wub. It does serve pages, but does not convert the content-type text/x-html-fragment into text/html as it passes through the mason and expansion steps. Neither Safari, firefox, nor Opera likes x-html-fragment. Is there a 'logical' place that fragments turn into full fledged html? Is that in mason or somewhere else? [CMcC] 25Apr07 Found and solved that problem. Turns out that some browsers send the header 'Accept: */*' which means 'the user will accept any type at all', but of course that's not true. Now, Wub assumes '*/*' means text/html. There have been a lot of changes to get [wikit] up under Wub. I'll have to go back and fix the default Site.tcl file to reflect those changes. ---- [tedickey] - 2007-05-01 It delivers text/html, but lynx and w3m are not seeing the connection close in a timely fashion. ---- [RLH] 2007-05-02: I do not see anything on the Google site as far as "source" goes. [LV] As was mentioned about, you download the [starkit] version of wub, and use [sdx] to unwrap it. The source will be in the extracted directory. [MHo] Can't locate '''wub.kit''', too... [CMcC] There is a 'source' link on the google site given. The kit version is totally out of date. ---- [LES] 2007-05-31: How do you use it? I downloaded the source and there is no documentation, not even about how to run it. It '''looks like''' I am supposed to run ../wub/Wub/wub-daemon, but nothing happens when I run it. [CMcC] Yeah, rushed release - sorry. Try grabbing the wikit code from http://code.google.com/p/wikitcl/ into a sibling directory, and running tclsh Site.tcl from that (wikit) directory. [MHo]: Hmmm... I tried ''tclkit site.tcl''. That results in ''can't find package snit''..... [Vtk]: Several suggestions: 1. Documentation -- the major reason why projects die or never take off is the lack of documentation. I see you are using Snit in your code so you are familiar with the way Snit documantation is done. It is the best way to do it -- by giving working examples of each command/option. DO NOT do a man page type documentation. It is a waste of time and is NOT helpful. DO NOT tell people to read the source code as a way of figuring out how to use it. Some experienced people can do it but still it takes a lot of time and most users are not willing to do that. So, they leave in search of a better documented project (let's take PHP, for example. It is not the best programming language but the creator(s) had lots of documentation from the start and lots of working examples. That's the main reason (IMHO) PHP has become popular). 2. Lego type plugins/modules -- consider using XOTcl instead of Snit. It is more friendly for creating modular design (look at filters/mixins, for example) and is more advanced. Once you know Snit, XOTcl is easy to learn and converting your code from Snit to XOTcl is not difficult. I have myself invested a lot of time into Snit, to find out later that it was easier and shorter to write my code in XOTcl. Snit is a great OO language to start with (especially for learning OO way of programming), but XOTcl is a step up from there. Good luck and I hope your project doesn't stagnate like many other promissing projects out there. [CMcC] Thanks, Vtk, that's all good advice - particularly the idea of using examples. In the interim I've set up http://wiki.tcl.tk/_doc/ which rips namespace, class and proc/method level docs straight out of the code and formats it. That covers the man-style documentation with minimal effort. I'll think about examples in-line as well. [apw] 2007-12-21 - These are the steps I used to make a local wiki running: * download wub and wikittcl as described above * if you use the described svn commands rename directory wub-read-only to Wub * you need a tclsh8.5 with threads enabled * eventually you have to fix tcllib1.8/snit/snit2.tcl change all {expand} to {*} (best way to fix it is to upgrade to [tcllib] 1.10, released in Sept 2007). * download [wiki database for offline use] and copy it to wikit-read-only/wikit/wikit.tkd (I had problems with the link, so I used http://heanet.dl.sourceforge.net/sourceforge/tclerswikidata/ instead of http://heanet.dl.sourceforge.net/sourceforge/tclerswikidata/wikit-20071217.tkd and selected an available version with a different name) * edit wikit-read-only/wikit/WubkitWub.tcl about line 1884, comment out ''WikitRss init wdb "Tcler's Wiki" http://wiki.tcl.tk/'' otherwise I got an error from mk about page 0 not found * now run: tclsh Site.tcl (from wikit-read-only/wikit) * hopefully local wiki now runs, as it did in my case ... [Vtk] 2008-03-30 - Re: the steps to take to make it run by apw. The installation steps above are exactly what drives people away. If you have to go through all kinds of dancing to make it work, who knows what else is the problem. From my many experiences with other code, I don't feel too excited to install it. Getting old and tired of spending hours to (may be) make it work. And that "may be it will work" stops me right there. I would prefer the three steps: 1.download 2. unpack 3. run ;) ;) - I bet everyone would like that - that's why I like tclhttpd: download, gunzip, "tclsh httpd.tcl -debug 1" - is all I need to do. ---- [Sarnold] 2008-07-05 - At a first look, Wub runs on Microsoft Windows XP with the constraint that the path of the Tcl files does not contain spaces. In short, don't install Wub to C:\Program Files\ or C:\Documents and Settings\. The demo application seems to work, at least it is serving files. [Larry Smith] Or refer to it as Progra~1 [LV] Perhaps there are coding changes in Wub that would enable it to be free of this restriction? ---- fhou 2008-07-20 - The QuickStart seems to work fine for me. The mailing list ''wub-users@lists.sourceforge.net'' ("the best source of current information about Wub") does not seem to exist anymore. Is there another one, or is ''this'' the place? boune 2009-01-09 - Why not other wiki system? See http://www.wikimatrix.org/, there are dozens of good wiki systems. [CMcC] 2009-01-09 - Wub's the web server, not the wiki. Why another wiki system, something wrong with wikit? If so, what? Finally, there's some virtue to it all being written in tcl. ---- [EMJ] 2009-02-17 I'm writing an application that has [tclhttpd] incorporated, so I should look at Wub, yes? So... * Set up a side install of Tcl 8.5 (too much stuff on the machine to retest when I move off 8.4 - one of these days...) * Get and build Subversion - if I can't build it myself I don't use it on this server, period. (There are exceptions, but this is not enough to justify one.) Luckily, I don't have to actually install it to use it to ... * Get Wub into a suitable directory. * Try to run Application.tcl, find I need TclOO. * Get and install TclOO. * Now I have ''invalid command name "coroutine"'', does this mean I need 8.6 - can't do that, the application I'm writing is for someone else and I can't use beta releases! * Oh, well, I didn't want that evening anyway (I had a break to do some ironing - wow!). [CMcC] Yes, Wub uses [coroutine]s extensively now in Httpd.tcl (the protocol package.) I'm sorry that bit you, but if you saw how much simpler the code is, you'd understand. [MHo]: * Is there a chance to turn the server into a windows service with the help of [winserv] and to wrap it as a single executable [starpack]? In theory, it should not be more complex as with [tclhttpd], I guess... * Just tried to start wub with the tclkitsh.exe v8.6 from http://www.patthoyts.tk/tclkit.html - and get a '''can't find package fileutil''' error... is there a predefined place where I can copy the routines into or should I hack elsewhere...? ---- !!!!!! %| [Category Internet] | [Category Webserver] | [Category Application] |% !!!!!!