AMG: This page lists to-dos, feature requests, wishes, etc. for the Wibble web server.
dzach 2011-3-30 I think wibble has to honor header "Connection: close" when requested and close the socket immediately. This happens e.g. when wibble is used as a proxy behind another server which only speaks HTTP/1.0 to the backend, in my case nginx [L1 ] (with a proxy module), and inserts the above header into the stream. Unless wibble closes the socket, nginx keeps the client connection alive (this side speaks HTTP/1.1) without sending anything and then times out.
A fast workaround is to replace in ::wibble::process:
# Flush the outgoing buffer. chan flush $socket
with:
# Flush the outgoing buffer.
if {[dict exists $request header connection] &&
[dict get $request header connection] eq "close"} {
chan close $socket
} else {
chan flush $socket
}which works but gives the error:
errorinfo: can not find channel named "sock8"
while executing
"chan configure $chan -translation crlf"
(procedure "getrequest" line 3)
invoked from within
"getrequest $port $socket $peerhost $peerport"AMG: Wibble currently doesn't support character encodings (neither Accept: nor Content-Type: charset); it's hard-coded to only use ISO8859-1. It should at least support UTF-8.
I think the zone handlers should be able to encode their responses however they like, so long as they correctly identify the encoding, then a general facility can convert encodings if necessary. Or I could standardize on UTF-8, but this becomes a problem when serving files from disk which may not be UTF-8. If the client will accept ISO8859-1 and the file is in ISO8859-1, it doesn't make sense to read the file, convert it to UTF-8, convert it back to ISO8859-1, then send it. Just use chan copy.
By the way: I don't like how HTTP crams charset into the Content-Type: header; I'd rather it have a separate header, so that I can parse and update it more easily. There's a lot not to like about HTTP. :^)
AMG, 2010-11-06: I enhanced header parsing so that charset and other parameters are available as dict entries. Nothing actually uses charset yet, but this was a necessary and very difficult first step.
AMG: The Wibble static file server doesn't send Content-Type:, so the client has to guess. (Internet Explorer second-guesses the text/plain Content-Type: anyway.) I think a contenttype zone handler might be able to fill the gap, but I'm not sure what heuristics it should use to detect the type.
makr (2009-11-19): A pure-Tcl solution could employ fumagic (fileutil::magic). tmag on the other hand is an extension built on top of libmagic (only available for Unix so far).
AMG: Sounds good, thanks for the pointer. So that's two possible approaches, and I can think of a third one: file extension matching. Since zones stack, it's possible (but untested) to override contenttype (or its arguments) for individual files, by declaring single-file zones. Anyway, I might try implementing all three, once I have made a separate Wiki page for extra zone handlers.
AMG, 2010-11-06: Now I have the separate Wiki page! See Wibble zone handlers.
dzach: Here is another one: Wibble at present throws an error if it receives a cross-site request [L2 ], [L3 ] with a "preflighted" OPTION method. A minimal cross site request implementation would accept silently the OPTION method (and the other HTTP/1.1 methods currently not handled by wibble) without throwing the error and present a simple cross site request interface to the browser (i.e. handle headers Origin and Access-Control-Allow-Origin), which could possibly be done by using the rawheader field in the request data.
AMG: I should write more zone handlers for sessions, Content-Type:, etc. This page is getting fairly long, so they'll be collected on a separate page of this Wiki. Also the current zone handlers are very bare-bones, particularly dirlist, and maybe could stand to be improved. Originally I wrote them to be as short and simple as possible so that they clearly demonstrate the concept without being cluttered with features.
AMG, 2010-11-06: Again, see Wibble zone handlers. There's not much on it at this point, but hopefully that will change.
AMG: I'm not necessarily thinking of caching entire pages; I wonder if I can be a bit more general than that. Perhaps I cache zone handler outcomes, using the incoming request as part of the key. (See memoizing.)
One tricky part is identifying the rest of the key, that is to say, all other inputs that play a part in determining the outcome. A closely related issue is identifying the irrelevant parts of the request. For example, if the socket name affects the outcome, there's no point in caching, since that changes all the time. Except for debugging, I can't think of a reason why the socket name would matter. Likewise a dependency on the peer's network name would impede caching, and sometimes that dependency exists legitimately.
I have to be very careful with this one, because caching can incur overhead that exceeds the cost of recalculating from scratch every time. In my line of work (real-time simulation), this is nearly always the case. For that reason I don't want to experiment with caching until performance becomes a problem and other optimizations cease to bear fruit.
AMG, 2010-11-06: [wibble::template] now caches the scripts it generates.
AMG: I really don't know how this is done...
jbr 2010-12-18 Here is a bit of code I tested a couple weeks ago, cribbed from A Server Template....
package require Trf
package require tls
proc wibble::secure { certfile keyfile args } {
::tls::init -certfile $certfile -keyfile $keyfile \
-ssl2 1 -ssl3 1 -tls1 0 \
-require 0 -request 0 {*}$args
proc ::wibble::socket { args } {
::tls::socket {*}$args
}
}AMG: Thanks. I guess I still need to research the meaning of all those options, especially $certfile and $keyfile.
AMG: I have yet to research the WebSocket page written by AGB. It may be worth incorporating into Wibble, or at least updating to match the latest version. I want to see how much of it can be made into zone handlers and how much of it requires modifications to the core.
AMG: (I don't really have a use for this at the moment, but I'll add it while I'm thinking about it.) The Host: HTTP header can be used to serve multiple "sites" from a single server process listening on a single IP address. Wibble can do this by prepending the hostname to the zone name. But I want this to be optional, with the ability to script the mapping from hostname to zone prefix. I mean, it would suck to require that the entire zone handler table be replicated for each of "example.com" and "web.example.com" and "www.example.com" and "example.com." and every other variant name of the same site. Also it doesn't make sense to pay for this feature when it's not needed.
jcw 2011-03-25 - It would be nice to be able to override the error handler in some standard way one day. The current ::wibble::process always generates a stack-trace, but sometimes you just want to redirect to a normal page , with perhaps an error box inserted (as well as logging and/or emailing the techie-info). Perhaps errors could be "caught" by a specially-marked zone handler?
AMG: The zone handler approach sounds interesting. The error handler exists outside of the zone handler loop (the I/O loop too), since it's there to catch my coding errors more than anything else, and coding errors can happen anywhere! I don't see a good way to reenter the loop and try again. Maybe instead have two levels of error handling, one to catch Wibble coding errors and one to catch zone errors. Then the error information can be placed in a special "error" zone (no leading slash means that it's distinct from all possible URIs). What do you think?
jcw - I can't quite wrap my mind around the different trade-offs, to be honest. Ideally, you'd want to be able to catch errors in different ways, i.e. various zone handlers may need specific error handling (an Ajax zone would not return an error page, for example). Then again, maybe the obvious solution is also the most flexible for such cases: try / on-error within the zone handlers themselves. Hm - I think a single settable handler might then be sufficient after all, perhaps simply a ::wibble::onerror proc to set the error handler? Or even simpler: define and call ::wibble::onerror - anyone can then redefine it, in the same way as ::wibble::log works.
AMG: That last sentence sounds good to me. If you like, you can make your [onerror] call selected zone handler procs directly, e.g. to put standard headers and footers on the page. Since you're customizing the code, and you want to have a simple path for error handling, dynamic dispatch isn't needed.
AMG: Insert feature request here.