Version 11 of Tclhttpd Hints and Tips

Updated 2004-09-29 04:34:54 by CMCc

Scott Gamon explains how to expose SOAP and XMLRPC methods from tclhttpd at: [L1 ]. Also, exposing those same methods through REpresentational State Transfer, REST: [L2 ].


Look at Wikit configuration to see how to integrate the Wikit (with static page-access) into TclHttpd.


Prevent results of method calls and variable assignments from being output in a template - WJR Occasionally I have a need to set variables in the context of a template - session variables for example. The result of this action is output in the template. Some means to prevent this would be nice as I don't always want to show this result to a user.

schlenk Try 'package require html', 'html::set var value'. Should do what you want.

WJR - That works perfectly, thank you!

CMCc - Alternatively, wrap it in <!-- and --> or use the following form which is quite useful for whole blocks of code:

    [
      set var val
      return ""
    ]

WJR - Yes, I was using the former method previous to the html::set method. In a way it's a nice technique since it helps with debugging.

CMCc - it's worth noting that the new subst semantics allows you to use continue and break inside the [] as well, giving you more control over what's evaluated and rendered.


Direct_Url multithreading - TV: A mechanism or description of how to make multi-sink DirectUrls... A multi sink Direct_Url is a Direct_Url which somehow can serve more than on thing at the time. A Direct_Url links a procedure to a Url. The procedure returns the Html which is fed back to the requester of that page, which is very handy; but when a lengthy or stalling procedure is used as directurl, the whole server hangs, and certainly also there can not be two (semi) parallel requests for such a page, as there can be for files. Lets say one wants to slowly feed a video file with a directurl, than the whole server stalls until that request is completely downloaded!

CMcC I believe [Httpd_Suspend $sock] and [Httpd_Resume $sock] will do what you want. You suspend a socket and allow the server to process other sockets, you can later resume a socket and send a little more data, and so on. To find the current socket, use [Httpd_CurrentSocket] as you don't have the socket in a Direct_Url.

This should work. Of course you'll have to write your service in an event driven way, and generate the event yourself (perhaps with an [after]). This probably warrants a TclHttpd Threading page.


Analog LOGFORMATs for TclHttpd

(02-12-04) Here's the latest Analog [L3 ] LOGFORMATs for TclHttpd. These should work with TclHttpd 3.4.3+:

 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c - "%f" "%B")
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c %b "%f" "%B")
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c %b - "%B")

LOGFORMATs for Pre 3.4.3:

 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c - "%f" "%B" -)
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c - - "%B" -)
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c - - - -)
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c %j "%f" "%B" -)
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c %j %j "%B" "%j")
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c %j - - -)
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] - - - - - -)
 LOGFORMAT (%s - - [%d/%M/%Y:%h:%n:%j] "%j%w%r%w%j" %c %j - "%B" -)

Logging in general is much improved in the latest versions.

-WJR

SS: I wonder if Tclhttpd log files for default work with Visitors [L4 ]. If somebody will try Visitors against Tclhttpd logs please tell me if it works vanilla or with the Analog LOGFORMAT so that I can add it in the documentation.


LES on Mar 21, 2004: Before delving into documentation, one is likely to want to get started with TclHttpd right away, i.e. use the server and build a site. And at that, a Windows user may wonder: if I run it with wish, I get this widget floating permanently; if I run it with tclsh, I get this DOS console box floating permanently; how do I make TclHttpd run "invisible"?

PT 21-Mar-2004: My advice (without checking the sources yet) would be to start it with wish. Then you get a window. If you then also [package require dde; dde servername tclhttpd] you could withdraw the toplevel window and later on control it via dde. eg: [dde eval tclhttpd wm deiconify .]

LES I don't have Tcl Dev Kit. And I am on Windows 98. I think I'll just follow PT's advice, but hacking srvui.tcl directly instead.

schlenk I once just withdrew . and put an icon in the system tray with winico, that popped up a control panel for tclhttpd. Works just fine with Win 9x.


Does Threading to Work in Linux? - NB: Was the setuid/getuid linux pthreads stuff ever resolved?

CMCc Threading works in TclHttpd under Linux. Only sf bug relating to this suggests that there may be a problem with TclX and threading. yahalom removing 'package require TclX' makes threading possible on linux

The setuid/getuid problem in Linux pthreads is an artefact of what I think is a broken threading implementation under Linux, imposed by the evils of the posix threading model (don't get me started.) IIRC, Linux created a thread with the initial uid, a so-called 'master' thread which seems to mediate inter-thread communications and synchronisation. The problem arises when you change the uid and start up a new thread - the new thread gets the right uid, but it can no longer communicate with the master thread. I personally put it down as broken by design and forgot about it.

The only need I could see to change the uid is to run as root then surrender privileges, primarily to grant access to privileged ports (specifically port 80.) I found it more satisfactory, and safer, to map port 80 to port 8015 with xinetd (or ipchains) and run as user www-data.

BBW Yeah, CMCc is correct - the setuid() syscall only applies to the current thread, and Tcl creates a Notifier thread very early. So, if you do a setuid you end up with a Tcl thread that cannot usefully communicate with the Tcl notifier thread because that involves signaling deep under the covers. What AOLserver does is to fork a listener thread very early and let that run as root. The rest of the world does a setuid to something else, and then the listener thread uses OS-dependent facilities to pass an accepted connection over to the rest of the server. In short, to do this you need to have a fairly modified Tcl shell that does this. It sounds like the port mapping with xinetd/ipchains may be more feasible, but I haven't tried that.

CMcC 20040609 - Here's my iptables setup. It may well be sub-optimal ... someone should come up with a tcl wrapper for iptables :)

    # map local port 80 to local port 8015
    iptables -A INPUT -j ACCEPT -p tcp --dport 80
    iptables -A FORWARD -j ACCEPT -p tcp --dport 80
    iptables -t nat -A PREROUTING -p tcp -d $EXTERNAL_INTERFACES --dport 80 -j REDIRECT --to-ports 8015
    iptables -t nat -A PREROUTING -p tcp -d $INTERNAL_INTERFACES --dport 80 -j REDIRECT --to-ports 8015
    iptables -t nat -A PREROUTING -p tcp -d lo --dport 80 -j REDIRECT --to-ports 8015

Content type and file extension -- CMcC 20040929

It should be realised by users of tclhttpd by any web author, really, that the file extension (.htm, .html, .pdf, .exe, whatever) bears only a passing relationship to the type of file sent to a browser, and to how the browser will process it.

Conventionally, .html means the document represents a file with mime type text/html, but this is actually determined by the HTTP protocol header Content-Type whose value is expected to be a mime type.

It is quite possible, in other words, to have a file named fred.pdf which contained a HTML file, and deliver it to a browser as an HTML file, and have it processed as an HTML file, merely by sending the appropriate Content-Type. The converse is also true. In fact, MSIE had a security flaw in which a file named fred.html could be an executable, and would be executed (I think they checked permission to render on the .html file suffix, not the actual content type.)

Why does this matter?

tclhttpd takes care of the conventional mappings of .suffix to mime-type, so normally a .html file will be returned with the expected type. However, it does this by means of a file mime.types in the lib directory, and this mapping is limited. You may need to extend the table from time to time, or you can directly register a mapping via [Mtype_Add $suffix $type].

tclhttpd also provides several ways to dynamically determine the content type of a file or data it's about to return:

  • [Httpd_Return*] procs all take a type argument, which usually defaults to text/html
  • Templates can set the element data(contentType) to their preferred content-type.
  • [Doc_$type] commands call [Httpd_Return*] directly, so can specify their content-type.
  • Each Direct domain can set its content-type by storing it in a per-proc global variable, e.g. [set Device/a/b/c text/plain] for a direct domain Device and procedure /a/b/c.

It should also be noted that browsers (in theory) negotiate permissable content type with the server by specifying an ordered list of the mime types they are prepared to handle.

In practice, this is much less useful than it might be, because most browsers claim to be able to support anything, with a default acceptable mime type */*. You're not invalidating any standards by returning an application/postscript file in response to a request for index.html (for example) although I'm not necessarily recommending it :)


Category Tclhttpd