Version 45 of Wibble discussion

Updated 2011-04-01 19:42:18 by dzach

AMG: This page is for general discussion on the subject of the Wibble web server. For discussion of bugs, see Wibble bugs. Older discussion can be found in the Wibble discussion archive.

Fetching backrefs...

htmlgen + wibble

jblz: UPDATE 1-1-11 (HAPPY NEW YEAR!)

I may move this to its own page if i end up using htmlgen in a serious way, but for the meantime, this is how to use htmlgen with wibble:

  1. Download xmlgen from sourceforge
  2. Decompress the directory, and place it in a directory listed on your $::tcl_pkgPath
  3. Because htmlgen creates a proc named "script" (for the "script" html tag, of course), you will need to rename the "script" zonehandler to something like "ascript"
  4. Create a file such as the following in your wibble $root directory, with a name like foo.html.script
# this is a reimplementation of the code in the /vars zonehandler
# adapted to using htmlgen

package require htmlgen
namespace import ::htmlgen::*

dict set response header content-type text/html

::htmlgen::buffer ::DOC {
    html ! {
        head ! {
            title - Playing Vars
            style type=text/css - {
                body {font-family: monospace}
                table {border-collapse: collapse; outline: 1px solid #000}
                th {white-space: nowrap; text-align: left}
                th, td {border: 1px solid #727772}
                tr:nth-child(odd) {background-color: #ded}
                tr:nth-child(even) {background-color: #eee}
                th.title {background-color: #8d958d; text-align: center}
            }
        }
        body ! {
            div id=wrapper ! {
                table ! {
                    foreach {dictname dictvalue} $state {
                        if {$dictname eq "request"} {
                            set dictvalue [dumprequest $dictvalue]
                        }
                        tr ! {
                            th class=title colspan=2 + {
                                [enhtml $dictname]
                            }
                        }
                        foreach {key value} $dictvalue {
                            tr ! {
                                th + {[enhtml $key]}
                                td + {[enhtml $value]}
                            }
                        }
                    }
                }
            }
        }
    }
}
    
dict append response content $::DOC

A few points worth mentioning

  • the default zonehandler sends .script files as text/plain, so make sure you change the "response header content-type" to text/html
  • because we have to send the finished page to the response dict, you'll need to use ::htmlgen::buffer to capture the output

Thanks to kbk in tcler's chat for helping me with wrangling htmlgen.


Cookie Convenience Procs

jblz: These procs are taken from the ngci package, and modified for use with wibble.

proc ::wibble::cookie {cookie} {
    upvar header cheader
    if {[dict exists $cheader cookie $cookie ""]} {
       return [dict get $cheader cookie $cookie ""]
    }
}

proc ::wibble::setcookie {args} {
    upvar response cresponse
    array set opt $args
    set line "$opt(-name)=$opt(-value) ;"
    foreach extra {path domain} {
        if {[info exists opt(-$extra)]} {
            append line " $extra=$opt(-$extra) ;"
        }
    }
    if {[info exists opt(-expires)]} {
        switch -glob -- $opt(-expires) {
            *GMT {
                set expires $opt(-expires)
            }
            default {
                set expires [clock format [clock scan $opt(-expires)] \
                    -format "%A, %d-%b-%Y %H:%M:%S GMT" -gmt 1]
            }
        }   
        append line " expires=$expires ;"
    }
    if {[info exists opt(-secure)]} {
        append line " secure "
    }
    dict set cresponse header Set-Cookie $line
}

Right now i can't seem to add more than one cookie per request. Changing dict set cresponse header Set-Cookie $line to dict lappend cresponse header Set-Cookie $line does put two Set-Cookie dicts in the response dict, but only the last one seems to be recognized by the browser. Any edits by an http guru to support multiple cookies per request would be super :).

Come to think of it, this may be a limitation of using a dict. Maybe only one pair of key/values named Set-Cookie are possible.


jblz 2011-1-03

Here is some proof-of-concept code for sinatra-like url-routing within wibble:

# zone-handler for sinatra-like routing
proc wibble::quibble {state} {
    dict with state request {}; dict with state options {}
        
        GET /foo/:bar {
                dict set response content "<h1>:bar is [dict get $data bar]</h1>"
        }
        
        GET /foo/:bar/baz/:fozz {
                append res "<h1>:bar is [dict get $data bar]</h1>"
                append res "<h1>:fozz is [dict get $data fozz]</h1>"
                dict set response content $res
        }
}

proc ::wibble::qrouter {mthd dpath body} {
        upvar state cstate
        upvar response qresponse
        upvar data rdata
        
        dict with cstate request {}; dict with cstate options {}

        # check if the HTTP method is correct
        if { $method ne $mthd} return
        
        set def_path [lrange [split $dpath /] 1 end]
        set def_length [llength $def_path]        
        set req_path [lrange [split $path /] 1 end]        
        set req_length [llength $req_path]
        
        # check if the number of elements match
        if {$def_length != $req_length} return
        
        set data {}
        foreach d $def_path r $req_path {
                if {$d ne $r} {
                        if {[string index $d 0] ne ":"} {
                                return
                        } else {
                                set varname [string range $d 1 end]
                                dict set rdata $varname $r
                        }
                }
        }
        dict set qresponse status 200
        dict set qresponse header content-type text/html
        uplevel 1 $body
        sendresponse $qresponse
}

interp alias {} GET {} ::wibble::qrouter GET
interp alias {} POST {} ::wibble::qrouter POST

Fossil

AMG: Several folks have suggested that Wibble has outgrown its current ad-hoc revision control and issue management system, which would be this wiki. I mostly agree. The current system requires a lot of manual labor, and I don't often have the time. Also I would like to expand the discussion, examples, documentation, etc. without feeling guilty that I'm hijacking somebody else's wiki. But I'm conflicted; there's a lot that I like about hosting Wibble on this wiki. In particular, it's very open; anyone can make changes and comments without getting an account or using a special client or anything like that. It's popular, it's maintained, it looks nice, it has a couple spiffy features, there are many great local pages to link to, other pages link back to Wibble, you can search page contents as well as titles, etc.

Fossil comes closest to the current system, and I am seriously considering it. In fact, I've already made a couple abortive attempts to import Wibble into a Fossil repository. The main thing that's stopping me is Fossil's markup interpretation for commit and ticket messages. If I could get a decoration mode instead of a markup mode, I'd be happy, but the current system requires a lot of work in order to get correct formatting (e.g. not losing square brackets, which all by itself is a deal-breaker). Also those workarounds break the command-line interface in order to get the common web interface pages right, and I will have to undo them again later when I finally get the decoration mode I need. (Tcl/Tk's own Fossil import has the same issues that Wibble does.)

Fossil has a built-in wiki which is nice for hosting discussion. I just played with the Fossil wiki a bit. It works, but it's as minimal as can be. However, it does support most HTML, so tables and such are possible. All in all, I'd prefer a few extra features, mostly to ease importing the pages and history from this wiki into Fossil. For instance, italic text and bold text are bracketed with <i>...</i> and <b>...</b> instead of double and triple apostrophes, respectively. Likely I will have to write a script to convert the markup. Also there's no double-bracket rule, so inserting brackets verbatim (which will be a very common task) requires that the brackets be spelled "&\#91;" and "&\#93;" (without the \'s; I had to insert those to dodge a bug in our wiki [L1 ]), or "<nowiki>[</nowiki>" and "<nowiki>]</nowiki>", which is even uglier. In fact, I think the lack of a double-bracket rule is also a deal-breaker, the same as the message markup problem described above.

Fossil also has a versioned documentation system, which makes it easy to get the documentation that matches any particular version of the code. That's a definite plus over the current ad-hoc system Wibble currently has, where you have to do some sleuthing to find the right version of the documentation pages. Of course, I haven't been good about updating the documentation anyway. ;^) But if I could version the documentation alongside the code, I would definitely make the effort to keep them in sync.

The Fossil wiki versioning is separate from the code/documentation versioning. They're on the same timeline, which makes correlation possible but not automatic. But they're still essentially separate. This creates a problem. I'd like for the documentation, examples, etc. pages to be in the wiki, open to editing and discussion. But for me to version them with the code, they have to be files. The only thing I can figure is to have manually synchronized parallel copies, both as files and as wiki pages. The wiki pages will only be regarded as documenting the latest version of Wibble, whereas the files are authoritative for whichever version of Wibble they came with. This seems like I'm trading one problem for another. Right now I have to tie all the pages together manually, but that's easier than manually copying everything between the wiki and files. I don't have to change the formatting of the files, but I might still want to, since if they're wiki-formatted then the user will need Fossil to properly read them.

I think what I need is a tool to extract a snapshot of the wiki, which I can commit as files. Maybe this tool can also prerender everything into HTML. If I am to open up the official documentation pages for editing and commenting, there will be discussion interspersed with normative verbiage. Most of the time when reading documentation you will want to skip the discussion, but it can be valuable on occasion. So it makes sense to be able to specially bracket the discussion and show/hide it using JavaScript/CSS trickery or simply by exporting two copies of each file, the official version and the annotated version.

Suggestions? Comments?

escargo 2011-03-19 - OK. These comments are from somebody sitting on the sidelines, but maybe they will be helpful, or at least thought-provoking.

Some of the problems you are having sound like features missing from Fossil. Perhaps they can be added. Where you say, "I will have to write a script...," that sounds like where a feature needs to be added to Fossil. Alternatively, maybe there needs to be a feature added to this wiki that makes exporting easier for other programs to import, an exchange format almost.

Where you mention, "manually synchronized parallel copies," that sounds like a need for a function that can link file and wiki page contents. (If I recall correctly, Fossil is really storing things in a data base, and probably exporting them or serving them as files or pages respectively as required. That would mean making Fossil understand some embedded query string that tells it to provide some tagged version of data as an exported file or current wiki page, if I understand correctly.) Perhaps the way to proceed is to enhance the underlying software so that moving to Fossil is less painful and that Fossil would better support how you need it to work. Your requirements might not be unique.

dzach: What is the purpose of this line in ::wibble::process?

 } elseif {[dict exists $response content]} {
   dict set response content [encoding convertto iso8859-1 [dict get $response content]]

Content that is already in iso8859-1 is not converted. Content that has been already converted to utf-8 by a user defined zone handler cannot be converted back to iso8859-1 with this code.