Wiki Announcements

18-jun-2010 Converted keywords as detected by the syntax highlighting into links to the corresponding Wiki page. Also changed the highlighting color for { and } back to black.

14-jun-2010

• Light gray background for code blocks
• Syntax highlighting for code blocks based on http://shjs.sourceforge.net/
  • Lars H: I officially hate it! It seems to be seriously wrong (if taken as hinting about how the code should be interpreted) at least as often as it is right, and that's when it's colouring Tcl code. We have a lot of code blocks that aren't Tcl code, and no markup to tell which is which. Please disable soonest possible.
  • stevel I don't like it either, and it was my idea originally :) The intention was to make the Tcl commands links to the relevant manual pages, to make it easier for noobs to see code then get details on the language. It's an idea worth pursuing, just not sure if the fugliness of the syntax highlighting kills it. Just to give some perspective, we have one person (Jos Decoster) maintaining the wiki markup code, and Jos tries to keep everyone happy. But with this level of resource, one of the ways that ideas are filtered is to try them and gauge the response, rather than attempt consensus before every change (and therefore stagnate). Mostly he gets it right, occasionally there's a good idea that turns out not to be in practice.
  • AMG: See the syntax highlighting page for a more conservative proposal, including a few examples. I wonder if this highlighting scheme would be less odious, to the point that we can live with it without needing to add Yet More Markup to turn it off and on. (We already have four different ways to do fixed-width text, five if you count the left column of the rarely-used option table markup.)
  • nscerqueira: Apart from the posted problems I like the syntax highlighting a lot. It turns the code more intuitive and easy to read. ( I thought that a positive thought was missing here;) )
  • AMG: I haven't decided if I welcome syntax highlighting on this Wiki or not, but I think it's worth a try. However, I am thoroughly unimpressed with the current implementation. See [L1 ] (derived from [L2 ]) for the definition being used: notice the misspelled keywords, highlighting of single quotes, lack of support for namespaces and arrays, and oversimplified detection of quotes and comments. It seems to be somewhat inherited from C, for instance look at the u, L, int8, etc. suffixes that numbers are allowed to have. I understand that the syntax highlighting engine has intrinsic limitations that can't be overcome by a clever syntax definition, but even given that I think there's a ton of room for improvement. See my Graffiti experimentation: [L3 ].
  • CMcC: I think syntax highlighting is a good thing, on balance. The existing implementation needs tweaking, yes. Also, I think its default .css is fairly high-chroma and ill-considered (highlight braces in red? bad ergonomics, IMO, as it draws the eye ... I imagine this has some bearing on peoples' initial aversion) The existing implementation has the considerable benefit that it is in-client, and able to target different languages. It seems to me that a shallow parser for Tcl in JS+regexps isn't a bad project to consider developing communally over time, and this isn't a bad venue for exposing it to contribution and successive improvement.
• The subtitle on line or word difference pages now has a link to the word or line difference page
  • AMG: The page titles are now a bit wonky: "Word difference between version 61 and 60 for Word" or "Line difference between version 61 and 60 for Line". jdc: fixed.
• Reworked history page so you can easily get a diff between non consecutive pages

1-feb-2010 A link to create a new page was added to the left hand menu. A link to revert a page to a specific version was added to the history table. Both new links are ReCAPTCHA protected using Wub's ReCAPTCHA domain.

14 dec 09 CETDST New markup to incorporate documentation written using doctools or tcl-nroff markup in the Wiki. Check the doctool and nroff sections on the formatting rules page for more information about the new markup. Page tclMIDI is an example of using doctools markup on the wiki.

AMG: Why is the tag spelled <<doctool>> and not <<doctools>>? Also, is this protected against [exit], [exec rm -rf /], extremely deep recursion, infinite loops, very complex expressions, etc.?

jdc There was no special reason to choose <<doctool>> over <<doctools>>. I had the impression doctools was refering to the group of tools in tcllib, not to the markup. All rendering is done in slave interpreters, and wiki text is not eval-ed. Input which would cause problems with the doctools tcllib library or with the tcltk-man2html.tcl will cause trouble with wiki too.

3 dec 09 CETDST Image history, reworked search results

Images can now be updated and history is kept in the database. A page can not change its type. Once a text pages, always a text page. Once an image, always an image.

The search results will now show images (rescaled to 100x100 pixel thumbnails) and is now showing the results in a table. Check e.g. https://wiki.tcl-lang.org/_/search?S=wub .

30 nov 09 CETDST In-database images

Images can now be store in the db. When editing a page for the first time, you can choose between uploading or editing. When uploading, you can upload an images. This image will be stored in the database and can be displayed in another page by using a wiki [link]. Clicking on the inlined image will bring you to the image page, where you can check the back references for the images.

Search works on both text and image titles. Images will be shown in the search results. Check [L4 ].

30 Nov 09 AEST Robot problem should be fixed now.

Cookie standard *really* sucks.

28 Nov 09 AEST we don't really think you're a robot. [L5 ]

There's a problem caused by my misuse of cookies which is incorrectly informing people that we think they are robots. We don't really. Although can anyone really be sure? I mean, what if robot technology was advanced enough that we are all simulations? Metaphysics aside: the problem will be fixed in the next few days. I think it's now fixed.

Meanwhile, shift-reload seems to clear it enough to allow you to access the functions you may need.

24 Nov 09 AEST Time to Grasp the Nettle.

The Wiki currently resides directly under the / URL. This has been a niggling problem for ages, but more recently has prevented effective caching of static content. I intend to move the wiki pages down to /wiki/, and expect this will ameliorate that problem (and others.) This will cause breakage of bookmarks, and some other disturbances. I will endeavour to redirect not-found links to the wiki, for reprocessing. I expect peoples' bookmarks, and google's cache, will adapt over time.

New: I've been asked to hold off, and reconsider this (which I'm happy to do.) Other possibilities present themselves, such as having a distinct hostname for non-page content.

escargo - Please don't break bookmarks.

CGM - Please, please don't break existing references. Many of us have posted wiki links as answers to questions on usenet, stackoverflow, reddit, etc. If a new potential Tcl user tries to follow one of these links and finds it broken they will just conclude that Tcl is dead.

21 Nov 09 AEST the Ooops, I guess we'd better commit then modset

• major refactoring of Wub
• major mods to Wikit: removed magic numeric pages, added types, added rudimentary permission system for admin tasks.

AMG: the front page links are now all broken. The /_/ part of the URL is getting lost. The problem is that the links in ADMIN:Welcome are relative, not absolute. I'd fix this myself, but the permission system is working. :^) CMcC not quite, but close. I changed the rewrite to a redirect, now welcome always comes up in the right place, and relative links work as intended.

27 Oct 09 AEST the TDBC sounds interesting modset.

• Wiki backend has been rewritten as an abstract API. Behind the API jdc implemented a new TDBC-based database. The current backend target is sqlite, but it should be easy enough to retarget it to any supported database. One virtue of this approach is that we get to provide a testbed for TDBC. Please report any issues arising. We're particularly interested in worst-case page save performance.

31 Jul 09 AEST the this should be fun modset

• Mainly for fun, created https://wiki.tcl-lang.org/_tiny/ which is a tinurl-like facility, I call it MiniscURL, which generates persistent unique small URLs from longer ones. You can use it if you like to generate URLs for use on the wiki.

29 Jul 09 AEST the let's try this new refchan thing modset

• per-ip connection is back. Using the new reflected channels facility. Limits of 20 connections, but that's a bit fluid. Any strange unexpected network behaviour may be down to this new approach.
• in the development, there are some bugs, we will be exercising and diagnosing them, some strange behaviour may be present. The specific behaviour appears as 'connection reset' on FF.

24 Jul 09 AEST the bots really suck modset

• per-ip connection limit of 20 - should be enough. Not yet working, backed out, investigating.
• performance instrumentation feedback - anything over 100mS is too slow, and is logged. Exceptions enumerated.
• finally found a way to detect bots pretty reliably. it requires cookies, so just accept 'em.
• bots are now denied access to expensive/slow operations as they don't need 'em and some bots don't obey robots.txt
• rss.xml sped up and cached.

20 Jul 09 AEST the why is this thing running so *slow* modset

• page caching - experimental
• new instrumentation on page generation (not visible to users, but nice to note :)
• simplified some page generation with concommittant speedups.

23 Jun 09 CETDST

• New markup to specify categories:

           <<categories>> Category A | Category B | Category C

• This will be rendered requivalent to:

           ----
           !!!!!!
           %|[Category A]|[Category B]|[Category C]|%
           !!!!!!

• Don't go updating pages where categories are specified in another way, this will be done in batch.

16 Jun 09 CETDST

• Link in left hand menu to add comments to a wiki page. You can enter a comment, which will be added at the end of the wiki page, preceded by an horizontal rule, your nick-name and a time-stamp. When the page ends with new style category specifiers, comments are insert right above that.
  LV so what's the philosophical difference between adding a comment to a page versus using the wiki's Edit link to just make changes in place like this? I'm wondering when one is going to be used versus the other. I know that I prefer to see people make corrections directly to the content as opposed to making comments at the bottom that leave me having to read through everything on the page to be certain that what I am reading is correct.
  DKF: The big difference is that you don't get edit conflicts if two people are adding comments to a page at the same time, unlike if they're both editing. Probably could do with some JS/Ajax cleverness so that we can see the page we're commenting on while writing the comment, but that's easier to add once the basic functionality is done (well, it is if you're JDC and know that sort of thing. :-))
  jdc Adding a comment has the same restrictions as a plain edit. Only it is added at the end of the page, in a fixed format. I see it as making remarks about the contents of the page, discussing about the content without clobbering it. Not as a replacement of fixing and adding content in place.
  ferrieux Agreed. One extra value I see to the fixed-form comment style is to deprecate the habit of writing a comment (or whole discussion) without a date.

22 Apr 09 CETDST

• More toolbar buttons on edit page (save, preview, cancel, help)
• Move preview and help up so they appear above the edit area

Jos, I tried to submit a bug report, twice, on the following problem, but the bug tracker input form is failing. Today I am seeing weird tooltip behavior when I edit a wikit page. I see one of two related things. I either see 2 tooltip labels containing the same text, but at different sizes, appear, or I see one label, and when I move the cursor a pixel or two that label disappears and the second one, with different size text, appearing.

I'm using IE 7.

jdc Larry, this is IE7 using the alt text of the buttons as the larger toolt-tips. I'll remove the alt text.

14 Feb 09 AEST

Some disruption likely to some functions for a few days.

• major Wub mods to provide http:/_nub/nubs and .ini based configuration
• edit preview not currently working (I screwed up the js :}) but will be fixed
• contemplating move to jQuery for javascripting needs

8 Dec CETDST

• CSS for media screen, print and handheld.
• Default for media screen is now with left-hand menu. Alternative CSS without left-hand menu for screen, can be toggled from footer, choice is stored in a cookie.
• Media print and handheld are without left-hand menu.
• Some space between general and page links in the left-hand menu.
• Made ribbons on /4 nofollow links.

24 Nov CETDST

• Activity indicator on Recent Changes which is a measure for the editing done on the page. The activity is shown as a gray ribbon. The longer the ribbon, the more edit activity on the page. Clicking on the ribbon will bring you to the new editing summary page.
• The edit summary lists the latest edits, most recent at the top, newly added lines in green, removed lines in red.
• Some browsers convert sequences of 8 spaces into a tab. This introduced a lot of fake edits. When saving, tabs are now converted into sequences of 8 spaces.
• The RSS feed now contains all changes, not just the latest one. The description field now also contains the newly added lines.

LV The activity bar is still a bit of a mystery to me. For instance, sometimes, an addition of a Category seems to result in a single block - like 21910 right now. But 21911 has also had a category (well, technically 2 categories) added, and it has a 5 block indicator. Is there some sort of expontial relationship, depending on the time span of the changes?

jdc The activity is calculated as the sum of (numberOfLinesChanged*numberOfCharsChange/(clock seconds-timeOfChange)) for all changes to a page for the last 10 days. The length of the ribbon is calculated as int(log10($sum*10000)).

5Nov08 CETDST Edit preview and extended help on the edit pages, markup for nested numbered/unnumbered lists (see Formatting Rules), all headers are now shown in the page-TOC, color visited links differently, top align items of +++ blocks, draw border around items in +++ blocks, make <h2> smaller than <h1> and bigger than <h3>.

21Oct08 AEST Experimenting with changes to the way saved pages are returned to the editor, removed a redirect which sometimes lead to stale (pre-edit) pages, and thence to confusion. It's a bit of a hack, because other copies of the page may still be served from cache, but using the Content-Location should invalidate those. ZB Could be possible to set different colour for "visited links"?

2Mar08 AEST Testing cookies on WAP devices - seems to work on normal browsers.

27Feb08 AEST Some problems with edit cookies - should be fixed now

21Feb08 AEST Once again, session experimentation in progress, some disruption possible.

17Feb08 AEST Experimenting with an entropy measure for calculating significant recent changes and RSS feed. For a while, RSS will probably be empty until it adapts to the new measure.

16Feb08 AEST After a lot of learning about cookies and caching, I'm going to be playing with sessions again. You may experience some discomfort.

29Jan08 AEST I've broken Table of Contents, trying to fix these cookies. It'll be offline until I can get back to it in a couple of hours. I need a break.

TOC is back, can be toggled. The changes were mainly performance motivated. If there are problems with the new cookies, please indicate.

28Jan08 AEST Trying to remove spurious cookies, some instability may be evident with table of contents, and possibly you'll have to re-login. If you're having problems, try clearing wiki.tcl.tk cookies.

22Jan08 AEST Sessions for Wikit, take 2.

21Jan08 AEST Tentatively: I think editing is working again. Found the bug - unexpected side-effect of an uplevel dict with. Bleah.

20Jan08 AEST Experimenting with new cookie/session code. I am moving all cookies to a more restricted domain (just /_edit) which should reduce bandwidth and round trips a little.

This has been quite difficult. I'm hoping it's back up now for editing. You will need to re-login.

17Jan08 AEST aggressively blocked some presumed bots: 196.203.126.23 and 80.116.113.53 which together consumed about 38% of total wiki bandwidth. If this is you, and you think you have a reason for this usage (random traversal of links per page, ignoring robots.txt) please contact me and we'll see what we can do.

17Jan08 AEST Made _toc caching more aggressive. Should improve latency and reduce bandwidth.

17Jan08 AEST EYE CANDY! https://wiki.tcl-lang.org/_stats/

escargo 16 Jan 2008 - GMT-600 - How much would the access numbers be reduced because of successful Varnish[L6 ] caching?

CMcC Not at all - these logs are from varnish. The downside is that malicious client blocking isn't working. The upside is I know, better, who they are.

16Jan08 AEST I'm trialling Varnish caching on the Wub. There may be some problems - specifically edits may not immediately show up until I get the interface right; this is under active test. Please be vigilant and patient. It is hoped that Varnish will provide high-speed caching and increase the speed of the wiki. Sometimes using C to solve a problem just makes sense :)

• Problem interaction with Spider Blocking solved. You may have seen Forbidden responses for a while.
• Caching and fetching after edit are uncoupled, so your client may receive a redirect after edit before the cache has processed the cache-invalidation request. This means you may not see your changes immediately. I've got to mull this over.

04Jan08 AEST The Wub distribution now comes with a small simple demo application (called Application.tcl, found in the Wub/ directory). Enjoy. Now I'm off to bed; it's 4am.

03Jan08 AEST Removed snit from some modules (turned them into namespace ensembles instead) with surprising apparent speedups. Simplified and generalized some internal interfaces.

02Jan08 AEST Modified server to block sites which have a large number of simultaneous connections persistently. Modified server to make block persistent. Client sites may be blocked for being obvious spiders, malicious hole seekers, etc. https://wiki.tcl-lang.org/_block will give the current list of blockages.

31Jul07 AEST made front end single-threaded, allowed a single front-end worker to handle multiple connections. Seems a little slower to me, but having multiple front-end threads handling multiple connections seemed fastest of all. Update: the wikit is currently running entirely within one thread.

16Jul07 AEST added some CSS button styling to the wikit, also added some sorted tables where it seemed useful. Please report issues here: [L7 ]. Also added a space [L8 ] for documenting Wub.

10Jul07 AEST Fixed bug preventing Lynx from saving. Tested.

9Jul07 AEST excluding MSIE from gzip content-encoding. It's totally broken. People should use FireFox.

8Jul07 AEST Implemented and deployed gzip content-encoding. Please be alert to any problems which may arise. Of course, the transfer speed ought to be much faster now. Update: Unfortunately there are some problems with MSIE, so gzip is turned off for now.

7Jul07 AEST Collecting core dumps now whenever there's a sigsegv. Upgraded to HEAD tcl8.5, metakit, tcllib.

6Jul07 AEST Added another Save button on editing. Really only added this to see if it works. :) It did.

4Jul07 AEST Caching was interfering with edit reporting and recent changes. Fixed.

3Jul07 AEST Found/fixed a utf-8 bug only triggered by Safari (as far as I can tell.) Removed general robot nofollow,noindex - the intention is that web spiders can now follow local (in-wiki) links, but not external links. Working on potential races at HTTP processing level.

21Jun07 AEST Fixed the conflict-with-self bug. Turned out one shouldn't return a 302 response to a POST. Returning a 201 now.

13Jun07 Thanks to the efforts of kbk and jdc, Wiki now has a functional in-db diffs module - you can see it in operation by using the Revisions links at the bottom of most pages (for example, see [L9 ].)

Happy Wiki Fun Plots:

A plot of last change date for page number for pages with content.

Vertical axis is unix-time (over 8 years of wikit.) Horizontal axis is wiki page number.

You can see some artifacts

• the empty line across the top is the great outage of 2007
• the big empty space in the middle is a lot of stuff which has been changed or deleted.
• the dark horizontal lines are probably wikignoming (which seems to be a seasonal event :)
• a significant number of pages are edited for creation, then left (that's the 'crust')
• What is that empty space from 6000-8000? (DKF: I think that's remnants from when the FAQ was partially-transferred into the wiki; it generated a very large number of empty/virtual pages, and then the person working on it was asked not to do that, so that many have never had content added to them.)
• And the empty space in the 19000's ???

This graph was created using plotchart with the following script:

package require Tk
package require Mk4tcl
package require Plotchart

mk::file open db [lindex $argv 0]

canvas .c  -background white -width 800 -height 600
pack .c -fill both

set s [::Plotchart::createXYPlot .c {0.0 22000.0 2000.0} {900000000.0 1210000000.0 100000000.0}]

$s dataconfig series1 -colour "red" -type symbol -symbol cross
$s ytext "Time"
$s xtext "Page number"
$s title "Page/last-edit graph"

for { set i 0 } { $i < [mk::view size db.pages] } { incr i } {
    lassign [mk::get db.pages!$i date page] date page 
    if { [string length $page] > 1 } { 
        $s plot series1 $i [mk::get db.pages!$i date]
    }
}

mk::file close db

A plot of number of residual changes per day:

This plot was created with the following script:

package require Tk
package require Mk4tcl
package require Plotchart

mk::file open db [lindex $argv 0]

canvas .c  -background white -width 800 -height 600
pack .c -fill both

set s [::Plotchart::createXYPlot .c {900000000.0 1210000000.0 100000000.0} {0.0 80.0 10.0}]

$s dataconfig series1 -colour "red" -type symbol -symbol cross
$s xtext "Time"
$s ytext "Page number"
$s title "Residual changes per day"

for { set i 0 } { $i < [mk::view size db.pages] } { incr i } {
    lassign [mk::get db.pages!$i date page] date page 
    if { [string length $page] > 1 } { 
        incr rd([expr $date-($date%86400)]) 1
    }
}

mk::file close db

foreach k [array names rd] {
    $s plot series1 $k $rd($k)
}

jdc Top 20 of most edited pages (from data snapshot on 10-jul-2009)

This table was generated with the following script:

package require Mk4tcl

mk::file open db [lindex $argv 0]

set edl {}

for { set i 0 } { $i < [mk::view size db.pages] } { incr i } {
    lassign [mk::get db.pages!$i date page name] date page name
    set ed 0
    if { [string length $page] > 1 } { 
        set ed 1
        incr ed [llength [mk::select db.pages!$i.changes]]
        lappend edl [list $i $ed $name]
    }
}

mk::file close db

set edl [lsort -decreasing -index 1 -integer $edl]

set cnt 0

puts "%|'''Number of page'''|'''Number of edits'''|'''Page name'''|%"

foreach edsl $edl {
    lassign $edsl page count name
    puts "&|[format %5d $page]|[format %5d $count]|\[$name\]|&"
    incr cnt
    if { $cnt > 20 } { 
        break
    }
}

escargo 10 Apr 2008 - This might be true, but is it useful? How about the 20 most edited pages changed in the last 30 days? Or, alternatively, the 20 pages edited most often in the last 30 days? (Those are not necessarily the same pages.)

The 20 most edited pages in last 30 days (from data snapshot on 10-jul-2009):

This table was generated with the following script:

package require Mk4tcl

mk::file open db wikit.tkd

set edl {}

for { set i 0 } { $i < [mk::view size db.pages] } { incr i } {
    lassign [mk::get db.pages!$i date page name] date page name
    if {$date < [clock scan "30 days ago"]} continue
    set ed 0
    if { [string length $page] > 1 } { 
        set ed 1
        for {set j 0} {$j < [mk::view size db.pages!$i.changes]} {incr j} {
            lassign [mk::get db.pages!$i.changes!$j date] date
            if {$date < [clock scan "30 days ago"]} continue
            incr ed
        }
        lappend edl [list $i $ed $name]
    }
}

mk::file close db

set edl [lsort -decreasing -index 1 -integer $edl]

set cnt 0

puts "%|'''Number of page'''|'''Number of edits'''|'''Page name'''|%"

foreach edsl $edl {
    lassign $edsl page count name
    puts "&|[format %5d $page]|[format %5d $count]|\[$name\]|&"
    incr cnt
    if { $cnt > 20 } { 
        break
    }
}