Poorly written documentation, indeed no documentation at all, can turn completely unusable the best programs or packages. In what file format should documentation be written: man, chm, html, xml, PDF, or something else?
LV For presentation of scripting reference material for extensions and at least simple application references, if to be used on Unix, documentation in "man" format needs to be able to be generated.
For presentation of more detailed application doc, I prefer html over PDF simply because it is generally easier for me to read the docs that way.
Another potentially useful format is the wikit format; starkits have been making use of a wikit for help information for some time.
As of 24Nov2003'', Dr. Detlef Groth has presented a help viewer based on Tkhtml's widget and the "hv.tcl" demo code to the starkit mailing list. This starkit is simply a wrap-up of a directory full of HTML files, presented to the user with table of contents and index pages - much like the Windows help viewer. As an example kit, he presents http://goblet.molgen.mpg.de/dgHelpBrowser.kit with 3Mb (compressed) of docs for Tcl/Tk and many extensions.
However, the Tcl doctools format is pretty good for preparation purposes.
About documentation tools: editors, screenshot utilities, i18n...
The simpler the better, for editors.
kroc According to me, a perfect doc generator would be:
Something based on expand maybe?
About content itself: What is desirable to do and what to avoid to write a good user manual?
Please give here your opinion, your own choices and why.
RS: Like many of us, I document less than I code. Luckily, I rarely have to deploy to nonexpert end-users. But even running my own apps after some time, I like concise online help best, e.g., htext, and sticking to accepted standards, like having the Exit command at the bottom of the File menu...
But if it has to be documentation that can also be printed out, I prefer HTML as the most portable, even to PocketPC, editable, simple, yet decent in printout. Oh, and especially pages at the Tcl'ers Wiki - most easily reached from any online box :)!
LES Nov 25 2003: I find the PDF format one of the most offensive things ever made for computers.
RJM finds this sounding quite negative. I feel free to emphasize that pdf is intended as a platform independent printable document format, where printers and pdf screen rendering programs will exactly represent the document regardless which system is used, at least when fonts are also included. Please do consider the fact that pdf document keeps pictures and text always according to the author's intention (e.g. on the same page), where printed html may yield unclean picture page rollovers. In one word: HTML and pdf have different targets.
Plus: the pdf format might have been contributed to better proliferation of non-MSword textprocessors when niche-type textprocessors can output pdf. Or in other words: MS-word ".doc" type document exchange is bad practice, since it supposes everybody having MS-word (or a viewer).
kroc As I've found nothing really adapted for it, I planned to do it myself.
Purpose: Create a very simple editor for end-users manuals (let's us call it Tkdoc). This editor itself will use a tcl-like syntax that will be parsed on output to produce html. The parsing will be done according to predefined macros (see below).
Goal of Tkdoc: Make writing end-users' manuals easier and quicker. Make manuals easier to translate, to update and to correct. Finally give a kind of uniformity for look and feel of tcl and tcl apps manuals.
Functions envisaged: Very simple GUI with an edit window (notebook based). Menu with standard file operations, "parsing" button, "preview" button and "embed image" button.
Tkdoc file format: A single macro+text file with screenshots, images, data at the end. At the point, predefined macros could be:
Please feel free to add your own ideas before I start it, so the result could be as good as expected for everybody.
Lars H: I like the command substitution like syntax, but it brings a couple of questions to mind. Is this what expand uses? Are there any other names being used for the same idea? (I've often felt the need for an introduction that explains the various formats and the communities in which they are used, but I haven't found any.) What projects currently have documentation marked up in that way?
A nice thing with using a Tcl-like syntax for markup is that you don't have to worry about making it so crude that some things are impossible (or nearly impossible) to write -- the Wiki syntax has some dusty corners in that regard -- because the Tcl syntax has already been tried in actual programming and has been shown to make do for almost everything. Also, you might want to consider supporting also backslash substitution, since a simple \uXXXX is a reliable way of encoding exotic characters (in people's names and such) while keeping your source 7-bit clean.
As for the actual macros suggested, stylistic variations such as italic, bold, and underline (underscore is a character) are probably a mistake. What would be needed is higher level markup, such as emphasis (which would probably be rendered as italic) and computer text (known as "typewriter" in the TeX world, probably best rendered as monowidth text where that is an option). The reason for this is that different output formats have very different capabilities. Printed text/PS/PDF handle bold, italic, monowidth, and many other things beautifully, but underlining is usually a mess. Character-based terminals may OTOH handle underlining well, but fail miserably with italic. By designing the markup to convey what you mean rather than what you expect to see in your favourite output format, the documentation becomes more likely to render well into a broad variety of output formats.
The [index] thingie also looks a bit wrong; it seems you are rather thinking about a table of contents than about an index. There's nothing wrong with a [tableofcontents], but one should also think about indexing, because that is a very valuable kind of markup. The meaning of
should rather be to expand to nothing, but also make a note that at this particular position in the document there is useful information about the string. For large projects whose documentation is split on several files, an index with cross-references covering the entire project is an efficient way of finding the information one is looking for.
LV What is the difference between a level 1 and 2 chapter? Are you meaning the difference between a chapter title and a section title? link says that it's argument can be a section - what does that mean?
I myself would rather see use of the italic/bold/underscore type things minimized and instead have semantic tags. That would make things easier to to translate to xml.
Are the macros mentioned above the doctools names, the TMML names, or something else?
kroc 26 May 2003 It's something else and, once again, it would be better to use doctools existing tags describe here : http://tcllib.sourceforge.net/doc/doctools_fmt.html and evaluate what must be added to write a descent end-user manual.
Michael Schlenker Tkdoc looks very much like doctools, but probably with a slightly different focus. Could those efforts be merged? There is not yet a GUI editor for doctools would be a good way to start.
AK: I would welcome a GUI editor for doctools. This could start out separately, but ultimatively this should be made a subcommand of dtp (IMHO).
AK: Regarding Tkdoc, it is quite similar, and thus a bit of a redundant effort. From my POV we can use parts of doctools (call, arg, method, option, ...) to describe a procedure and place that as comments in the code, either before a procedure, or inside. Then we just need a tool to extract the information and weave it together into a complete manpage. See also autodoc for such a tool, except that it uses it own markup for the embedded documentation (it predates doctools by some years).
kroc: All this is more adapted to tcl developers' manpages than real end-users' manuals Tkdoc is suppose to produce.
ro It's a waste of time to create yet another format. Instead of documenting, you will be coding (a common pitfall). Just start documenting now, in whatever you know (roff, html, LaTeX...). You can always hack it into another format later. Docs now is better than perfect docs later.
AK Note that doctools as base format makes conversion to Latex, nroff, html easier. IOW, not hacking into a different format later, but automatic conversion without hassle.
LV Over the years, people have suggested various SGML or XML/DTD solutions to documentation. The problem with many/most/all (??) of these has been that while getting the words of the documentation is awkward enough, then the effort of adding appropriate formatting results in a loss of focus. If one is formatted as they write, as much time is spent worrying about the formatting as is about the content. If one attempts to format afterwards, the resulting text becomes difficult to edit. HTML, Tex, nroff and PostScript are likewise an increasingly difficult series of formatting languages.
Some would argue that MS Word resolves this - the text can be input directly, editing can, in general, remain focused on content. However, the problem with Word is that it may work well for Windows platform users, but is not easily generated and maintained on other platforms.
What do other language communities use? Perl has their '''Plain Old Documentation" format (aka POD).
NEM 25May2003: I'm personally quite fond of LaTeX these days for writing documentation (and dissertations). I tend to just write the absolute minimum of formatting, just defining chapters and sections. Generally, LaTeX will then produce a pretty good quality output from this bare minimum. It's then a relatively quick (and quite fun!) job to add in specific formatting commands where needed. Also, importing a few packages here and there can make a really professional result. The big advantage that LaTeX has for me (over things like Word) is that the authors of it obviously knew a lot more than I ever will about professional typesetting. The results are generally excellent, and take account of so many things I wouldn't think of (like the fact that humans find reading lines of greater than 66 characters difficult). LaTeX also has the advantage that you can define new commands and environments to suit the current task (much like Tcl!). The basic idea is to leave all formatting commands out of a document, and just define the structure. Works well.
RWT 24Nov2003: I also have just completed a set of online documentation for an application. It consisted of a technical manual and a users guide, instead of simply man pages. I also needed images (screen shots) in the users guide, and the technical manual is rife with equations. LaTeX turned out to be the best input format. doctools looked good, but didn't have images. For printable versions, I used pdflatex to get PDF files with TeX's famous beauty. For online versions, I used latex2html to generate HTML with table of contents, list of figures, internal reference links, and a navigation panel on each page. From the application help menu, I launch a web browser or (if that fails) the "hv.tcl" hypertext viewer that comes with Tkhtml.
kroc: As AK wrote above, Tkdoc also could be a simple dtp frontend. Of course, I'll do that with pleasure instead of starting from scratch. But in this case, I'll need to add/remove some doctools tags (like image support, restricted nested levels, etc..). Moreover, AK more or less planned to update doctools tags. Maybe we could define here what should be kept, add or remove for Tkdoc particular purpose?
LV What did you have in mind that would need to be removed from doctools?
kroc I don't really need to remove something to doctools. In fact, it will be enough for Tkdoc purpose to use only a limited set of doctools tags. The less choices the writer will have, the quicker the manual will be written. I think the list I wrote above is enough to write a comprehensive end-user manual. We also can consider a full doctools support for tcl-aware user manual.
kroc 27 May 2003: What should be present in an effective help system (a beginner point of view).
Help have to be found easily. Here we can list three distincts kinds of help:
Online help - One or few html pages to explain:
Command line help (pure Tcl only) - at least the two following:
Contextual help (Tk only) - at least one of the three following:
Printable detailed help - should be shipped in application/package/library archive, if possible using a platform standard help file format (manpage for unix, html or chm for windows, etc..). It should be the most detailed part of the help system. Globally, it could be the same content than online help, but with detailled features, subcommands, switches and arguments explained.
Help content, some simple rules:
One recent (July 2003) issue that came up was the fact that Tcl has established the precedence of documenting every command of tcl as a seperate man page, where as perl, python, ksh, etc. have taken other approaches - in fact, tcl appears to be the only commonly installed language which has taken this approach. Despite this, perl at least comes with much more documentation than does tcl. python comes with tutorials, etc. So the general impression by someone new to tcl is that there is too little doc spread across too many pages.
Subsequent discussions pointed out that this was not entirely the case. Perl, for instance, has hundreds of man pages in section 3 as well as over 100 man pages in section 1! Amazingly enough, the section 3 man pages are mostly packages, while the section 1 man pages are FAQs, etc. The perl language itself appears to be spread across a number of chapter files.
LES Nov 25 2003: Perl's documentation is atrocious. How is one supposed to find anything in those endless single rolls of parchment? Talking about split docs, don't forget PHP. I never checked if PHP has man pages, but its help system, usually in HTML, has always been split into thousands of pages. I have read most of it in the CHM format and I think it's the best piece of documentation I have seen. Beautiful, clear, handy and well written.
LES 21 Aug, 2003 - Wouldn't txt2tags [L1 ] be an excellent tool to help develop documentation in many formats?
24 Nov, 2003: ...or aurigadoc [L2 ].
Anyway, I see a lot of discussion about the format and nothing about the content. I think Tcl's documentation is mostly confusing, too economical and/or inadequate to newcomers. It really could use some rewriting, IMO.
DKF: Frankly, I'd advise people to stop worrying about the format. Formats can be changed. The key problems all have to do with understanding when the content is insufficient. If you (or anyone else) know of a problem, file a bug report on SF; so long as we know which page has a problem and what is missing/wrong, we can probably fix it in short-order.
LES: filing a "bug" is tricky. I am not talking about a problem here or there. It's the overall approach and tone. Everyone here says that Tcl is cool, simple and elegant. All true, but not its documentation. I find it excruciatingly boring, with a very technical and monotonic style. It often sounds like it is just getting rid of the task of specifying what the command does. Well, maybe it is. Programmers are notorious for hating to write documentation. Many times do I read one command description in the manual and have to run for the wiki to read it in "human-ish" and finally understand what it does. And more: what else it is capable of doing. I don't think I have found any errors yet, but there is plenty of what I consider some form of omission. Most notably, examples are quite rare. Or an example of at least one application of each command. Many of the commands descriptions are so vague and abstract that even if they were clear it would still be difficult to realize what I can do with them.
AM (25 november 2003) Part of your critique is due to the long-standing practice with man pages: they are meant to be written that way. But i do agree with you that examples are important.
I think it would help a lot if you listed specific examples of where the documentation could be improved.
LES 25 Nov 2003: Oh, well, let's see an example, picked at random:
fblocked - Test whether the last input operation exhausted all available input SYNOPSIS fblocked channelId DESCRIPTION The fblocked command returns 1 if the most recent input operation on channelId returned less information than requested because all available input was exhausted. For example, if gets is invoked when there are only three characters available for input and no end-of-line sequence, gets returns an empty string and a subsequent call to fblocked will return 1. ChannelId must be an identifier for an open channel such as a Tcl standard channel (stdin, stdout, or stderr), the return value from an invocation of open or socket, or the result of a channel creation command provided by a Tcl extension.
Let's see what I (don't) see:
fblocked - Test whether the last input operation exhausted all available input
Wrong: it's not about the "last input operation", but about a certain input operation identified by channelId, which is not necessarily the last one.
"...exhausted all available input"
I would say that differently. I don't know how because I don't really know what it means. I have a good clue, but the choice of words confuses me. "Exhaust", "available" and "input" are all words that can be interpreted in multiple ways, especially if you're new in the block.
SYNOPSIS fblocked channelId
Synopsis??? Why not syntax? Isn't that what fblocked channelId demonstrates?
The fblocked command returns 1 if the most recent input operation on channelId returned less information than requested...
"Returned"? I am reading it carefully, following this explanation and (hopefully) a sequence of events and you tell me that it's happened already? I was reading so carefully, how did it happen and I didn't see it? You force me to read it again and realize you're talking about something that ought to be pretty common in Tcl, but I wouldn't know because I am a newbie. I resign, humbly. BTW, what is "less information than requested"? I have no idea of how that can happen. This all has to do with "the most recent input operation", but "input operation" sounds so vague.
For example, if gets is invoked when there are only three characters available for input and no end-of-line sequence, gets returns an empty string and a subsequent call to fblocked will return 1.
Oh, an example. But it's a) in the middle of the command's description - bad place for an example and good proof that the author himself recognizes that his description is not clear enough; b) a rather long-winded sentence. Pay attention: "only three characters available for input", "no end-of-line sequence", "gets returns an empty string", "a subsequent call to fblocked" and "return 1". That's a lot to happen in one single sentence. Notice that as you start reading this sentence, you cannot guess what is going to happen. If you do, you don't need to be reading it. This is how it sounds to my ear:
IF something happens WHEN something else happens AND yet another thing happens, THEN something AND something else will happen. Doesn't it sound a lot like those rules in the inside face of boxes of games like Monopoly? :-)
"Gets returns an empty string"? I didn't know that. But are we talking about gets or fblocked? "...and a subsequent call to fblocked will return 1"? Excuse me, I still have my mind set on the gets thing, so don't give me big words like "subsequent" because my mind is not listening anymore. "Subsequent" is something that comes after. But if you talk about what comes after in the same sentence without a full stop, it sounds like it's all happening at the same time to me. It jumbles. You know when you take a very large bite and can't swallow but can barely chew either? Summing up, this entire sentence tries to shove information onto the reader. It almost feels like "let's get done with it". Well, maybe that was the case. We never know in what circumstances something was written.
On two counts, it is said when fblocked will return 1. But it never says what happens when it does not return 1. I suppose it returns 0, but that's not said. I shouldn't have to guess. It could also return -1, who knows? And is "1" equivalent to "TRUE"? Can I use TRUE? That's common in that other language (whatever it is) that I use, so I find that a reasonable question. BTW, I miss synonyms. These channels you talk about all the time, isn't it closely related to something that goes by the name of "handle" wild and at large? Why not mention it, at least once?
ChannelId must be an identifier for an open channel such as a Tcl standard channel (stdin, stdout, or stderr), the return value from an invocation of open or socket, or the result of a channel creation command provided by a Tcl extension.
Not too bad, but why not break the above sentence into a bulleted list? A list would give the newbie an always opportune overview of all possible "Tcl standard channels". And, again, the "handles" could be mentioned here.
Of course, I sprinkle a bit of exaggeration here and there, but I am serious about most of it. Like this phrase: "...exhausted all available input". It still echoes in my mind and means nothing. Too technical, too vague, too... er...
AM (27 november 2003) I wonder whether you really picked this page at random, but after filtering out the exaggeration I too conclude that this deserves a thorough rewrite.
I am not yet convinced that this is true for all or most manual pages, but we (the community at large) should make an effort at identifying such muddied descriptions - perhaps especially us non-Anglosaxons!
The wonderful thing about Tcl is: it is open for improvements, so let us get started :)
Here is a formulation of such a project:
LES swears and crosses his heart that it was picked at random: It was the very first page I opened in the ActiveState CHM. I really like the idea of rewriting the documentation. I can't program, but I can write. :-) I will rewrite one or two command descriptions today, as an exercise. Whom should I submit it to?
AM Post a patch on the SourceForge project - http://tcl.sourceforge.net will get you there
AM (25 november 2003) In a recent discussion on c.l.t. it was concluded that a user's guide of some sort is lacking. (This is just a reminder for myself :))
AM (27 november 2003) I have started the page How to make a Tcl application
Peter Newman 8 April 2004: Just noticed this page. But the points made above re. Tcl's documentation are bang on the money. Tcl's documentation is absolute crap. It totally sucks. With the possible exception of the (almost totally undocumented) Event Loop, it's undoubtedly the worst feature of Tcl.
I started a project a few days ago that requires reading a text file line by line. Nothing strange about that. A read of the Tcl docs suggests you use "gets". And presumably "eof" to detect EOF? ...Or should that be "fblocked"? ...Or possibly you need both? ...But maybe it depends on the "fconfigure" settings?
If you read the relevant man pages you soon realise the Tcl docs simply provide NO reliable description of how you're supposed to detect EOF when reading a text file line by line (which, after all, is a really common programming problem).
And worse, there are straight-out descrepancies. For example:-
From the "gets" manpage:-
And from the "fblocked" manpage:-
So which is it? Do you get the last incomplete line? Or don't you? --- And do you waste hours trying to figure (or test) it out? --- Or do you give up on Tcl? And go back to Perl! Or on to Python?
IMHO getting the words right is far more important than the format (PDF, HTML, doctools or whatever) used. The problem is that almost every Tcl manpage needs an almost complete re-write. And that would take literally man years.
The simplest solution to me, would be that we put the manpages up on this Wiki. So that clarification, re-writes, explanations, examples and tutorials could be added.
NEM 8Apr04: While I can see that there is a case for improving Tcl's documentation somewhat - especially for beginners - I actually quite like Tcl's terse man pages. Examples would be the single biggest improvement (Chris Nelson's Tcl/Tk Programmer's Reference is excellent in this regard, but slightly out of date now). Terse man pages are very useful when you already know what you are looking for, and just need some specific information. Usually, this is just when I'm using the C-API these days. For Tcl commands, I find it's generally easier to just fire up tkcon and see what happens with a trial-and-error approach. I can't remember a situation where I've had to "waste hours trying to figure (or test) it out...". Usually a few seconds, up to a minute, is sufficient. This is one of the benefits of Tcl's interactive nature, and it's gentle handling of errors. This perhaps isn't the best situation, but it suffices, once you know what you are doing. The key here is getting good docs for people who don't (yet) know what they are doing. That's going to rely on good-natured volunteers, as by the time you've learnt enough to be able to write such documentation, you no longer have a need for it! One of the classic problem areas for free open-source projects (no-one's paying for a documentation effort)...
The discrepancy between the gets and fblocked man pages described above is clearly a doc bug, and should be reported as such on sourceforge.
DKF: Specific suggestions for improvements to documentation are always welcome - and I'm a really big fan of examples, especially when pithy - as are references to overview documents of some subsection. One of the things which I have been hoping for in the TIP process, and which hasn't happened anything like as much as I want, is for documents describing these high-level pictures to enter the Informational track.
LES: That last sentence didn't sound too clear to me. Care to elaborate a little?
DKF: TIPs describe three sorts of things. They describe TCT rules, changes to Tcl and Tk (the most well-known kind of TIP), and provide other sorts of background document. It's this last type (so-called Informational TIPs) that's not flourished as I wanted, but I want it to expand to include descriptions of what constitutes good practice, why major subsystems are the way they are, etc.
LkpPo 2012-08-24 - If the Tcl / Tk Programmer's Reference book of Chris Nelson is so good from this point of view, why not improve the official documentation, with what we can recover from it, with his permission?