Don also suggests http://tmml.sourceforge.net/doc/tcl/tcltest.html as a superior formatting of the same information.

Anyone have some good examples of making use of this code?

A wonderful addition to this page would be instructions on how to run one particular test, using verbose mode, for the purpose of reporting test suite failures to a package developer. [A Pythoneer has written an article--which DGP answers:

 make TESTFLAGS='-verbose tpse -file safe.test' test

CL can locate if helpful--illustrating use of PyUnit with Roman numeral conversion. In the absence of any other inspiration, maybe someone wants to steal that example ...]

RS, off page topic: See also Roman numbers for Tcl routines.

AK: Most modules in tcllib come with .test files using tcltest for regression testing.

VI : 2003/10/02. We at [L1 ] use Tcl (and Tk) extensively. We use Tcl for hardware testing in simulation (think multimillion-gate , multifunction asic verification). One of the major factors in our initial decision to use Tcl and our continuing to use tcl is tcltest. We use most tcltest features including constraints and are very pleased with the easy configuration of tests and the reporting. I do have gripes, but relatively minor, and since it is pure Tcl, we can change it anyways.. davidw 2003-10-03: I am doing some work to improve tcltest, specifically to give it an API so that you can programmatically fetch information about the test results. I would love to hear what sorts of features you would find useful - feel free to drop me email.

davidw 2003/10/03: I am doing some work to improve tcltest, specifically to give it an API so that you can programmatically fetch information about the test results. I would love to hear what sorts of features you would find useful - feel free to drop me email.

[tcltest is a truly great and wonderful thing. We should make a point of explaining its virtues and uses.]

escargo 16 Dec 2003 - I have started using tcltest, but I had to figure a few things out on my own in order to get started. Clearer documentation could have helped me out.

The man page portion for -body says (in part), "The -body attribute indicates the script to run to carry out the test. It must return a result that can be checked for correctness." My first reading of this was confused by the meaning of return a result. My first thought was that there ought to be a return at the end of the specified body. That is not the case. Is there a conventional meaning of return a result in Tcl that is not related to the return command?

DGP Good point. Would produce a result or yield a result be less likely to confuse?

escargo - My background with Icon leads me to expect terminology like produce a value or produce a result (not to be confused with generate a value or generate a result, which are semantically distinct from produce in Icon). (I can see where other people think about commands returning results, but I don't think of scripts as returning results, since they don't use the return command. Once the conceptual leap has been made, this way of thinking is no longer a problem, but the gulf has to be recognized before it can be crossed.)

WHD: The return value of a Tcl script is the return value of the last command executed in the script. Many of my tcltest cases end with a call to "set" (which returns the value of a variable) or with a call to "list" (to return a number of results at once).

A second question I had (which is not addressed in the documentation for -body) is, "What scope is the body code executed in?" Is it executed in the scope containing the test command that the -body option belongs to? (That would seem to be the only reasonable choice, since there are no parameters to the test command, but I suppose execution could be in the global scope.)

DGP All of the scripts, -setup, -body, and -cleanup are evaluated in the caller's scope. This is the same rule as for other control structure commands like if or while.

escargo - Do you think of test as a control structure?

DGP For the most part, yes. Any command that takes a script as an argument and controls something about how that script is evaluated is a kind of control structure in my view.

escargo - Looking at it that way, and thinking about how the -constraints make execution of the other scripts (e.g., -body) supplied to the command conditional, I can agree that test is a control structure.

How to write tcltest result values [Where to obtain ...]

disneylogic It would be useful to have a callback option, prefixed determine the correctness of a test. In lieu of demanding a real value to compare or abusing return codes, this would adjust 'tcltest' to perform more like SUnit [L2 ] in the Beck testing framework [L3 ].

I encountered this when I was trying to write a routine to do sampling without replacement and could not specify a specific result to use to compare.

If there is already some way of doing this, please, please specify it here! I waded through the man page for a bit and couldn't find anything, particularly in the usage of 'test' section.

DGP: Can you provide an example of what a test DGP Can you provide an example of what a test for test ? Assume the reader knows nothing about for test ? Assume the reader knows nothing about

RHS One way to handle tests that the -result option can't handle is to do something like:

 test mytest-1.1 {
     Calling my proc should always return a list of at least length 3
 } -body {
     set result [myProc a b c d e]
     # -result can't handle {>= llength 3}, so we test it here
     expr { [llength $result] >= 3 }
 } -result {1}

Ie, perform the test for passing inside the actual test body, and have the -result be 1/0 if that test passed/failed.

escargo 16 Aug 2005 - Isn't this exactly the problem that customMatch is supposed to solve? At the start of your test script, you define a script to evaluate to determine if your result is correct. You then configure a customMatch for your user-defined mode, and then your specific test specifies that same match mode, which in turn calls your script to determine if the answer is right or not.

LV: I have a programming itch and am wondering if tcltest will help me scratch this itch. escargo 16 Aug 2005 - Is there a way to tell if there have been any tests? I'm looking at a script where users might be using tcltest. If they did, then if critical errors are detected, I want to call cleanupTests before exiting. I could check to see if cleanupTests was a defined proc, or if the ::tcltest namespace exists, but those are not sufficient conditions. - RS How about just writing

 catch tcltest::cleanupTests

It's not that cleanupTests fails, it's just that if no test commands have been executed, there is not much point to running cleanup.

What are folks looking for in a New Test Package?

"Your first tcltests"

LV I have a programming itch and am wondering if tcltest will help me scratch this itch. Problem: set up regression testing of a client/server command line and stdout/return code related pair of applications.

Currently, I'd love to find some help, tutorials, examples, etc. of how others are successfully doing this sort of thing with the result being test cases that are nearly trivial to read and write. I'd like to not have to teach people all of tcl before they can write new test cases. Ideally, having a half dozen or so examples should provide enough context to write additional code.

I'd also like suggestions for best practices relating to this sort of use of tcltest.

Thanks!

2006-03-31 JR, I would like to customize the output format of cleanupTests. Is that possible ?

2006-04-24 Bryan Oakley has written brief articles on how to get started with tcltest [L4 ] and how to install tcltest [L5 ].

A look at the Tcl test suite with gcov

2007-08-05 MB: A simple alternative to tcltest for the small test in between is tcltap.

LV 2007 Sep 25 Anyone have info on how I might use the Tcl source code test suite, but run it against a binary distribution of Tcl? That is, I have a machine with tcl installed on it, and I'd like to run the test suite against it, to see if the installed version has any problems.

DGP tclsh all.tcl