Purpose: collection point for information regarding the tcl testsuite support code. [http://www.tcl.tk/man/tcl/TclCmd/tcltest.htm%|%official reference]: The tcltest package provides the user with utility tools for writing and running tests in the Tcl test suite. It can also be used to create a customized test harness for an extension. Tcltest is one of the packages that is bundled into the [Tcl] software core distribution. http://www.purl.org/tcl/home/man/tcl8.4/TclCmd/tcltest.htm [PYK] 2015-04-09: One strategy for some complex tests is to set `-result` to [Donald Porter] advises "Please note that version 2.x of the tcltest package (included with Tcl 8.4) is much more flexible in the kind of matching it permits." <> Don also suggests http://tmml.sourceforge.net/doc/tcl/tcltest.html as a superior formatting of the same information. ---- Anyone have some good examples of making use of this code? A wonderful addition to this page would be instructions on how to run one particular test, using verbose mode, for the purpose of reporting test suite failures to a package developer. [[A Pythoneer has written an article--which [DGP] answers: make TESTFLAGS='-verbose tpse -file safe.test' test ---- [CL] can locate if helpful--illustrating use of PyUnit with Roman numeral conversion. CL can locate if helpful--illustrating use of PyUnit with Roman numeral conversion. maybe someone wants to steal that example ...]] [RS], off page topic: See also [Roman numbers] for Tcl routines. ---- [AK]: Most modules in [tcllib] come with .test files using tcltest for regression testing. [VI] : 2003/10/02. We at [http://www.comit.com] use Tcl (and Tk) extensively. We use Tcl for hardware testing in simulation (think multimillion-gate , multifunction asic verification). One of the major factors in our initial decision to use Tcl and our continuing to use tcl is tcltest. We use most tcltest features including constraints and are very pleased with the easy configuration of tests and the reporting. I do have gripes, but relatively minor, and since it is pure Tcl, we can change it anyways.. [davidw] 2003-10-03: I am doing some work to improve tcltest, specifically to give it an API so that you can programmatically fetch information about the test results. I would love to hear what sorts of features you would find useful - feel free to drop me email. ---- [davidw] 2003/10/03: I am doing some work to improve tcltest, specifically to give it an API so that you can programmatically fetch information about the test results. I would love to hear what sorts of features you would find useful - feel free to drop me email. ---- [[tcltest is a truly great and wonderful thing. We should make a point of explaining its virtues and uses.] ---- ''[escargo] 16 Dec 2003'' - I have started using tcltest, but I had to figure a few things out on my own in order to get started. Clearer documentation could have helped me out. The man page portion for '''-body''' says (in part), "The -body attribute indicates the script to run to carry out the test. It must return a result that can be checked for correctness." My first reading of this was confused by the meaning of ''return a result.'' My first thought was that there ought to be a '''return''' at the end of the specified body. That is not the case. Is there a conventional meaning of ''return a result'' in Tcl that is not related to the '''return''' command? '''[DGP]''' Good point. Would ''produce a result'' or ''yield a result'' be less likely to confuse? ''[escargo]'' - My background with [Icon] leads me to expect terminology like ''produce a value'' or ''produce a result'' (not to be confused with ''generate a value'' or ''generate a result'', which are semantically distinct from ''produce'' in Icon). (I can see where other people think about ''commands'' returning results, but I don't think of ''scripts'' as returning results, since they don't use the '''return''' command. Once the conceptual leap has been made, this way of thinking is no longer a problem, but the gulf has to be recognized before it can be crossed.) [WHD]: The return value of a Tcl script is the return value of the last command executed in the script. Many of my tcltest cases end with a call to "set" (which returns the value of a variable) or with a call to "list" (to return a number of results at once). A second question I had (which is not addressed in the documentation for '''-body''') is, "What scope is the body code executed in?" Is it executed in the scope containing the '''test''' command that the -body option belongs to? (That would seem to be the only reasonable choice, since there are no parameters to the test command, but I suppose execution could be in the global scope.) '''[DGP]''' All of the scripts, ''-setup'', ''-body'', and ''-cleanup'' are evaluated in the caller's scope. This is the same rule as for other control structure commands like [if] or [while]. ''[escargo]'' - Do you think of '''test''' as a control structure? '''[DGP]''' For the most part, yes. Any command that takes a script as an argument and controls something about how that script is evaluated is a kind of control structure in my view. ''[escargo]'' - Looking at it that way, and thinking about how the '''-constraints''' make execution of the other scripts (e.g., '''-body''') supplied to the command conditional, I can agree that '''test''' is a control structure. ---- [How to write tcltest result values] [[Where to obtain ...]] ---- [disneylogic] It would be useful to have a callback option, prefixed determine the correctness of a test. In lieu of demanding a real value to compare or abusing return codes, this would adjust 'tcltest' to perform more like SUnit [http://sourceforge.net/projects/sunit/] in the Beck testing framework [http://www.xprogramming.com/testfram.htm]. I encountered this when I was trying to write a routine to do sampling without replacement and could not specify a specific result to use to compare. If there is already some way of doing this, please, please specify it here! I waded through the man page for a bit and couldn't find anything, particularly in the usage of 'test' section. [DGP]: Can you provide an example of what a test '''[DGP]''' Can you provide an example of what a test for `test` ? Assume the reader knows nothing about for [test] ? Assume the reader knows nothing about ---- [RHS] One way to handle tests that the ''-result'' option can't handle is to do something like: test mytest-1.1 { Calling my proc should always return a list of at least length 3 } -body { set result [myProc a b c d e] # -result can't handle {>= llength 3}, so we test it here expr { [llength $result] >= 3 } } -result {1} Ie, perform the test for passing inside the actual test body, and have the ''-result'' be 1/0 if that test passed/failed. [RHS] Tcltest is an amazing little package for writing simple ''unit'' type tests. However, there are a lot of things that tcltest makes it very hard to do. There are other things that aren't hard to do, but make the tests that do them hard to read. I'd like to put together a list of desired functionality for a new test package, in hopes of eventually writing such a beast ''(or, someone else coming along and writing it)''. The things I'm looking for are: * Simple to write simple tests, like tcltest. If I just want to call a proc and test the return value, it should be a very small bit of code to do so * Things that aren't part of the actual testing framework should be in a seperate package. For example, things like makeFile and makeDirectory should be in a seperate package that comes with the framework. * Testing the errorCode should be as easy as testing the return code * Complex test cases should still be easy to read and understand. * Assertions should be available, and the test cases should know that the test failed if any assertions failed * It should be easy to say ''If this assertion fails, the test is finished, don't bother with the rest'' test myComplexTest-1.1 { A sample of a complex test, with comments } -setup { set data {a 1 b 2 c 3} set filename [extrapackage::makeFile $data myComplexTestFile-1.1] catch {unset myArray} catch {unset expectArray} ; array set expectArray $data } -body { set code [readFileToArray $filename myArray] assertEquals -nofail 1 $code "The read failed, don't bother with other assertions" assertEquals -nofail 1 [info exists myArray] "The array did not get created" assertArrayEquals expectArray myArray "The array results were incorrect" } * Tests w/o a ''-result'' flag are assumed to return with a code of 0/2, and their actual result doesn't matter. The success/failure of such a test case (if it returns with code 0 or 2) depends on the assertions in the test * It should be possible to create ''"Test Suites"'' that group a bunch of tests/test files together * It should be possible to programatically run a test, a whole file of tests, a test suite, a while directory of tests * It should be possible to retrieve the results of programatically run tests ---- Back to [Tcl core extension syntax] ---- [[ [Category Package] | [Category Command] from [Tcl] ]] [[ [Tcl syntax help] | [Arts and crafts of Tcl-Tk programming] ]]