Purpose: collection point for information regarding the tcl testsuite support code. [http://www.tcl.tk/man/tcl/TclCmd/tcltest.htm%|%official reference]: The tcltest package provides the user with utility tools for writing and running tests in the Tcl test suite. It can also be used to create a customized test harness for an extension. Tcltest is one of the packages that is bundled into the [Tcl] software core distribution. http://www.tcl.tk/man/tcl/TclCmd/tcltest.htm [PYK] 2015-04-09: One strategy for some complex tests is to set `-result` to [Donald Porter] advises "Please note that version 2.x of the tcltest package (included with Tcl 8.4) is much more flexible in the kind of matching it permits." See [tcltest customMatch] for some examples. <> Don also suggests http://tmml.sourceforge.net/doc/tcl/tcltest.html as a superior formatting of the same information. Anyone have some good examples of making use of this code? A wonderful addition to this page would be instructions on how to run one particular test, using verbose mode, for the purpose of reporting test suite failures to a package developer. [[A Pythoneer has written an article--which [DGP] answers: make TESTFLAGS='-verbose tpse -file safe.test' test ---- [CL] can locate if helpful--illustrating use of PyUnit with Roman numeral conversion. In the absence of any other inspiration, maybe someone wants to steal that example ...]] [RS], off page topic: See also [Roman numbers] for Tcl routines. ---- [AK]: Most modules in [tcllib] come with .test files using tcltest for regression testing. [VI] : 2003/10/02. We at [http://www.comit.com] use Tcl (and Tk) extensively. We use Tcl for hardware testing in simulation (think multimillion-gate , multifunction asic verification). One of the major factors in our initial decision to use Tcl and our continuing to use tcl is tcltest. We use most tcltest features including constraints and are very pleased with the easy configuration of tests and the reporting. I do have gripes, but relatively minor, and since it is pure Tcl, we can change it anyways.. [davidw] 2003-10-03: I am doing some work to improve tcltest, specifically to give it an API so that you can programmatically fetch information about the test results. I would love to hear what sorts of features you would find useful - feel free to drop me email. ---- [davidw] 2003/10/03: I am doing some work to improve tcltest, specifically to give it an API so that you can programmatically fetch information about the test results. I would love to hear what sorts of features you would find useful - feel free to drop me email. ---- [[tcltest is a truly great and wonderful thing. We should make a point of explaining its virtues and uses.] ---- ''[escargo] 16 Dec 2003'' - I have started using tcltest, but I had to figure a few things out on my own in order to get started. Clearer documentation could have helped me out. The man page portion for '''-body''' says (in part), "The -body attribute indicates the script to run to carry out the test. It must return a result that can be checked for correctness." My first reading of this was confused by the meaning of ''return a result.'' My first thought was that there ought to be a '''return''' at the end of the specified body. That is not the case. Is there a conventional meaning of ''return a result'' in Tcl that is not related to the '''return''' command? '''[DGP]''' Good point. Would ''produce a result'' or ''yield a result'' be less likely to confuse? ''[escargo]'' - My background with [Icon] leads me to expect terminology like ''produce a value'' or ''produce a result'' (not to be confused with ''generate a value'' or ''generate a result'', which are semantically distinct from ''produce'' in Icon). (I can see where other people think about ''commands'' returning results, but I don't think of ''scripts'' as returning results, since they don't use the '''return''' command. Once the conceptual leap has been made, this way of thinking is no longer a problem, but the gulf has to be recognized before it can be crossed.) [WHD]: The return value of a Tcl script is the return value of the last command executed in the script. Many of my tcltest cases end with a call to "set" (which returns the value of a variable) or with a call to "list" (to return a number of results at once). A second question I had (which is not addressed in the documentation for '''-body''') is, "What scope is the body code executed in?" Is it executed in the scope containing the '''test''' command that the -body option belongs to? (That would seem to be the only reasonable choice, since there are no parameters to the test command, but I suppose execution could be in the global scope.) '''[DGP]''' All of the scripts, ''-setup'', ''-body'', and ''-cleanup'' are evaluated in the caller's scope. This is the same rule as for other control structure commands like [if] or [while]. ''[escargo]'' - Do you think of '''test''' as a control structure? '''[DGP]''' For the most part, yes. Any command that takes a script as an argument and controls something about how that script is evaluated is a kind of control structure in my view. ''[escargo]'' - Looking at it that way, and thinking about how the '''-constraints''' make execution of the other scripts (e.g., '''-body''') supplied to the command conditional, I can agree that '''test''' is a control structure. ---- [How to write tcltest result values] [[Where to obtain ...]] ---- [disneylogic] It would be useful to have a callback option, prefixed determine the correctness of a test. In lieu of demanding a real value to compare or abusing return codes, this would adjust 'tcltest' to perform more like SUnit [http://sourceforge.net/projects/sunit/] in the Beck testing framework [http://www.xprogramming.com/testfram.htm]. I encountered this when I was trying to write a routine to do sampling without replacement and could not specify a specific result to use to compare. If there is already some way of doing this, please, please specify it here! I waded through the man page for a bit and couldn't find anything, particularly in the usage of 'test' section. [DGP]: Can you provide an example of what a test '''[DGP]''' Can you provide an example of what a test for `test` ? Assume the reader knows nothing about for [test] ? Assume the reader knows nothing about ---- [RHS] One way to handle tests that the ''-result'' option can't handle is to do something like: ====== test mytest-1.1 { Calling my proc should always return a list of at least length 3 } -body { set result [myProc a b c d e] # -result can't handle {>= llength 3}, so we test it here expr { [llength $result] >= 3 } } -result {1} ====== Ie, perform the test for passing inside the actual test body, and have the ''-result'' be 1/0 if that test passed/failed. ''[escargo] 16 Aug 2005'' - Isn't this exactly the problem that '''customMatch''' is supposed to solve? At the start of your test script, you define a script to evaluate to determine if your result is correct. You then configure a customMatch for your user-defined mode, and then your specific test specifies that same match mode, which in turn calls your script to determine if the answer is right or not. [LV]: I have a programming ''itch'' and am wondering if tcltest will help me scratch this itch. ''[escargo] 16 Aug 2005'' - Is there a way to tell if there have been any tests? I'm looking at a script where users ''might'' be using '''tcltest'''. If they did, then if critical errors are detected, I want to call '''cleanupTests''' before exiting. I could check to see if cleanupTests was a defined proc, or if the ::tcltest namespace exists, but those are not sufficient conditions. - [RS] How about just writing ====== catch tcltest::cleanupTests ====== It's not that '''cleanupTests''' fails, it's just that if no '''test''' commands have been executed, there is not much point to running cleanup. ---- What are folks looking for in a [New Test Package]? ---- "[Your first tcltests]" ---- [LV] I have a programming ''itch'' and am wondering if tcltest will help me scratch this itch. Problem: set up regression testing of a client/server command line and stdout/return code related pair of applications. Currently, I'd love to find some help, tutorials, examples, etc. of how others are successfully doing this sort of thing with the result being test cases that are nearly trivial to read and write. I'd like to '''not''' have to teach people all of tcl before they can write new test cases. Ideally, having a half dozen or so examples should provide enough context to write additional code. I'd also like suggestions for '''best practices''' relating to this sort of use of tcltest. Thanks! ---- 2006-03-31 JR, I would like to customize the output format of cleanupTests. Is that possible ? ---- ---- 2006-04-24 [Bryan Oakley] has written brief articles on how to get started with tcltest [http://www.tclscripting.com/articles/apr06/article1.html] and how to install tcltest [http://www.tclscripting.com/articles/apr06/article2.html]. ---- [A look at the Tcl test suite with gcov] ---- 2007-08-05 MB: A simple alternative to tcltest for the small test in between is [tcltap]. ---- [LV] 2007 Sep 25 Anyone have info on how I might use the Tcl source code test suite, but run it against a binary distribution of Tcl? That is, I have a machine with tcl installed on it, and I'd like to run the test suite against it, to see if the installed version has any problems. [DGP] tclsh all.tcl ---- !!!!!! %| [Category Testing] | [Category Package] | [Category Command] from [Tcl] |% [[ [Tcl syntax help] | [Arts and crafts of Tcl-Tk programming] ]] !!!!!!