ABU 2024-08-07: - TTXN-1.0.2 - adapted for Tcl8 and Tcl9.
Download:
package require Tcl 8.5
package require TTXN ?1.0?
TTXN is a package for the construction of test suites. TTXN development is the evolution of R.Seeger and P.Maker's ideas for alternative notations of tcltest scripts. These original works are available on the TclTk wiki-site at the following links:
Like the original Pretty Test Language, TTXN is a wrapper of the standard tcltest package; it does not completely hide the tcltest commands, but provides an alternative notation for writing more readable tests. Being a wrapper of tcltest, TTXN commands can be inserted in a normal tcltest script. Therefore with TTXN you can write a tcltest suite (e.g. *.test) interspersing tcl commands, tcltest commands and TTXN commands. Aim of TTXN is not to replace tcltest commands but just to provide a simpler notation for test-case specs. Simpler test-cases are easy to read and easy to maintain. Just an example to get the flavor of how to use TTXN: First, let's consider a (structurally) complex test-case for tcltest
tcltest::test demo-1.0.0 {
list / lappend
} -body {
set L {}
lappend L a
lappend L b
lappend L c
} -cleanup {
unset L
} -result [list a b c]and now the same test-case written with the TTXN notation
Name: demo-1.0.0
Descr: list / lappend
Test: {
set L {}
lappend L a
lappend L b
lappend L c
}
Cleanup: {unset L}
Expected: [list a b c]As you can see, whilst tcltest provides a single command (tcltest::test) with many options (-setup, -body, -result, ...), in TTXN the same test-case can be written with a sequence of simpler, more readable commands (Name:, Setup:, Test:, Expected:, ...)
A test-case in TTXN can be expressed as a sequence of commands. Although the recommended sequence for a test-case is the following:
Name: Description: (optional) Only: (optional) Setup: (optional) Test: OutputChannel: (optional) ErrorChannel: (optional) Cleanup: (optional) Expected:
commands can be specified in any order with just these exceptions:
Note that all the special TTXN commands start with an upper-case letter, and have a trailing ":" . The trailing colon is part of the command (it's not a token separator), therefore at least one space is required after it.
Name:FirstTest ; # this is wrong Name: FirstTest ; # ok
TTXN supports the following commands:
Let's start with a small test-suite made of 2 test-cases. Save the following script in a file named "hello.test".
package require TTXN
# load all the modules for the Software Under Testing
# ---------------------------------------------------
# package require hello ; # fake loading
# init_hello ; # fake init
# begin of test-cases
# --------------------
Name: helloworld
Test: { string toupper "Hello world" }
Expected: "HELLO WORLD"
# Note: this test sometimes may fail... deep analysis required !
Name: weatherForecast
Test: {
#
# ... forecast in progress ...
#
return "Rainy"
}
Expected: "Sunny"
# test-suite standard cleanup
# ---------------------------
::tcltest::cleanupTeststhen run
tclsh hello.test
Note: be sure TTXN package is installed under a tcl-library path. Result should be something like this:
==== weatherForecast FAILED
==== Contents of test case:
#
# ... forecast in progress ...
#
return "Rainy"
---- Result was:
Rainy
---- Result should have been (exact matching):
Sunny
==== weatherForecast FAILED
hello.test: Total 2 Passed 1 Skipped 0 Failed 11 test-case passed, 1 failed ! Try to adjust "hello" package ( or better, "hello.test" ) !
If test-case returns (a list of) two values, the Expected: command should specify a list of two values ...
...
Name: test-split
Test: {
set str "alpha:beta"
return [split $str ":"]
}
# Expected: alpha beta ; # this is wrong !
Expected: {alpha beta}
...When a test-case returns nothing, the Expected: value should be written as {} (or "")
...
Name: test-lassign
Descr: first element of an empty list is {}
Test: {
lassign {} x
return $x
}
Expected: {}
...By default the value returned by the body of a test-case (Test:) is exactly compared with the expected-result, but you may specify other criteria for comparison. You can use one of the predefined criteria
These comparison-criteria (matchMode) may be specified as the first optional parameter of the "Expected:" command:
Expected: ?matchMode? expectedValue
All the previosly seen "Expected:" commands such as
Expected: expectedValue
have an implicit matchMode whose default is "exact" and are therefore equivalent to:
Expected: exact expectedValue
Example - express expectedValue as a "glob" expression :
...
Name: test-string-reverse
Test: { string reverse "abcdefgz" }
Expected: glob "z*a" ; # note the "glob" match-mode"
...Your test-case body can be designed not only for testing normal expectedValues, but for trapping exceptions, too. Let's write a test-case for checking the exception thrown when we try to open a non-existing file: Here is an old (discouraging) trick for such test-case
Name: test-open-exception
Test: {
list [catch {open "not-existing-file.txt" r} f] $f
}
Expected: {1 {couldn't open "not-existing-file.txt": no such file or directory}}And here is a more polite alternative making use of the extended syntax of the "Expected:" command:
Name: test-open-exception
Test: {
set f [open "not-existing-file.txt" r]
}
Expected: error {couldn't open "not-existing-file.txt": no such file or directory}Note that this testcase body is much more readable and that the Expected: command has been extended to support the following full syntax:
Expected: ?expectedCodeList? ?matchMode? expectedValue
All the previosly seen "Expected:" commands such as
Expected: expectedValue
have an implicit ?expectedCodeList? and ?matchMode? and are therefore equivalent to:
Expected: {ok return} exact expectedValue As a final improvement, we could rewrite our last test-case, replacing the (implicit) "exact" matching with a simpler-to-test "glob" matching:
Name: test-open-exception
Test: {
set f [open not-existing-file.txt r]
}
Expected: error glob {couldn't open *}the following TTXN commands
OutputChannel: expectedValue ErrorChannel: expectedValue
allow you to specify the expectedValue against which any output sent to stdout or stderr during evaluation of the test-case body script will be compared. Note that (as for tcltest) only output printed using ::puts is used for comparison. If OutputChannel: (or ErrorChannel:) is not specified, output sent to stdout (or stderr) is not processed for comparison.
Let's suppose we have an instrument returning a decimal value close to 1.0 with a "random" error of +/- 0.0001. How can we compare the expected-value "1.0" with a "random" but very close value ? We should add a new comparison criteria, just by using some "classic" tcltest features
tcltest::customMatch approx approxCompare
proc approxCompare {expected actual} {
expr {abs($expected - $actual) < 0.001 }
} With this new matchMethod, we could write the following test-case:
...
Name: instrument-reset
Test: {
# ... do something and compute a result ...
return 0.999997 ;# of course this is not a random value. It's just a demo
}
Expected: approx 1.0
...tcltest, testing
dcd - 2012-08-09 21:46:17
There's a bug in your last example, presumably a holdover from PTL:
Result: approx 1.0
s.b.
Expected: approx 1.0
ABU - 4 hours later Thanks, fixed with other small wiki-formatting errors. A new package 1.0.1 will be available tomorrow.