Tcl has long traditions in testing [put in some general-purpose testing reference here--Beizer, ...]. Especially notable are Expect, Oracle, Sybase, NASA , and other projects.
A Windows-specific DLL that can simulate key presses, mouse movements, and so on. MPJ: I added the Ffidl wrapper code to access all the AutoIt functions.
A Java compiler testing framework that was developed for open-source Java compiler projects because the official Sun development kits have not been released. Jacks rivals the Sun kits in capability, and may soon even have the same number of compatibility tests.
Harness for Algorithm Testing and Evaluation. Can do both regression tests and statistical analysis of different algorithms applied to accomplish a single task.
A Tcl library for unit testing iRules which are used when configuring F5 BigIP devices. The goal of this library is to make it easy to unit test iRules used when load balancing HTTP traffic.
A transparent application input fuzzer. changes "random bits in" the file and socket accesses it detects on its own. There's no particular connection to Tcl, I believe, except that combines with it nicely.
testing system for Sybase. One of the largest Tcl code bases.
Description
JCG writes: "All of tcltest, simpletest and DejaGNU have a similar goal, IMHO: that of providing a /framework/ for testing, although both tcltest and simpletest are conceived for Tcl testing while DejaGNU seems more suited towards C/C++. None of them provide a full fledged environment useful for non-programmers, though."
And don't forget about the tcltest package, which is a test harness for writing and running tests in the Tcl test suite or for creating a customized test harness for an extension. It includes both documentation and a huge set of examples (the Tcl and Tk test suites).
RS: A basic testing skeleton (further developed in e.g.) might look like this:
proc test {command expected} {
catch {uplevel 1 $command} result ;# so we can use variables in caller's scope
if {$result ne $expected} {error "$command->$result, expected $expected"}
}
Instead of error, you might use puts, so the test suite doesn't stop at the first surprise. Example for testing the test:
test {expr {2+3}} 5
The Suitability of Tcl for Testing
In response to a question about testing C and Java from Tcl, Cameron Laird responded eloquently:
"Tcl is the single most successful language for test automation of the kind you're describing. That's so undeniable that I'd forgotten it deserves to be documented for new generations of developers. DejaGnu is the testing framework that's made gcc possible. Companies like Oracle and Sybase have MILLIONS of lines of Tcl testing code on which they rely to assure the operation of their 'flagship' products.
Given enough motivation, I could do a whole book on use of Tcl in testing. Tcl tests automobile engines, emergency telephone circuits, chemical sensors, microprocessors, rocket components, industrial ovens, and much, much more. In the absence of any other knowledge, ANY software project should think of Tcl as its first choice for testing."
The original Dart framework was written in Tcl. Recently, it was rewritten in Java.
The Dart framework produces the pretty test status dashboards that one sees for Vtk. More information about Dart is at http://public.kitware.com/Dart/HTML/Index.shtml . Some examples of Dart's output are at [L2 ] and [L3 ]. Dart, like Vtk, is open-source.
Test in a simulated environment that makes all external effects deterministic. See discrete event simulation
Execute the program in a single-threaded mode for testing.
Produce multiple implementations of a function, and test that the produce substantially similar results.
Introduce errors and challenge the testing suite to find them.
Create a test case generator that produces tests a fuzz tester can use for reference.
Fuzz test with as much tracing as possible enabled in order to quickly see where things are going wrong.
test a variety of possible inputs, particularly the edge cases. See mutation testing .
Articles
Articles:
CL's article about Mo DeJong, detailing the benefits of a strong regression test policy. [L4 ]
Michael J. Norton's "Network Test Automation with Mac OS X and Tcl" [L5 ]
Ahmet C. Keskin, Till I. Patzchke, and Ernst vonVoight. "TclTk: A Strong Basis for Complex Load Testing Systems." Proc. 7th Intl. Tcl/Tk Conf. Austin, Texas: USENIX, 14--18 February 2000, pp. 53--60. [L6 ]
Carsten H. Lawrenz and Rajkumar Madhuram. "Using Tcl to Build a Buzzword-Compliant Environment That Glues Together Legacy Analysis Programs" Proc. 7th Intl. Tcl/Tk Conf. Austin, Texas: USENIX, 14--18 February 2000, pp. 61--70. [L7 ]
Paul Amaranth. "A Tcl-based Multithreaded Test Harness." Proc. 6th Intl. Tcl/Tk Conf. San Diego, California: USENIX, 14--18 September 1998, pp. 69--78. [L8 ]
C. Allen Flick and S. Dixson. "Using TCL/TK for an Automatic Test Engine." Proc. 6th Intl. Tcl/Tk Conf. San Diego, California: USENIX, 14--18 September 1998, pp. 79--88. [L9 ]
David E. Smyth. "Tcl and concurrent object-oriented flight software: Tcl on Mars." Proc 2nd Tcl/Tk Workshop. New Orleans: CPU, 23--25 June, 1994, pp. 3--10. [L10 ]
Arjen Markus' paper "Generating test programs with TestMake" [L11 ] Dead link US