[Tcl] has long traditions in testing [[put in some general-purpose testing reference here--Beizer, ...]]. Especially notable are [Expect], [Oracle], Sybase [http://www.usenix.org/publications/library/proceedings/tcl96/grady.html], NASA [ftp://ftp.procplace.com/pub/tcl/sorted/misc/Tcl_on_Pathfinder/Tcl_on_Pathfinder.ps.gz], and other projects. The GNU [C] Compiler uses Tcl scripts and [Expect] to do regression testing. - [tomk] Please use this page to document success stories and information on how Tcl has been used and can be used to build a working test environment. "All of tcltest, simpletest and DejaGNU have a similar goal, IMHO: that of providing a /framework/ for testing, although both tcltest and simpletest are conceived for Tcl testing while DejaGNU seems more suited towards C/[C++]. None of them provide a full fledged environment useful for non-programmers, though", writes [JCG]. And don't forget about the [tcltest] package, which is a test harness for writing and running tests in the Tcl test suite or for creating a customized test harness for an extension. It includes both documentation and a huge set of examples (the Tcl and Tk test suites). [SimpleTest] is a [tcltest] work-alike for projects using [The Simple Library]. See also [Stress testing]. [RS]: A basic testing skeleton (further developed in [e.g.]) might look like this: proc test {command expected} { catch {uplevel 1 $command} result ;# so we can use variables in caller's scope if {$result ne $expected} {error "$command->$result, expected $expected"} } Instead of [error], you might use [puts], so the test suite doesn't stop at the first surprise. Example for testing the test: test {expr {2+3}} 5 ---- [Jacks] is a Java compiler testing framework that was developed for open source Java compiler projects because the official Sun development kits have not been released. Jacks rivals the Sun kits in capability, and may soon even have the same number of compatibility tests. ---- The Dart framework produces the pretty test status dashboards that one sees for [Vtk]. More information about Dart is at http://public.kitware.com/Dart/HTML/Index.shtml . Some examples of Dart's output are at [http://public.kitware.com/VTK/Testing/Dashboard/MostRecentResults-Nightly/Dashboard.html] and [http://public.kitware.com/VTK/Testing/Dashboard/MostRecentResults-Nightly/TestOverviewByTest.html]. Dart, like Vtk, is open-source. The CVS repository for Vtk is cvs -d :pserver:anonymous@public.kitware.com:/cvsroot/VTK checkout VTK Password is "vtk". The repository for Dart is cvs :pserver:anonymous@public.kitware.com:/cvsroot/Dart checkout Dart Password is "dart" ---- In response to a question about testing C and Java from Tcl, [Cameron Laird] responded eloquently: "Tcl is the single most successful language for test automation of the kind you're describing. That's so undeniable that I'd forgotten it deserves to be documented for new generations of developers. [DejaGnu] is the testing framework that's made gcc possible. Companies like Oracle and Sybase have MILLIONS of lines of Tcl testing code on which they rely to assure the operation of their 'flagship' products. Given enough motivation, I could do a whole book on use of Tcl in testing. Tcl tests automobile engines, emergency telephone circuits, chemical sensors, microprocessors, rocket components, industrial ovens, and much, much more. In the absence of any other knowledge, ANY software project should think of Tcl as its first choice for testing." What more do you need? [RLH] How about the book? : ) ---- [LV] responds - ''I know what '''I''' need - help knowing where and how to get started.'' ---- Articles: * [CL]'s article about [Mo DeJong], detailing the benefits of a strong regression test policy. [http://softwaredev.earthweb.com/sdopen/article/0,,12421_630531,00.html] * Michael J. Norton's "Network Test Automation with Mac OS X and Tcl" [http://www.oreillynet.com/pub/a/mac/2001/10/26/tcl_osx.html] * Ahmet C. Keskin, Till I. Patzchke, and Ernst vonVoight. "TclTk: A Strong Basis for Complex Load Testing Systems." ''Proc. 7th Intl. Tcl/Tk Conf.'' Austin, Texas: USENIX, 14--18 February 2000, pp. 53--60. [http://www.usenix.org/publications/library/proceedings/tcl2k/full_papers/keskin/keskin.pdf] * Carsten H. Lawrenz and Rajkumar Madhuram. "Using Tcl to Build a Buzzword-Compliant Environment That Glues Together Legacy Analysis Programs" ''Proc. 7th Intl. Tcl/Tk Conf.'' Austin, Texas: USENIX, 14--18 February 2000, pp. 61--70. [http://www.usenix.org/publications/library/proceedings/tcl2k/full_papers/lawrenz/lawrenz.pdf] * Paul Amaranth. "A Tcl-based Multithreaded Test Harness." ''Proc. 6th Intl. Tcl/Tk Conf.'' San Diego, California: USENIX, 14--18 September 1998, pp. 69--78. [http://www.usenix.org/publications/library/proceedings/tcl98/full_papers/amaranth/amaranth.pdf] * C. Allen Flick and S. Dixson. "Using TCL/TK for an Automatic Test Engine." ''Proc. 6th Intl. Tcl/Tk Conf.'' San Diego, California: USENIX, 14--18 September 1998, pp. 79--88. [http://www.usenix.org/publications/library/proceedings/tcl98/full_papers/flick/flick.pdf] * Steven Grady, G. S. Madhusudan, and Marc Sugiyama. "QuaSR: A Large-Scale Automated, Distributed Testing Environment." ''Proc. 4th Intl. Tcl/Tk Conf.'' Monterey, Calif.: USENIX, 10--13 July 1998, pp. 61--68. [http://www.usenix.org/publications/library/proceedings/tcl96/grady.html] * David E. Smyth. "Tcl and concurrent object-oriented flight software: Tcl on Mars." ''Proc 2nd Tcl/Tk Workshop.'' New Orleans: CPU, 23--25 June, 1994, pp. 3--10. [ftp://ftp.procplace.com/pub/tcl/sorted/misc/Tcl_on_Pathfinder/Tcl_on_Pathfinder.ps.gz] * [Arjen Markus]' paper "Generating test programs with TestMake" [http://www.tu-harburg.de/skf/tcltk/papers.html] ''Dead link [US]'' * ... * [[ [Android], ...]] * ... * http://www.commsdesign.com/design_corner/OEG20020123S0058 ---- The Open CASCADE 3D modeling application development platform relies on Tcl for its testing framework [http://www.opencascade.org/getocc/require/]. [GS] - Tcl can do more with OCC: it is used for prototyping applications with a tool called Draw. ---- [AutoIt] [http://www.hiddensoft.com/AutoIt/] is a Windows-specific DLL that can simulate key presses, mouse movements, and so on. It '''looks''' like something that would be fun to wrap with Tcl. I know of no one who's done that, though ... [MPJ] ~ I just added the [Ffidl] wrapper code to access all the AutoIt functions here [http://mini.net/tcl/4016]. ---- More testing products: [cwind], [WinTclSend], ... [MR] A good directory of systems... http://www.opensourcetesting.org/ which unfortunately only lists 3 tools for doing tcl unit testing... ---- "[Techniques for 'driving' Windows applications]"; [FIT]; [TETware] One product that addresses the same need in the [Mac OS X] world is Redstone Software's Eggplant. ---- [AM] Here is another page on testing: [Generating random tests] ---- [Joel Spolsky] writes on testing sometimes: "Top Five (Wrong) Reasons You Don't Have Testers" [http://joelonsoftware.com/articles/fog0000000067.html], ... ---- There is the [tclwebtest] package for testing web applications. And the [Pipeline programming] driver. ---- An important landmark in the testing vista is "IEEE 829" [http://en.wikipedia.org/wiki/IEEE_829]. ---- See also [STAF], [tclunit]. ---- See also FDR http://www.fsel.com/software.html , which Stephan Kuhagen says (on comp.lang.tcl) "makes heavy use of Tcl/Tix". ---- 'Nother testing project: describe a Tcl perspective on * unit * component * system and * functional testing, then do the same for * standard conformance * dependency analysis * coverage * complexity then integrate stress testing and other performance metrics. ---- ... and, in quite a different direction, zzuf [http://packages.debian.org/unstable/devel/zzuf] is a fuzzing package which changes "random bits in" the file and socket accesses it detects on its own. There's no particular connection to Tcl, I believe, except that combines with it nicely. ---- [COMPANY: Deutsche Börse Systems] ---- Today, while looking at an email about Tcl, my gmail account pointed me to http://www.fanfaresoftware.com/info/FunctionalTestingWhitePaper.htm which is a product which is test automation software for validating the functionality of software embedded in high-tech equipment... ---- !!!!!! %|[Category Debugging] | [Category Dev. Tools] | [Category Testing]|% !!!!!!