Version 0 of A little experiment with table-driven testing

Updated 2009-03-30 08:26:00 by arjen

Arjen Markus (30 march 2009) As I am fascinated by all the possibilities that exist to automate testing (our tcltest package is a wonderful example of that), I thought I'd experiment a bit with the tabular format of specifying tests as promoted by FIT. Here is the result.

Some notes:

• Right now procedures are expected to behave a simple functions: they take arguments and return a value, without any side effects (relaxing that last condition would be quite problematic ...)
• Some details might be improved upon - there can be no values containing newlines now
• There can be no argument to the proc under test that is called "expected"
• There is no possibility to anticipate error conditions

Well, just a quick-and-dirty experiment, what do you expect?

---

# table-test.tcl -- # Experiment with FIT # #

# table-test -- # Generate tests based on a table of cases # # Arguments: # procname Name of the procedure to test # varnames List of variable names (arguments to the procedure # and the expected result) # values Values - each line is a test case # args Arguments to be passed to the test command directly # # Result: # None # # Side effects: # Runs the individual test cases and thus influences the test # statistics # # Note: # The test that is generated is simply of the following form: # - set the arguments according to the line in the "values" argument # - run the procedure to be tested # - the expected result is the column "expected" # - the lines in the table "values" must form a valid list # proc table-test {procname varnames values args} {

    package require tcltest

    #
    # Examine the procedure under test and the variable names
    #
    if { [lsearch $varnames "expected"] < 0 } {
        return -code error "The table does not contain a column \"expected\" - the expected results"
    }
    if {[catch {set arglist [info args $procname]} msg] } {
        return -code error $msfg
    }

    set argtest ""
    foreach arg $arglist {
        if { [lsearch $varnames "expected"] < 0 } {
            return -code error "The table does not contain a column \"$arg\" - one of the arguments to procedure \"$procname\""
        }
        append argtest " \$$arg"
    }

    #
    # We are ready now ...
    #
    set testidx 0
    set nvars   [expr {[llength $varnames] - 1}]
    set expidx  [lsearch $varnames "expected"]
    foreach line [split $values \n] {
        if { [string trim $line] == "" } continue
        set body ""
        foreach var $varnames value [lrange $line 0 $nvars] {
            append body "set $var \"$value\"\n"
        }
        append body $procname $argtest

        set expected [lindex $line $expidx]
        ::tcltest::test $procname-$testidx "$procname-$testidx" -body $body -result "$expected" {*}$args

        incr testidx
    }

}

# main -- # Test the whole idea #

# # Define a simple procedure # proc simpleProc {a b} {

    expr {$a*$b}

}

table-test simpleProc {

   a     b    expected} {
   1     1    1       # Okay
   2     2    4.0     # Not okay: we test strings!
   2     2    4       # Okay

}

::tcltest::cleanupTests

---