NaviServer Testing

Main ✸ Source Code ✸ Downloads ✸ API Documentation ✸ Mailing Lists ✸ Bugs ✸ Developers

NaviServer ships with almost 1000 tests[L1 ]. Writing a test is a great way to communicate with other developers about a bug you may have found. Some one's going to have to test the bug fix anyway, right? So why not write a test to describe the bug, which will then also ensure the fix is to your satisfaction (and doesn't creep back accidentally in the future).

Check out a copy of the latest code:

 $ hg clone http://bitbucket.org/naviserver/naviserver/ naviserver-test
 $ cd naviserver-test

You can keep your clone up-to-date with the pull command (-u means also update the working directory, see: hg help pull):

 $ hg pull -u

Build the server with debugging symbols. It's a good idea to have a copy of Tcl around also built with debugging symbols, and possibly memory debugging available. I keep one in ~/local/tcl8.4.15g, one in ~/local/tcl8.4.15gmem and a plain one in ~/local/tcl8.4.15.

We're going to specify an alternate --prefix where the server would be installed, if we were installing it. Make sure it doesn't point to an existing install, or when the server runs during the tests it will link against any already installed libraries in preference to the freshly built ones. This can be surprising! It would really suck to spend an hour chasing a bug, wondering why your changes are not taking effect because you're testing the installed libraries.

 $ ./autogen.sh --with-tcl=/home/me/local/tcl8.4.15g/lib --enable-symbols --prefix=/tmp/ns
 $ make

Now you can run the tests.

 $ make test

There are a couple of variations on this. If you're debugging C code and the server is crashing, you'll want to use the debugger to find out what went wrong. We've already built Tcl and NaviServer with debugging symbols enabled, so all that's left is to run the server under gdb and make sure all the right libraries are loaded.

 $ make gdbtest
 (gdb) run

Easy. You might also want to do some interactive testing in naviserver command mode, with all ns_* commands available.

 $ make runtest
 % ns_log notice "Hello, World!"

This is actually a fully-fledged running server (port 8000 by default) with a driver thread, conn threads, and so on. The command prompt runs independently in it's own thread, and from there you can poke at the internals of the server as you use a browser to fetch pages. If you're trying to interactively crash the server, then this is what you want:

 $ make gdbruntest
 (gdb) run
 % ns_log notice "Hello, World!"

Only interested in the tests in one file? The full test suite can take a little while to run. It's not too bad, but if you're making multiple changes while chasing down one particular bug, the wait can become annoying. All the tests live in tests/*.test[L2 ] -- locate the file of interest and run it like this:

 $ make test TCLTESTARGS="-file http.test"

Want to fix one of our known bugs? Thanks! Here's how: tests which expose known but unfixed bugs are marked with the knownBug tcltest constraint. They don't run by default. You can enable a constraint, and thus enable those failing tests like so:

 $ make test TCLTESTARGS="-file http.test -constraint knownBug"

Uh-oh, your server crashed. What now? Unless you know C there may be not much you can do. But it's worth while narrowing down the possibilities, and the code is reasonably clean, so feel free to poke around. What failed?

 (gdb) backtrace

This gives you the equivalent of a Tcl stack trace. The function at the top of the stack is the most recent, but it may not be the most interesting if it's something from the C library or Tcl (unless it's a bug in Tcl). You probably want to move up the stack into some NaviServer code:

 (gdb) up 3

Moves up 3 frames (down the screen) and makes the new function the current one. It should tell you the source file name and the line number. Open that up in an editor side by side with your shell. Now you can investigate:

 (gdb) info locals

Prints out all the local variables. You can investigate further with the print command and basic C expressions. You can often spot a problem by printing out structures and looking closely at the contents.

 (gdb) print servers
 $3 = (Ns_Set *) 0x92df180

 (gdb) print *servers
 $4 = {name = 0x92df1a0 "ns/servers", size = 3, maxSize = 10, fields = 0x92f14c8}

Found something fishy? Now would be a good time to ask for help on the mailing list[L3 ].

Don't forget:

 $ make help
 Commonly used make targets:
   all          - build program and documentation
   install      - install program and man pages to PREFIX ()
   test         - run all tests in the automatic test suite
   gdbtest      - run all tests, under the control of the debugger
   runtest      - start the server in interactive command mode
   gdbruntest   - start the server in command mode, under the debugger
   memcheck     - run all tests, under the valgrind memory checker
   build-doc    - build the html and nroff documentation
   dist         - create a source tarball naviserver-4.99.3.tar.gz
   clean        - remove files created by other targets

 Example for a system-wide installation under /usr/local/ns:
   make all && su -c "make install"

 Example for running a single test in the test suite, under the debugger:
   make gdbtest TCLTESTARGS="-file tclconnio.test -match tclconnio-1.1"