V0.95 Regression Tests

The regression tests are a collection of test suites designed to test PREDATOR's functionality. Each suite consists of multiple tests which check small portions of PREDATOR. For instance, there is a suite which tests relational join functionality, and it includes many tests which test different kinds of joins.

The regression tests are designed as a development tool. Before committing new code to the PREDATOR codebase, the regression tests must be run to insure that the new code does not upset previous functionality. In addition, when new code is added, so too must a regression test which tests that functionality.

The mode of testing is simple:

Start a database server.
Start the regression tester program (java\regression\RegressionTester.java)
The regression tester runs multiple queries and spits out any deviations from expected results.
When the tester is done, shutdown the server.

The specifics of how to actually run these steps varies on WindowsNT and Solaris.

Running on WindowsNT

Open a command prompt and cd to P:\. Type setvar to establish the environment. Then run Q:\suites\runserver which starts the server (this is a script which runs predserver.exe with the appropriate command-line flags.)
Within DevStudio, set the regression project as your active project and run it (F5 key or CTRL-F5 key). This opens a command prompt window and starts running the regression tests. You should see a number of test names appear in the window. In the server window, you should see execution times appearing as commands are executed.
If there are any differences from the expected output, a "failed" message will be appear in the command prompt indicating the name of the failed test.
When the tester is done, shutdown the server by typing shutdown in its window.

Running on Solaris

In one windown, start the server by running $PREDATOR_LAIR/suites/runserver.csh (this script starts the server with the appropriate command-line flags).
In a separate window, 'cd' to $XXXROOT/java/regression. Make sure to source your ~/.cshrc file that sets the appropriate environment variables. Ensure that Gnu make appears ahead of any other make program in your path. Run make auto which will run the regression tests. You should see a number of test names appear in the window. In the server window, you should see execution times appearing as commands are executed.
If there are any differences from the expected output, a "failed" message will be appear in the test window indicating the name of the failed test.
When the tests are done, shutdown the server by typing shutdown in its window.

The tests are organized within a suites directory (Q:\suites on NT and $PREDATOR_LAIR/suites on Solaris). Each test suite is a subdirectory, which contains a number of specific tests. Each test is defined by a .run file and a .correct file with the expected answer. When the tests are run, the output generated is written to a suitesout directory (Q:\suitesout on NT and $XXXROOT/suitesout on Solaris), with a separate file for each test Each output filename has a prefix with the suite name and the test name.

Tips and TroubleShooting

Make sure that you followed all the stages of the setup procedure, especially setting the environment variables, before running the regression tests.
It may take a while to run all of the tests; a few of the tests are particularly large, so it may appear as if the system has hung. It most likely has not, so don't kill the regression tester.
Another possible problem is that you may not have enough disk space. The regression tests have to create test "disks" and log files for them. These occupy about 100MB totally. If you do not have enough disk space, you could reclaim some space by deleting the object files and libraries from your compilation.
If the server crashes, you have a serious error that needs to be reported to us.
If one or more tests fail, you can look at the cause of the failure by comparing the output files with the correct expected output. Sometimes, the difference is due to variations in tuple order in a relational result (which is actually not an error).
In the current V0.95 pre-release, a few tests may "fail", but these are not really errors in the server (but rather, errors in the regression testing script which is too naive in checking for correct answers).
To selectively run a few tests, edit the files Q:\suites\NT.txt or $PREDATOR_LAIR/suites/UNIX.txt respectively

Solaris-Specific

If the server fails to start, it is probably an error in the shared-memory setup on your machine (see the FAQ for ways to fix it).
If you kill the tester midway (or it dies for some reason), run $PREDATOR_LAIR/suites/cleanup.csh which will kill the abandoned processes.

If all (or most) of the tests succeed, your installation is valid, and you can proceed to use the system. An introductory tutorial may help you get started. If you have problems with the regression tests, send mail to predator-support@cs.cornell.edu with details of the problems that arise.

Mail user support: predator-support@cs.cornell.edu .... Back to PREDATOR Home Page