V0.1 Regression Tests

[ Running | Creating a new Test ]

The regression tests are a collection of test suites designed to test PREDATOR's functionality. Each suite consists of multiple tests which test small portions of PREDATOR. For instance, there is a suite which tests relational join functionality, and it includes many tests which test different kinds of joins. The tests are performed by starting a server for each suite and a client for each test in that suite which sends queries to the server. The responses the clients receive are then compared against "correct" output, and errors are reported if something went wrong. [The actual comparison is more complex: the output is first sorted by lines (since different algorithms produce the same results in different orders), then whitespace is eliminated, and the resultant files are compared.]

The regression tests are designed as a development tool. Before adding code to the PREDATOR codebase, the regression tests must be run to insure that the new code does not upset previous functionality. In addition, when new code is added, so too must a regression test which tests that functionality.

Running

All suites

If you have not already done so, create an environment variable called XXX_SERVER_PORT that has some random port number above 5000. (You may wish to add this variable to your .cshrc file.) Then in your $XXXROOT/regression/ directory, type make.

setenv XXX_SERVER_PORT 5776
cd $XXXROOT/regression
make

Each of the test suites (located in their respective directories in the $XXXROOT/regression/suites directory) will be run in turn, and error messages will be output to the screen. If a test fails, you will want to examine the contents of regression/suites/suitename/out/testname.out to determine what went wrong. You can find the instructions piped to the client in regression/suites/suitename/run/testname.run. The output of the server can be found in regression/suites/suitename/out/serverrun.out.

Some Useful Tips

It may take a few minutes to run all of the regression tests; a few of the tests are particularly large, so it may appear as if the system has hung. (It most likely has not, so don't hit Ctrl-C! If you do, run the script cleanup, which will kill off the abandoned processes.)

However, if it you do not see progress of any kind, your setup might have been problematic. Often, this manifests itself as an endless sequence of dots on the screen. If you have more than about 15 dots without any other kind of message, it means that your server failed to start. In that case, type Ctrl-C and then run cleanup. Then examine the output files described above; if you cannot determine the problem, contact predator-support@cs.cornell.edu.
Another possible problem is that you may not have enough disk space. The regression tests have to create test "disks" and log files for them. These occupy about 100MB totally. If you do not have enough disk space, you could reclaim some space by deleting the object files and libraries from your compilation. Another option is to edit the runsuites file in the regression directory, changing 50000 to a smaller number. A couple of the test suites may not run because of insufficient disk space, but the rest will be fine.

If all the tests succeed, your installation is valid, and you can proceed to use the system. An introductory tutorial may help you get started. If you have problems with the regression tests, send mail to predator-support@cs.cornell.edu with details of the problems that arise.

Selected suites

You can run a subset of the suites by providing their names as parameters to runsuites, e.g.,

runsuites image audio counter

Creating

Adding a new regression suite

A new regression test suite should be added to test an entirely new functionality of the PREDATOR system, e.g., a new data type or a new SQL statement. To do this, follow these steps:

In your $XXXROOT/regression/ directory, type
```
makesuite <suite_name>
```
where suite_name is the name of the new suite. This script not only creates a local directory tree for your script, but an RCS tree for it too. For this reason, please do not run the script unless you are creating a real regression suite.
In the suite's run directory, ($XXXROOT/regression/suites/suite_name/run) create the test files, each comprised of commands you wish to be piped directly to the client. Append a .run extension to the name of each test script. Remember that all scripts will be run by the same server, so if two scripts create the same table and neither drops it, an error will result.
Run the suite by typing the following:
```
       cd $XXXROOT/regression
       runsuites <suite_name>
       
```
It will not work, but it will provide the .out files in the out/ subdirectory.
Check that the .out files associated with your test scripts are correct, and then move them to your correct/ subdirectory.
Run the suite again. Now it should work.
Check in the .run and .out files.

Adding a new test to an existing suite

To add a new test to an existing suite, follow these steps:

Create a new test file in the run/ subdirectory. Be sure to append a .run extension to its name.
Create a new correct output file in the correct/ subdirectory as described above.
Check in both files.

Mail user support: predator-support@cs.cornell.edu .... Back to PREDATOR Home Page