Performing Configuration Testing Using HTTPD-Test's Perl Framework

by Martin Brown

Martin Brown plunges deeper into the world of HTTPD-Test with an overview of the Perl Framework, a module that tests the configuration and the components of Apache to ensure it is working as it should and that the modules are properly compiled and installed.

HTTPD-Test is a collection of tools that provide ways of testing HTTP servers in general. The previously covered Flood (see Staying Out of Deep Water: Performance Testing Using HTTPD-Test's Flood), is one such tool. As the name implies, Flood enables administrators to test the performance of a Web server by flooding it with requests.

The Perl Framework, which is part of the same suite, concentrates on testing the configuration and components of Apache (the core binary and its associated support modules) on your platform to ensure the configuration works and the modules have compiled and installed properly.

By now you're probably thinking this sounds pretty useless -- once you've installed Apache, especially from a pre-built binary installer, why would you want (or need) to test it?

The simple answer is that many modules -- even the core ones -- rely on the functionality of external libraries. If you periodically update your system, either manually or automatically through something like the RedHat Network, then you could break your Apache installation without even realizing it. The problem could be so obscure that it never turns up in standard tests and investigations, and you probably wouldn't know about it until a user complains.

In a deployment situation, whether installing Apache on a number of identical machines or on a range of different platforms, you must ensure that your Apache installation is as stable and compatible as possible. Often simply starting Apache or using the configtest argument to apachectl will not be enough to highlight a potential problem.

The Perl Framework was originally designed to help test the mod_perl v2.0 extension for Apache. While the framework is compatible with both Apache 1.3.x and Apache 2.0, the modular nature of Apache 2.0 makes the process significantly easier, and the tests provided by the framework in 2.0 are much more extensive.

Installing the Framework

To use the Framework you'll need Perl and a variety of additional modules, as well as the main HTTPD-Test distribution itself. If you haven't already downloaded the HTTPD-Test distribution, use the following to download the latest version from the Apache CVS server (the password is 'anoncvs'):

$ cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic login
$ cvs -d :pserver:anoncvs@cvs.apache.org:/home/cvspublic co httpd-test

Once you have that available, the easiest way to get the necessary Perl modules is through CPAN:

perl -MCPAN -e 'install Bundle::ApacheTest'

The above will download and install all the modules required by the Perl framework. If you've got CPAN set to follow prerequisite modules, it will download or update all of the other required modules.

The next step is setting up and building the Perl framework. First, change to the perl-framework directory within the httpd-test directory. If you are using a dynamic version of Apache, you should have apxs tool, and you can build the suite using the -apxs command line option and specify the tools location, e.g.,:

perl Makefile.PL -apxs /export/http/apache2/bin/apxs

Otherwise, just use:

Perl Makefile.PL

This will build the necessary components for you to continue and run the actual tests.

The next step is run make to create all the necessary files and get ready to run the tests.

Running the Tests

The tests themselves are located in the t directory below the main perl-framework directory. One global script, TEST, provides an interface to the underlying test routines and summarizes the output of the various tests for you. You can also run tests individually by specifically requesting them on the command line.

Before you start, bear in mind how the tests actually work. The Perl framework is designed to test the modules and components in an Apache installation. That means it uses the Apache installation directly -- it doesn't build or install a copy for you. Therefore, your Apache configuration should be exactly as you are planning to use it, and you must ensure it's the right version of Apache, configured during build time with the right options, and that your DSO modules are up to date and installed (assuming you are using them).

The framework does not use either your configuration file or documents from your Web site. Test documents are included with the distribution, located in the t/htdocs directory. The configuration file is generated just before the test sequence starts running the actual tests, which is almost immediately before the Apache server is started. You can set certain options when you start the TEST script. For more in-depth modifications, the best way is to add or edit the file t/conf/extra.conf.in with your desired settings. This will be incorporated into the temporary config file generated before the tests are executed.

Running All the Tests

You can run all the available tests by simply typing:


The above will use the httpd server identified by apxs, if you have it. If you want to specify an alternative server, use the -httpd command line option:

t/TEST -httpd /export/http/apache2/bin/httpd

Even on a relatively fast machine, it will take a while for all the tests to be run on the entire suite and for a summary to be generated. Also, be aware that the first time you run the tests it will probably compile a few modules and create a few tests. Once all the tests have been run, the output will likely be similar to this:

*** setting ulimit to allow core files
ulimit -c unlimited; t/TEST 'modules'
*** root mode: changing the fs ownership to 'nobody'
/export/http/apache2/bin/httpd  -d
/export/contrib/build/solaris/httpd-test/httpd-test/perl-framework/t -f
using Apache/2.0.44 (prefork MPM)
waiting for server to start: .
waiting for server to start: ok (waited 0 secs)
server localhost:8529 started
server localhost:8530 listening (mod_nntp_like)
        all skipped: cannot find
module 'info'
        all skipped: cannot find
module 'rewrite'
        all skipped: cannot find
module 'vhost_alias'
All tests successful, 7 tests and 6 subtests skipped.
Files=19, Tests=1086, 104 wallclock secs (57.64 cusr + 9.89 csys = 67.53 CPU)
*** server localhost:8529 shutdown

Note that this is truncated; typical output will be about two pages long.

What you are looking for here is a failure when running the tests. Therefore, you can generally ignore "skipped" notices, they just indicate that a required module is unavailable. However, take notice of them if you thought the module was installed correctly.

If one of the module sets fails, it's time to start drilling down and running test sets and, ultimately, individual tests.

Running a Specific Test

If you look in the t directory, you'll see a number of subdirectories that contain the tests for specific areas. For example, the standard toolkit incorporates tools for checking the main Apache setup, the configuration files, and PHP. To select which tests to run, just specify the name of the test on the command line to the TEST script. For example, to run the tests for the additional modules in Apache:

t/TEST modules

To run the tests on the include module:

t/TEST modules/include

The further down you drill, the more likely you are to find the precise location of the problem.

Tracing a Problem

Tracing a problem is actually a bit more difficult than it at first seems. Even if you can determine which test within a particular set failed, the only way you are going to find out the reason for the problem is to check the test scripts and associated documents and determine where the problem lies. With about 25,000 lines of code and sample documents to work through in the entire set, it's not a job for faint hearted

You also need to be careful -- just because the script returns an apparently fine execution doesn't mean there aren't problems. It's always worth checking the error_log generated by Apache, located in the t/logs directory. Often this can highlight errors raised by the server that the test harness never picks up. But, always compare it to the test scripts -- you might find that the script is actually testing for a failure, in which case, an error in the error log really indicates a success, not a failure!

This article was originally published on Monday Jul 7th 2003
Mobile Site | Full Site