Running a test plan
You have a test plan. Now it's time to execute and see what this whole smoke-test thing is all about. Here is all that is required:
- Make sure the demo application is running and that you can hit it using
your browser or other appropriate client of the Web application (remember
Puffin isn't just for Web sites -- it can even test Web services).
- On the command line, navigate to the folder containing puffin.py and
type the following:
If not specified, Puffin will use the following (enter
for a full run-down on command-line arguments):
You should see the following:
Note that at #4, Puffin will appear to hang. This is exactly right as this script should take 30 seconds to execute (thus failing in terms of the TimerResponseAnalyzer).
If you got an error, check the following:
- All Puffin requirements are installed properly on your machine (Python, eGenix Base, PyXML, Puffin itself).
- Your puffinConfig.xml is both a valid XML document and correct.
- You don't have any spurious spellings in your test.plan file.
Here's what Puffin did:
- Loaded the config file (in this case the default) and initialized itself.
- Loaded the test plan and initialized it with the information from the configuration file.
- Executed each test action in the order it was listed in the test plan.
For each test action, this involves the following sub-steps:
- It processed any inputs.
- It made the call to the Web application and retrieved its results.
- It analyzed the results using both the StatusResponseAnalyzer and the TimerResponseAnalyzer.
- It processed any outputs and placed their values into the current token dictionary.
- Reported the status for each action on the command line (using the StreamHandler, remember).
- Wrote a file (puffinResults.log) containing a full report of the entire test plan's execution. If this file already exists, Puffin appends the results onto the end of the file.
- For each failure, Puffin wrote out a failure alert to a file in this case (puffinFailures.log). This could just as easily have been an e-mail address or similar.
- Puffin closed down.