Task: Run Tests
Execute the appropriate collections of tests required to evaluate product quality. Capture test results that facilitate ongoing assessment of the product.
Discipline:  Test
Purpose
To run the system test suite.
Relationships
RolesPrimary Performer: Additional Performers:
InputsMandatory: Optional:
  • None
Outputs
Main Description
Run the system test suite which addresses functional and system integration tests, and potentially user acceptance tests as well.
Steps
Schedule test execution
Your system test suite should be run as often as possible.  Ideally, you should run your tests whenever new code is checked into your version control tool.  For larger systems this will prove to be too costly, your test suite may take several hours to run, and therefore it needs to be scheduled less often.  If possible run the test suite several times a day, minimally run the suite each night but if possible also try to run it during working hours.
Execute test suite

Run the test suite at the scheduled time.  This should be automated.

Good practices:

  1. Run the test suite in a separate test environment.
  2. Ensure that you run the test suite against the latest clean build.
  3. The first step of the test suite should be to setup the test environment (e.g. ensure that the network is available, that the test database is available and reset to a known state, and so on).
Close test suite run

The last step of the test suite should be to close the run. To do this, you must:

  1. Close the test log(s).  The appropriate test log file(s) should be closed and if appropriate placed in the appropriate folder or directory.
  2. Announce results.  A notice should be sent out to any concerned person(s) informing them of the result of the test run and where the test log(s) have been placed.
Key Considerations
  1. The test suite should be run regularly, ideally whenever the code changes but minimally once a day.
  2. It should be possible for anyone on the test team to run the test suite at any time.
More Information