Guideline: Developer Testing
This guideline describes approaches for performing developer testing on the software units.
Relationships
Main Description

Testing Philosophies

Here are some important philosophies with regards to testing:

  1. The goal is to find defects. Successful tests find bugs, but correcting them falls into other areas.
  2. Test often and early. The cost of change to rises exponentially the longer it takes to find and then remove a defect. The implication is that you want to test as early as possible (the earliest you could possibly test is first).
  3. Testing builds confidence. Many people fear making a change to their code because they are afraid that they will break it, but with a full test suite in place if you do break something you know you will detect it and then fix it.
  4. One test is worth a thousand opinions. You can tell me that your application works, but until you show people the test results they will not believe you.
  5. Test to the risk. The riskier something is, the more it needs to be reviewed and tested. In other words you should invest significant effort testing in an air traffic control system but nowhere near as much effort testing a “Hello World” application.
  6. You can validate all artifacts. You can test all your artifacts, not just your source code, although the focus of this guidance is testing code.

Qualities of a Good Developer Test

These are the qualities of a good developer test:
  • It runs fast. They have short setups, run times, and break downs.
  • It runs in isolation. You should be able to reorder your tests.
  • It is understandable. Good tests have consistent and informative names and use data that makes them easy to read and to understand.
  • It uses real data. E.g. Use copies of production data when appropriate, but remember that you'll also have to create some specific "artificial" test data as well.
  • It is cohesive. The test represents one step towards your overall goal. The test should address one and one only issue.

Approaches for Test Setup

To successfully run a test, the system must be in a known state.  To do this you will need objects or components in memory, or rows in your database, etc. which that you will test against.  The easiest approach is to hardcode the required data and setup code within the test itself.  The primary advantages are that all the information that you need about the test is in one place and that they test is potentially self sufficient.

Another approach is to define an external data set which is loaded into memory or into the database at the beginning of the test run.  There are several advantages to this approach.  First, it decouples the test data from the test.  Second, more than one test could use the same data set.  Third, it is easy to modify and/or multiply the test data.  The primary disadvantages are the increased complexity for maintaining the external data and the potential coupling between test cases (when they share a common test data bed it becomes very easy to write tests which depend on other tests running first, thereby coupling them together).

Coding for Testability

Instrument the code for testing and debugging.  Pay special attention to the implementation of the observation/control points, as these aspects might need special support that has to be implemented in the component under test.

Reviewing Tests

If a test will be long-lived, ask a person with less inside knowledge to run it and check if there is enough support information. Review it with other people within the development team and other interested parties as needed.