Testing Philosophies
Here are some important philosophies with regards to testing:
-
The goal is to find defects. Successful tests find bugs, but correcting them falls into other areas.
-
Test often and early. The cost of change to rises exponentially the longer it takes to find and then remove a
defect. The implication is that you want to test as early as possible (the earliest you could possibly test is
first).
-
Testing builds confidence. Many people fear making a change to their code because they are afraid that they will
break it, but with a full test suite in place if you do break something you know you will detect it and then fix
it.
-
One test is worth a thousand opinions. You can tell me that your application works, but until you show people the
test results they will not believe you.
-
Test to the risk. The riskier something is, the more it needs to be reviewed and tested. In other words you should
invest significant effort testing in an air traffic control system but nowhere near as much effort testing a “Hello
World” application.
-
You can validate all artifacts. You can test all your artifacts, not just your source code, although the focus of
this guidance is testing code.
Qualities of a Good Developer Test
These are the qualities of a good developer test:
-
It runs fast. They have short setups, run times, and break downs.
-
It runs in isolation. You should be able to reorder your tests.
-
It is understandable. Good tests have consistent and informative names and use data that makes them easy to read
and to understand.
-
It uses real data. E.g. Use copies of production data when appropriate, but remember that you'll also have to
create some specific "artificial" test data as well.
-
It is cohesive. The test represents one step towards your overall goal. The test should address one and one only
issue.
Approaches for Test Setup
To successfully run a test, the system must be in a known state. To do this you will need objects or components
in memory, or rows in your database, etc. which that you will test against. The easiest approach is to hardcode
the required data and setup code within the test itself. The primary advantages are that all the information that
you need about the test is in one place and that they test is potentially self sufficient.
Another approach is to define an external data set which is loaded into memory or into the database at the
beginning of the test run. There are several advantages to this approach. First, it decouples the
test data from the test. Second, more than one test could use the same data set. Third, it is easy to
modify and/or multiply the test data. The primary disadvantages are the increased complexity for maintaining the
external data and the potential coupling between test cases (when they share a common test data bed it becomes
very easy to write tests which depend on other tests running first, thereby coupling them together).
Coding for Testability
Instrument the code for testing and debugging. Pay special attention to the implementation of the
observation/control points, as these aspects might need special support that has to be implemented in the component
under test.
Reviewing Tests
If a test will be long-lived, ask a person with less inside knowledge to run it and check if there is enough support
information. Review it with other people within the development team and other interested parties as needed.
|