Test Reflexology, Part 1 (second post)

Download the Podcast

...continued from previous post...


Overly Protective Test
Sometimes when examining a test we find it to be much larger in size than the production class. Oftentimes we can just split the test into multiple tests – but not in this case – remember our initial assumption is that the tests are as good as they can get. What could be the cause then? It could be because the test is overly protective.

In a protective test we end up testing not only the specified behavior, but we are also testing to ensure that another behavior implemented by the tested unit does not interfere with the original behavior. For example, if unit X deals with a computation and with data caching, we will need to ensure that as the results of the computation are independent of, for example, inserting a new item into the cache. 

The need for such a test is often a result of perfect hindsight – a bug.   For instance, we discover later that certain computations accidentally alter the items in the cache.

Before fixing the bug, a developer practicing TDD will update the test to ensure that the secondary behavior never again interferes with the primary behavior. This is a good idea, but the need to create this overly protective test indicates a design issue – yes, you’ve guessed it; we have another problem with cohesion. The lack of cohesion leads to missing encapsulation[5] which has allowed the secondary behavior to couple unexpectedly to the primary behavior and affect it. The solution is to extract the behaviors into individual, encapsulated entities and prevent the coupling from occurring. Encapsulated entities cannot encroach on each other’s state.

Once we see an overly protective test we are sure to see it many times, whenever the primary behavior is tested conjunction with a different secondary behavior. This is done to insure that none of other activities of the unit interferes with the primary behavior we really want to test. The number of discrete test scenarios will increase geometrically because all behaviors need to be tested in conjunction with all of the other behaviors. Even if it were possible to create all these scenarios and test them in a reasonable period of time, there is an obvious redundancy in the tests which is highly undesirable[6]. 

Combinatorial Scenarios
We often see tests that do not test all the possible scenarios but rather a selected subset of the possible scenarios. This is because there are just too many of these scenarios to reasonably go through all of them. For example, let’s assume that during the week an employee is allowed to be late at most once. If we treat the week as five distinct days: Mon, Tue, Wed, Thu, and Fri, we will need to test the 5 scenarios where the employee is late once to prove that there is no action taken; we also need to test the 20 scenarios where the employee is late twice to make sure that an action is taken. This test reeks of repetition and would either be very long or would require some parametrization or iteration built into it to reduce the number or redundant scenarios. Alternatively, we may choose to test only a subset of the scenarios which leads to incomplete testing.

The test is shedding a light on a problem, namely that we are not choosing the correct abstractions in our design. In the example above, we should have considered the week to be a collection of days and verified that if in that collection we have only one day of tardiness – no action is taken, and if two – an action is taken. In this case we only need 2 tests to guarantee that the behavior is correct. This makes the complexity of the test the same as the complexity of the code and proves that the correct abstraction in this case is of a collection of days rather than individual days.

Stay Tuned for Test Reflexology, Part 2
Part one has focused on how a given unit test can provide you with insights about the quality of your design.  Part two will extend this notion into the entire suite of tests; how the nature of the suite can also let you know when your design may be wanting.  Coming soon!


-----

[5] Inside the scope of a class, we really cannot encapsulate much.  “Private” means nothing.  Temporary method variable are really the only encapsulation available “inside the curly braces”.

[6] This will be the subject a future blog, we promise.