Testing Best Practices: Test Categories, Part 3

download the podcast

Continued from Test Categories, Part 2

Constant Specification

We often find values that are significant to a given problem domain, but are otherwise arbitrary.  For example, the sales tax rate in a state might be .08 (8 percent), but this is just the rate as currently defined by that state’s government.

Similarly, we sometimes have enumerations that are also significant to a given problem domain.  In a baseball team, you have a Pitcher, Catcher, First Baseman, Second Baseman, Third Baseman, Shortstop, Left Fielder, Center Fielder, and Right Fielder.  Always these nine positions exist, but this is only because the rules of baseball say so.

Good coding practice says to avoid “magic numbers” in our code.  If we were developing a taxation application, we would not want to hard code .08 every time we needed the tax rate.  If we did, this would cause several problems:

• It would be in more than one place (redundant) and thus if it changed would have to be changed in more than one place.
• It would be meaningless to someone who did not happen to know it was a tax rate.
• Other systems depending on the same value would likely hard code it as well.

Because of this we tend to create system constants to represent these values.  Something like System.TAX_RATE which, once established in a single place, can be used throughout the code in the place of the literal value.  Or, for an enumerated type, we create a discrete enumeration to represent it: Positions.PITCHER, Positions.CATCHER, and so forth.

This is good practice, and in the past we simply created these constants as needed.  In TDD, however, we have to look at the issue differently.

In TDD we do not create implementation code until we first have a failing test, and we always create only that code needed to make the test pass.  In TDD as we are defining it, the “test” is actually not a test at all, but a specification of the system.  Specifications must be complete.

In other words, we must create a test that specifies these constants should exist before we create the code that means they do exist.  This is extremely simple to do, but is very often left out of the process when people first start doing TDD.

For a constant:

public void specifyTaxConstant() {
 Assert.AreEqual(.08, System.TAX_RATE);
}

Not a tremendous amount of work here, just more the realization that it needs to be done.  With an enumeration, it might be a tad more involved:

public void specifyPlayerPositions() {
 string[] players = Enum.getValues(Positions);
 Assert.AreEqual(9, players.length());
 Assert.Contains(“PITCHER”, players);
 Assert.Contains(“CATCHER”, players);
 Assert.Contains(“FIRST_BASEMAN”, players);
 Assert.Contains(“SECOND_BASEMAN”, players);
 Assert.Contains(“THIRD_BASEMAN”, players);
 Assert.Contains(“SHORTSTOP”, players);
 Assert.Contains(“LEFT_FIELDER”, players);
 Assert.Contains(“CENTER_FIELDER”, players);
 Assert.Contains(“RIGHT_FIELDER”, players);
}

Why would we do this?  It does seems silly at first.  But, as easy as this is to to, it is actually pretty important.

• This is s specification, remember.  In a traditional specification, these values would certainly be included.  In a tax application’s specification, you would certainly have, somewhere, “the tax rate is currently 8 percent” or words to that effect.  Or, “the positions in Baseball are:...”.
• If a value needs to be changed (if, for example, the state raises the tax rate), then we want to make the change to the specification first, watch the test fail now, and then change the implementation to make the test pass again.  This ensures that the spec and the code are always in sync, and that we always have tests, going forward, that can fail.  It is quite easy, in test maintenance, to change a test in such a way as to make it impossible for it to fail (a bug, in other words).
• If a value needs to be added (for example, Positions.DESIGNATED_HITTER) then, again, we have a place to change the spec first.
• Other developers on other teams will now know about these constants, and will use them too instead of magic numbers.

So a constant specification, as simple as it is, tells you four very important things (espcially later, when reviewing the spec).

1. That a constant was used
2. Where it is
3. What it’s called
4. What its current values are

That’s a lot of value for very little work!  It is important to note, however, that all other tests that access these constant values must use the constants rather than the literal values.  Just like any code we have to maintain, we don’t want redundancies.

Creational

Instantiating objects is the first thing that we need to do in any system. After all, before we use them, they have to be there, right?
So what is there to test (that is, specify) about the creation of objects?

There are two scenario we need to deal with

1. Simple, single objects
2. Compound objects.

Simple Objects
When it comes to simple objects we need to specify two things:

• The type of the object
• Its observed initial state

What is a type (as in ‘the type of the object is …’)? The type is a concept that we have identified as existing in our domain. By the word concept we mean that we identify it but we have no idea how it is implemented. The concept could be a high level concept such as shape or vehicle, or a very specific concept such as an equilateral triangle or a 2005 Honda Accord.

In order to use an object it must be instantiated, and that is done through some factory mechanism. We may use a factory object, a factory method (such as GetInstance()) or the new operator (which is, after all, the most primitive factory method).

The simplest case
Lets assume we run a furniture business and in our catalog we have a desk named Aaarty,
AartyDesk is a low level concept in our model. AartyDesk will need to be created, and here is the test for it:

//If in our language the new operator cannot fail and still return (as is the case with C# or Java) then the test is:
public void testCreateAartyDesk{
 newAartyDesk();
}

//if the construction could fail and still return a value (as is the case with c++)
public void testCreateAartyDesk {
 Assert.NotNull(new AartyDesk());
}

This is not too interesting, but we have just specified that the concept of an Aarty desk exists in our domain and that it can be created. The next step, to make it more interesting (and useful) is to specify the behavior associated with AartyDesk immediately after it was created -- what we call the initial behavior of the object, which, naturally, depends on the initial state of the object. Note that we should have no visibility to that initial state, only the effect it has on the initial behavior.

public void testCreateAartyDesk {
 AartyDesk aartyDesk = new AartyDesk();
 Assert.False(aartyDesk.isLocked());
 Assert.True(aartyDesk.hasProtectiveCover());
}

Note that it is up to you to decide whether this initial behavior should be captured in single or multiple test; we will discuss that in a later blog.

At this point, we can note that the actual instantiation could vary, for example using a static factory method:

public void testCreateAartyDesk {
 AartyDesk aartyDesk = AartyDesk.GetInstance();
 Assert.False(aartyDesk.isLocked());
 Assert.True(aartyDesk.hasProtectiveCover());
}

or a factory object:

public void testCreateAartyDesk {
 AartyDesk aartyDesk = DeskFactory.GetInstance();
 Assert.False(aartyDesk.isLocked());
 Assert.True(aartyDesk.hasProtectiveCover());
}

The latter two options are significantly better from design perspective as they allow change to occur more easily by encapsulation the construction.

Creating a higher-level abstraction
Our domain analysis revealed that there will be more than one type of desk in our store -- we were told that a new model is being introduced. We want to capture this knowledge through the higher level Desk abstraction. There are several ways to capture this in the tests, depending on the creation mechanism. Firstly let's consider the new operator:

public void testCreateAartyDesk {
 Desk desk = new AartyDesk();
}

This simple test tells the reader that an AartyDesk behaves like a Desk. The same holds for if we use the GetInstance method.

public void testCreateAartyDesk {
 Desk desk = AartyDesk.GetInstance();
}

And the same for a factory object:

public void testCreateAartyDesk {
 Desk desk = DeskFactory.GetInstance();
}

But wait! How can we be sure that the DeskFactory.GetInstance() actually returns an AartyDesk? For that matter, how do we know that AartyDesk.GetInstance()actually returns an AartyDesk? We need to specify that in the test. For the purpose of this code we will assume the existence of an AssertOffType assertion.

public void testCreateAartyDesk {
 Desk desk = AartyDesk.GetInstance();
 AssertOfType<AartyDesk>(desk);
}
And the same for a factory object:
public void testCreateAartyDesk {
 Desk desk = DeskFactory.GetInstance();
 AssertOfType<AartyDesk>(desk);
}

Choice
Factories have two responsibilities. The obvious one is the creation of objects. Before this, however, is the responsibility of choosing which type to instantiate. In the example so far, there was only one type of desk -- Aarty, so there was no need to make any selection. What happens when we add the next variation -- Shmaag. Now the factory needs to choose between Aarty and Shmaag.

There are different mechanisms that can be used to enable factory selection. In our case we will have a helper class that returns the type of buyer: Cheap or Refined. Someone who knows the buyer category needs to set this information before the factory is called. In the test case, that someone is that test itself. We will also assume that the test used to specify these two categories of buyer via BuyerCategory was already written . We will also assume that the buyerHelper object was likewise already tested..

public void testCreateAartyDesk {
 BuyerHelper.getInstance().BuyerCategory = BuyerCategory.Cheap;//[4]
 Desk desk = DeskFactory.GetInstance();
 AssertOfType<AartyDesk>(desk);
}

public void testCreateAartyDesk {
 BuyerHelper.getInstance().BuyerCategory = BuyerCategory.Refined;
 Desk desk = DeskFactory.GetInstance();
 AssertOfType<ShmaagDesk>(desk);
}

As before, further tests, or assertions within these tests are needed to specify the initial behavior of these Desk objects.

Composition
The next level of complexity arrives with the introduction of collections. There are 2 things that need to be specified when collections are created:

• The content of the collection
• The behavior of the collection

Specifying the content of the collection is done by specifying what the expected values are and what their initial behavior is. For example, in a football team you have many players with specific roles. Consider part of the offense: Quarterback (QB), Center (C), Running Back (RB), Fullback (FB), Tight End (TE), Left and Right Tackles (LT, RT), Left and Right Guards (LG, RG) and Wide Receivers (WR).

The specific makeup of the offensive team depends on the specific play that is called.  For example, if we’re going to run the ball we may choose to have no WR and if we’re gonna pass it we may elect to have 3.

The OffenceFactory object will return the 11 players that should be on the field based on the type of play, which will be a parameter to the GetInstance method. We can have a whole playbook’s worth of plays (which need to be defined in their own tests, of course). Here is one possible test -- We assume here that Type is the type of the object and it can be passed around. There are many possible implementation in various languages. The code below is highly pseudocodish, but easily implementable...

public void testShotgunOffenceComposition() {
 Type[] expectedOffenseRoles = new Type[]
   {QB, C, LG, RG, LT, RT, TE, WR, WR, WR, WR}
 Player[] offense = OffenceFactory.GetInstance(Playbook.Shotgun);
 Type[] actualOffenseRoles = GetTypes(offense)
 expectedOffneseRoles.sort();
 actualOffenseRoles.sort();
 Assert.Equal(expectedOffenseRoles, actualOffenseRoles);  
}

The sort method sorts in place and the Assert.Equal when applied to collections uses iterators to iterate and compare the individual elements. The reason the collections are sorted before the comparison is to make is clear that the order does not matter, just the content.

Once we establish the role composition, we can iterate over the returned collection (offense in this case) and specify the initial behavior of all its contained Players.

Lastly, we need to specify the behavior of the collection as a collection. There are two ways to do that.

• Assert that the type of the collection if some specific type. This ensures all the behavior implied by the type.
• Specify the behavior of the iterators using ordering mocks. We will talk more about indexing mocks when we discuss mocks, but for now just remember that they are used to ensure that an iterator returns them in the correct order.

A Note on asserting the type of objects
Asserting types requires what may seem to some as an unsavory approach in some languages, which is the use of metadata; for example, .net reflection, Java introspection or c++ RTTI. In the context of asserting the type of objects, it is OK. As we have already discussed, the type of an object is not an implementation detail, but rather derived from the problem domain. Metadata just a means of retrieving this data, and many times, the only way.

Here is an example of implementing AssertType using C# and .net reflection. It is but one of the many ways that it can be done.

public void AssertType<Expected>(object actual)
{
    Assert.AreEqual(typeof(Expected).Name, actual.GetType().Name,
                     "type mismatch");
}

Work-Flow

Consider the following code:

class SalesOrder {
 public double CalculateCharges(TaxPolicy taxCalc) {
   double charge = // Some algorithm that totals up the sales
   return charge + taxCalc.getTax(charge); //Add the tax
 }
}

interface TaxPolicy {
 double taxCalc(double amount);
}        
 
TaxPolicy is an interface that we can create different implementations of for the various locales where we have to calculate tax.  We might have one for VAT (value added tax) for our European customers, another that calculated PST (provincial sales tax) for Canadian customers and so forth.  Each of these implementations would have its own test, of course.

But in testing SalesOrder, we would not want to use any of those “real” TaxPolicy implementations.  The reasons may be obvious, and we certainly cover this elsewhere, but just to summarize:

1. We don’t want to couple the test of SalesOrder to any particular TaxPolicy.  If we just “picked one” then subsequently retire that object, we’ll break the test for SalesOrder even though SalesOrder was unchanged.
2. We could see the test of SalesOrder fail when the object is operating correctly; the failure could be caused by a bug in the particular TaxPolicy implementation we chose.
3. When that happens, we would have two tests failing for one reason (the test of SalesOrder and the test of the particular TaxPolicy implementation).  This is a bad road to be on.

The issue is easy to deal with... we could simply create an implementation of TaxPolicy that returns zero.  This would be a mock object[3], and would actually be a part of the test.

So, let’s say we did that.  One morning we start our day by running our tests.  They all are green.  We also happen to note our code coverage measurement and see that it is 100%.  Great!  100% of the code is covered, and all the tests are green!  
     
We have a strong sense confidence that nothing bad has been done to the code since we last encountered it.  Nobody has introduced a bug, and nobody has added code without an accompanying test.

Unfortunately, this confidence may be faulty.  Perhaps some other developer, innocently, made the following change.        

class SalesOrder {
 public double CalculateCharges(TaxPolicy taxCalc) {
   double charge = // Some algorithm that totals up the sales
   return charge;
 }
}

Bascially, SalesOrder does not call the getTax method on the TaxPolicy implementation at all.  This can happen for any number of reasons... perhaps the developer felt that the comment //Add the tax was not necessary, and so backspaced it out... and simply went too far.  Or, perhaps he inaccurately believed that the tax was being added elsewhere.  There are lots of reasons why mistakes like this are made.

Unfortunately, all the tests still pass. The real TaxPolicy implementations still work, and the SalesOrder not calling the mock produces the same additional tax change as calling it: zero.  Also, removing code will not lower the code coverage percentage.

What we’re missing, therefore, is a test that ensures that SalesOrder does, in fact, make use of the TaxPolicy object it is given.  This is a question of proper workflow between these objects, and it must have a test because it is required behavior.  

This is, in our experience, a very commonly-missed test.  This is not because the test is particularly difficult or laborious to write, it is because the developers simply don’t think of it.  

class MockTaxPolicy : TaxPolicy {
 bool gotCalled = false;
 public double getTax(double amount) {
   gotCalled = true;
   return 0.0;
 }

 public bool didGetCalled() {
   return gotCalled;
 }
}

…. and then we would add a new test:

public void testSalesOrderToTaxPolicyWorkflow() {
 MockTaxPolicy mock = new MockTaxPolicy();
 SalesOrder order = new SalesOrder();

 order.CalculateCharges(TaxPolicy mock);

 Assert.IsTrue(mock.didGetCalled());
}

This works because the test holds the mock by its concrete type (making didGetCalled() available to it) while SalesOrder holds it in an implicit upcast to TaxPolicy.  This encapsulates the “for testing only” method from the production code.

You should take care not to overdo this.  Simply because you can do something does not necessarily mean you should.  If a given workflow is already covered by a test of behavior, than you don’t want to test it again on its own.  We only want to create workflow tests where breaking the workflow per-se does not cause any existing test to fail.

Another way to think if it is this:  some workflows are specified, some are simply an implementation choice that might be changed in refactoring (without, in other words, changing the resulting behavior).  Only the specified (must be done) workflows should have tests written specifically for them.  Creating tests of workflows that are only implementation details will couple your test suite too tightly to your current implementation and will impede your ability to refactor.

Conclusions

The purpose in creating categories of tests was fourfold:

1. Ensure that we remember to include all the types of tests needed to form a complete specification, including those that are often missed (constant and workflow specification being very typical)
2. Capture the best practices for each category
3. Promote consistency across the team/organization by creating a shared set of test categories that all developers know to create
4. Add to the language of testing, improving communication

The list we presented here may not be comprehensive, but it serves as a good starting point.  Once we have the notion of these categories in place, we can enhance them and add new ones as we discover more ways to fully specify a system using tests.

-----

[1] In point of fact, we don’t actually completely agree with this method of categorizing the Design Patterns, but it does serve as a reasonable example of categorization in general.
[2] There are other ways to do this.  In another blog we will discuss the use of an “Any” class to make these “I don’t care” values even more obvious.
[3] We have lots to say about mocking.  For now, we’re keeping it simple.
[4] This is the Singleton pattern.  If you are unfamiliar with it:  
     http://www.netobjectives.com/PatternRepository/index.php?title=TheSingletonPattern