Testing Best Practices: Test Categories, Part 2

download the podcast

Continued from Test Categories, Part 1

5. Behavior with related boundaries (“and”) 

Sometimes a behavior varies based on more than one factor.  For example, let’s say that in order for the system to allow someone to be hired (returning a true from the canHire() method, perhaps), the individual in question must both be at least 18 years old and must be a US citizen.

Similar to a boundary within a range, here we need more than one assert to establish the rule.  However, here 2 is not enough.

Let’s further stipulate that there is a constant, System.MINIMUM_HIRING_AGE, and a enumeration named System.Citizenship with members for various countries.  The specifying test code would look like this:

class HiringRuleTest {
   public void testOnlyAppropriateCitizensOfSufficientAgeHired() {
       System.Citizenship anyOtherThanUS =
           System.Citizenship.OUTER_MONGOLIA;

       HiringRule testHiringRule = new HiringRule();

       Assert.True(testHiringRule(System.MINIMUM_HIRING_AGE,
           System.Citizenship.US));
       Assert.False(testHiringRule(System.MINIMUM_HIRING_AGE - 1,
           System.Citizenship.US));
       Assert.False(testHiringRule(System.MINIMUM_HIRING_AGE,
           anyOtherThanUS));
   }
}

A typical question that  might occur to you on reading this might be:  

Should we not show that a person under the minimum age and also not a citizen of the US will get a false when we call this method?  We have shown that:

1. A US citizen of sufficient age will return a true (hire)
2. A younger US citizen will not be hired (false)
3. A non-US citizen “of age” will not be hired (false)

But should we not show that 2 and 3 combined will also produce a false?

This is a perfect example of the difference between testing and specification.  If we were to ask an expert in testing this question, they would likely talk about the four possible conditions here as “quadrants” and would, in fact, say that the the test was incomplete without the fourth case.


In TDD, we feel differently.  We don’t want to make the test suite any larger than necessary, ever, because we want the tests to run as fast as possible, and also because we don’t want to create excessive maintenance tasks when things change.  The ideas of “thoroughness” and “rigor” that naturally accompany the testing thought process become “sufficiency” in TDD.  This probably seems like a trivial point, but it becomes less so over the long haul of the development process.


Put another way, we never add a new assert or test unless doing so makes a new distinction about the system that was not already made in the existing assertions and tests.  What makes non US citizens distinct from US citizens is that they will get a false when the age is sufficient.  That a non-US citizen with insufficient age will get a false is no different than a US citizen, it is the same distinction.


Again, TDD does not replace traditional testing.  We still expect that to happen, and still respect the value of testing organizations as much as we ever have.


You may also wonder “why Outer Mongolia to represent ‘some other country’?  Why not something else?”  We don’t really care what country we choose here, hence we named the temporary method variable “anyOtherThanUS”.  Elsewhere in this book, we’ll look at other options for representing “I don’t care” values like this one.


6. Behavior with repeated boundaries (“or”)


Sometimes are have boundaries that are related in a different way.  Sometimes it is a matter of selecting some values across a set, and specifying that some of them must be in a given condition.  If we don’t address this properly, we can seriously explode the size of our tests.


Let us say, for example, that we have a system of rules for assigning employees to teams.  Each team consists of a Salesperson, a Customer Support Representative (CSR), and an Installation Tech. The rule is that no more than one of these people can be a probationary employee (“probie”).  It’s okay if none of them are probies, but if one of them is a probie the other two must not be.  If we think if this in terms of the various possible cases, we might envision something like this:

This would indicate eight individual asserts.  However, if we take a lesson from the previous section on “and” boundaries we note that zero probies is the same behavior (acceptance) as any one probie; it is not a new distinction and we should leave that one off.  Similarly, we don’t want to show that all three being probies is unacceptable, since any two are already not acceptable and so, again, this would not be a new distinction.


That makes for six asserts.  Doesn’t that feel wrong somehow?  Like it s a brute force solution?  Yes, and this instinct is something you should listen to.  Imagine if the teams had 99 different roles and the rule was that no more than 43 of them could be probies?  Would you like to work out the truth table on that?  Me neither, but we can do it with math:

That would be a lot of asserts!  Painful.


Pain is almost always diagnostic.  When something feels wrong, this can be a clue that, perhaps, we are not doing things in the best way.  Here the testing pain is suggesting that perhaps we are not thinking of this problem correctly, and that an alternate way of modeling the problem might be more advantageous, and actually more appropriate.


A clue to this lies in the way we might decide to implement the rule in the production code.  We could, for instance, do something like this:


class TeamBuilder {
    public const MAX_PROBIES = 1;

    public boolean isTeamProperlyConfigured(
        Employee aSalesPerson,
        Employee aCSR,
        Employee aTech) {

        int count = 0;

        if(aSalesperson.type == “probationary”) count++;
        if(aCSR.type == “probationary”) count++;
        if(aTech.type == “probationary”) count++;

        if(count > MAX_PROBIES) return false;

        return true;

}

}

This also seems like an inelegant, brute-force approach to the problem, and we might refactor it to be something like this:


class TeamBuilder {
    public const MAX_PROBIES = 1;

    public boolean isTeamProperlyConfigured(
        Employee aSalesPerson,
        Employee aCSR,
        Employee aTech) {

        int count = 0;
        Employee[] emps = new Employee[]

{aSalesPerson, aCDR, aTECH};

        forEach(Employee e in emps)
            if(e.type == “probationary”) count++;

        if(count > MAX_PROBIES) return false;


        return true;

}

}


In other words, if we stuff all the team member into a collection, we can just scan it for those who are probies and count them.  This suggests that perhaps we should be using a collection in the first place, in the API of the object:


class TeamBuilder {
    public const MAX_PROBATIONARY = 1;

    public boolean isTeamProperlyConfigured(Employee[] emps) {
        int count = 0;
        forEach(Employee e in emps)
            if(e.type == “probationary”) count++;

        if(count > MAX_PROBATIONARY) return false;

        return true;

}

}


Now we can write a test with two asserts: one that uses a collection with just enough probies in it and one with MAX_PROBATIONARY + 1 probies in it.  The boundary is easy to define because we’ve changed the abstraction to that of a collection.


Here is another example of a test being helpful.  The fact that a collection would make the testing easier is the test essentially giving you “design advice”.  Making thing easier for tests is going to tend to make things easier for client code in general since, in test-first, the test is essentially the first client that a given object ever has.


7. Technically-induced boundaries

Addition is an example of a single-behavior function:

val = op1 + op2

As such, there is little that needs to be specified about it. This, however is the mathematical view of the problem. In reality there’s another issue that presents itself in different ways -- the bit-limited internal representation of numbers in a computer or the way a specific library that we consider using is implemented.


Let’s assume, for starters, that op1, op2 and val are 32 bit integers. As such there are a maximal and minimal values that the can take: 231-1 and -231. The tests needs to specify the following: 

1. What are the largest positive and negative numbers that can be passed as arguments? The boundary in this case is on the “value of the operand”
2. What happens if the sum of the arguments is more, or less than these technical limits?   (231-1) + 1 = ?

The boundary in this case is on the “sum of the operands”

When it comes to floating point calculation the problem is further complicated by precision both is the exponent and the mantissa. For example:

                           100000000000.0 + 0.00000000001 = 100000000000.00000000001

In a computer, because of the way floating point processors operate, the calculation may turn out differently:

                            100000000000.0 + 0.00000000001 = 100000000000.0

Note that the boundary in this case is on the “difference between the operands”

In many cases the hardware implementation will satisfy the needs of our customer. If it does not, this does not mean we cannot solve the problem. It means that we will not be able to rely on the the system’s software or hardware implementation but would need to roll our own. Surely this is a tidbit of information worthwhile knowing prior to embarking on implementation?

Continued in Test Categories, Part 3...