Redefining Test-Driven Development, Pt. 2

Download the Podcast 


In part 1 we said “How you do something new is often influenced to a great extent by what you think you are doing.”  Let’s add that, similarly, changing the way you do something you are already doing can come from a new understanding of its nature.

Something development teams already do (or, in our opinion, really should be doing) is to write a specification of the system before they create it.  This specification comes from an analysis of requirements, and reflects the development team’s understanding of the business value of the system from the customer’s perspective and the technology used to create the solution.  “The spec” is then referred to throughout the development process as fundamental guidance for everything the team does.

Specifications have great value; this value, however, is not persistent.  

Let’s say you created a specification in a traditional way: you wrote a document, embedded some design diagrams charts and graphs, and so forth.  This would form an artifact that expressed your understanding of the system.

Let’s further say that you used this specification to work from, completed the development process, released the system, and moved on.  

Now, eighteen months later, the customer wants to make changes to the system.  You’ve been away from the system for quite a while, and you’re fuzzy on the details, so job one is to re-acquaint yourself with it.  Should you re-read that specification you created way back when?  You could, but how do you know it is still accurate?  Someone could easily have made changes to the system and not updated the spec accordingly.

We all know we should not do that, but as a practical matter it happens all the time.  People make changes with limited time and resources, and under pressure... and often they simply neglect the spec entirely, or they update it incompletely or incorrectly.

And even if you don’t have any reason to suspect this has happened, how can you know, really know for sure, that it has not?  The only way is to examine the system in detail and compare it to the spec.  If you have to do this, they what good did having the written spec really do you?

So, consider this, a typical unit test:

// pseudocode
public class AccountTest {
    public void testAccountAmortizesCorrectly() {
        double value = Any.value();
        int term = Any.term();
        int yearToWriteOff = Any.yearUpTo(term);

        Account testAccount = new Account(value, term);
        double expectedAmount = max(value/term, 100.00);

        double actualAmount = testAccount.amortize(yearToWriteOff);

        assertAreEqual(expectedAmount, actualAmount, 1);
    }
}

Look closely.  What does this tell you?

1. There is an object called Account that can amortize itself
2. Account takes a value and a term via its constructor
3. Value is double, term is int, and neither are constrained (“Any”) [1]
4. Amortize means “write off”
5. All years amortize in the same way (“Any” again)
6. You call an amortize() method and pass the year to write off (an int) to it
7. The way you know how much to write off is value/term, but no more than 100.00
8. We do not care about pennies (the tolerance for the assertion is 1)

Would you not say that this could serve, at least for the development team, as a specification?  It tells you how the system should work, how it is structured, the API specifics (both constructor and public methods), etc... everything that a traditional spec would record.

Compare now, in the scenario where you’re coming back eighteen months later, this kind of specification to the document you would normally create.  You can run this “unit test” immediately, watch it compile (the API’s have not changed if it does), watch it pass (the behavior of the system has not changed if it does), and thus confirm that it is still accurate with no effort at all.  If we then further stipulate that every behavior of the system has a test like this, and we can run them all with a single click of the mouse, then we know our test suite is accurate to the code.  Now run your code coverage measurement... is it 100%?  Now you know that there is no additional behavior that has been added by someone else without that person adding such a unit test.

So, in TDD we do not write tests.  We write specifications.  Executable specifications.

Note that the testing framework itself (with just about every tool you’ll encounter) uses the term “assert.”  Look that one up:

Assert(v) to state with assurance, confidence, or force; state strongly or positively; affirm; aver: He asserted his innocence of the crime. [2]

Note this not “check” or “examine to determine if” or “confirm”.  When we assert something we do not say “this should be true” we say “this is true”.  It’s a statement of truth not an investigation.  It is not a test, but a fact about the system.

This simple shift in thinking from “I am writing a test” to “I am writing a specification” changes so many things about how you’ll write them, what you write and won’t write, what qualities you will look for and emphasize, how you’ll name things... and on and on... that we won’t even try to enumerate them here.  We’ll write an entire posting just about this (Testing as Specification).

So, why do we still call them tests?  Two reasons.

1. First, “Test-Driven Development” is the term we are stuck with.  Language is a living thing, a shared thing, and we cannot dictate on our own what things are called.  We’d love to call it what it is: “Behaviour-Based Analysis and Design”, and we think of it that way, but at the end of the day...
2. We’re not going to throw these executable specifications away when we’re done driving our development with them.  Why would we?  It took effort to make them, and we want to be able to refer to them later.  But you know what else they magically turn into at this point?  Tests!  We can used them to test against system regression when we need to refactor it.  These are regression tests we got for no extra effort, by the way.[3]

So, does TDD add new work to the development team?  No.  We were going to write a specification anyway, we’re just doing it in a different way now.  A better way, because it will be written in cold, hard code (rather than vaguely in human language), and it will be automatically verifiable against the real system at any point we desire, with no effort on our part.

And additionally, for free, it will produce a regression suite at well.  Most teams struggle mightily and do all sorts of shenanigans (see our upcoming blog Lies, Damn Lies, and Code Coverage) to achieve 75% to 80% code coverage.  We will have 100% [4] and we don’t have to do anything additional to get it.

All this leaves is the third objection from part 1...  what about the maintenance burden we take on when we have to keep the test suite up to date?  What about new requirements that cause dozens or even hundreds of tests to break, and have to be repaired?

Yes indeed, what about that?  Must have something to do with the word... Sustainable.

Stay tuned.




----

[1] We’ll talk about Any in a future blog
[2] http://dictionary.reference.com/browse/assert
[3] Not that we are saying our test suite will replace all traditional testing.  It will not.  But as a regression test suite it has a lot of value both for developers and testers alike
[4] ...or very close to it.  Nothing is ever perfect, after all