BDD: TDD Done Right

A recent discussion I had about TDD lately has drawn some interest about BDD, and I thought what I was going to respond would deserve its own post in this blog.

Anyone in TDD circle would agree that the very first thing one needs to know about Test-Driven-Development is: it has nothing to do with test. Doing TDD to replace other means of application tests is probably just as harmful as not having any test at all. What I keep hearing from times to times is people who start practicing TDD complaining that they’re still staring at many bugs crawling around before their eyes, and they will start questioning the relevance of TDD in software development. In different occassions, I hear people using their keyboard and mouse on screens to do “unit-test”.

I think the word “test” in our profession is one of those unfortunate legacies we inherit from the past. It’s totally misleading. And there has been many attempts to fix this. New terms have been introduced, like “Design by Examples”, and “Behavior Driven Development”.
Behavior Driven Development is often claimed as an evolution from Test-Driven-Development, a sexy buzzword, a total paradigm shift that lets application code speaks in business language. And it’s suddenly becoming the greatest thing since sliced bread.

Well, the fact is, BDD is no other than our good old TDD. Or at least TDD done right, to be precise. BDD is not quite the evolution from TDD. It is TDD.

While some frameworks tackle BDD from completely different angle (e.g. NBehave), many BDD frameworks (e.g. SpecUnit, MSpec) are just a guideline of applying best practices in writing good xUnit tests. They are refinements of xUnit patterns (e.g. Four Phase Test). I use SpecUnit myself, and classes in SpecUnit actually inherit directly from nUnit.

One of the common pattern (anti-pattern?) in xUnit is Testcase Class per Class and Testcase Class Per Method, which usually leads to ineffective tests. BDD promotes Testcase Class Per User Story.

There are many BDD examples out there. But one remark came up during the discussion was that most examples about TDD/BDD around are usually centered around the framework and end-results, instead of the process itself. I think that’s a good point. The process might illustrate the key difference between the 2 approaches, and what BDD really means for us. I’ll try to cover some TDD process here, it’s going to be verbose. I’ll highlight the code changes between each steps.

At the risk of being totally boring, I’m going to use the ubiquotous shopping-cart story as an example.
Creating a test called ShoppingCartTest is perhaps too obviously baaaaadd.. So let’s make a test that’s “reasonably bad”, so common that lots of people actually do that in TDD, like AddingProductToShoppingCartTest. Then later on we’ll try to refactor this into BDD way.

Let’s say, for discussion sake, products that go into the shopping cart will be acquired from the inventory-system. Not realistic perhaps, but should be good enough for our purpose.

Test Driven Development

Let’s start with our first test-case:

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        cart = new ShoppingCart(
                this.customer = new Customer(),
                this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
        customer.OrderHistory.Add(createPreviousOrder());

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);

        cart.Add(nokia, 12);

        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(20);
    }
}

Nice. Now the second test: AddingSameProductsShouldGroupThemTogether, and you realize that certain portion of the code could be reused, so you are refactoring that bit into a SetUp method.

While we’re here, there’s 1 thing that you don’t normally observe from TDD examples that only give you the end-result. Here you notice that refactoring into SetUp method is a “reactive” activity. We create SetUp method after couple of testcases. SetUp method is meaningless, and its purpose is merely to eliminate repetition by consolidating the least-common-denominator of all testcases.

Further down in this post, you will see that this is in contrast with BDD, where SetUp method has its own place as a first-class citizen. In BDD, SetUp method (aka Context) is created proactively before you start writing your first testcase.

Anyway, so here’s our first 2 testcases after refactoring:

[TestFixture]
public class AddingProductToShoppingCartTest
{
	// [ADDED] --------------------------- {
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
        customer.OrderHistory.Add(createPreviousOrder());

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);
    }
	// } -------------------------- [/ADDED]

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
		// [REMOVED /]
        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

	// [ADDED] --------------------------- {
    [Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }
	// } -------------------------- [/ADDED]
}

So far so good. You go ahead and write the next 8-9 testcases, before you stumble across our next testcase: CustomerFirstTimeOrder_ShouldAllowOnly1Product. Bugger! Our SetUp method no longer fits to cover this scenario. You will need to take line 10 from SetUp method back to each of the testcases.

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);

		// [REMOVED /]

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);
    }

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        customer.OrderHistory.Add(createPreviousOrder()); // <- ADDED

        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

	[Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        customer.OrderHistory.Add(createPreviousOrder()); // <- ADDED

        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }

    // [ADDED] --------------------------- {
    [Test] public void CustomerFirstTimeOrder_ShouldAllowNotAllowMoreThan1Product()
    {
        Assert.Throws<IllegalRequest>( ()=> cart.Add(nokia, 2));
        cart.GetLineFor(nokia).ShouldBeNull();
    }

    [Test] public void CustomerFirstTimeOrder_ShouldAllow1Product()
    {
        cart.Add(nokia, 1);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(1);
    }
	// } -------------------------- [/ADDED]
}

OK let me say it again. SetUp method consolidates the least-common-denominator of all testcases. Well, after you cover various scenarios for your test, you will no longer have too much common-denominator. Consequently, you will find yourself keep changing large number of testcases and refactor your test-code every time you add a new test-case that covers slightly different scenario.

For completeness sake, let’s add 1 more test-case: WhenInventoryRunsOutOfStock_ShouldRejectProduct(). Alas, once again we need to refactor the SetUp method. So this concludes the final result of our TDD process:

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
    }

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 12)).ThenReturn(true);

        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

    [Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 10)).ThenReturn(true);
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 20)).ThenReturn(true);

        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }

    [Test] public void CustomerFirstTimeOrder_ShouldAllowOnly1Product()
    {
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 2)).ThenReturn(true);

        Assert.Throws<IllegalRequest>( ()=> cart.Add(nokia, 2));
        cart.GetLineFor(nokia).ShouldBeNull();
    }

    [Test] public void WhenInventoryRunsOutOfStock_ShouldRejectProduct()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 20)).ThenReturn(false);

        Assert.Throws<IllegalRequest>(()=> cart.Add(nokia, 20));
        cart.GetLineFor(nokia).ShouldBeNull();
    }
}

Everytime we cover a new test-case, we change the equilibrium of our common-denominator, and you will spend most of your time refactoring. After 99 testcases, you have virtually nothing in common. Your SetUp method is practically empty. Each of your testcases are a giant repetitive set of arrage-expect-action-assert, all in every single method. Effort to maintain this test is out of control. The test is no longer readable. Try reading again the test we just wrote. Still readable? Try again tomorrow morning.

Behavior Driven Development

Let’s try converting the exact same testcases we just wrote into SpecUnit grammar. I’ll also switch my mock-framework from record-replay mode into AAA-syntax.

public class when_adding_a_product:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    public override void Because()
    {
        cart.Add(nokia, 12);
    }

    [Observation]
    public void should_acquire_product_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 12));
    }

    [Observation]
    public void cart_should_only_contain_1_line()
    {
        cart.Lines.ShouldHaveSize(1);
    }

    [Observation]
    public void cart_should_contain_line_for_added_product()
    {
        cart.Lines.First().Product.ShouldBe(nokia);
    }

    [Observation]
    public void cart_item_should_have_added_quantity()
    {
        cart.Lines.First().Quantity.ShouldBe(12);
    }
}

public class when_adding_same_product_twice:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    public override void Because()
    {
        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
    }

    [Observation]
    public void should_acquire_both_requested_products_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 10));
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 20));
    }

    [Observation]
    public void cart_should_only_contain_1_line()
    {
        cart.Lines.ShouldHaveSize(1);
    }

    [Observation]
    public void cart_should_contain_line_for_added_product()
    {
        cart.Lines.First().Product.ShouldBe(nokia);
    }

    [Observation]
    public void cart_item_quantity_should_be_sum_of_added_quantities()
    {
        cart.Lines.First().Quantity.ShouldBe(10 + 20);
    }
}

public class when_inventory_runs_out_of_stock:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    Exception exception;
    public override void Because()
    {
        inventoryService.When(x=>x.AcquireProduct(nokia, 2)).ThenReturn(false);

        thrownException = ((MethodThatThrows)()=>
            cart.Add(nokia, 2))
        .GetException();
    }

    [Observation]
    public void should_reject_the_request()
    {
        thrownException.ShouldNotBeNull();
    }

    [Observation]
    public void should_not_acquire_product_from_inventory()
    {
        inventoryService.AssertWasNotCalled(x=>x.AcquireProduct(nokia, Arg.AnyInt));
    }

    [Observation]
    public void should_not_add_any_product()
    {
        cart.Lines.ShouldBeEmpty();
    }
}

public class when_a_first_time_customer_add_more_than_1_product_quantity:
    Behaves_like_an_empty_shopping_cart
{
    Exception exception;
    public override void Because()
    {
        thrownException = ((MethodThatThrows)()=>
            cart.Add(nokia, 2))
        .GetException();
    }

    [Observation]
    public void should_reject_the_request()
    {
        thrownException.ShouldNotBeNull();
    }

    [Observation]
    public void should_not_acquire_product_from_inventory()
    {
        inventoryService.AssertWasNotCalled(x=>x.AcquireProduct(nokia, Arg.AnyInt));
    }

    [Observation]
    public void should_not_add_any_product()
    {
        cart.Lines.ShouldBeEmpty();
    }
}

public class when_a_first_time_customer_add_1_product_quantity:
    Behaves_like_an_empty_shopping_cart
{
    Exception exception;
    public override void Because()
    {
        cart.Add(nokia, 1);
    }

    [Observation]
    public void should_acquire_product_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 1));
    }

    [Observation]
    public void cart_item_should_contain_the_product()
    {
        cart.GetLineFor(nokia).Quantity.ShouldBe(1);
    }
}

[Concern("Shopping Cart")]
public class Behaves_like_an_empty_shopping_cart: ContextSpecification
{
    public override void Context()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);

        inventoryService.When(x=>x.AcquireProduct(Arg.Any<Product>(), Args.AnyInt)).ThenReturn(true);
    }
}

public class Behaves_like_an_empty_shopping_cart_and_a_repeat_customer:
    Behaves_like_an_empty_shopping_cart
{
    public override void Context()
    {
        base.Context();
        customer.OrderHistory.Add(createPreviousOrder());
    }
}

So this time, every user-story is no longer written in 1 (or several) methods. Instead, each user-story is represented as 1 class with a clear context. And each test-method only contains one single line of Assert statement. As a rule-of-thumb, 1 assert turns to 1 test-method. As such, we can quickly lookup the definition of each expected behavior. E.g., we can lookup that the definition of cart_item_quantity_should_be_sum_of_added_quantities is cart.Lines.First().Quantity.ShouldBe(10 + 20).

Same goes for the classes that represent each user-story. They are defined in a clear business vocabulary, comes with each definition. Programmers can lookup the definition of each business context instantly. E.g. the definition of when_inventory_runs_out_of_stock is inventoryService.When(x=>x.AcquireProduct(nokia, 2)).ThenReturn(false).
Our test becomes a handy place for programmers to lookup business glossary just by following Context() and Because() methods. So the definition of “inventory running out of stock” in the context of a shopping cart is when AcquireProduct() returns false.

Usually I stick all these classes together into a single file (e.g. AddingProductToShoppingCartSpecification.cs). A file where we can lookup a whole aspect of a little sub-functionality in our system in a neat grammar of Behaves_like, When, Should, Should, Should… where every one of them is immediately followed by its definition.

Now everytime we need to write a new scenario, we’re no longer worried if this will affect the other testcases. If the new scenario doesn’t fit with our current contexts, we simply create a new context. Just create a new when_xxx class… without needing to refactor any existing test-case.
This helps us to avoid the temptation to apply lazy-workaround. For instance, in our previous TDD example about “first-time-customer”, developers could actually just create a new method in Customer class to clear all OrderHistory to turn a repeat-customer back into a fresh first-time-customer. This makes the SUT fits with existing SetUp method scenario and test-cases dependency hell… saving them from refactoring effort. But this leads to smelly design, and probably doesn’t fit with the domain they’re representing. (In domain requirement, they probably will never have a customer clearing his order history).

When you execute this “testcases”, SpecUnit provides a tool that converts this into a pretty HTML document with nice given-when-then grammar, structured in an organized fashion. This document is readily consumable by non-technical people. After the business people see this document quite several times, they will start to understand how you work. They are able to see how we (developers) translate the requirement they convey into an executable specification in given-when-then fashion. Soon enough, they will get excited and start to learn to speak in the same grammar.

They already have a general feeling of the way we talk, so they will start expressing their requirement in the same given-when-then language, which will immediately appear as our BDD specification. Closing the room for ambiquity. Bridges the gap of communication.