BDD: TDD Done Right

A recent discussion I had about TDD lately has drawn some interest about BDD, and I thought what I was going to respond would deserve its own post in this blog.

Anyone in TDD circle would agree that the very first thing one needs to know about Test-Driven-Development is: it has nothing to do with test. Doing TDD to replace other means of application tests is probably just as harmful as not having any test at all. What I keep hearing from times to times is people who start practicing TDD complaining that they’re still staring at many bugs crawling around before their eyes, and they will start questioning the relevance of TDD in software development. In different occassions, I hear people using their keyboard and mouse on screens to do “unit-test”.

I think the word “test” in our profession is one of those unfortunate legacies we inherit from the past. It’s totally misleading. And there has been many attempts to fix this. New terms have been introduced, like “Design by Examples”, and “Behavior Driven Development”.
Behavior Driven Development is often claimed as an evolution from Test-Driven-Development, a sexy buzzword, a total paradigm shift that lets application code speaks in business language. And it’s suddenly becoming the greatest thing since sliced bread.

Well, the fact is, BDD is no other than our good old TDD. Or at least TDD done right, to be precise. BDD is not quite the evolution from TDD. It is TDD.

While some frameworks tackle BDD from completely different angle (e.g. NBehave), many BDD frameworks (e.g. SpecUnit, MSpec) are just a guideline of applying best practices in writing good xUnit tests. They are refinements of xUnit patterns (e.g. Four Phase Test). I use SpecUnit myself, and classes in SpecUnit actually inherit directly from nUnit.

One of the common pattern (anti-pattern?) in xUnit is Testcase Class per Class and Testcase Class Per Method, which usually leads to ineffective tests. BDD promotes Testcase Class Per User Story.

There are many BDD examples out there. But one remark came up during the discussion was that most examples about TDD/BDD around are usually centered around the framework and end-results, instead of the process itself. I think that’s a good point. The process might illustrate the key difference between the 2 approaches, and what BDD really means for us. I’ll try to cover some TDD process here, it’s going to be verbose. I’ll highlight the code changes between each steps.

At the risk of being totally boring, I’m going to use the ubiquotous shopping-cart story as an example.
Creating a test called ShoppingCartTest is perhaps too obviously baaaaadd.. So let’s make a test that’s “reasonably bad”, so common that lots of people actually do that in TDD, like AddingProductToShoppingCartTest. Then later on we’ll try to refactor this into BDD way.

Let’s say, for discussion sake, products that go into the shopping cart will be acquired from the inventory-system. Not realistic perhaps, but should be good enough for our purpose.

Test Driven Development

Let’s start with our first test-case:

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        cart = new ShoppingCart(
                this.customer = new Customer(),
                this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
        customer.OrderHistory.Add(createPreviousOrder());

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);

        cart.Add(nokia, 12);

        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(20);
    }
}

Nice. Now the second test: AddingSameProductsShouldGroupThemTogether, and you realize that certain portion of the code could be reused, so you are refactoring that bit into a SetUp method.

While we’re here, there’s 1 thing that you don’t normally observe from TDD examples that only give you the end-result. Here you notice that refactoring into SetUp method is a “reactive” activity. We create SetUp method after couple of testcases. SetUp method is meaningless, and its purpose is merely to eliminate repetition by consolidating the least-common-denominator of all testcases.

Further down in this post, you will see that this is in contrast with BDD, where SetUp method has its own place as a first-class citizen. In BDD, SetUp method (aka Context) is created proactively before you start writing your first testcase.

Anyway, so here’s our first 2 testcases after refactoring:

[TestFixture]
public class AddingProductToShoppingCartTest
{
	// [ADDED] --------------------------- {
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
        customer.OrderHistory.Add(createPreviousOrder());

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);
    }
	// } -------------------------- [/ADDED]

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
		// [REMOVED /]
        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

	// [ADDED] --------------------------- {
    [Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }
	// } -------------------------- [/ADDED]
}

So far so good. You go ahead and write the next 8-9 testcases, before you stumble across our next testcase: CustomerFirstTimeOrder_ShouldAllowOnly1Product. Bugger! Our SetUp method no longer fits to cover this scenario. You will need to take line 10 from SetUp method back to each of the testcases.

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);

		// [REMOVED /]

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);
    }

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        customer.OrderHistory.Add(createPreviousOrder()); // <- ADDED

        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

	[Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        customer.OrderHistory.Add(createPreviousOrder()); // <- ADDED

        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }

    // [ADDED] --------------------------- {
    [Test] public void CustomerFirstTimeOrder_ShouldAllowNotAllowMoreThan1Product()
    {
        Assert.Throws<IllegalRequest>( ()=> cart.Add(nokia, 2));
        cart.GetLineFor(nokia).ShouldBeNull();
    }

    [Test] public void CustomerFirstTimeOrder_ShouldAllow1Product()
    {
        cart.Add(nokia, 1);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(1);
    }
	// } -------------------------- [/ADDED]
}

OK let me say it again. SetUp method consolidates the least-common-denominator of all testcases. Well, after you cover various scenarios for your test, you will no longer have too much common-denominator. Consequently, you will find yourself keep changing large number of testcases and refactor your test-code every time you add a new test-case that covers slightly different scenario.

For completeness sake, let’s add 1 more test-case: WhenInventoryRunsOutOfStock_ShouldRejectProduct(). Alas, once again we need to refactor the SetUp method. So this concludes the final result of our TDD process:

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
    }

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 12)).ThenReturn(true);

        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

    [Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 10)).ThenReturn(true);
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 20)).ThenReturn(true);

        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }

    [Test] public void CustomerFirstTimeOrder_ShouldAllowOnly1Product()
    {
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 2)).ThenReturn(true);

        Assert.Throws<IllegalRequest>( ()=> cart.Add(nokia, 2));
        cart.GetLineFor(nokia).ShouldBeNull();
    }

    [Test] public void WhenInventoryRunsOutOfStock_ShouldRejectProduct()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 20)).ThenReturn(false);

        Assert.Throws<IllegalRequest>(()=> cart.Add(nokia, 20));
        cart.GetLineFor(nokia).ShouldBeNull();
    }
}

Everytime we cover a new test-case, we change the equilibrium of our common-denominator, and you will spend most of your time refactoring. After 99 testcases, you have virtually nothing in common. Your SetUp method is practically empty. Each of your testcases are a giant repetitive set of arrage-expect-action-assert, all in every single method. Effort to maintain this test is out of control. The test is no longer readable. Try reading again the test we just wrote. Still readable? Try again tomorrow morning.

Behavior Driven Development

Let’s try converting the exact same testcases we just wrote into SpecUnit grammar. I’ll also switch my mock-framework from record-replay mode into AAA-syntax.

public class when_adding_a_product:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    public override void Because()
    {
        cart.Add(nokia, 12);
    }

    [Observation]
    public void should_acquire_product_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 12));
    }

    [Observation]
    public void cart_should_only_contain_1_line()
    {
        cart.Lines.ShouldHaveSize(1);
    }

    [Observation]
    public void cart_should_contain_line_for_added_product()
    {
        cart.Lines.First().Product.ShouldBe(nokia);
    }

    [Observation]
    public void cart_item_should_have_added_quantity()
    {
        cart.Lines.First().Quantity.ShouldBe(12);
    }
}

public class when_adding_same_product_twice:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    public override void Because()
    {
        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
    }

    [Observation]
    public void should_acquire_both_requested_products_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 10));
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 20));
    }

    [Observation]
    public void cart_should_only_contain_1_line()
    {
        cart.Lines.ShouldHaveSize(1);
    }

    [Observation]
    public void cart_should_contain_line_for_added_product()
    {
        cart.Lines.First().Product.ShouldBe(nokia);
    }

    [Observation]
    public void cart_item_quantity_should_be_sum_of_added_quantities()
    {
        cart.Lines.First().Quantity.ShouldBe(10 + 20);
    }
}

public class when_inventory_runs_out_of_stock:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    Exception exception;
    public override void Because()
    {
        inventoryService.When(x=>x.AcquireProduct(nokia, 2)).ThenReturn(false);

        thrownException = ((MethodThatThrows)()=>
            cart.Add(nokia, 2))
        .GetException();
    }

    [Observation]
    public void should_reject_the_request()
    {
        thrownException.ShouldNotBeNull();
    }

    [Observation]
    public void should_not_acquire_product_from_inventory()
    {
        inventoryService.AssertWasNotCalled(x=>x.AcquireProduct(nokia, Arg.AnyInt));
    }

    [Observation]
    public void should_not_add_any_product()
    {
        cart.Lines.ShouldBeEmpty();
    }
}

public class when_a_first_time_customer_add_more_than_1_product_quantity:
    Behaves_like_an_empty_shopping_cart
{
    Exception exception;
    public override void Because()
    {
        thrownException = ((MethodThatThrows)()=>
            cart.Add(nokia, 2))
        .GetException();
    }

    [Observation]
    public void should_reject_the_request()
    {
        thrownException.ShouldNotBeNull();
    }

    [Observation]
    public void should_not_acquire_product_from_inventory()
    {
        inventoryService.AssertWasNotCalled(x=>x.AcquireProduct(nokia, Arg.AnyInt));
    }

    [Observation]
    public void should_not_add_any_product()
    {
        cart.Lines.ShouldBeEmpty();
    }
}

public class when_a_first_time_customer_add_1_product_quantity:
    Behaves_like_an_empty_shopping_cart
{
    Exception exception;
    public override void Because()
    {
        cart.Add(nokia, 1);
    }

    [Observation]
    public void should_acquire_product_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 1));
    }

    [Observation]
    public void cart_item_should_contain_the_product()
    {
        cart.GetLineFor(nokia).Quantity.ShouldBe(1);
    }
}

[Concern("Shopping Cart")]
public class Behaves_like_an_empty_shopping_cart: ContextSpecification
{
    public override void Context()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);

        inventoryService.When(x=>x.AcquireProduct(Arg.Any<Product>(), Args.AnyInt)).ThenReturn(true);
    }
}

public class Behaves_like_an_empty_shopping_cart_and_a_repeat_customer:
    Behaves_like_an_empty_shopping_cart
{
    public override void Context()
    {
        base.Context();
        customer.OrderHistory.Add(createPreviousOrder());
    }
}

So this time, every user-story is no longer written in 1 (or several) methods. Instead, each user-story is represented as 1 class with a clear context. And each test-method only contains one single line of Assert statement. As a rule-of-thumb, 1 assert turns to 1 test-method. As such, we can quickly lookup the definition of each expected behavior. E.g., we can lookup that the definition of cart_item_quantity_should_be_sum_of_added_quantities is cart.Lines.First().Quantity.ShouldBe(10 + 20).

Same goes for the classes that represent each user-story. They are defined in a clear business vocabulary, comes with each definition. Programmers can lookup the definition of each business context instantly. E.g. the definition of when_inventory_runs_out_of_stock is inventoryService.When(x=>x.AcquireProduct(nokia, 2)).ThenReturn(false).
Our test becomes a handy place for programmers to lookup business glossary just by following Context() and Because() methods. So the definition of “inventory running out of stock” in the context of a shopping cart is when AcquireProduct() returns false.

Usually I stick all these classes together into a single file (e.g. AddingProductToShoppingCartSpecification.cs). A file where we can lookup a whole aspect of a little sub-functionality in our system in a neat grammar of Behaves_like, When, Should, Should, Should… where every one of them is immediately followed by its definition.

Now everytime we need to write a new scenario, we’re no longer worried if this will affect the other testcases. If the new scenario doesn’t fit with our current contexts, we simply create a new context. Just create a new when_xxx class… without needing to refactor any existing test-case.
This helps us to avoid the temptation to apply lazy-workaround. For instance, in our previous TDD example about “first-time-customer”, developers could actually just create a new method in Customer class to clear all OrderHistory to turn a repeat-customer back into a fresh first-time-customer. This makes the SUT fits with existing SetUp method scenario and test-cases dependency hell… saving them from refactoring effort. But this leads to smelly design, and probably doesn’t fit with the domain they’re representing. (In domain requirement, they probably will never have a customer clearing his order history).

When you execute this “testcases”, SpecUnit provides a tool that converts this into a pretty HTML document with nice given-when-then grammar, structured in an organized fashion. This document is readily consumable by non-technical people. After the business people see this document quite several times, they will start to understand how you work. They are able to see how we (developers) translate the requirement they convey into an executable specification in given-when-then fashion. Soon enough, they will get excited and start to learn to speak in the same grammar.

They already have a general feeling of the way we talk, so they will start expressing their requirement in the same given-when-then language, which will immediately appear as our BDD specification. Closing the room for ambiquity. Bridges the gap of communication.

Advertisements

Data Access Test with Sqlite

I have mentioned briefly in previous post about 5 different approaches to write unit-test for data-access code. So now I will try to cover the first method in more detail. And among the 5, this method seems to be the most common one so far.
I will be using Sqlite as in-memory database. Some other arguably better alternative is available, that is if you are happy to fork out some extra bucks on VistaDB.
So let’s reemphasize the desired characteristics of a good unit test, especially in the context of data-access code.
1. Isolated. Database (by definition) persists any change to its state. However, this is usually not desirable in unit-test. A good test case runs in complete isolation from any changes made in other test-cases. I.e., any changes to the database from one test case should not be visible from other test-cases.
2. Repeatable. Regardless how many times a unit test is executed, a consistent result is expected. For this to happen, unit-test should not rely on presumption on external condition, especially shared database.
3. Fast. If your test fixture cannot be executed in every several minutes or so, there is only one thing that can possibly happen: developers will start abandoning it. And unfortunately, this is usually the case if you are connecting to database system from test code, where the whole test-suite can take up to an hour to execute, if you are lucky. No one can run it frequent enough to make it useful, and hence no one will keep on maintaining it.

IN MEMORY DATABASE
The main reason why using in-memory database for unit-test is that it is incredibly fast! Both in restoring zero state, execution, and cleaning up. Speaking of which, let’s take a look on typical cycle of in memory database in unit-test.
We are going to have NHibernate to build the database schema from scratch before each test case, and dispose it at the end of the test case. Then start again with building the schema on clean database again for the next test-case. This way, we always have an empty sheet to work on for each test-case without affecting (or being affected by) any database changes in other test-cases.
Here is some NUnit test case example using NHibernate.

[TestFixture]
public class CustomerRepositoryTest: InMemoryDBTestFixture
{
	private ISession session;
	private CustomerRepository repository;
	
	[TestFixtureSetuUp]
	public void FixtureSetUp()
	{
		InitialiseNHibernate(typeof(Customer).Assembly);
	}
	
	[SetUp]
	public void SetUp()
	{
		this.session = this.CreateSession();
		this.repository = new CustomerRepository(
			new FakeSessionManager(session));
	}
	
	[TearDown]
	public void TearDown()
	{
		this.session.Dispose();
	}
	
	[Test]
	public void CanQueryCustomerByLastname()
	{
		var customers = new List<Customer>(){
			new Customer(){
				FirstName="Peter",
				LastName="Griffin"},
			new Customer(){
				FirstName="Other",
				LastName="Lads"},
			new Customer(){
				FirstName="Meg",
				LastName="Griffin"},
			new Customer(){
				FirstName="Yet Another",
				LastName="Bloke"}};
		
		foreach(var cus in customers)
			session.Save(cus);
		session.Flush();
		foreach(var cus in customers)
			session.Evict(cus);
		
		var loaded = repository.QueryByLastname("Griffin");
		
		Assert.That(loaded.Count, Is.EqualTo(2));
		AssertLoadedDataEqual(loaded[0], customers[0]);
		AssertLoadedDataEqual(loaded[1], customers[2]);
	}
	private static void AssertLoadedDataEqual(Customer loaded, Customer saved)
	{
		AssertThat(loaded, Is.NotEqualTo(saved)); // Make sure it's not cached data
		
		Assert.That(loaded.ID, Is.EqualTo(saved.ID));
		Assert.That(loaded.FirstName, Is.EqualTo(saved.FirstName));
		Assert.That(loaded.LastName, Is.EqualTo(saved.LastName));
	}
}

The plumbing for initializing NHibernate and building in-memory database is managed by base-class InMemoryDBTestFixtureBase. It is a very common practice to have this kind of base class for all database test-fixtures within a project, so we we can turn our back on setting up test database and concentrate on testing what we care about. Let’s take a look at the base class code.

public abstract class InMemoryDBTestFixtureBase
{
	protected static ISessionFactory sessionFactory;
	protected static Configuration configuration;

	public static void InitialiseNHibernate(params Assembly [] assemblies)
	{
		if(sessionFactory!=null)
			return;

		var prop = new Hashtable();
		prop.Add("hibernate.connection.driver_class", "NHibernate.Driver.SQLite20Driver");
		prop.Add("hibernate.dialect", "NHibernate.Dialect.SQLiteDialect");
		prop.Add("hibernate.connection.provider", "NHibernate.Connection.DriverConnectionProvider");
		prop.Add("hibernate.connection.connection_string", "Data Source=:memory:;Version=3;New=True;");

		configuration = new Configuration();
		configuration.Properties = prop;

		foreach (Assembly assembly in assemblies)
			configuration = configuration.AddAssembly(assembly);
		sessionFactory = configuration.BuildSessionFactory();
	}

	public ISession CreateSession()
	{
		var session = sessionFactory.OpenSession();
		new SchemaExport(configuration)
			.Execute(false, true, false, true, session.Connection, null);
		return session;
	}
} 

This base class takes care of configuring NHibernate with Sqlite in-memory provider, and registering all the assemblies where our hbm files are located. CreateSession method will load the new session with a fresh in-memory database, and it gets NHibernate to build it with the schema from those hbm files.

BOTTOM LINE
We have achieved our 3 objectives for isolation, repeatability, and speed. Additionally, compared to the other 4 unit-test approaches, in-memory database offers a unique advantage.
Each test-case is self sufficient: specifying its own pre-condition (initial data), and verifying the final outcome. Each test case is self explanatory in revealing the intention of the test. Test readers will find it remarkably easy to follow each of the test-cases independently without having to switch back and forth between Visual Studio and database IDE or dataset XML (as in the case with nDbUnit).
Having self-sufficient test-case might as well come as a disadvantage considering how bloated the test-code ends up, even for this rather simplistic example. In practice, this can get worse since you have to deal with populating the data for all chain of unrelated tables as well only to satisfy foreign key constraints when setting up initial data. Not to mention the frustration when the database schema changes every now and then. It is very likely that data initialization would typically take up majority of the test code, just to support mere couple lines of real test logic that we really care about.
Another disadvantage is that not all functionalities will work (or behave the same way) between in-memory and targetted database. Not to mention various subtle idiosynchracies with Sqlite.
And if you don’t use data-access framework that offers cross database portability (like NHibernate does), this approach is not even an option.

Mocking Helper for Pipe and Filter

In previous post, I was questioning the mockability of query filters in Rob Conery’s Pipe and Filter pattern. I left the post open with a hint about one possible approach to do it, hence this post.

So I have written a simple helper class, QueryableMock, specifically to address this issue, and apparently it is much more complex than I ever expected. The idea is basically not to really mock away the filter implementation, but instead to record the final result of expression-tree generated by the filter. During the playback, the SUT will run the filter methods again, and again, we capture again its expression-tree, and finally compare it with the previously recorded expression. If it matches, then we can safely say that our SUT has passed the test by running exactly all expected filters.

So just to remind ourselves, let’s take a look at the code we want to test from the last post, a Spammer class that sends spam emails to all Tazmanian teens.

public void SendTazmanianTeenSpam(string spamMessage)
{
	var targets = this.customerRepository.All()
		.LivesInCity("Tazmania")
		.YoungerThan(18);

	foreach(var customer in targets)
		emailSender.Send(customer.Email.Address, spamMessage);
}

So let’s check out how we write the unit-test using our helper QueryableMock class.

[Test]
public void CanSendSpamToTazmanianTeen()
{
	var targetStubs = new List<Customer>
	{
		new Customer(1) {Email = new Email("test@email.com")},
		new Customer(2) {Email = new Email("anotherTest@email.com")}
	};

	var customerQueryMock = new QueryableMock<Customer>();
	customerQueryMock.Expect()
		.LivesInCity("Tazmania")
		.YoungerThan(18);
	customerQueryMock.StubFinalResult(targetStubs);

	using (mockery.Record())
	{
		Expect.Call(customerRepository.All()).Return(customerQueryMock);
		emailSender.Send("test@email.com", spamMessage);
		emailSender.Send("anotherTest@email.com", spamMessage);
	}

	using (mockery.Playback())
	{
		spammer.SendTazmanianTeenSpam(spamMessage);
	}
}

Look closer at line #10:

customerQueryMock.Expect()
	.LivesInCity("Tazmania")
	.YoungerThan(18);
customerQueryMock.StubFinalResult(targetStubs);

What it does is building expected filtration expression. It is done by executing the real filter methods and record the final Queryable Expression. This recorded Expression will later on be compared with the Expression being executed later on in SUT: spammer.SendTazmanianTeenSpam(spamMessage);

UNDER THE HOOD
That was all we need to do in the unit-test. Here’s some relevant bit of the code behind QueryableMock.

public class QueryableMock<T> : QueryableProxy<T>
{
	// ... Other stuffs ...
	protected override IEnumerator GetFinalEnumerator()
	{
		VerifyExpression();
		return stubbedFinalEnumerable.GetEnumerator();
	}
	public void VerifyExpression()
	{
		if(!ExpressionComparer.CompareExpression(
			Provider.QueriedExpression, recorderQuery.Provider.QueriedExpression))
			throw new Exception("Query expression does not match expectation");
	}
}

The idea is so far pretty straightforward: when the SUT is about to execute (enumerate) the query filter, it will try to do VerifyExpression to compare the filter expression against expected expression.
The biggest challange here is to compare the equality of 2 expression trees. I.e., how to verify if x=>x+1 is equal to x=>x+1. I was quite surprised that there is no easy way of doing it. Life cannot be as simple as calling Expression.Equals(). So I wrote a quick brute-force comparison logic in ExpressionComparer class. You can check the detail on the attached source-code. If you know of any easier way to do it, please do let me know.

EVEN FURTHER
Finally, this QueryableMock utility is very specifically intended for unit-testing around Pipe-Filter pattern. It does a very limited thing, but so far it works very well to tackle my need (namely: to mock query-filters). Another interesting scenario that is also covered is:

customerQueryMock.Expect()
	.LivesInCity("New York")
	.SelectEmailAddresses()
	.InDomain("gmail.com");
customerQueryMock.StubFinalResult(emailList);

Note that the filtration switched from IQueryable<Customer> to IQueryable<Email> its half way through.

You may argue that TypeMock can provide much elegant solution for this. It uses bytecode instrumentation that can therefore intercept even static and extension methods. But I think the problem is not really whether I can do it or not. But if I cannot easily test a piece of code without relying on bytecode magic, I will question if the code design is even worth doing at all.

Admittedly, I am not 100% comfortable with this approach of mocking. Let me know what you think.

SOURCE CODE
You can download the full source-code of this post here to see what on earth I have been talking about.

How Do You Test Pipe and Filter?

In his notable MVC Storefront series, Rob Conery brought up a very intereting pattern, called Pipe and Filter. My first reaction to that was a bit anxious if it might hurt testability. Before I start with the problem, I think this pattern deserves a little bit introductory words, just in case you have stayed under the rock for the last 12 months, and haven’t checked Rob Conery’s posts.

Pipe and Filter is a pattern recently made popular by Rob’s MVC Storefront episodes. Also known as Filtration pattern, it is a very nice trick to produce a highly fluent Linq2Sql data repository (although, well, it’s not quite repository anymore).

Basically, instead of having several specific-purpose querying filters on repository interface like this:

var list = customerRepository
	.FindByStateAndAgeBelow("Tazmania", 20);

We can now use a far more fluent and flexible syntax through chained filtering statements like this:

var list = customerRepository.All()
	.WithState("Tazmania")
	.WithAgeBelow(20).List();

The magic behind it is .Net 3.5’s Extension Methods.

public static class CustomerFilter
{
	public IQueryable<Customer> WithState (this IQueryable<Customer> query, string state)
	{
		return from cus in query 
			where cus.HomeAddress.State == state
			select cus;
	}
	public IQueryable<Customer> WithAgeBelow(this IQueryable<Customer> query, int age)
	{
		var maxDob = Date.Now.AddYear(-age);
		return from cus in query 
			where cus.BirthDate < maxDob 
			select cus;
	}
}

Testing both of these filters is easy. Just create a collection of customer object, execute the filter, then go ahead and check the result. Here is the unit-test code for WithState filter.

// Stub Customer List
var list = new List<Customer>()
{
	new Customer() {HomeAddress = new Address() {State = "Illinois"}},
	new Customer() {HomeAddress = new Address() {State = "Tazmania"}},
	new Customer() {HomeAddress = new Address() {State = "NSW"}},
	new Customer() {HomeAddress = new Address() {State = "Tazmania"}}
};
var query = list.ToQueryable();

// Execute
var filtered = query.WithState("Tazmania").ToList();

// Verify
Assert.That(filtered.Count, Is.EqualTo(2));
Assert.IsTrue(filtered.Contains(list[1]));
Assert.That(filtered.Contains(list[3]));

Now imagine I work in an ambitious evil project, where I have a small piece of method in my business logic that sends spam emails to all Tazmanian teens.

public void SendAdvertisement(string message)
{
	foreach(var customer in 
		customerRepository.All()
		.WithState("Tazmania").WithAgeBelow(20))
	{
		emailSender.Send(customer.EmailAddress, message);
	}
}

The question is now, how do we write unit test to verify this logic?

Had we used customerRepository.FindByStateAndAgeBelow(“Tazmania”, 20), we would be able to just mock away the call to customerRepository.FindByStateAndAgeBelow(), (hence, behavior or interaction-based verification), and we are sorted.

But here, the problem with Pipe and Filter pattern is the fact that it uses Extension Methods, which are essentially static methods! (And we all know, static methods are the villains in TDD world). They are not mockable, and we have to deal with the real filter implementations.

True, I could just use the same state-based verification approach that we have done above on WithState unit-test by stubbing up a list of customer objects, and then verify the emailSender based on the expected filtered customers. But this is really gross.
I don’t want to care about the internal behavior of the filters here. As the matter of fact, each filter has already had its own unit-test (we have written one above), and I don’t want to repeat myself here. All that I really want to care is if my business logic makes the correct calls to the correct filters, and put the filtration result as our spam targets.

I want to gather how you write unit-test for Pipe|Filter pattern. How you mock out filter logic from your business-logic tests. I have a quick thought in mind about writing a Rhino-Mock helper to catter this scenario. I have yet to try it out, and I will write it on the next post as soon as I do. But first, I would like to hear what other people think about this. Any comment?

Unit Testing Data Access Code

There are a lot of challanges to write unit-tests for database facing code. It typically requires a lot of setup code, a lot of effort to maintain, and hard to make it repeatable. It has seemed to be a universal excuse for a lot of teams to drop TDD practice.

So I gather here all various approaches that I am aware of when dealing with database unit-testing. In the next several posts, I will cover each of the implementations in detail, but let’s just have a quick overview on all the options. More than anything else, it’s just a matter of taste really, rather than situational. Hopefully after the next several posts you can make out the good and the bad of each approach.

  1. In-memory database using Sqlite
  2. Transaction rollback
  3. Restorable file-based DB
  4. Preset state with NDBUnit
  5. Fake DB provider with Linq

Throughout the major part of these posts, I will assume the use of nHibernate. Why? Simply because we need somewhere to work on. These approaches are however equally applicable to virtually any ORM or data-access framework.

PS: It is perfectly valid to argue that these are all integration tests, rather than unit-tests. But I digress.

Unit Testing Web UI with Monorail

3 rules that always apply in regard to unit-test:

1.       Do NOT unit-test your infrastructure

2.       Test should be automated without user intervention

3.       Design is easy to test, by human eyes. Not by machines.

Let’s now see how our UI fits into this equation. Firstly, user interface involves a lot of infrastructure. To name a few: Http layers, GDI, session management, control callback, and all kind of web security manners. These things are not within our comfort zone and should not be coupled with our unit-test.

Secondly, there is virtually no way to navigate UI without having your hands on mouse and keyboard. Remember, the first word in UI is “User”.

And last, UI is all about design. The UI elements themselves have little meaning to machines, but it means everything on human perception. You might not even notice few slight changes on UI design, but it’s more than likely to break all automated UI test scripts.

Try unit testing your UI, and you’ll quickly break all those 3 rules. User interface code has seemed to repel all but the most serious attempts at test automation. Therefore, automated testing against UI code was just flat out deemed too much work for the gain.

The equation has changed over the last several years with the emergence of MVC architecture. MVC is UI pattern that improves UI testability.  UI is an ugly untestable part of the application, and using MVC, we want to isolate the ugliness in one place (the View), and put the logic in a place that’s easier to test (the Controller). We will slice the View as thin as possible. Put as little code as possible in the View. Remember, any code in a View is almost automatically harder to test than it would be in a Controller.

Castle Monorail is one of the first MVC alternatives to ASP.Net web-form. It’s based on Ruby on Rails, and brings 2 most important features to Asp.Net: simplicity and separation of concerns. To demonstrate how it can improve web development, I will show an overly simplistic demo application using untestable ASP.Net web form, and demonstrate how easy it is to switch to Monorail counterpart, and make it a fully testable web application.

The application is “The Pester”. It allows users (employees) to login into the system, so their boss can assign them with tasks and keep bugging them for the rest of the day. Hence the application name.

First screen is a pretty basic login screen.

It asks for username and password, authenticate to SecurityService. If fails, display a message the the user. And if authentication passes, it will redirect to “View Task Summary” screen. It’s not that complex of a requirement really. The code behind of the webform:

public partial class login : System.Web.UI.Page

{

   protected void Page_Load(object sender, EventArgs e)

   {

      this.lbLoginFailure.Visible = false;

   }

 

   protected void btLogin_Click(object sender, EventArgs e)

   {

      SecurityService securityService = new SecurityService();

                    

       bool isSuccess = securityService.AuthenticateLogin(

       this.txUsername.Text, this.txPassword.Text);

 

       if(!isSuccess)

          this.lbLoginFailure.Visible = true;

       else

          Response.Redirect(“viewTask.aspx”);

    }

}

That’s pretty self-explanatory. And now the “View Task Summary” screen looks like the following:

 

It starts with a very friendly welcoming message, which is followed by a list of all the tasks for current user fetched from the database. The aspx code:

<%@ Page Language=”C#” AutoEventWireup=”true”

         CodeBehind=”taskSummary.aspx.cs”

         Inherits=”SheepTask.WebForm.viewTask” %>

 

<html xmlns=”http://www.w3.org/1999/xhtml&#8221; >

<head runat=”server”>

   <link rel=”stylesheet” href=”css/default.css”

         type=”text/css” />

   <title>Tasks From Pester</title>

</head>

<body>

   <form id=”form1″ runat=”server”>

     <h1>Now Get Back to Work!</h1>

     <asp:DataGrid id=”dgTasks”

       HeaderStyle-CssClass=”gridHeader” CssClass=”grid”

       runat=”server” AutoGenerateColumns=”False”>

       <Columns>

          <asp:BoundColumn ItemStyle-Width=”100″

              DataField=”SlaDate” HeaderText=”SLA”

              DataFormatString=”{0:dd/MM/yyyy}” />

          <asp:BoundColumn ItemStyle-Width=”120″

              DataField=”AssignerName” HeaderText=”Assigner” />

          <asp:HyperLinkColumn ItemStyleWidth=”500″

              DataNavigateUrlField=”TaskID”

              DataNavigateUrlFormatString=

                     “taskDetail.aspx?taskId={0}”

              DataTextField=”Title” HeaderText=”Task”/>

       </Columns>

     </asp:DataGrid>

   </form>

</body>

</html>

And the code behind of the webform:

public partial class viewTask : System.Web.UI.Page

{

   protected void Page_Load(object sender, EventArgs e)

   {

      string username = (string)Session[“username”];

      TaskRepository taskRepository = new TaskRepository();

      IList<Task> tasks =

           taskRepository.GetAllTasks(username);

             

      this.dgTasks.DataSource = tasks;

      this.dgTasks.DataBind();

   }

}

That code really isn’t that complex, but let’s think about how we could automate a test for this requirement to run within our regression test suite. That little bitty coupling with web control and coupling to HttpResponse that results is a serious pain to deal with in an automated test. Furthermore, the dependency with SecurityService and TaskRepository makes it impossible to be unit-tested in isolation to backing LDAP and database. But I think the worst part is that to test this logic you have to fire up the browser, navigate through login page, then trigger the btnLogin Click event. That’s a lot of work just to get to the point at which you are exercising the code you care about.

Now, let’s rewrite these screens with Monorail. The structure of our Monorail project looks like the following.

Other than default configuration to activate Monorail in Web.config, we won’t need any more configuration to get this working. This simplicity is one of the main strengths of Monorail that is well borrowed from Rail’s concept of “Convention over Configuration”. In our structure, we have 2 views split across 2 folders: home and task. By convention, the controllers will need to be defined with name HomeController and TaskController, each of which will have a method with a name corresponding to the filename of its respective view template: Login and TaskSummary. Each of the views looks like the following.

<html>

<head>

  <link rel=”stylesheet” href=”/css/default.css”

        type=”text/css” />

  <title>Pester Login</title>

</head>

<body>

<form>

   <% if isLoginFailed: %>

      <div class=”error”>Failed to match username and password</div>

   <% end %>

   Username: <input name=”username” type=”text” /> <br />

   Password: <input name=”password” type=”password” /> <br />

   <input id=”login” value=”Login” type=”submit” />

 </form>

</body>

</html>

 

<html>

<head>

    <link rel=”stylesheet” href=”/css/default.css”

          type=”text/css” />

    <title>Pester Login</title>

</head>

<body>

   <h1>Now Get Back to Work!</h1>

   <table class=”grid”>

     <tr class=”gridHeader”>

        <td width=”100″>SLA</td>

        <td width=”120″>Assigner</td>

        <td width=”500″>Task</td>

      </tr>

      <% for task in taskList: %>

      <tr>

        <td>${task.SlaDate.ToString(“dd/MM/yyyy”)}</td>

        <td>${task.AssignerName}</td>

         <td>

           <a href=”taskDetail.aspx?taskId=${task.TaskId}”>

               ${task.Title}

           </a>

         </td>

      </tr>

      <% end %>

   </table>

</body>

</html>

Now let’s implement the controller for our first page: login. First, to allow the code to be tested without concrete implementation of LDAP SecurityService, we refactor the SecurityService to extract an interface. The following is the code for HomeController.

public class HomeController: SmartDispatcherController

{

   private ISecurityService securityService;

 

   public HomeController(ISecurityService securityService)

   {

       this.securityService = securityService;

   }

 

   public void Login()

   {

       PropertyBag[“isLoginFailed”] = false;

   }

 

   public void Login(string username, string password)

   {

       if (securityService.AuthenticateLogin

             (username, password))

       {

           Session[“username”] = username;

           Response.Redirect(“task”, “taskSummary”);

       }

       else

           PropertyBag[“isLoginFailed”] = true;

   }

}

Notice the ISecurityService. We no longer tie the UI with concrete implementation of SecurityService. This will allow good isolation for our unit-test later on. I also use SmartDispatcherController that gives us a slick feature for automatic data-binding. It automatically transform query string and POST values into method’s parameters with corresponding names. We have therefore 2 overload of Login method: a parameter-less method for initial visit to the login screen, and another method that takes username and password to handle user’s POST login request. The form information will automatically gets passed to both parameters. SecurityService is injected through constructor.

Our second controller follows. There’s nothing special compared to the first controller.

public class TaskController: SmartDispatcherController

{

    private ITaskRepository taskRepository;

 

    public TaskController(ITaskRepository taskRepository)

    {

       this.taskRepository = taskRepository;

    }

 

    public void TaskSummary()

    {

       string username = (string)Session[“username”];

       PropertyBag[“taskList”] =

            taskRepository.GetAllTasks(username);

    }

}

How is this model better? Since I’ve been claiming that this style of structuring the user interface improves the testability of our web, let’s take a look at how we can write our unit-test for login. Astute readers will now note that I should be writting the unit-test code before started the controller code. But it’s a bit hard to explain things in that order. 

[TestFixture]

public class HomeControllerTest : BaseControllerTest

{

   private MockRepository mockery;

   private ISecurityService service;

   private HomeController controller;

 

   [SetUp]

   public void BeforeTest()

   {

      mockery = new MockRepository();

      service = mockery.CreateMock<ISecurityService>();

      controller = new HomeController(service);

      PrepareController(controller);

   }

 

   [Test]

   public void ShouldAlertUserForInvalidLogin()

   {

      using (this.mockery.Record())

      {

         Expect.Call(

            service.AuthenticateLogin(“myUser”, “myPassword”))

            .Return(false);

      }

 

      using (mockery.Playback())

         controller.Login(“myUser”, “myPassword”);

 

      Assert.AreEqual(true,

         controller.PropertyBag[“isLoginFailed”]);

   }

 

   [Test]

   public void SuccessfulLoginShouldRedirectToTaskSummary()

   {

      using (mockery.Record())

      {

         Expect.Call(

            service.AuthenticateLogin(“myUser”, “myPassword”))

            .Return(true);

      }

 

      using (mockery.Playback())

         this.controller.Login(“myUser”, “myPassword”);

 

      Assert.AreEqual(“/task/taskSummary.rails”,

         Response.RedirectedTo);

   }

}

Now you notice that since we have extracted out the interface for SecurityService, we can use Rhino-Mock to mock out the SecurityService implementation from the controller. I use BaseControllerTest from Monorail test-helper to intercept Response and Request object in order to evaluate URL redirection (to /task/taskSummary.rails) without tying the test to web infrastructure. PrepareController is one of the method comes from the BaseControllerTest that does the mock-out for us.

TaskController can be as easily unit tested in similar fashion.

[TestFixture]

public class TaskControllerTest: BaseControllerTest

{

   [Test]

   public void ShouldDisplayUsersTaskSummary()

   {

      MockRepository mockery = new MockRepository();

      ITaskRepository taskRepository =

         mockery.CreateMock<ITaskRepository>();

      TaskController controller = new

         TaskController(taskRepository);

      

      PrepareController(controller);

 

      Context.Session[“username”] = “myUser”;

      List<Task> taskList = new List<Task>();

      

      using(mockery.Record())

      {             

         Expect.Call(taskRepository.GetAllTasks(“myUser”))

            .Return(taskList);

      }

 

      controller.PropertyBag[“tasks”] = taskList;

   }

}

That wasn’t bad, now was it? But hang on, you ask. Where is the actual logic for the login and fetching the task? For right now, I’m just worried about wiring the screen itself. Yes, this unit-test covers very little ground, but we have created some level of trust that this particularl linkage in the code does work. I can now turn my back on the UI and test the actual SecurityService and TaskRepository.

One last thing missing from this article. We unwired our controller from concrete implementation of SecurityService and TaskRepository for the sake of unit-test. So now we need to inject the dependency through the constructor of the controller. Fortunately, this can be easily solved by integrating Monorail with Castle Windsor. We need to register both controllers and backing servicees to the Windsor container.

windsor.AddFacility(“rails”, new RailsFacility());

 

windsor.AddComponent<HomeController>();

windsor.AddComponent<TaskController>();

windsor.AddComponent<ISecurityService, SecurityService>();

windsor.AddComponent<ITaskRepository, TaskRepository>();

The first line is to register the facility to allow Monorail to resolve its controllers from Windsor container. The rest are to register each of the controllers and services. It uses auto-wiring to automatically introspect all necessary services to instantiate a controller. Now everytime Monorail instantiate the controller, we will get all necessary logic in place. The resulted application is pretty much the same as the original webform application. But with improved separation of concern under the hood.

An incisive reader will note that we didn’t write any unit tests for the View.  The philosophy in general is to make the hard to test view code so simple as to be reliably verified by inspection alone.  I.e., you should make the View code so simple that it’s almost impossible to screw up.

TDD Screencast – Episode 1

This is the beginning of series of screencasts about applying TDD in ‘real-world’-ish .Net application. This series is my answer to recent discussion in .net-Indonesia list about using Spring and Rhino-Mock to test-drive .net applications.

Throughout the series, we will be building an example application using common Agile frameworks:
– NUnit
– Rhino-Mock
– Castle Monorail to front the web MVC
– Castle Windsor for the IoC, as opposed to Spring
– Castle Active Records backed by nHibernate.
– SqlLite to cover in-memory database-testing topic

I will also try exploring about using Linq for nHibernate, although I haven’t really tried it in real project.
For the first episode, I decided not to start with the application. This episode covers nothing but the most basic ground of Test-Driven-Development from naivitae point of view. Those with fair awareness of TDD might not get anything useful out of this. But this will set the base principle we will follow for the rest of the series.
It also (hopefully) gives me opportunity to get feedbacks and suggestions from you guys for the next episodes.

You can download the screencast at http://mihd.net/s0ad6i. (invalid)
The screencast is available at http://mihd.net/o6sx13w

You will need Techsmith codec that can be downloaded for free at http://www.techsmith.com/download/codecs.asp.