Mocking Helper for Pipe and Filter

In previous post, I was questioning the mockability of query filters in Rob Conery’s Pipe and Filter pattern. I left the post open with a hint about one possible approach to do it, hence this post.

So I have written a simple helper class, QueryableMock, specifically to address this issue, and apparently it is much more complex than I ever expected. The idea is basically not to really mock away the filter implementation, but instead to record the final result of expression-tree generated by the filter. During the playback, the SUT will run the filter methods again, and again, we capture again its expression-tree, and finally compare it with the previously recorded expression. If it matches, then we can safely say that our SUT has passed the test by running exactly all expected filters.

So just to remind ourselves, let’s take a look at the code we want to test from the last post, a Spammer class that sends spam emails to all Tazmanian teens.

public void SendTazmanianTeenSpam(string spamMessage)
{
	var targets = this.customerRepository.All()
		.LivesInCity("Tazmania")
		.YoungerThan(18);

	foreach(var customer in targets)
		emailSender.Send(customer.Email.Address, spamMessage);
}

So let’s check out how we write the unit-test using our helper QueryableMock class.

[Test]
public void CanSendSpamToTazmanianTeen()
{
	var targetStubs = new List<Customer>
	{
		new Customer(1) {Email = new Email("test@email.com")},
		new Customer(2) {Email = new Email("anotherTest@email.com")}
	};

	var customerQueryMock = new QueryableMock<Customer>();
	customerQueryMock.Expect()
		.LivesInCity("Tazmania")
		.YoungerThan(18);
	customerQueryMock.StubFinalResult(targetStubs);

	using (mockery.Record())
	{
		Expect.Call(customerRepository.All()).Return(customerQueryMock);
		emailSender.Send("test@email.com", spamMessage);
		emailSender.Send("anotherTest@email.com", spamMessage);
	}

	using (mockery.Playback())
	{
		spammer.SendTazmanianTeenSpam(spamMessage);
	}
}

Look closer at line #10:

customerQueryMock.Expect()
	.LivesInCity("Tazmania")
	.YoungerThan(18);
customerQueryMock.StubFinalResult(targetStubs);

What it does is building expected filtration expression. It is done by executing the real filter methods and record the final Queryable Expression. This recorded Expression will later on be compared with the Expression being executed later on in SUT: spammer.SendTazmanianTeenSpam(spamMessage);

UNDER THE HOOD
That was all we need to do in the unit-test. Here’s some relevant bit of the code behind QueryableMock.

public class QueryableMock<T> : QueryableProxy<T>
{
	// ... Other stuffs ...
	protected override IEnumerator GetFinalEnumerator()
	{
		VerifyExpression();
		return stubbedFinalEnumerable.GetEnumerator();
	}
	public void VerifyExpression()
	{
		if(!ExpressionComparer.CompareExpression(
			Provider.QueriedExpression, recorderQuery.Provider.QueriedExpression))
			throw new Exception("Query expression does not match expectation");
	}
}

The idea is so far pretty straightforward: when the SUT is about to execute (enumerate) the query filter, it will try to do VerifyExpression to compare the filter expression against expected expression.
The biggest challange here is to compare the equality of 2 expression trees. I.e., how to verify if x=>x+1 is equal to x=>x+1. I was quite surprised that there is no easy way of doing it. Life cannot be as simple as calling Expression.Equals(). So I wrote a quick brute-force comparison logic in ExpressionComparer class. You can check the detail on the attached source-code. If you know of any easier way to do it, please do let me know.

EVEN FURTHER
Finally, this QueryableMock utility is very specifically intended for unit-testing around Pipe-Filter pattern. It does a very limited thing, but so far it works very well to tackle my need (namely: to mock query-filters). Another interesting scenario that is also covered is:

customerQueryMock.Expect()
	.LivesInCity("New York")
	.SelectEmailAddresses()
	.InDomain("gmail.com");
customerQueryMock.StubFinalResult(emailList);

Note that the filtration switched from IQueryable<Customer> to IQueryable<Email> its half way through.

You may argue that TypeMock can provide much elegant solution for this. It uses bytecode instrumentation that can therefore intercept even static and extension methods. But I think the problem is not really whether I can do it or not. But if I cannot easily test a piece of code without relying on bytecode magic, I will question if the code design is even worth doing at all.

Admittedly, I am not 100% comfortable with this approach of mocking. Let me know what you think.

SOURCE CODE
You can download the full source-code of this post here to see what on earth I have been talking about.

How Do You Test Pipe and Filter?

In his notable MVC Storefront series, Rob Conery brought up a very intereting pattern, called Pipe and Filter. My first reaction to that was a bit anxious if it might hurt testability. Before I start with the problem, I think this pattern deserves a little bit introductory words, just in case you have stayed under the rock for the last 12 months, and haven’t checked Rob Conery’s posts.

Pipe and Filter is a pattern recently made popular by Rob’s MVC Storefront episodes. Also known as Filtration pattern, it is a very nice trick to produce a highly fluent Linq2Sql data repository (although, well, it’s not quite repository anymore).

Basically, instead of having several specific-purpose querying filters on repository interface like this:

var list = customerRepository
	.FindByStateAndAgeBelow("Tazmania", 20);

We can now use a far more fluent and flexible syntax through chained filtering statements like this:

var list = customerRepository.All()
	.WithState("Tazmania")
	.WithAgeBelow(20).List();

The magic behind it is .Net 3.5’s Extension Methods.

public static class CustomerFilter
{
	public IQueryable<Customer> WithState (this IQueryable<Customer> query, string state)
	{
		return from cus in query 
			where cus.HomeAddress.State == state
			select cus;
	}
	public IQueryable<Customer> WithAgeBelow(this IQueryable<Customer> query, int age)
	{
		var maxDob = Date.Now.AddYear(-age);
		return from cus in query 
			where cus.BirthDate < maxDob 
			select cus;
	}
}

Testing both of these filters is easy. Just create a collection of customer object, execute the filter, then go ahead and check the result. Here is the unit-test code for WithState filter.

// Stub Customer List
var list = new List<Customer>()
{
	new Customer() {HomeAddress = new Address() {State = "Illinois"}},
	new Customer() {HomeAddress = new Address() {State = "Tazmania"}},
	new Customer() {HomeAddress = new Address() {State = "NSW"}},
	new Customer() {HomeAddress = new Address() {State = "Tazmania"}}
};
var query = list.ToQueryable();

// Execute
var filtered = query.WithState("Tazmania").ToList();

// Verify
Assert.That(filtered.Count, Is.EqualTo(2));
Assert.IsTrue(filtered.Contains(list[1]));
Assert.That(filtered.Contains(list[3]));

Now imagine I work in an ambitious evil project, where I have a small piece of method in my business logic that sends spam emails to all Tazmanian teens.

public void SendAdvertisement(string message)
{
	foreach(var customer in 
		customerRepository.All()
		.WithState("Tazmania").WithAgeBelow(20))
	{
		emailSender.Send(customer.EmailAddress, message);
	}
}

The question is now, how do we write unit test to verify this logic?

Had we used customerRepository.FindByStateAndAgeBelow(“Tazmania”, 20), we would be able to just mock away the call to customerRepository.FindByStateAndAgeBelow(), (hence, behavior or interaction-based verification), and we are sorted.

But here, the problem with Pipe and Filter pattern is the fact that it uses Extension Methods, which are essentially static methods! (And we all know, static methods are the villains in TDD world). They are not mockable, and we have to deal with the real filter implementations.

True, I could just use the same state-based verification approach that we have done above on WithState unit-test by stubbing up a list of customer objects, and then verify the emailSender based on the expected filtered customers. But this is really gross.
I don’t want to care about the internal behavior of the filters here. As the matter of fact, each filter has already had its own unit-test (we have written one above), and I don’t want to repeat myself here. All that I really want to care is if my business logic makes the correct calls to the correct filters, and put the filtration result as our spam targets.

I want to gather how you write unit-test for Pipe|Filter pattern. How you mock out filter logic from your business-logic tests. I have a quick thought in mind about writing a Rhino-Mock helper to catter this scenario. I have yet to try it out, and I will write it on the next post as soon as I do. But first, I would like to hear what other people think about this. Any comment?

Unit Testing Data Access Code

There are a lot of challanges to write unit-tests for database facing code. It typically requires a lot of setup code, a lot of effort to maintain, and hard to make it repeatable. It has seemed to be a universal excuse for a lot of teams to drop TDD practice.

So I gather here all various approaches that I am aware of when dealing with database unit-testing. In the next several posts, I will cover each of the implementations in detail, but let’s just have a quick overview on all the options. More than anything else, it’s just a matter of taste really, rather than situational. Hopefully after the next several posts you can make out the good and the bad of each approach.

1. In-memory database using Sqlite
2. Transaction rollback
3. Restorable file-based DB
4. Preset state with NDBUnit
5. Fake DB provider with Linq

Throughout the major part of these posts, I will assume the use of nHibernate. Why? Simply because we need somewhere to work on. These approaches are however equally applicable to virtually any ORM or data-access framework.

PS: It is perfectly valid to argue that these are all integration tests, rather than unit-tests. But I digress.