Logging is So Old School

Microsoft has revealed Visual Studio 2010 (aka Rosario) and its.Net Framework 4. One thing that immediately intrigues me is its new recorded debugging feature that allows us to “debug the past”, especially to tackle non-reproducable bugs, sort of like watching yesterday’s rugby match from your TiVo, with that remote in your hand to fast-forward and rewind as you like. I think this is a really brilliant idea, and I will nominate it for nobel prize for saving millions of developers nights. It might soon eliminate the word “non-reproducable bugs” completely from English vocabulary.
In several years time we might be living in a different world. Debugging is an activity to look into the past. Soon you might find yourself saying “What time yesterday did you see your button went missing? I will attach my debugger to there“. You will probably never spend any day staring behind thousands line of stack trace and logging information. Logs are for dinosaurs (let’s just forget about production issues for a moment). And I probably wouldn’t be bothered too much about writing logging in applications anymore.

Data Access Test with Sqlite

I have mentioned briefly in previous post about 5 different approaches to write unit-test for data-access code. So now I will try to cover the first method in more detail. And among the 5, this method seems to be the most common one so far.
I will be using Sqlite as in-memory database. Some other arguably better alternative is available, that is if you are happy to fork out some extra bucks on VistaDB.
So let’s reemphasize the desired characteristics of a good unit test, especially in the context of data-access code.
1. Isolated. Database (by definition) persists any change to its state. However, this is usually not desirable in unit-test. A good test case runs in complete isolation from any changes made in other test-cases. I.e., any changes to the database from one test case should not be visible from other test-cases.
2. Repeatable. Regardless how many times a unit test is executed, a consistent result is expected. For this to happen, unit-test should not rely on presumption on external condition, especially shared database.
3. Fast. If your test fixture cannot be executed in every several minutes or so, there is only one thing that can possibly happen: developers will start abandoning it. And unfortunately, this is usually the case if you are connecting to database system from test code, where the whole test-suite can take up to an hour to execute, if you are lucky. No one can run it frequent enough to make it useful, and hence no one will keep on maintaining it.

IN MEMORY DATABASE
The main reason why using in-memory database for unit-test is that it is incredibly fast! Both in restoring zero state, execution, and cleaning up. Speaking of which, let’s take a look on typical cycle of in memory database in unit-test.
We are going to have NHibernate to build the database schema from scratch before each test case, and dispose it at the end of the test case. Then start again with building the schema on clean database again for the next test-case. This way, we always have an empty sheet to work on for each test-case without affecting (or being affected by) any database changes in other test-cases.
Here is some NUnit test case example using NHibernate.

[TestFixture]
public class CustomerRepositoryTest: InMemoryDBTestFixture
{
	private ISession session;
	private CustomerRepository repository;
	
	[TestFixtureSetuUp]
	public void FixtureSetUp()
	{
		InitialiseNHibernate(typeof(Customer).Assembly);
	}
	
	[SetUp]
	public void SetUp()
	{
		this.session = this.CreateSession();
		this.repository = new CustomerRepository(
			new FakeSessionManager(session));
	}
	
	[TearDown]
	public void TearDown()
	{
		this.session.Dispose();
	}
	
	[Test]
	public void CanQueryCustomerByLastname()
	{
		var customers = new List<Customer>(){
			new Customer(){
				FirstName="Peter",
				LastName="Griffin"},
			new Customer(){
				FirstName="Other",
				LastName="Lads"},
			new Customer(){
				FirstName="Meg",
				LastName="Griffin"},
			new Customer(){
				FirstName="Yet Another",
				LastName="Bloke"}};
		
		foreach(var cus in customers)
			session.Save(cus);
		session.Flush();
		foreach(var cus in customers)
			session.Evict(cus);
		
		var loaded = repository.QueryByLastname("Griffin");
		
		Assert.That(loaded.Count, Is.EqualTo(2));
		AssertLoadedDataEqual(loaded[0], customers[0]);
		AssertLoadedDataEqual(loaded[1], customers[2]);
	}
	private static void AssertLoadedDataEqual(Customer loaded, Customer saved)
	{
		AssertThat(loaded, Is.NotEqualTo(saved)); // Make sure it's not cached data
		
		Assert.That(loaded.ID, Is.EqualTo(saved.ID));
		Assert.That(loaded.FirstName, Is.EqualTo(saved.FirstName));
		Assert.That(loaded.LastName, Is.EqualTo(saved.LastName));
	}
}

The plumbing for initializing NHibernate and building in-memory database is managed by base-class InMemoryDBTestFixtureBase. It is a very common practice to have this kind of base class for all database test-fixtures within a project, so we we can turn our back on setting up test database and concentrate on testing what we care about. Let’s take a look at the base class code.

public abstract class InMemoryDBTestFixtureBase
{
	protected static ISessionFactory sessionFactory;
	protected static Configuration configuration;

	public static void InitialiseNHibernate(params Assembly [] assemblies)
	{
		if(sessionFactory!=null)
			return;

		var prop = new Hashtable();
		prop.Add("hibernate.connection.driver_class", "NHibernate.Driver.SQLite20Driver");
		prop.Add("hibernate.dialect", "NHibernate.Dialect.SQLiteDialect");
		prop.Add("hibernate.connection.provider", "NHibernate.Connection.DriverConnectionProvider");
		prop.Add("hibernate.connection.connection_string", "Data Source=:memory:;Version=3;New=True;");

		configuration = new Configuration();
		configuration.Properties = prop;

		foreach (Assembly assembly in assemblies)
			configuration = configuration.AddAssembly(assembly);
		sessionFactory = configuration.BuildSessionFactory();
	}

	public ISession CreateSession()
	{
		var session = sessionFactory.OpenSession();
		new SchemaExport(configuration)
			.Execute(false, true, false, true, session.Connection, null);
		return session;
	}
} 

This base class takes care of configuring NHibernate with Sqlite in-memory provider, and registering all the assemblies where our hbm files are located. CreateSession method will load the new session with a fresh in-memory database, and it gets NHibernate to build it with the schema from those hbm files.

BOTTOM LINE
We have achieved our 3 objectives for isolation, repeatability, and speed. Additionally, compared to the other 4 unit-test approaches, in-memory database offers a unique advantage.
Each test-case is self sufficient: specifying its own pre-condition (initial data), and verifying the final outcome. Each test case is self explanatory in revealing the intention of the test. Test readers will find it remarkably easy to follow each of the test-cases independently without having to switch back and forth between Visual Studio and database IDE or dataset XML (as in the case with nDbUnit).
Having self-sufficient test-case might as well come as a disadvantage considering how bloated the test-code ends up, even for this rather simplistic example. In practice, this can get worse since you have to deal with populating the data for all chain of unrelated tables as well only to satisfy foreign key constraints when setting up initial data. Not to mention the frustration when the database schema changes every now and then. It is very likely that data initialization would typically take up majority of the test code, just to support mere couple lines of real test logic that we really care about.
Another disadvantage is that not all functionalities will work (or behave the same way) between in-memory and targetted database. Not to mention various subtle idiosynchracies with Sqlite.
And if you don’t use data-access framework that offers cross database portability (like NHibernate does), this approach is not even an option.