Roll Your Own COP, Part 2: Concern and SideEffect

It has been a while, and this is my first blog post since I bought an XBox 360 with bunch of highly addictive games. Combined with some unusual working-weekends, and a recent discovery of a very nice pub in neighbourhood, I have really been given an awfully hard time to even get anywhere near my laptop.

It’s now time to get back to my dusty Visual Studio, and I will start with the second instalment of “Roll Your Own COP” saga. If you haven’t read the first episode of this series, I recommend you to do so. And if you haven’t heard of Composite Oriented Programming at all, you might want to visit Q14j website who initiated this rather eccentric concept.

Today we will be implementing the next features of COP, namely Concern and SideEffect. They both are similar in the idea, and in AOP they are known with an only single concept: Interceptor. In COP, they are split in two, which differ subtly in 2 ways:

  • Concern is a method interceptor that can be applied to a composite, and is called BEFORE each method call. It therefore is able to amend the method argument, or even decide not to proceed the method call. SideEffect in the other hand, is called AFTER the completion of all Concerns and method invocations. I.e., SideEffect can never mess around with the method invocation, its arguments, or return value in any way.
  • Concern is intended to be applied specifically within mixin boundary. For cross-mixins or even cross-composites operations, SideEffect is to be used.

We will be implementing both features using an example from Qi4j 10 minutes introduction.

[Concern(typeof(PurchaseLimitConcern),typeof(InventoryConcern))]
[Mixin(typeof(OrderImpl), typeof(HasLineItemImpl))]
public interface OrderComposite: Order, HasLineItem
{
	
}

The following is the definition of each mixin:

public interface HasLineItem
{
	void AddLineItem(LineItem item);
	void RemoveLineItem(LineItem item);
}
public interface Order
{
	string CustomerName{ get; set; }
	void Submit();
}

We need to be able to define one or more Concerns chained together to intercept methods on a composite. For instance;

public class InventoryConcern: HasLineItem
{
	private InventoryRepository inventoryRepository 
		= new InventoryRepository();

	[ConcernFor] 
	private HasLineItem target;

	public void AddLineItem(LineItem item)
	{
		inventoryRepository.Reduce(item.ProductCode, item.Quantity);
		target.AddLineItem(item);
	}

	public void RemoveLineItem(LineItem item)
	{
		inventoryRepository.Add(item.ProductCode, item.Quantity);
		target.RemoveLineItem(item);
	}
}
public class PurchaseLimitConcern: HasLineItem
{
	[ConcernFor] private HasLineItem target;

	public void AddLineItem(LineItem item)
	{
		Console.WriteLine("Checking quantity below 100");

		if (item.Quantity >= 100)
			throw (new ArgumentException(
"Can't exceed 100 units of product in a line item"));
	}

	public void RemoveLineItem(LineItem item)
	{
		target.RemoveLineItem(item);
	}
}

The InventoryConcern is implemented as a class, that implements HasLineItem (mixin) interface to intercept each of its methods.

We can see that the InventoryService is instantiated straight in the field definition. In further stage, we will provide [Service] attribute decoration on it to hook it with dependency injection framework (e.g. Windsor or Spring), as also shown in Qi4j version of this example. But dependency injection is not of our interests today.

The [ConcernFor] attribute is to create the interceptor call chain and each Concern is required to have only one such field. Each interceptor method can (but not necessarily) forward the invocation to the next object in interception chain. In this case, InventoryConcern to PurchaseLimitConcern then finally to HasLineItemImpl. This gives Concern the flexibility to change method arguments to next invocation, or even to neglect forwarding the call.

Note that [ConcernFor] will only return specific mixin interface and you can’t cast it back to the composite or one of its mixins. It is intended by design since a Concern should not cross its mixin boundary.
Now let’s start implementing the Composheep code to support these. Before we start, let’s refactor our code from previous episode. CompositeInterceptor.AddMixin() method:

public void AddMixin(object mixin)
{
	Type targetType = mixin.GetType();

	MethodInfo[] methods =
	targetType.GetMethods(
	BindingFlags.Instance |
	BindingFlags.Public);

	foreach(MethodInfo method in methods)
	{
		// Skip members declared in System.Object
		if (method.DeclaringType == typeof(object))
			continue;
			methodTargetMap.Add(
				string.Intern(method.ToString())
				, mixin);
	}
}

Some of the code to introspect public methods and the implementation detail of dictionary hashing method can be extracted out for reusability and improved readability.

public void AddMixin(object mixin)
{
	MethodInfo[] methods = GetPublicMethods(mixin);

	foreach(MethodInfo method in methods)
	{
		// Skip members declared in System.Object
		if (method.DeclaringType == typeof(object))
			continue;
			
		methodTargetMap.Add(HashKeyOf(method), mixin);
	}
}
private MethodInfo[] GetPublicMethods(object mixin)
{
	Type targetType = mixin.GetType();

	return targetType.GetMethods(
		BindingFlags.Instance | BindingFlags.Public);
}
		
private static string HashKeyOf(MethodInfo method)
{
	return string.Intern(method.ToString());
}

Let’s now add new method to add Concern:

public void AddConcern(object concern)
{
	MethodInfo[] methods = GetPublicMethods(concern);

	foreach (MethodInfo method in methods)
	{
		string key = HashKeyOf(method);

		if(!methodTargetMap.ContainsKey(key))
			continue;

		object target = methodTargetMap[key];

		LinkConcernChain(concern, target);
		methodTargetMap[key] = concern;
	}
}
private void LinkConcernChain(object concern, object target)
{
	foreach (FieldInfo field in
		concern.GetType().GetFields(
			BindingFlags.Instance | 
			BindingFlags.Public | 
			BindingFlags.NonPublic))
	{
		if (field.IsDefined(typeof(ConcernForAttribute), true))
		{
			field.SetValue(concern, target);
			break;
		}
	}
}

This method simply take an invocation target for each method, and slip in the Concern instance as the new target, and put the original target in Concern’s invocation chain. And also, we can’t put a concern on unimplemented methods (i.e., those without targets), therefore we skip methods that doesn’t exist in methodTargetMap.

Finally, I also refactored CompositeBuilder.BuildComposite<T>() method solely for improved readability:

public T BuildComposite<T>() where T : class
{
	ProxyGenerator generator = new ProxyGenerator();

	CompositeInterceptor interceptor = new CompositeInterceptor();
	T composite =
		generator.CreateInterfaceProxyWithoutTarget<T>
			(interceptor);
			
	PopulateMixins<T>(interceptor);
	PopulateConcerns<T>(interceptor);
	
return composite;
}

All the codes about mixin has been extracted out into PopulateMixins<T>() method. Additionally, we have also defined new method PopulateConcerns<T>().

private void PopulateConcerns<T>(CompositeInterceptor interceptor)
{
	object[] attributes =
		typeof(T).GetCustomAttributes(
			typeof(ConcernAttribute), true);
	foreach (ConcernAttribute concern in attributes)
	{
		foreach (Type concernType in concern.Types)
			interceptor.AddConcern(
				Activator.CreateInstance(concernType));
	}
}

I suppose this code is self explanatory and is very similar to PopulateMixins<T>() method. And we’re done with Concern. A quick run of the following program:

orderComp.CustomerName = "Hendry";

LineItem burger = new LineItem("Burger", 3);
LineItem kebab = new LineItem("Kebab", 2); 
			
orderComp.AddLineItem(burger);
orderComp.AddLineItem(kebab);

orderComp.RemoveLineItem(burger);

orderComp.Submit();

Console.ReadLine();

Will gives us this:

Now, to start with SideEffect, let’s say that we want to send a mail to sales@mycompany.com when the order is submitted. This is a SideEffect, and will execute after the Constraints, Concerns and Mixins. We add the SideEffect to the OrderComposite;

[SideEffect(typeof(MailNotifySideEffect))]
[Concern(typeof(PurchaseLimitConcern),typeof(InventoryConcern))]
[Mixin(typeof(OrderImpl), typeof(HasLineItemImpl))]
public interface OrderComposite: Order, HasLineItem
{
	
}
public class MailNotifySideEffect: Order
{
	[This]
	private Order order;

	private MailService mailer = new MailService();
		
	public string CustomerName {get; set;}

	public void Submit()
	{
		mailer.Send("sales@mycompany.com", "An order has been made for " + order.CustomerName);
	}
}

We have also introduced a new concept of This. [This] attributes decorates a field that will be injected at runtime with instance of the Composite of current fragment. Yes, I did mention fragment, meaning that the keyword can be used in Mixin, Concern, and SideEffect, although it’s generally a bad idea since it tighten the coupling to its composite that makes them unreusable.

To implement this into our Composheep, I will follow with the same path as that for Concern. Let’s start with adding new method to CompositeInterceptor ;

readonly Dictionary<String, IList<Object>> methodPostHandlerMap = 
new Dictionary<String, IList<Object>>();
		 
public void AddSideEffect(object sideEffect)
{
	MethodInfo[] methods = GetPublicMethods(sideEffect);

	foreach (MethodInfo method in methods)
	{
		string key = HashKeyOf(method);
				
		if (!methodTargetMap.ContainsKey(key))
			continue;

		IList<object> sideEffects;
		if (methodPostHandlerMap.ContainsKey(key))
			sideEffects = methodPostHandlerMap[key];
		else
		{
			sideEffects = new List<object>();
			methodPostHandlerMap[key] = sideEffects;
		}
				
		sideEffects.Add(sideEffect);
	}
}

We define a new field methodPostHandlerMap which contains method invocation target that is called AFTER the original method invocation finishes (hence the field name). One method can have more than 1 sideeffects, so in contrast to methodTargetMap, methodPostHandlerMap is defined as dictionary of lists. I am not sure yet at this stage if I can merge together methodTargetMap as part of this handler list. But we will come back to it later when we find they are conceptually the same. But for now, I am happy with it. Now let’s move on to the CompositeBuilder.

private void PopulateSideEffect<T>(CompositeInterceptor interceptor, T composite)
{
	object[] attributes =
typeof(T).GetCustomAttributes(
typeof(SideEffectAttribute), true);
	foreach (SideEffectAttribute SideEffect in attributes)
	{
		foreach (Type sideEffectType in SideEffect.Types)
		{
			object sideEffectInstance =
				Activator.CreateInstance(sideEffectType);
			interceptor.AddSideEffect(sideEffectInstance);
			ResolveReferenceToComposite(
				sideEffectInstance, composite);
		}
	}
}

This method, in many ways, are almost identical to PopulateConcern<T>() and PopulateMixin<T>(). The only thing we are interested here is a call to a new method ResolveReferenceToComposite(). This method is responsible to handle [This] functionality. Therefore, I have also added this in both PopulateMixin<T>() and PopulateMixin<T>(). The logic behind the method is plain straightforward:

private void ResolveReferenceToComposite(object instance, object composite)
{
	foreach (FieldInfo field in
		instance.GetType().GetFields(
			BindingFlags.Instance | 
			BindingFlags.Public | 
			BindingFlags.NonPublic))
	{
		if (field.IsDefined(typeof(ThisAttribute), true))
			field.SetValue(instance, composite);
	}
}

Run the code again, and here’s what we got:

Mission accomplished. All calls are intercepted by defined Concerns and SideEffects. And this marks the end of our second episode. And as usual, the source code for this episode is available at Composheep release in CodePlex.

Roll Your Own COP. Part I: Mixins

A new programming paradigm has been born in distant land of Java. Qi4j introduces the first Composite Oriented Programming (COP) concept to the world, and has drawn massive interest among Java crowd ever since.
It drove me to start a work on implementing Composite Orientation on .Net as a project named Composheep. This will be an open-source lightweight COP framework (as defined by Qi4j) that I will build incrementally as I write each step on this blog as “Roll Your Own COP” series. I publish the project in CodePlex as LGPL.
Let’s start from the mission of COP. One of the biggest flaws of OOP is that it’s not object oriented at all. If anything, OOP is Class-Oriented, since class is the primary citizen that all objects are derived from. Not the other way round.
As an example, I am a programmer at work, a driver in a car, a researcher in kitchen, and a hunter and a pray in the jungle. As an object, my role (class) constantly changes depending on contexts. Some objects also traverses different scope boundaries. For instance, a Person will have its classes changing over time. New abilities are learnt, from Kid, Student, Dancer, Ninja, and he will eventually due, but it doesn’t mean that the Person object should be deleted from the system since the “memory” of him may live for long time. In a conventional OOP system, we will need to transfer some of the states across instances of different classes. In Composite Oriented Programming, they are all ONE instance. We assign role (Class) to an object. Not instantiating objects based on a destined class. Object is therefore the primary citizen of COP. Please visit Qi4j site for more detail.
One of the most important COP concepts is mixin, in which multiple reusable classes are mixed to form a solid composite. It relies much on Java class generation technique, in which .net is always known far inferior. But, as the first part of the series, I will show how easy it actually is to build our own Mixins implementation on .net using Castle DynamicProxy, in only 15 minutes.
Our objective is to achieve a robust Mixins builder that is smart enough to:
– Handle both properties and methods
– Differentiate parameters overload. E.g. foo() and foo(string)
– Handle generic methods. E.g. foo<T>()
– Differentiate generics overload. E.g. foo<T>() and foo<T,Y>()
– Declarative programming using attributes
As a sample use-case, first I will define a composite of Person that is composed from 2 mixins: HasName and CanFight. I want to define the composite in the following fashion:

[Mixin(typeof(HasNameImpl), typeof(Fighter))]
public interface Person: HasName, CanFight
{
}

Note that Person, as a composite, is defined as an interface. We won’t write any implementation of Person since it will be automatically derived by composing 2 mixin implementations together. We declaratively specify that we wish to use HasNameImpl and Fighter as mixin implementor by using Mixin attribute.
Here is the definition of the mixins:

public interface HasName
{
	string FirstName{ get; set;}
	string LastName { get; set;}

	string IntroduceSelf();
	string IntroduceSelf(string target);
}

public interface CanFight
{
	string Kick<Target>();
	string Kick<LTarget, RTarget>();
}

And the following is the implementation of each mixins:

public class HasNameImpl: HasName
{
	public string FirstName {get; set;}
	public string LastName {get; set;}

	public string IntroduceSelf()
	{
		return Console.Writeln(
			"Hi there, I am {0} {1}",
			FirstName, LastName);
	}

	public string IntroduceSelf(string target)
	{
		return Console.Writeln(
			"Hi {0}, I am {1} {2}",
			target, FirstName, LastName);
	}
}

public class Fighter: CanFight
{
	public string Kick<Target>()
	{
		return Console.Writeln
			("Roundhouse kick to {0}",
		typeof (Target).Name);
	}

	public string Kick<LTarget, RTarget>()
	{
		return Console.Writeln
			("Left foot to {0}, and right foot to {1}",
		typeof (LTarget).Name,
		typeof (RTarget).Name);
	}
}

The following is how I want the client code to be:

CompositeBuilder builder = new CompositeBuilder();
Person person = builder.BuildComposite<Person>();

person.FirstName = "Hendry";
person.LastName = "Luk";
person.IntroduceSelf());
person.IntroduceSelf("Goofy"));
person.Kick<Dog>());
person.Kick<Dog, DebtCollector>());

We instantiate an object of Person, whose implementation is generated dynamically by mixing HasNameImpl and Fighter together to form a complete Person. This way, we will be able to separate a class into smaller chunks of fragment (mixin) that can be reused effectively.
I think this use simple use-case provides a good coverage to all our 5 requirements. It exploits the use of properties, methods with overloaded parameters, and methods with overloaded generics.
Now, to make this work, we will need to write the CompositeBuilder as part of our Composheep solution. We will be using Castle DynamicProxy, which is a lightweight proxy generator used in many open-source frameworks like nHibernate, Windsor, Aspect#, etc.
Here is the implementation of CompositeBuilder. It searches for MixinAttributes in supplied type, then grab the mixin implementer types that was provided as attribute parameter.

public class CompositeBuilder
{
	private ProxyGenerator generator = new ProxyGenerator();

	public T BuildComposite<T>() where T : class
	{
		CompositeInterceptor interceptor = new CompositeInterceptor();

		object[] attributes =
		typeof(T).GetCustomAttributes(typeof(MixinAttribute), true);
		foreach (MixinAttribute mixin in attributes)
		{
			foreach (Type mixinType in mixin.Types)
				interceptor.AddMixin(Activator.CreateInstance(mixinType));
		}

		return generator.CreateInterfaceProxyWithoutTarget(interceptor);
	}
}

ProxyGenerator is an API from Castle DynamicProxy that we use to dynamically define implementation of composite interface. The most important part of this only method is that in each mixin type defined in mixin attribute, we use default constructor to instantiate the mixin, and pass it to CompositeInterceptor. It’s a custom proxy interceptor that we will create to handle each composite method invocation to its corresponding mixin implementation.
Here is the code for CompositeInterceptor.

internal class CompositeInterceptor : IInterceptor
{
	Dictionary methodTargetMap =
		new Dictionary();

	public void AddMixin(object mixin)
	{
		Type targetType = mixin.GetType();

		MethodInfo[] methods =
		targetType.GetMethods(
		BindingFlags.Instance |
		BindingFlags.Public |
		BindingFlags.NonPublic);

		foreach(MethodInfo method in methods)
		{
			// Skip members declared in System.Object
			if (method.DeclaringType == typeof(object))
				continue;

			methodTargetMap.Add(method.ToString(), mixin);
		}
	}

	// Implementing IInterceptor.Intercept method
	public void Intercept(IInvocation invocation)
	{
		object target = FindMixin(invocation.Method);
		if (target == null)
			throw (new NotImplementedException());

		invocation.ReturnValue =
		invocation.Method.Invoke(
		target, invocation.Arguments);
	}

	private object FindMixin(MethodInfo callMethod)
	{
		if (callMethod.IsGenericMethod)
			callMethod = callMethod.GetGenericMethodDefinition();

		foreach (String method in methodTargetMap.Keys)
		{
			if (method == callMethod.ToString())

				return methodTargetMap[method];
		}
		return null;
	}
}

The idea is that AddMixin method will map each method signature with a mixin instance in a Dictionary. Therefore, when we intercept a proxy method invocation, we will be able to lookup the dictionary for the method signature, and get the mixin instance. Finally, the invocation will be redirected to that mixin instance.
The easiest way to lookup matching method signature (in FindMixin method) is by using MethodInfo.ToString() since it gives us the method name, parameters types, return type, and generic parameters. So we will be using this as the key of the Dictionary as well.
The only problem with generic parameter is that we will be storing open-generic method signature, for example, string Kick() in AddMethod method during interface introspection. But during invocation, we will get passed with a closed-generic method, for instance, string Kick(). To get around this, we put 2 lines on top of FindMixin:

if (callMethod.IsGenericMethod)
callMethod = callMethod.GetGenericMethodDefinition();

It converts void Kick() back into void Kick(). And this is all we need to build our mixin builder! Run the application, and this is what we got:

Just few minutes of pretty straightforward code and we’ve got the building block for our Composheep in place. You can download the code for this episode here. Coming next, in the second episode, we will be building the second features of COP: concerns and side-effects.