SAQL and SheepAOP Remake

SAQL?

Say hello to SheepAOP-Query-Language (hereafter referred to as ‘SAQL’), a new addition to SheepAOP, serving as the main language to query against your source-code for your SheepAOP pointcut definitions.

I know I said external-DSL would NOT be in my priority list, but the latest SheepAOP enhancement I was working on (outlined later) has made it strongly imperative that an external DSL needed to be brought onto the game, right now.

Initially, I took the path which follows the current trend in .Net libraries designs that have been increasingly moving away from external DSL in favor of lambda Fluent APIs. To name a few: NHibernate’s HQL taken over by NH2Linq, HBM by Fluent-NHibernate, Binsor by fluent-configuration, and I myself have also built lambda API in lieu of RhinoETL’s boo scripts. Lambda seems to be the new black. Therefore, at the beginning of SheepAOP project, it was very clear to me that Fluent lambda was the way to go about this whole AOP thing. With lambda API, I would get the intellisense and refactoring support from Visual-Studio, and I actively avoided the tedious effort of building an external DSL from scratch.

Over the weeks, I have been noticing few minor issues with lambda API that have been irritating me, albeit not to a point that bothers me enough to do anything about it.

  1. Safety. SheepAOP weaving is done during compile-time whenever you build your project. I have not been particularly happy about the fact that SheepAOP compiler needed to execute your actual application (i.e. your configuration routines) during compilation, as opposed to reading only its metadata. If you’re not careful, you might accidentally (or deliberately) execute certain unintended code-blocks (from within your config routine) during compile time, which might be harmful to anyone merely opening the project in their Visual-Studios. It makes it way too easy to abuse.
  2. File lock. Since your application are executed (by SheepAOP) during VisualStudio compilation, they would get loaded into your VS process, and will never get release (There’s no such thing in the .Net framework as “unloading” an assembly). It also means it will lock your output file, something you don’t want on an IDE. Spawning separate AppDomain normally solves this, but that doesn’t do the trick on VS custom-tasks for whatever reason. My current solution is to spawn a separate (console) process. Either way, lambda configuration makes communications difficult since they are not transmittable between AppDomains/processes. A plain DSL text-script, on the other hand, is.  Hence giving me a lot more flexibility in my approaches.
  3. Verboseness. Lambda API introduces too much noise for the signal. We will see this in comparison shortly.

Those are 3 minor drawbacks that I could happily live with, rather than spending my effort on an external DSL that could be spent elsewhere. Plus I was reluctant to give up IDE intellisense with fluent API.
But as I worked on the next enhancements (covered below), it quickly became obvious that fluent API no longer finds its place in the system, and I came to a tough decision to ditch the whole thing, and started working on a brand new querying language.

SheepAOP Enhancements

After exploring several common patterns and use-cases frequently seen on other more mature AOP tools like AspectJ, and then reviewed the current (previous) implementation of SheepAOP, I determined that the old SheepAOP design wasn’t up for the job. A total rework had to be done. The 3 major features that inspired the rework, which have now been delivered on the new version of SheepAOP, are:

  1. Aspect Lifecycle. This is a very important feature that single-handedly makes AOP useful in far broader sets of real-world problems. So far I have only seen this in AspectJ. I will post more elaborately on this one particular topic, but in essence, it allows you to determine the lifespan of your aspects, so you can maintain contextual states.
    For example, imagine you just defined a new client-facing method (say: MethodA) for your POS system, exposing a transactionTime as a parameter that the client (human or another app) can specify, so that all sales-order objects constructed at any point within the scope of that method will be initialized with this transaction-date. Normally this means that you will have to pass this date from MethodA all the way through all methods beneath its flow, to be used in every sales-order instantiation it reaches.
    With PerFlow lifecycle (new!), you can define that your aspect’s life will start when the MethodA starts. It will then begin intercepting all sales-order instantiations within the flow, and assign them with the transaction-date (captured from before). When MethodA completes, this aspect will also be disposed out of the way, which also ends its intercepting business. With a pseudo-code, your aspect looks like:

    [Lifecycle (PerFlow of "MethodA_called")]
    aspect TransactionDateAspect:
    {
       // Normal instance field, stays here within the life of this aspect object
       private Date _transactionDate;
    
       pointcut MethodA_called;  // The "life trigger" for this aspect, as declared on the top
       pointcut SalesOrder_instantiation;
    
       [Before(MethodA_called)]
       void OnMethodA_called(Date transactionDate)
       {
           // Aspect initiates here. Let's keep the transaction-date within the context
           this._transactionDate = transactionDate;
       }
    
       [After(SalesOrder_instantiation)]
       void  SalesOrder_instantiation (SalesOrder order)
       {
          // This advice is only active within the life of this aspect
          order.TransactionDate = this._transactionDate;
       }
    }
    

    This ability enables you to compose an aspect-oriented workflow. There are many other lifecycles, for example: PerTarget aspects allow you to cache method-calls to the same object instance. PerThis allows you to keep all dirty states within that instance. Singleton and Transient are the other options.
    The reason this could not be done on the previous SheepAOP design was because all advice-methods had to be static since they merely acted as code templates to be copied to the injected targets (which also causes the problem in point#3).

  2. Inheritance, Polymorphism, and Abstract Aspects. Now that we have aspects and pointcuts as normal living instances (as opposed to a bunch of static instruction templates), you can use inheritance and polymorphism around your aspects for reusability and extensibility. For instance, you can define the following abstract aspects (this one is an actual SheepAOP code):
    public abstract class LoggingAspectBase
    {
       protected abstract string Title();
    
       protected abstract void LoggablePointcut(); // Override your pointcut here!
    
       [Around("LoggablePointcut")]
       public object LogExecution(JoinPoint jp)
       {
          try
          {
             _logger.Log("Entering {0} with [{1}]" + Title(), jp.Args);
             var result = jp.Proceed();
             _logger.Log("Returning {1} from {0} " + Title(), result);
             return result;
          }
          catch(Exception e)
          {
             _logger.Log("Error on {0}: {1}", Title(), e);
          }
       }
    }
    

    And you can reuse this base-aspect class by simply implementing it.

    public class OrderSubmissionLogging: LoggingAspectBase
    {
       protected override string Title()
       { return "Order Submission"; }
    
       // Overriding your pointcut
       [SelectMethods("Name:'SubmitOrder' & InType:(Namespace:'Sheep.Sales.*')")]
       protected override void LoggablePointcut(){}
    }
    
  3. No method-copier, means better debugger. As explained in the previous posts, the old SheepAOP used your advice-methods as a code-template, in a literal sense, copying the body line by line to surround the target code-blocks. (Think T4). Because your advice-methods were only templates and never got executed, it naturally means your debugger would never arrive there. I suggested event-based advices to “solve” that, which I have to admit, a bit clunky, unpredictable, and error-prone.
    The new SheepAOP will no longer employ this “code-template” technique. It now makes use of .net delegates. SheepAOP now pulls your code-blocks into statically-generated methods, and keeps the delegates of them, so your application will literally jumps back and forth between these delegates. Your debugger will be able to cope with that. It also means that you can even make your advice call your target methods several times. E.g.:

    public int Advice(JoinPoint jp)
    {
       var returnValue= jp.Proceed();
       returnValue+= jp.Proceed();
       returnValue+= jp.Proceed();
       return returnValue;
    }
    

    That advice would execute your target methods 3 times, and yup, this time is for real: your advice method actually gets executed during runtime. When you call jp.Proceed(), beneath the skin it will simply make a delegate call that points back to your hidden underlying target code-block.

These changes demanded a major change. I had to rewrite the project pretty much from scratch. How does this affect Fluent API? Let’s see.

The shift from static advices and pointcuts into active instances (especially to support inheritance at point#2) means that SheepAOP will now have to instantiate your actual aspect classes in order to execute its configuration routines (and their inheritances). The same aspect classes that are also used during the runtime.
I don’t like this. Your classes typically have certain instantiations routine required during the runtime (e.g. application-settings, dependency-injects, even sometimes database-connections). The fact that now I have to execute the same runtime routines as part of my compilation process really bothers me. And once you get the actual runtime instance of your object, it really becomes inevitable to make “accidental” abuse within your AOP configuration code. It was aggravated by the ever swelling number of generics tags and lambda arrows noise thanks to the fast growing need for a more expressive configuration system, I eventually determined that this is a good time for the birth of SAQL.

SAQL!

Ok, this will be brief. I’ll just give a comparison of the same pointcut between the old syntax and the new one. I will deliberately use a dreadfully complex example to magnify the difference.

Old Fluent API:

[RegisterPointcut]
public static void MyPointcut(PointcutRegistry reg)
{
   reg.Advise(r=> r.Method(m=>
      m=> m.Name("Save").And(m1=>
            m1.IsStatic(true)
            .DeclaringType(t=> t.Namespace("Sheep.Data.*"))
            .Arg(t=> t.Any())
            .Arg(t=> t.AssignableFrom(i=> i.FullName("NHibernate.ISession")))
         .Or (m2=>
            m2.IsStatic(false)
            .DeclaringType(t=> t.Implements(i=> i.FullName("Sheep.IRepository`1")))
           ))
      ), MyAdvice);
}

SAQL:

[SelectMethods(
    @"Name:'Save' &(
       IsStatic & InType:Namespace:'Sheep.Data.*' & Args:( , (AssignableFrom:'NHibernate.ISession'))
       || !IsStatic & InType:Implements:'Sheep.IRepository`1'
      )"
)]
public void MyPointcut() {}

The 2 codes above are querying exactly for the same pointcuts, resulting in exactly the same joinpoints. But I believe the second one using SAQL is much easier on eyes. The noise is kept minimum in the syntax. That SAQL query will then be lexed, parsed, and walked using ANTLR3 into a workable AST, that’s in turn translated into SheepAop Pointcut objects, ready to crunch your assemblies (fed by mono-cecil) to find the matching code-blocks within your assemblies.

As for the grammar, I actively avoided using words as part of the syntax (like in SQL and HQL) , because even though it makes it easier to read, it makes it harder to write. You might sometimes find yourself wondering whether you have to use ‘where‘, or ‘is‘, or ‘not in‘, or ‘is not in‘ in specific instances. I believe a consistent use of small number of punctuations makes for a more intuitive experience. But I’m open to feedback regarding the language grammar.

Where’s The Stuff?

No surprise here, everything is in the Codeplex repository (http://sheepaop.codeplex.com/). I haven’t produced a release for this yet since this is still very unstable, especially in the error-detection department (EDIT: and Generics situations, which is unexpextedly very tricky).

I will post next articles to explore the various features I’ve only briefly alluded above:

  • SAQL basics
  • Abstract aspects and polymorphism
  • Aspects lifecycle (real world examples of “contextual aspects”)
  • Aspect factory (connecting your beloved IoC to SheepAOP)
  • Debugging tips
Advertisements

4 thoughts on “SAQL and SheepAOP Remake

  1. Yes it’s quite a migraine-inducing post, mainly because I (scruffily) cover much of the internal details, so as to give enough information in its earliest stage for those interested in participating in the source-code.

    But I’m planning to cover more user-oriented topics and practical “in-action” examples in upcoming posts, that are bound to be safer for everybody’s mental health and general well-being.

    Podcast, yea always love to 🙂

  2. Hey,

    Just when I was trying to get the hang of Event Listener’s you introduced this, it looks solid and hope its stable soon so we can try it.

    Keep it up

  3. Zubair, thanks for checking it out.
    Yap, event listeners madness is completely gone now ;). There’s a similar thing in the new version called Interceptor, which is much simpler.
    In general, the new SheepAOP strives to stay faithful to the AspectJ design (which is a good thing, by the way).
    The code is “relatively” stable as per today, and the surface API won’t change much anymore, so feel free to give it a go. I’m planning a preview-release hopefully this weekend, weather permitting 🙂
    Some introductory articles will follow.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s