Friday, September 17, 2010

Heartysoft.com | Anonymous Types are Internal, C# 4.0 Dynamic Beware!

Just found this interesting article:
Heartysoft.com | Anonymous Types are Internal, C# 4.0 Dynamic Beware!

That explained why my spark templates where failing at runtime because the default dynamic object only works for public and non-internal members.
I implemented the following class in order to use dynamic and non-dynamic models in our spark templates that are used in standalone mode for emailing templating:

public class DynamicViewData : DynamicObject
{
private readonly object _model;

public DynamicViewData(object model)
{
_model = model;
}
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
var member = _model.GetType().GetMember(binder.Name,
BindingFlags.Instance |
BindingFlags.Public |
BindingFlags.NonPublic).FirstOrDefault();
if (member != null)
{
switch (member.MemberType)
{
case MemberTypes.Field:
result = ((FieldInfo)member).GetValue(_model);
return true;
case MemberTypes.Property:
result = ((PropertyInfo)member).GetValue(_model, null);
return true;
}
}
return base.TryGetMember(binder, out result);
}
}

And a custom AbstractSparkView which has a dynamic property where I'm passing an instance of this class together with the anonymous model.

Now I don't have to define in my spark templates the model type, and also I don't have to define in my code every single viewmodel.


Wednesday, September 15, 2010

Logging and Metrics using Gibraltar – A Windows Form Sample App

Introduction

Hello dear reader, have been such a long time. As usual the amount of work at the office I’ve to do on a daily basis, side projects, the occasional colleague asking for help, and last but not least, family life, made almost impossible to write any post on this blog lately, luckily today I’ve found a bit of spare time to do so and I hope you like this blog post.

So, what’s Gibraltar? To my understanding, Gibraltar is an step forward to create components that are categorized as those that work as cross cutting concerns in an application, in plain English that would mean that those components are part of the application but do not belong to the business domain of it.
A simple example could be Logging, or User Auditing in a CRM application, because the reason of existence of a CRM application is not to perform any of those tasks, but to handle your contacts, calendars, events and so on.
This is one of the keys concepts when doing AOP programming.

what are the MAIN components of gibraltar

Basically Gibraltar is based on two components:

  1. Gibraltar Agent
    This is a .Net library that provides the API to perform most of the job needed. It’s the principal component, without this you wouldn’t be able to collect the precious information from your application.
  2. Gibraltar Analyst
    This is a user friendly standalone application used to analyze the data sent by your application. Provides several built-in tools to display/filter/sort/group the data generated by the Gibraltar Agent.

There’s a third optional component, the Gibraltar Hub, which is a service to host in a centralized place all the data generated by your applications.

For an in-deep look of what those three components are all about, I recommend you to check the following resources:

Getting Started Guide
Gibraltar Architecture
Gibraltar Hub FAQ
Gibraltar Try It FAQ

The sample app

This is a fictitious application that allows you to manage your contacts from several sources like GMail, Facebook and so on.

image

As you can see, we’ve a simple IFeature interface that is used by concrete classes to be recognized in the system as a feature of it. This interface has a Execute method which is used by the UI to make the feature to run.
I’ve created a base class (FeatureBase) to simplify the implementation of some of the interface members.
Below I’ll show you the code of this class and of the DisplayContactsFromGMail class, the other two classes are pretty identical and do not make much sense to repeat myself over and over.

    public abstract class FeatureBase : IFeature
{
public abstract Guid Id { get; }

public string Name
{
get { return GetType().Name; }
}

public abstract string Description { get; }

public virtual void Execute()
{
MessageBox.Show(@"Executing Feature " + Name);
}
}



    public class DisplayContactsFromGMail : FeatureBase
{
public override Guid Id
{
get { return Guid.Parse("614DFAA4-70F7-4F5F-82D4-9598C0F39F60"); }
}

public override string Description
{
get { return "Display Contacts from your GMail Account"; }
}
}




As you can see, very simple classes according with the scope of this post.



The second part of the application is the component in charge of discovering and providing the features. It could be implement in many ways, for instance by using a DI tool like StructureMap or Autofac. I choose a simple reflection approach, that is, look at the IFeature assembly for types implementing this interface.



image



The FeatureCatalog is implemented as follows:



    public class FeatureCatalog : IFeatureCatalog
{
private readonly IEnumerable<IFeature> _features;

public FeatureCatalog()
{
_features =
typeof(FeatureCatalog)
.Assembly
.GetTypes()
.Where(x => x.IsPublic)
.Where(x => x.IsClass)
.Where(x => !x.IsAbstract)
.Where(x =>
typeof(IFeature).IsAssignableFrom(x))
.Select(
Activator.CreateInstance)
.Cast<
IFeature>()
.ToList();
}

public IEnumerable<IFeature> Features
{
get { return _features; }
}
}

Did I said I really love LINQ? :)


Ok, now we’ve everything in place let’s create a simple UI to use the implemented features; if you think this the worst UI you’ve seen in you whole life, please bear with me, UI design is not one of my strong skills :)


image 


We’ll load the list of features in the listbox control, and the button at the right side will just execute the selected feature.

Let’s take a look at the code used to load the features into the listbox and the code of the button click event.



        private void Form1_Load(object sender, EventArgs e)
{
FeaturesList.DataSource =
new FeatureCatalog().Features;
}

private void ExecuteFeatureButton_Click(object sender, EventArgs e)
{
((
IFeature) FeaturesList.SelectedItem).Execute();
}


Pretty simple uh? Lets see how the app behaves when executing the DisplayContactsFromGMail feature.



image



Selecting other features and clicking the button at the right side produces the same effect (display a messagebox with a generic description).

Introducing Gibraltar

Now you may be wondering, “WTF? I’ve not seen any Gibraltar usage anywhere!!”.

I know, I know…please be patience.

Well, let’s say your boss tells you that the users are complaining about the large list of items being displayed on the listbox (menu, tabs or any other method you real app would use to show the available features). Your boss ask you to determine which are the most used features from the application and display the top X in your original control (the listbox) and the rest somewhere else.

So, the first thing you’ve to do is to update the app to track the features usage and determine which are the most used in a given period of time and deploy this new version of the app to your clients, after a few days of monitoring you’ll know what to do and what features put apart.

We could use several different options here, like Dynamic Proxy, Postsharp for IL Weaving, and a few more. But what we can’t do is obviously start manually changing one class at a time to do this, that would be a daunting,tedious and error prone task.

The question is…where should we inject this? In the button click handler…for the scope of this article would probably do the trick, but I suppose in a real project a feature X could call to another feature from the catalog.

So if the catalog is the entry point to retrieve features, then the catalog is the place where this new code should be implemented.

What I’m going to do is to create an IFeature implementation that will act as a wrapper of the real implementations. Each IFeature member implemented in this wrapper will simply forward the call to the real feature undercover, excepting the Execute method where I’m going to record the the feature usage and after that the wrapper will continue with the execution flow.

The usage recording will be performed in a helper class where the Gibraltar related code will be dropped.

Gibraltar besides the normal logging of messages it has something called Metrics, that as its name implies, is specially designed to this kind of scenarios where we need to measure certain aspect of our applications, for this scenario, we will sum the number of times a feature is used, another could be determine what’s the average number of emails our users sent per day and so on.

Is time to modify our application to include the required Gibraltar configuration settings and references…but don’t worry, this step is the easiest one (and the most fun of all of them).

First of all, you’ll need to download and install the Gibraltar bits from here.

Once you have everything in place, go and execute the Gibraltar Analyst program. Bellow are the steps required to enable Gibraltar in your app:


  1. Tools / Add Gibraltar to an Application.

    image





  2. Click on the “…” button and navigate and select your project file and click the “Open” button in the window dialog.

    image




  3. A set of options will get enabled on this screen, choose “Add the Gibraltar Agent to this Application”.


  4. The next step will ask you how do you want that the Agent categorize your Application, just pick up the option “Automatically Determine Options”.


  5. In the step “How should session data be reported to you?” we will choose to use the Gibraltar Hub just for fun, you can leave the other options unchecked.

    image


  6. Now in this step it will ask you for an account name or the settings of your private hub service. We will choose to provide an account name for our hub service. You can get a 30 days renewable trial of the hub service from the Gibraltar web page here. Just check the options you see below, follow the instructions and in a matter of minutes you’ll have a hub and the client side tooling required to get your app up and running.


    image


    Once you’ve registered your hub service trial, introduce the name of your service in the input box and you’re almost ready to go.


    image




  7. This step is about configuring how do you like to sent your sessions data, I just leave the suggested options checked.

    image




  8. And that’s all.

    image



Once you perform the steps I outlined above, your project will reload itself with the references and all you need to start integrating your application with Gibraltar.




image


Now that you’ve configured your project to work together with the Gibraltar Agent. It’s time to configure your Gibraltar Analyst to fetch the generated data from the Gibraltar Hub.


  1. Tools/Connect to Hub… image




  2. image





  3. image



  4. image

And that’s all folks, you can start fetching data from your hub and view it locally in the Analyst client.


Let’s continue with the implementation of our wrapper.

using System;
using ContactsManager.Metrics;
using Gibraltar.Agent;

namespace ContactsManager.Features
{
public class FeatureWrapper : IFeature
{
private readonly IFeature _feature;

public FeatureWrapper(IFeature feature)
{
_feature = feature;
}

public string Name
{
get { return _feature.Name; }
}

public Guid Id
{
get { return _feature.Id; }
}

public string Description
{
get { return _feature.Description; }
}

public void Execute()
{
FeatureUsageRecorder.Record(_feature, 1);
Log.Information("Contacts Manager", "Feature Usage", "Feature {0} executed", _feature.Name);
try
{
_feature.Execute();
}
catch (Exception ex)
{
Log.Error(ex, "Contacts Manager", "{0} Error {1}", _feature.Name,ex.ToString());
throw;
}
}
}
}

As you can see, it forwards the calls to the real feature (_feature member), and when it gets the time to call to Execute, it uses the FeatureUsageRecorder static class and records this action, logs entrance to the method, it continues by calling the Execute method of the underlying feature, and it case of failure logs the exception.

Now we need to modify the catalog to wrap the features using this new class.


    public class FeatureCatalog : IFeatureCatalog
{
private readonly IEnumerable<IFeature> _features;

public FeatureCatalog()
{
_features =
typeof(FeatureCatalog)
.Assembly
.GetTypes()
.Where(x => x.IsPublic)
.Where(x => x.IsClass)
.Where(x => !x.IsAbstract)
.Where(x =>
typeof(IFeature).IsAssignableFrom(x))
.Where(x => !
typeof(FeatureWrapper).IsAssignableFrom(x))
.Select(
Activator.CreateInstance)
.Cast<
IFeature>()
.Select(x =>
new FeatureWrapper(x))
.ToList();
}

public IEnumerable<IFeature> Features
{
get { return _features; }
}
}



As you can see there, I’m excluding from the assembly scanning to the FeatureWrapper class, and once the IFeature implementations had been instantiated, I’m wrapping them all into the FeatureWrapper class just before calling to ToList in the IEnumerable. This ensure us that the application will still work as usual and that the features are not aware of the fact that we are generating metrics when our clients make use of them.

And what everyone was waiting, the code for the FeatureUsageRecorder class (including imported namespaces):

using ContactsManager.Features;
using Gibraltar.Agent.Metrics;

namespace ContactsManager.Metrics
{
public static class FeatureUsageRecorder
{
private const string MetricKey = "Feature name";
private const string MetricDataObjectType = "ContactsManager";
private const string CategoryName = "Features";
private const string MetricDescription = "Number of times this feature has been used.";
private const string UnitCaption = "Number of Calls";
private const string Caption = "Usage";
private const string CounterName = "Contacts Manager Feature Usage";
private static readonly EventMetricDefinition _featureUsageMetric;

static FeatureUsageRecorder ()
{
if (!EventMetricDefinition.TryGetValue(MetricDataObjectType, CategoryName, CounterName, out _featureUsageMetric))
{
_featureUsageMetric =
new EventMetricDefinition(MetricDataObjectType, CategoryName, CounterName);
_featureUsageMetric.AddValue(MetricKey,
typeof(int), SummaryFunction.Sum, UnitCaption, Caption,
MetricDescription);
EventMetricDefinition.Register(ref _featureUsageMetric);
}
}

public static void Record(IFeature feature, int count)
{
var metricEntry = EventMetric.Register(_featureUsageMetric, feature.Name);
var metricSample = metricEntry.CreateSample();
metricSample.SetValue(MetricKey, count);
metricSample.Write();
}
}
}



As you can see from the code above, Gibraltar lets you write custom Metrics and defining the fields that this metric will contain.

We are defining a “Contacts Manager Feature Usage” metric under certain category (“Features”), this metric belongs to the “ContactsManager” metric data object type. All of this are arbitrary names, you could choose whatever fits your needs.

Now this metric would be pointless if does not measure anything, so we register a unit that will be keyed with the value of the const MetricKey (“Feature name”), will have a int type, the summary function is defined as a sum of their column, and so on.

In the Record method, we take in a IFeature instance and just store the number of times this feature is used, currently hardcoded to 1, I thought it could be a good idea to call this method each X times a certain feature is used instead of calling it as soon as the feature is used as it is right now.

Now lets execute the application and start clicking the button many times for each feature.
Now close the application and check the Gibraltar Analyst application.
What you will see is something like this:

image


Clicking on the first row will open the following screen:

image



That’s an extensible and user friendly view of the log entries generated by our application (remember those Log.Information calls from the FeatureWrapper class??). Also this screen provides with filters in order to remove all the collected data that you don’t want to see.

What about our metrics, after all, that was the whole purpose of using Gibraltar. In this screen click onto the “New View” option at the top. You’ll see a pop-up like this one:


image


Please choose the “New Chart” option, later you can play with the other options.


That will open the metric reporting designer. From the threeview in the left side, expand the content under Metrics/Feature, drag and drop the item named “Contact Manager Feature Usage”:



image


After performing that command, you’ll get a dialog to define your report, choose the options I show you in the next screenshot:

image



Click “Ok” and you’ll get the following report:

Feature Usage Report



Final thoughts

As you can see, it was very easy to introduce Gibraltar into our App and get the benefits from a robust product that has almost all what you may need to trace and get rich information from your application without much effort.

For the record, Gibraltar comes with a set of built-in aspects (Postsharp attributes) that somehow, make even easier to work with it (GTrace, GTraceField, and so on). You can read about them here.

I didn’t use it those aspects because they require Postsharp in place for the IL weaving and that would complicated even more the infrastructure code required to run the sample app, and also my experience tells me that Postsharp will work only if you are in control of the application (that means, the source code)…the approach I used here would work in a environment where the features are loaded using a plugin architecture.

You can download the sample app from here.

See you soon.

Thursday, July 08, 2010

Dealing with legacy code when using TDD

 

Introduction

You may have heard about TDD lately and how great it is and so on, but most the time we work on projects that have a large code base and we are not allowed to make drastic changes.

Tools

Scenarios

In my experience some of the scenarios we face when refactoring legacy code are:

  • We are free to apply the changes we want, the world is perfect. :)
  • We can apply the changes we want but only with unit testing purposes, production code should work as usual.
  • We are not allowed to change any part of the code except infrastructure but remembering that things must work as always did, no breaking changes allowed.

Let’s elaborated on each one of those scenarios and how do we perform our changes to make legacy code more testable.

Concrete Example

    public static class DataBaseHelper
{
public static void Persist(object instance)
{
Console.WriteLine("Persisting object " + instance);
}
}


    public class ProductService
{
public void DisableProduct(Product product)
{
product.Enabled =
false;
DataBaseHelper.Persist(product);
}
}


As you can see, the Persist method from the DataBaseHelper does nothing, but we will assume we’ve some hard dependency with a database, perhaps an sqlconnection here an sqlcommand there and so on.



Now, what’s so bad with this piece of code? Most of us have seen,done and maintained  stuffs like that for years and if you’re fine with this, go ahead, I’m not here to tell you what’s wrong and what’s perfect, there’re no perfect solutions. However I would like to show you another way, a better way in my opinion.



I asked what was bad with that piece of code and I’ve not answered that yet…Let’s see:




  • Can you be really sure that in the following days/weeks/months/years that code will work as expected?


  • If in the future you found a bug in the system, how would you discard the problem cause is not related to this method? And when checking this fact, do you need to have your debugger/visual studio instance up and running?



I suppose that the answers didn’t were so pleasant at all, but don’t worry, the idea of this article is to help you to find a sane path when developing software by using TDD.



What should our tests cover



Well, there are a couple of things we should ensure in our tests:




  1. The product passed to the service method enters with its Enabled property with a True value, and must exits with a value of False.


  2. The service should persist the updated product instance.



Now lets back to the Scenarios section and supply a workaround for every one of those points.



We are free to apply the changes we want, the world is perfect. :)



This allows us to use a fantastic set of patterns, I’m talking about Inversion Of Control and Dependency Injection.



By using that, we can make the following changes:



 



    public class ProductService
{
private readonly IProductRepository _productRepository;

public ProductService(IProductRepository productRepository)
{
_productRepository = productRepository;
}

public void DisableProduct(Product product)
{
product.Enabled =
false;
_productRepository.Persist(product);
}
}




    public interface IProductRepository
{
void Persist(Product product);
}



    public class ProductRepository : IProductRepository
{
public void Persist(Product product)
{
DataBaseHelper.Persist(product);
}
}




    public static class DataBaseHelper
{
public static void Persist(object instance)
{
Console.WriteLine("Persisting object " + instance);
}
}


And introduce the following tests:


    public class Scenario1Tests
{
#region Mocks
private class NoOpProductRepository : IProductRepository
{
public void Persist(Product product)
{
// no-op
}
}

private class VerifiableProductRepository : IProductRepository
{
public event Action<Product> ProductPersisted;
public void Persist(Product product)
{
var productPersistedEvent = ProductPersisted;
if (productPersistedEvent != null)
productPersistedEvent(product);
}

}

#endregion


[Test]
public void when_disabling_a_product_its_enabled_property_is_set_to_false()
{
// Arrange
var product = new Product
{
ProductId = 1,
ProductName =
"Car",
Enabled =
true
};
// here we should use a mocking framework like Moq or Rhino.Mocks.
IProductRepository mockedRepository = new NoOpProductRepository();

var service = new ProductService(mockedRepository);

// Act
service.DisableProduct(product);

// Assert
Assert.AreEqual(false, product.Enabled);
}

[
Test]
public void when_disabling_a_product_it_persist_the_changes_using_the_product_repository()
{
// Arrange
var product = new Product
{
ProductId = 1,
ProductName =
"Car",
Enabled =
true
};
Product productPassedToPersistMethod = null;
var valueOfProductEnabledPropertyPassedToPersistMethod = false;
// here we should use a mocking framework like Moq or Rhino.Mocks.
var mockedRepository = new VerifiableProductRepository();
mockedRepository.ProductPersisted +=
instance =>
{
productPassedToPersistMethod = instance;
valueOfProductEnabledPropertyPassedToPersistMethod = instance.Enabled;
};

var service = new ProductService(mockedRepository);

// Act
service.DisableProduct(product);

// Assert
Assert.IsNotNull(productPassedToPersistMethod);
Assert.AreSame(product, productPassedToPersistMethod);
Assert.AreEqual(false, valueOfProductEnabledPropertyPassedToPersistMethod);
}


}


 



As you can see, those are quite drastic changes, in order to use again our ProductService now we have to supply an IProductRepository implementation because its constructor demands one. The usage of the static helper class to access to the database it has been moved our concrete implementation of the ProductRepository which is likely to be used at runtime. This is where the usage of a IoC/DI tool like StructureMap really shines because it will help us to configure such concerns (i.e. connect IProductRepository with ProductRepository) and make it transparent its usage.



Now what do we gain by doing this? Well, now we can test the ProductService.DisableProduct method at our pleasure by mocking its dependencies and ensuring everything works as expected. If someone in the future try to mess with the implementation of such method or if we’ve to introduce changes into our system, we’ll have those two unit tests to cover our backs and they’ll let us sleep tight at night. I think you can figure out what the tests does by just looking at the code, it is fairly simple and it does not take more than 10 minutes to create them.





 



We can apply the changes we want but only with unit testing purposes, production code should work as usual.



Here we can use something called Poor Man’s Dependency Injection, you can read about it here, and let me tell you something, I agree with the thoughts Jimmy Bogard give us in that article. And I really think dear developer we should strive all the time for option number one, push harder to let management change the infrastructure to use proper architecture and a DI/IoC tool. However, the world is not perfect and management or our work environment isn’t always too kind to accept changes.



So what we do?, we take the code from the first approach and modify the ProductService class and introduce a parameterless constructor which calls to the constructor that takes in an IProductRepository instance and pass to it an instance of the ProductRepository class, thus, removing the need of a DI/IoC tool to build the instances in our regards.



Here is the code to illustrate what I was mumbling in the previous paragraph:



    public class ProductService
{
private readonly IProductRepository _productRepository;

public ProductService()
:
this(new ProductRepository())
{

}

public ProductService(IProductRepository productRepository)
{
_productRepository = productRepository;
}

public void DisableProduct(Product product)
{
product.Enabled =
false;
_productRepository.Persist(product);
}
}


The tests and all the other stuff we added remain the same, we still have a testable ProductService class.



 



We are not allowed to change any part of the code except infrastructure but remembering that things must work as always did, no breaking changes allowed.



This means, no interfaces, no repositories, nothing, thus, no new concepts are allowed to be introduced into the principal code base, you should keep your changes as low as possible.



Well, I must reckon that does not let us with much choices to pick from…but even in such inflexible environment we can do better with simple approaches.



What we will do is to take our helper and make it flexible enough to allow us check the ProductService will work as expected in the future. Certainly this technique isn’t mine, at the moment I can find the resource where I first time read about it, so, if someone have a link, be my guest.



    public static class DataBaseHelper
{
private static Action<object> _persistenceDelegate;

static DataBaseHelper()
{
_persistenceDelegate = instance =>
Console.WriteLine("Persisting object " + instance);
}

public static void SetPersistenceDelegate(Action<object> persistenceDelegate)
{
_persistenceDelegate = persistenceDelegate;
}
public static Action<object> GetPersistenceDelegate()
{
return _persistenceDelegate;
}

public static void Persist(object instance)
{
_persistenceDelegate(instance);
}
}



As you can see, our static helper now can swap the logic used in the Persist method by way of the SetPersistenceDelegate method, the GetPersistenceDelegate is there only to satisfy the unit test needs, we would like to reset the value of the delegate on each test (don’t worry I’ll show this later). Another thing to note is that those two methods are public, well, I suppose we could make them internal and allow only the unit test assembly to use them, is up to the reader to do that too, for the sake of this demo I think it is ok (hint:InternalsVisibleToAttribute).





Now, let’s see how our unit tests look like:



    public class Scenario3Tests
{
#region Init, Before and After each test
private Action<object> _originalDelegate;

[
TestFixtureSetUp]
protected void Init()
{
_originalDelegate =
DataBaseHelper.GetPersistenceDelegate();
}

[
SetUp]
protected void BeforeEachTest()
{
DataBaseHelper.SetPersistenceDelegate(_originalDelegate);
}

[
TearDown]
protected void AfterEachTest()
{
DataBaseHelper.SetPersistenceDelegate(_originalDelegate);
}

#endregion

[Test]
public void when_disabling_a_product_its_enabled_property_is_set_to_false()
{
// Arrange
var product = new Product
{
ProductId = 1,
ProductName =
"Car",
Enabled =
true
};
var service = new ProductService();

// Act
service.DisableProduct(product);


            DataBaseHelper.SetPersistenceDelegate(instance =>
{
// no-op!
});
// Assert
Assert.AreEqual(false, product.Enabled);
}

[
Test]
public void when_disabling_a_product_it_persist_the_changes_using_the_product_repository()
{
// Arrange
var product = new Product
{
ProductId = 1,
ProductName =
"Car",
Enabled =
true
};
Product productPassedToPersistMethod = null;
var valueOfProductEnabledPropertyPassedToPersistMethod = false;
DataBaseHelper.SetPersistenceDelegate(
instance =>
{
productPassedToPersistMethod = (
Product) instance;
valueOfProductEnabledPropertyPassedToPersistMethod = productPassedToPersistMethod.Enabled;
});

var service = new ProductService();

// Act
service.DisableProduct(product);

// Assert
Assert.IsNotNull(productPassedToPersistMethod);
Assert.AreSame(product, productPassedToPersistMethod);
Assert.AreEqual(false, valueOfProductEnabledPropertyPassedToPersistMethod);
}
}



As you can see, the underlying idea of the tests remain the same, instead of using mocked repositories, we are using custom delegates and updating the helper delegate per test run to fit the test needs.



Finale



Well, I hope this article have gave you dear reader an idea of how to unit test legacy code. Please bear with me if I’ve made some silly statement, if so, please let me know so I can learn a few tricks too ;)



Also, I’ve not quite explained very well how a DI/IoC tool should be used to help us to resolve the dependencies at runtime, I’ll try to make another article to explain this, if you can’t wait there’s plenty of articles out there in the net from where you can learn of this subject.



You can download the sample code from here.

Monday, December 14, 2009

A quick look at the Reactive Extensions (Rx) for .Net

Introduction and some words about what I’ve been doing lately

Hello everyone, first of all, let me tell you that I’ve been very busy at work, we finally released a site for one of our customers and is a pleasure to see our hard work materialized in such a nice website.
We worked pretty hard particularly in the eCommerce module, which lead us to learn a lot of new paradigms and  even more tools and  frameworks around that module. Also I think the designers made a great work on the look and feel side of things, what can I say? I’m part of a great team. You can visit the site here.

Furthermore, there are a couple of personal projects for the next year which had get me very excited. So stay tuned for further info about this, I know you, dear reader, will really like it :)

Back to the subject, so, ¿what’s Rx?, again, I’m not so good at giving definitions or concepts, and in order to avoid any misunderstanding, here’s an excerpt from the project home page:

Rx is a library for composing asynchronous and event-based programs using observable collections.

Rx is a superset of the standard LINQ sequence operators that exposes asynchronous and event-based computations as push-based, observable  collections via the new .NET 4.0 interfaces IObservable<T> and IObserver<T>.  These are the mathematical dual of the familiar IEnumerable<T> and IEnumerator<T> interfaces for pull-based, enumerable collections in the .NET framework.

The Rx project is a new project from Microsoft and I hope you to play with it and when it becomes an stable library start using it in your projects together with other niceties that Microsoft is lately releasing to the public in general.

You may infer from the excerpt above that this is only available for .Net 4.0, but in fact it is also available for .Net 3.5. Check the downloadable installers and you’ll see there are 3 different versions of Rx: One for .Net 3.5, other for .Net 4.0 Beta 2, and another for Silverlight 3.

The code I’m going to present you today is using the .Net 3.5 bits (to reach a broader audience), but I think it is also applicable to .Net 4.0, I am not sure about Silverlight 3.

There’s a great set of articles which will introduce you into Rx way more articulated that I’m doing it, I’ve read them all and found them very useful:

Also, there’s a wiki with a great amount of information about the Rx API which is updated from time to time, you check it out here.

demonstrating Rx usage

As usual, I tend myself to explain things in code, together with a *common* scenario, well, at least one that could slightly reflect real world issues. However, this is completely relative from developer to developer.

The scenario is as follows, in our company we’ve a website that requires the following notifications:

  • Register all the visits.
  • When the user visiting our site is a new one, sent a notification to the webmaster about this.
  • When the user visiting our site is from USA or UK, sent a notification to the webmaster about this.

Those are roughly what the user tell to us, we as developer, add a couple of niceties that the user is not aware of:

  • In order to avoid spamming the webmaster email account, we sent those emails by chunks X, and each feature has a way to set the threshold. For our sample we are going to set this value to 4, but in a real app you must provide perhaps a UI with a database or some other way to configure this.

Now, lets define our ISiteVisitorsService interface:

 

    1 public interface ISiteVisitorsService

    2 {

    3     void StartMonitoring();

    4     IObservable<IEvent<VisitEventArgs>> NewVisitsObservable { get; }

    5     event EventHandler<VisitEventArgs> NewVisitEvent;

    6 }

The VisitEventArgs and its surrounding classes are structured as follows:

    1 public class VisitEventArgs : EventArgs

    2 {

    3     public VisitDetails VisitDetail { get; set; }

    4 }

 

    1 public class VisitDetails

    2 {

    3     public int VisitId { get; set; }

    4     public string FirstName { get; set; }

    5     public string LastName { get; set; }

    6     public VisitTypes VisitType { get; set; }

    7     public Contries Country { get; set; }

    8     public DateTime VisitDate { get; set; }

    9     public int VisitCount { get; set; }

   10 }

 

    1 public enum VisitTypes

    2 {

    3     FirstVisitor = 0,

    4     ReturningVisitor = 1,

    5 }

 

    1 public enum Contries

    2 {

    3     Usa = 0,

    4     Uk = 1,

    5     China = 2,

    6     Other = 3

    7 }

 

    1 public static class VisitorHandlerHelper

    2 {

    3     public static void LogVisit(IEnumerable<VisitDetails> visitDetails)

    4     {

    5         foreach (var visitDetail in visitDetails)

    6         {

    7             Console.WriteLine();

    8             Console.WriteLine("Registering visit => {0}: {1} {2} {3} {4} {5} {6}", visitDetail.VisitId,

    9                               visitDetail.FirstName, visitDetail.LastName, visitDetail.VisitType,

   10                               visitDetail.Country,

   11                               visitDetail.VisitDate, visitDetail.VisitCount);

   12         }

   13     }

   14 

   15     public static void SendEmailForEnglishSpeakersVisitors(IEnumerable<VisitDetails> visitDetails)

   16     {

   17         foreach (var visitDetail in visitDetails)

   18         {

   19             Console.WriteLine(

   20                 "Sending email to webmaster notifying an english speaker visitor => {0}: {1} {2} - {3}",

   21                 visitDetail.VisitId,

   22                 visitDetail.FirstName,

   23                 visitDetail.LastName,

   24                 visitDetail.Country);

   25         }

   26     }

   27 

   28     public static void SendFirstVisitorMessage(IEnumerable<VisitDetails> visitDetails)

   29     {

   30         foreach (var visitDetail in visitDetails)

   31         {

   32             Console.WriteLine("Sending email to webmaster notifying a new visitor => {0}: {1} {2} {3} {4} {5}",

   33                               visitDetail.VisitId,

   34                               visitDetail.FirstName,

   35                               visitDetail.LastName,

   36                               visitDetail.Country,

   37                               visitDetail.VisitType,

   38                               visitDetail.VisitCount);

   39         }

   40     }

   41 }

 

I provide two flavors to hook into the scenario where a new visitor arrives, the old fashioned will use the NewVisitEvent, and the others could use the Rx IObservable interface. I’ve added to StartMonitoring method to tell the service to enable firing events when new visitors land to our site. Obviously, I’m have no plans in implement a real monitoring service, rather you’ll see a faked service here. The implementations looks as follows:

    1 public class SiteVisitorsService : ISiteVisitorsService

    2 {

    3     private readonly IObservable<IEvent<VisitEventArgs>> _visitsObservable;

    4     private readonly string[] _names = new[] { "John", "Alex", "Jose", "Frank", "Laura", "Marco", "Mary" };

    5     private readonly string[] _lastNames = new[] { "Cruz", "Smith", "Gonzales", "Garcia", "Williams" };

    6 

    7     public event EventHandler<VisitEventArgs> NewVisitEvent;

    8 

    9     public SiteVisitorsService()

   10     {

   11         _visitsObservable = Observable.FromEvent<VisitEventArgs>(handler => NewVisitEvent += handler,

   12                                                           handler => NewVisitEvent -= handler);

   13     }

   14 

   15     public void StartMonitoring()

   16     {

   17         ThreadPool.QueueUserWorkItem(runSimulation);

   18     }

   19 

   20     public IObservable<IEvent<VisitEventArgs>> NewVisitObservable

   21     {

   22         get { return _visitsObservable; }

   23     }

   24 

   25     private void invokeNewVisitEvent(VisitEventArgs e)

   26     {

   27         var newVisitEvent = NewVisitEvent;

   28         if (newVisitEvent != null)

   29         {

   30             newVisitEvent(this, e);

   31         }

   32     }

   33 

   34     private void runSimulation(object state)

   35     {

   36         var visitId = 0;

   37         while (true)

   38         {

   39             Thread.Sleep(2000);

   40             var rnd = new Random();

   41             var visitType = (VisitTypes) rnd.Next(0, 2);

   42             var visitDetail = new VisitDetails

   43                                   {

   44                                       VisitId = visitId++,

   45                                       Country = (Contries) rnd.Next(0, 4),

   46                                       FirstName = _names[rnd.Next(0, _names.Length)],

   47                                       LastName = _lastNames[rnd.Next(0, _lastNames.Length)],

   48                                       VisitCount = visitType == VisitTypes.FirstVisitor ? 1 : rnd.Next(1, 2000),

   49                                       VisitDate = DateTime.Now.AddMonths(rnd.Next(0, 12)),

   50                                       VisitType = visitType

   51                                   };

   52             invokeNewVisitEvent(new VisitEventArgs()

   53                                     {

   54                                         VisitDetail = visitDetail

   55                                     });

   56         }

   57     }

   58 

   59 }

 

I think the code is pretty clear on what it does, what’s interesting about the code is the following snippet:

_visitsObservable = Observable.FromEvent<VisitEventArgs>(handler => NewVisitEvent += handler,

                                                  handler => NewVisitEvent -= handler);

 

What I’m doing there is creating an IObservable (thus, an instance that sent notifications) from our NewVisitEvent.
The FromEvent extension method has the following signature:

public static System.IObservable<IEvent<TEventArgs>> FromEvent<TEventArgs>(System.Action<EventHandler<TEventArgs>> addHandler, System.Action<EventHandler<TEventArgs>> removeHandler)
    where TEventArgs : System.EventArgs

As you can see, it’s a generic method which expect an inheritor of EventArgs, that would be our VisitEventArgs class, and also a delegate to add and remove handlers to this event.

Now, lets define our handlers, they should apply to the following interface:

    1 public interface IVisitorHandler

    2 {

    3     void RegisterListeners(ISiteVisitorsService service);

    4 }

 

The old way of do things

First, our old fashioned handler:

    1 public class OldFashionedVisitorHandler : IVisitorHandler

    2 {

    3     private static readonly List<VisitDetails> VisitsBuffer = new List<VisitDetails>();

    4     private static readonly List<VisitDetails> EnglishSpeakersVisitorsBuffer = new List<VisitDetails>();

    5     private static readonly List<VisitDetails> FirstVisitorBuffer = new List<VisitDetails>();

    6 

    7     public void RegisterListeners(ISiteVisitorsService service)

    8     {

    9         service.NewVisitEvent += serviceNewVisitHandler;

   10     }

   11 

   12     static void serviceNewVisitHandler(object sender, VisitEventArgs e)

   13     {

   14         var visitDetail = e.VisitDetail;

   15         VisitsBuffer.Add(visitDetail);

   16         if (VisitsBuffer.Count == VisitorHandlerHelper.LogVisitThreshold)

   17         {

   18             VisitorHandlerHelper.LogVisit(VisitsBuffer);

   19             VisitsBuffer.Clear();

   20         }

   21         if (visitDetail.Country == Contries.Uk || visitDetail.Country == Contries.Usa)

   22         {

   23             EnglishSpeakersVisitorsBuffer.Add(visitDetail);

   24             if (EnglishSpeakersVisitorsBuffer.Count == VisitorHandlerHelper.SendEmailForEnglishSpeakersVisitorsThreshold)

   25             {

   26                 VisitorHandlerHelper.SendEmailForEnglishSpeakersVisitors(EnglishSpeakersVisitorsBuffer);

   27                 EnglishSpeakersVisitorsBuffer.Clear();

   28             }

   29         }

   30         if (visitDetail.VisitType == VisitTypes.FirstVisitor)

   31         {

   32             FirstVisitorBuffer.Add(visitDetail);

   33             if (FirstVisitorBuffer.Count == VisitorHandlerHelper.SendFirstVisitorMessageThreshold)

   34             {

   35                 VisitorHandlerHelper.SendFirstVisitorMessage(FirstVisitorBuffer);

   36                 FirstVisitorBuffer.Clear();

   37             }

   38         }

   39     }

   40 }

 

Now, please look at the serviceNewVisitHandler static method, all those if statements, the thing about the three buffers to hold the VisitDetails instances (VisitsBuffer, EnglishSpeakersVisitorsBuffer and FirstVisitorBuffer), a complete mess!, also all that complexity add noises and hides the real purpose of the class. Also one last thing, in the RegisterListeners method we are using the NewVisitEvent event to handle when a new visitor arrives.
To summarize, this is how we’ve been doing this until today (I hope).

 

Using the Reactive Library

One of the benefits we get by using the RX library is that we can use events in a LINQ fashion, thus, that’s why some people name the Reactive Library as LINQ to Events.
Now, lets take a look to our reactive fashioned visitor handler:

 

    1 public class ReactiveVisitorHandler : IVisitorHandler

    2 {

    3     const int LogVisitThreshold = VisitorHandlerHelper.LogVisitThreshold;

    4     const int SendEmailForEnglishSpeakersVisitorsThreshold = VisitorHandlerHelper.SendEmailForEnglishSpeakersVisitorsThreshold;

    5     const int SendFirstVisitorMessageThreshold = VisitorHandlerHelper.SendFirstVisitorMessageThreshold;

    6 

    7     public void RegisterListeners(ISiteVisitorsService service)

    8     {

    9         var observerData = service.NewVisitObservable.Select(x => x.EventArgs.VisitDetail);

   10         observerData

   11             .Buffer(LogVisitThreshold, LogVisitThreshold)

   12             .Subscribe(VisitorHandlerHelper.LogVisit);

   13 

   14         observerData

   15             .Where(x => x.Country == Contries.Uk ||

   16                         x.Country == Contries.Usa)

   17             .Buffer(SendEmailForEnglishSpeakersVisitorsThreshold, SendEmailForEnglishSpeakersVisitorsThreshold)

   18             .Subscribe(VisitorHandlerHelper.SendEmailForEnglishSpeakersVisitors);

   19 

   20         observerData

   21             .Where(x => x.VisitType == VisitTypes.FirstVisitor)

   22             .Buffer(SendFirstVisitorMessageThreshold, SendFirstVisitorMessageThreshold)

   23             .Subscribe(VisitorHandlerHelper.SendFirstVisitorMessage);

   24     }

   25 }

 

The interesting stuffs happen in the RegisterListeners method.
First of all, if you look at the line 9, you’ll see we are projecting the  event arguments from the service IObservable to get only the stuff we are interesting in (the VisitDetail property).

Later on, we start  subscribing our delegates to the IObservable previously created.

From line 10 to 12, we are buffering the output of the event and after that we are subscribing the resulting IObservable to our VisitorHandlerHelper.LogVisit method. Way more simpler way than in our counterpart from the OldFashionedVisitorHandler.

From line 14 to 17 and from line 20 to 23, first we are filtering the IObservable by certain criteria, the first one will filter those events where the Country property is UK or USA, while the other IObservable is filtered only to be handled when the VisitType is equal to FirstVisitor. Next we are buffering the output as we did for the previous observer. And finally, we just subscribe the observable to its corresponding methods.

I don’t know about you, but I think that our ReactiveVisitorHandler is way more friendly than the OldFashionedVisitorHandler. It removes all that complexity and reduces drastically the amount of code you need to write in order to subscribe to an event and perform some special handling based on a complex requirement.

The output

Both versions will output the same thing, which in a console app may look like :

ReactiveSample

 

And one day we get a request for an improvement…

So, days go by, everything seems to work correctly. However we get a request telling us that the notification handling should work as always but now it must work in an async fashion, without blocking the service, thus, improving the performance.

So, what’s required to solve this problem? Let’s change our code to reflect just this:

    1 public void RegisterListeners(ISiteVisitorsService service)

    2 {

    3     var observerData = service.NewVisitObservable.Select(x => x.EventArgs.VisitDetail)

    4         .Asynchronous();

    5     observerData

    6         .Buffer(LogVisitThreshold, LogVisitThreshold)

    7         .Subscribe(VisitorHandlerHelper.LogVisit);

    8 

    9     observerData

   10         .Where(x => x.Country == Contries.Uk ||

   11                     x.Country == Contries.Usa)

   12         .Buffer(SendEmailForEnglishSpeakersVisitorsThreshold, SendEmailForEnglishSpeakersVisitorsThreshold)

   13         .Subscribe(VisitorHandlerHelper.SendEmailForEnglishSpeakersVisitors);

   14 

   15     observerData

   16         .Where(x => x.VisitType == VisitTypes.FirstVisitor)

   17         .Buffer(SendFirstVisitorMessageThreshold, SendFirstVisitorMessageThreshold)

   18         .Subscribe(VisitorHandlerHelper.SendFirstVisitorMessage);

   19 }

 

Do you note the difference (line 4)? So it took us just one line to achieve such a complex thing in a thread safe fashion.

What’s required to do the same in our OldFashionedVisitorHandler…well, I left that as an exercise to the reader, because from now on, frankly, I don’t care.

 

Summary

I hope dear reader you’ve noted the great improvement and less amount of code that you’ll get by using the Reactive Library, while it is in its initial stage, it is definitely something you should give a try.

The downloadable bits can be found here.

Bye bye.