Tuesday, January 6, 2009

LINQ And Interpreter Pattern : A tale of datasources

In a service oriented (SOA) environment, larger systems communicate/handshake with one and other with different varieties of data and formats.

These systems leverage different Data formats, contracts and pure XML (The RESTFULL way) hosted in a multi - platform environment.

Recently, in the real life, i came across a situation that i needed to fetch information from different data sources. The consuming application(s) does not want to know where these data's are being imported from and the producing application does not want to let go of the secret of the source to data. All it is concerned about is that the format must be consistent wherever it may be coming from.

I am supposed to fetch data from the following sources :

1. XML stored in a file system.
2. DataSet from another WebService
3. Service contracts from another service.

At first, this seems to be a mundane tasks, because you may have to create different code for different datasources and different translation for those datasources because an xml data is not the same as object data until we have a translator, we would not get anywhere near what we want to achieve.

First thing that comes to my mind was to leverage the Linq API and architecture to archive the above problem, how do i program my WCF service so that it will be extensible to other datasources, how do i ensure that the translation is not ambiguous (hence finding a better approach and framework that is consistent enough to do the job of translating data coming from different sources.

The Proposed architecture

I dig my hands into the dirty chests of patterns, i traveled from Gang of four to Microsoft pattern, i moved from different community, but there seem not to be a proper way of ensuring a consistent data translation strategy. After sometime, of digging, i eventually choose to use the interpreter pattern (Since it is used to interpret sentences in language elements). The reason for this choice is because i need a consistent language lexical structure that i can use for different data sources (as explained above) and Interpreter pattern seem to be the best choice to define my language grammar.

In this example, we want to interpret a Customer detail data structure coming from different sources, here is the C# structure of our Customer Poco class.

public class Customer
public string FirstName { get; set; }
public string LastName { get; set; }
public string Address { get; set; }
public int Age { get; set; }

We want to be able to translate several data sources to the Customer class defined above. This example will focus on the following data sources :

1. XML
2. Database
3. DataSet
4. object

If we are familiar with the Linq API, we would know that it supports for language elements across the named data sources above.

Interpreter Pattern Structure

The base interpreter is the interface to which we will use to interpret our language elements, because we need not know about the child classes or how they do their interpretations. The code snippet below is our base Interpreter.

public abstract class CustomerExpression
public Customer Customer { get; set; }
public abstract void Interpret();

The code above defines the CustomerExpression class and this class will define the structure of its child classes, the following child classes will be created :

  1. CustomerXMLExpression
  2. CustomerSQLExpression
  3. CustomerDataSetExpression
  4. CustomerObjectExpression
I will be discussing the first one only which is the CustomerXMLExpression. The code below is the snippet for the CustomerXMLExpression :

public class CustomerXMLExpression : CustomerExpression
public override void Interpret()
Customer = TranslateCustomer();

private Customer TranslateCustomer()
StringBuilder customerXML = new StringBuilder("XML Data");
XDocument customers = XDocument.Parse(customerXML.ToString());

return (from customer in customers.Descendants("Customers")
select new Customer
FirstName = customer.Element("FirstName").Value,
LastName = customer.Element("LastName").Value,
Age = (int) int.Parse(customer.Element("Age").Value),
Address = customer.Element("Address").Value,

We can use the same idea for the rest of the remaining data sources, for example we can search each data sources as follows :

CustomerExpression expression = new CustomerXMLExpression();
Customer customer = expression.Interpret();

CustomerExpression expression = new CustomerSQLExpression();
Customer customer = expression.Interpret();

CustomerExpression expression = new CustomerDataSetExpression();
Customer customer = expression.Interpret();

CustomerExpression expression = new CustomerObjectExpression();
Customer customer = expression.Interpret();

There is a long way you can go with this practise because it makes you to centralise your code and hides several dirty works away from developers.

No comments: