Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

Astoria, SSIS Adapters and BDD – Class Design

In the process of writing a post about one specific feature of the design, I realized that I hadn’t actually described the moving parts in the solution.  There are 3 projects in the VS solution: the SSIS-specific code, the core application logic, and the application logic tests (specifications).  I’ll talk more about the SSIS part later and focus here on the core application logic and tests.

At a high level, there are 2 major logical components that interact to support SSIS at design time and runtime – those are DataServicesQuery and DataServicesQueryBuilder. 

The DataServicesQueryBuilder Object

The query builder is responsible for getting the metadata from ADO.NET Data Services and constructing the resources that are used by SSIS during the design time experience (those resources are also passed to the query to be used at runtime for making sure that a data element gets put in the right place in the buffer).  The basic elements of the query builder are shown below:

core

As you can see, responsibilities are further decomposed as follows:

  • The DataServicesQueryBuilder class is responsible for understanding an ADO.NET Data Services query and producing a query that can be consumed by the SSIS runtime
  • The IReader interface is responsible for getting stream data from some data source – the concrete implementation used by the query builder is the HttpRequestReader.  This implementation is used by the query builder to get service metadata as follows:
var metadata = _metadataReader.GetData(
    string.Format("{0}/$metadata", ServiceURI));
  • The IResourceBuilder interface is responsible for producing Resource instances from whatever stream data is handed to it.  Since ADO.NET Data Services currently has a dependency on EDM (more specifically, CSDL), the only relevant concrete class at the moment is CSDLResourceBuilder.

The only other thing I want to point out from the query builder classes is that we have a class for both Property and for RefProperty.  Resource properties are the equivalent of columns (if looking at the data from a tabular perspective) and RefProperties are the equivalent of links.  This will become more obvious when I discuss use of the $expand keyword in a future post, but the difference between parent/child (and subsequently, the driver behind inheritance) is based on the following runtime behavior (when extracting data from results data):

public class Property
{
    ...

    public virtual object GetValue(XElement element) {
        var propertyElement = element.Element(XName.Get("content", XmlNamespaces.AtomXmlNamespace))
            .Element(XName.Get("properties", XmlNamespaces.DataServicesMetadataXmlNamespace))
            .Element(XName.Get(Name, XmlNamespaces.DataServicesXmlNamespace));

        var propertyVal = propertyElement.Value;

        var val = Convert.ChangeType(propertyVal, PropertyType);

        return val;
    }
}

public class RefProperty : Property
{
    public RefProperty(string name, Type type, int? maxLength, bool? unicode)
        : base(name, type, maxLength, unicode) { }

    public override object GetValue(XElement element) {
        return element.Parent.Name.LocalName=="feed" 
            ? base.GetValue(element.Parent.Parent.Parent.Parent) 
            : base.GetValue(element.Parent.Parent.Parent);
    }
}

The DataServicesQuery object

Ultimately, the SSIS code simply uses the query builder in order to get a DataServicesQuery object.  The query builder configures the query with the collection of resources that it created (called the data shape).  The query is responsible for executing the actual query and extracting the results from the returned XML (ATOM feed) into a tabular format which can be used to populate SSIS output buffers.  As you can see below, the query class layout is much simpler than the builder:

Query Classes

As you can see, the reader is again used to pull down stream data.  The query then uses it’s data shape (resources) to take the XML data stream and create a ResultSet for each resource.  1..n ResultSets are then handed back as the result of executing the query – this is what gets handed back to SSIS and allows the SSIS adapter code to populate the SSIS buffers.

You can download and look at the complete source at http://code.msdn.microsoft.com/ssisastoria.  As always, please let me know if there’s stuff that doesn’t make sense or that could be refactored to be improved.

About Howard Dierking

I like technology...a lot...
This entry was posted in ADO.NET Data Services, SSIS. Bookmark the permalink. Follow any comments here with the RSS feed for this post.