Greg Young [MVP]

State Pattern Misuse

2010-03-09T06:37:19Z

Just a quick run by posting from some discussions yesterday. I see many people using the State Pattern in their domains. More often than not though they are making a mistake that can be seen by analyzing the OO aspects of their code. Let’s go through a quick example of a Loan Application.

When its “In process” you can edit data on it or submit for approval.

When its “awaiting approval” you can no longer edit details data but you can approve or deny the application.

When its “completed” it has become immutable and can no longer be edited but may have some getters on it for viewing of historical data

I went through and made a quick little example of this

public interface ILoanApplicationState
{
    void Approve();
    void Deny();
    InterestRate GetApprovedInterestRate();
    InterestRate CalculateLikelyInterestRate();
    void SubmitForApproval();
    void ChangeDetails(LoanDetails details);
}

public class LoanApplicationState : ILoanApplicationState
{
    private LoanApplication parent;
    public LoanApplicationState(LoanApplication Parent)
    {
        parent = Parent;
    }

    public virtual void Approve() {}
    public virtual void Deny(){}
    public virtual InterestRate GetApprovedInterestRate() {}
    public virtual InterestRate CalculateLikelyInterestRate() {}
    public virtual void SubmitForApproval(){}
    public virtual void ChangeDetails(LoanDetails details) {}

}

public class InProcessState : LoanApplicationState
{
public InProcessState(LoanApplication Parent) : base(Parent) { }

    public override void Approve()
    {
        throw new InvalidOperationException("Cannot approve an inprocess application");
    }

    public override void Deny()
    {
        throw new InvalidOperationException("Cannot deny an inprocess application");
    }

    public override InterestRate GetApprovedInterestRate()
    {
        throw new InvalidOperationException("No approved interest rate exists on an in process application");
    }

    public override InterestRate CalculateLikelyInterestRate()
    {
        return InterestRate.Default;
    }

    public override void SubmitForApproval()
    {
        //change parent state to submitted
    }

    public override void ChangeDetails(LoanDetails details)
    {
        //change parent details
    }
}

public class AwaitingApprovalState : LoanApplicationState
{
public InProcessState(LoanApplication Parent) : base(Parent) { }

    public override void Approve()
    {
        //change parent state to approved and issue some behavior
    }

    public override void Deny()
    {
        //change parent state to denied and issue some behavior
    }

    public override InterestRate GetApprovedInterestRate()
    {
        throw new InvalidOperationException("No approved interest rate exists on an awaiting approval application");
    }

    public override InterestRate CalculateLikelyInterestRate()
    {
        return InterestRate.Default;
    }

    public override void SubmitForApproval()
    {
        throw new InvalidOperationException("Already awaiting approval");
    }

    public override void ChangeDetails(LoanDetails details)
    {
        throw new InvalidOperationException("No approved interest rate exists on an in process application");
    }
}

public class CompletedApplicationState : LoanApplicationState
{
public InProcessState(LoanApplication Parent) : base(Parent) { }

    public override void Approve()
    {
        throw new InvalidOperationException("Application has already been completed");
    }

    public override void Deny()
    {
        throw new InvalidOperationException("Application has already been completed");
    }

    public override InterestRate GetApprovedInterestRate()
    {
        //return parents approved interest rate
    }

    public override InterestRate CalculateLikelyInterestRate()
    {
        //return parents approved interest rate
    }

    public override void SubmitForApproval()
    {
        throw new InvalidOperationException("Already completed");
    }

    public override void ChangeDetails(LoanDetails details)
    {
        throw new InvalidOperationException("Application is already completed");
    }
}

OK enough yucky code you guys get the idea (the state pattern is also very verbose). The idea here is we plug in these state objects to our object and they handle the behavior changes from the original object.

My contention here is that you should not use the State Pattern like this. Where is the polymorphism? What if we think about the gross violation of LSP that we just created? This does however happen way too often.

The solution is that we really have three classes here InProcessApplication, AwaitingApprovalApplication, and CompletedApplication.

public class InProcessApplication {

InProcessApplication SubmitForApproval(ILoanProcessingService processor) { … }

}

this will force us to end up with smaller and easier to understand classes… and I mean hey, its not like polymorphism would have worked before anyways. It also makes our Ubiquitous Language more concise.

There is a particular code smell here that will help us distinguish whether we want the State Pattern or separate classes. Do some of the data/behavior only make sense in certain states? If so use three separate classes. Going along with this, if we find “throws” in our state implementors we should realize through LSP that we are doing something bad.

Why use Event Sourcing?

2010-02-20T20:06:10Z

Udi and I agree on probably 95% of what we talk about, one of the places that we have differing opinions is in the use of Event Sourcing I use the term as described previously to mean the rebuilding of objects based on events, not the definition that is currently on the bliki. To me this is an important distinction and I figured it would be worthwhile to write a post on why I feel the way I do, I explained parts of it in the previous post about CQRS and Event Sourcing but I wanted to talk not just about how the patterns are symbiotic but also some of the other reasons I use event sourcing.

Using a RDBMS

To start with let’s go through the alternative architecture, that is to rebuild objects from something that saves current state on the write side. I say something because that something could be many things; bigtable, mongodb, a relational database, xml files, photos of the objects that are then scanned in … It really doesn’t matter for the sake of this discussion, what matters is the storing of current state. For the sake of discussion let’s imagine that it is a relational database which Udi generally recommends here is the slide he generally uses to talk about it.

Data is stored, as normal in the relational database and the domain is instrumented to send events as well which can then be used with read model. There are some really good things about this architecture that I would like to go through before I talk about some of the issues I have run into with it in the past.

First of all this architecture is highly applicable to legacy systems that would like to move to a separated read model for some of their data (it is important to note that you may not move all of your data to a separated read model). it is also very familiar to development teams, operations, and management. We are well aware of how to deal with such a system, never underestimate the value of familiarity.

There are however some issues that exist with using something that is storing a snapshot of current state. The largest issue revolves around the fact that you have introduced two models to your data. You have an event model and a model representing current state. If you look in the above diagram you will see there are two operations, write and publish.

Any time that you represent things in more than one model you have to worry about keeping those two models in sync and this situation does not escape that. As an example, how do you rationalize that the data you saved to your database actually matches up with the events that you sent out? One can write unit tests to help try to keep things synchronized but they will eventually fall out of sync with each other and when they do, you have a problem.

Another problem with the having of two models is that it is necessarily more work. One must create the code to save the current state of the objects and one must write the code to generate and publish the events. No matter how go about doing these things it cannot possibly be easier than only publishing events, even if you had something that made storing current state completely trivial to say a document storage, there is still the effort of bringing that into the project.

Beyond all of that for me, the focus tends to be on the current state based model when the system is really event centric. This may sound like a nitpicky issue but I find teams doing this look at events with less importance than those who use events as storage as well as the latter only use events they are extremely event centric.

Using Event Sourcing

Using Event Sourcing does not change the architecture above much, the primary difference is that we have an Event Store holding the events to rebuild an object behind the domain as opposed to something storing the current state. There are however many interesting differences between the two architectures.

The first major difference is that it solves the problem of having two models. It removes the cost of having to deal with synchronization between the two and most importantly it removes the possibility that the two diverge. Having models diverge could be very bad as we use the event model as our integration model so others can create parallel models of our model. If we have a synchronization issue, they will have bad data while we have correct data, ouch. With Event Sourcing we only save the event (that can even be called “publishing it”)

There are however other benefits to using Event Sourcing as opposed to storing current state. I went through some of them briefly in another post. One of those benefits is that we can avoid having to use a 2pc transaction between the data model and the message queue (if we are using one)… the reason for this is that the event storage itself is also a queue, we could be trailing the event storage to place the items on the queue (or directly use the event storage as a queue).

Testing however is a big win with Event Sourcing, since all of your state changes are done through events you can simply test the events coming out the other side. This is particularly interesting when one considers that with events since you are being explicit about what is changing you are also testing what is not happening. Very few people write tests to show what doesn’t happen on behaviors (also a prime place where models can lose sync) eg: I am calling ScheduleAppointment on an object, do you check to make sure the address hasn’t changed? Since I in testing would assert that I only received an AppointmentScheduledEvent. Beyond that since we only deal with events on the other side we can view (and test) our domain as being a rather complex finite state machine which offers some very cool possibilities.

Of course we have left out what I think is one of the key values of Event Sourcing, we actually have the changes. Let’s say I get a concurrency violation (request is from version 5 while the current data is at version 12). With Event Sourcing I can ask for all of those events in between and see if any of those actually conflict with the command that I want to run (I will write a post on more about this works soon). In other words we provide intelligent merging…

You will note that I have not talked about any of the general benefits of Event Sourcing, such as the business value of the events, the value of having a log, the fact that the Event Store is additive only. I am leaving these out because many of these can exist in both systems. In the former you can save all of your events historically as well.

I hope this explains a bit about the differences between the models. I want to provide people as well a quick list of some of the pros and cons of each.

Current State Backing

Pros

Easier to retrofit on a legacy project

Well known technology/tools/skill sets

Easier sell for an organization

Cons

Dual models cost more and contain risk

Non-Event Centric

Event Storage Backing

Pros

Single Event Centric Model

Simplified/Better Testing

Conflict Management

Cons

Harder to sell to an organization

Less known tools/technologies (though you can implement the Event Store in a RDBMS which kind of mitigates this)

Very difficult to migrate a legacy app to

Hope this helps people.

CQRS and CAP Theorem

2010-02-20T18:30:11Z

The largest single benefit about CQRS is when you start running into problems with the CAP theorem. For those who are unaware, the basics of the CAP theorem are that there are three architectural properties that are linked together. Read through for a much better article describing CAP.

Consistency
Availability
Partitionability

The premise is that you can guarantee any two of these things. If you add one you do it by taking away from one (or both) of the others. In other words you can have a highly available consistent system but its partitionability will be low or you could have a highly available and partitionable system but you would likely need to give up on consistency.

Although it seems rather unrelated, CQRS and CAP have a very close relationship. You can use CQRS to get around problems you run into with CAP. The trick is that CQRS breaks up the system into multiple pieces that you can then adjust them separately. You are not avoiding CAP, you still have CAP limitations but you now have them in multiple places and you can make different choices in different places.

Let’s go through such a scenario. the customer needs all three; consistency, availability, and partitionability. Doing this in a single system according to CAP is impossible. But let’s get more into the problem and apply CQRS. We now have two distinct units, the first is the write side and the second is the read side. Where is consistency needed, is it everywhere or just for certain operations? How does our application scale? What is our ratio of reads to writes?

We can make different decisions on the read side and the write side. We can be ACID on the write side and BASE on the read side.

In most systems I see I find a ratio of reads to write between one and two orders of magnitude though I have seen this ratio up to six and higher orders of magnitude (consider a popular blog). So we can pretty quickly see that partitionability is important on the read side, it is what really controls our “perceived” partitionability, 90+% (in some cases up to five 9s) of our operations occur on the read side! On the write side we in most systems have a focus on not allowing in bad transactions and maintaining sanity within our data. We can give full consistency in most cases on the write side and give up a level of partitionability (not all systems but many, how many systems out there even process 1000 transactions per second? A modern RDBMS can easily handle this).

Coming back to the read side, if we have full consistency on the write model to validate transactions, do we actually need full consistency? What if the read model were 5 seconds behind the write model. Consider the popular blog example, what if when putting up a blog it took five seconds before it appeared throughout the whole site? This is in a human’s point of view a very short period of time and we will get very few conflicts introduced because of it (users generally take longer than this to edit data anyways, but that’s a post in and of itself)… but 5 seconds is an eternity in terms of a computers time. Introducing even just 1 second can allow our read model to become highly partitionable and available at very little cost in terms of user experience.

This is just one of the many examples of how we can make different decisions on the read side versus the write side. The ability to make these different decisions is really what CQRS is all about. Whether it be CAP decisions, storage model decisions, or hosting/platform decisions; all can vary independently. This is especially important because the architectural properties in all systems are different for reads than writes … I don’t have a formal proof but I cannot possibly think of a system where one will find an optimal solution that is the same for both.

Its important to note though, that these are things you can do, not necessarily things you should do. Separating the read and write models can be quite costly.

Using an ORM is like kissing your sister part 2

2010-02-19T16:02:16Z

I have to say, I love the reactions to the first post. I also expected them. People arguing about the merits of RDBMS, personal attacks, supporting comments… Its great. A few people did however actually understand the meaning of my post.

The whole point behind my post is … There are many factors in selecting a data store some functional, some non-functional. We need to actually be doing analysis of these requirements and leave behind the viewpoint of the RDBMS being the default. Examples of technical factors would be how well they match up with our functional specifications, examples in organizational factors might be what resources are currently available, whether we are fitting into a larger scheme of things, and of course cost.

Its also important to note that I never intended to speak out against the RDBMS but the storing of a relational model within one. One could as a commenter pointed out implement a non-relational model in a relational model. As an example I actually show people how to build an event storage system within a relational database quite often, it is not however a relational model.

ORMs have become the default architecture without people spending the time to actually analyze other technical and organizational needs. Frankly ORMs are highly inefficient when dealing with OLTP scenarios. Other systems have long since been shown to be an order of magnitude more efficient. They are not only more efficient but they also lower development costs significantly because there is not an impedance mismatch between the object model and the relational model (again non-functional requirements). If we are in say a service oriented system where data models are private, one can make a relatively good argument for not using a relational backing store, of course its rare that people actually make decisions like this.

For some antidotal evidence, I regularly when talking with groups of developers and architects ask them to raise their hands if they are using a relational data model. I then ask them to put their hands down if there was any actual analysis done to come to that decision. Significantly less than 1% will keep their hands up. I see the same trend in organizations that i work with, there is no rational thought process being performed. Its like people have ended up in that dreaded position where they have become comfortable with something so they just fall back on it for every problem, hammer anyone?

One of the largest things people should be looking at is using multiple models. Many people in comments talked about reporting, why not have a separate reporting model? There is a reason why there are the terms OLTP and OLAP, they are very different concepts and have drastically different needs. May this increase costs on your application? Of course, but decisions should be made based on rational discourse about your situation including both systemic and organizational needs.

To me, the blame is mainly with the vendors, spraying their kool-aid high and low throughout organizations. Organizations and developers are all too happy to accept the one size fits all answer as it allows the creation of cookie cutter systems which can be a good thing but is not always.

A RDBMS is a bad fit for many situations and a good fit for many scenarios … which is it in yours? Performing actual analysis could not really hurt could it?

Using an ORM is like kissing your sister

2010-02-18T14:25:37Z

I am pretty sure I have heard that phrase somewhere else before, no idea who to attribute it to but its a good one. If you are on a green field project and you are using an ORM, have you stopped and asked yourself why? Is your data model private? is someone integrating with you through the database? Why are you not just using an Object Database or a Document Database? Did anyone even ask these questions on your project? ORMs are being used to bridge the gap between object and relational models, do you actually even need a relational model?

It pains me to no end that developers more often than not make the single largest architectural choice on their system without applying any rigor to the decision. The choice to use an RDBMS drastically changes the options of where your architecture can go. Very often the decision is even made by people who have a background in marketing or “business” as opposed to computer science. Don’t get me wrong maybe you need a relational database for the OLTP side of your system … but was this actually a conscious decision on your project?

And we wonder why we have problems with so many systems.

CQRS, Task Based UIs, Event Sourcing agh!

2010-02-16T16:23:42Z

Many people have been getting confused over what CQRS is. They look at CQRS as being an architecture; it is not. CQRS is a very simple pattern that enables many opportunities for architecture that may otherwise not exist. CQRS is not eventual consistency, it is not eventing, it is not messaging, it is not having separated models for reading and writing, nor is it using event sourcing. I want to take a few paragraphs to describe first exactly what CQRS is and then how it relates to other patterns.

CQRS Command and Query Responsibility Segregation

Starting with CQRS, CQRS is simply the creation of two objects where there was previously only one. The separation occurs based upon whether the methods are a command or a query (the same definition that is used by Meyer in Command and Query Separation, a command is any method that mutates state and a query is any method that returns a value).

When most people talk about CQRS they are really speaking about applying the CQRS pattern to the object that represents the service boundary of the application. Consider the following pseudo-code service definition.

CustomerService

void MakeCustomerPreferred(CustomerId)
Customer GetCustomer(CustomerId)
CustomerSet GetCustomersWithName(Name)
CustomerSet GetPreferredCustomers()
void ChangeCustomerLocale(CustomerId, NewLocale)
void CreateCustomer(Customer)
void EditCustomerDetails(CustomerDetails)

Applying CQRS on this would result in two services

CustomerWriteService

void MakeCustomerPreferred(CustomerId)
void ChangeCustomerLocale(CustomerId, NewLocale)
void CreateCustomer(Customer)
void EditCustomerDetails(CustomerDetails)

CustomerReadService

Customer GetCustomer(CustomerId)
CustomerSet GetCustomersWithName(Name)
CustomerSet GetPreferredCustomers()

That is it. That is the entirety of the CQRS pattern. There is nothing more to it than that… Doesn’t seem nearly as interesting when we explain it this way does it? This separation however enables us to do many interesting things architecturally, the largest is that it forces a break of the mental retardation that because the two use the same data they should also use the same data model.

The largest possible benefit though is that it recognizes that their are different architectural properties when dealing with commands and queries … for example it allows us to host the two services differently eg: we can host the read service on 25 servers and the write service on two. The processing of commands and queries is fundamentally asymmetrical, and scaling the services symmetrically does not make a lot of sense.

Task Based UI

A task based UI is quite different from a CRUD based UI. In a task based UI you track what the user is doing and you push forward commands representing the intent of the user. I would like to state once and for all that CQRS does not require a task based UI. We could apply CQRS to a CRUD based interface (though things like creating separated data models would be much harder).

There is however one thing that does really require a task based UI… That is Domain Driven Design.

The Application Service Layer in Domain Driven Design represents the tasks the system can perform. It does not just copy data to domain objects and save them… It should be dealing with behaviors on the objects… Before going further let’s look at what happened if we did; there would be no verbs in our ubiquitous language except for “Create”, “Delete”, and “Change”. While there exist many domains where this is what the Ubiquitous Language is actually like, you probably should not be using Domain Driven Design for such systems.

The concept of a task based UI is more often than not assumed to be part of CQRS, it is not, it is there so the domain can have verbs but also capturing the intent of the user is important in general. Was this a managerial override or a normal update? Does it make a difference? It depends on what question you want to ask …

Moving on to the next pattern that gets confused into CQRS

Event Sourcing

For this I want to be clear, when I use this term I am not encompassing all of what is written on the bliki. I am referring to storing current state as a series of events and rebuilding state within the system by replaying that series of events. .

On the command side of the equation, since reads are no longer on the domain, storing events can be a great way of keeping the current state. The value increases more if you decide to have two separate models (a write model and a read model) and you have a need to integrate between the two of them as you will likely be doing that through events. Since you are building the eventing anyway, why not just use the one model to manage your state?

Messaging Patterns

There is no need to use messaging patterns with CQRS. That said, if you separate your data models you will more likely than not use messaging in the integration between the models because it offers interesting possibilities.

Finally I come to the last “pattern” I hate to call it a pattern when it is really a concept that people tend to put into their definitions of CQRS and it goes hand in hand with messaging.

Eventual Consistency

Eventual consistency is also quite often introduced between the services. It is done for many architectural reasons but the largest is that it allows you to increase your scalability and availability. If you remember CAP theorem consistency if given up allows increases in the other two. Eventual consistency is extremely useful in between the models if you have them in a separated fashion but is in no way a property of CQRS itself.

Going through all of these we can see that CQRS itself is actually a fairly trivial pattern. What is interesting around CQRS is not CQRS itself but the architectural properties in the integration of the two services. In other words the interesting stuff is not really the CQRS pattern itself but in the architectural decisions that can be made around it. Don’t get me wrong there are a lot of interesting decisions that can be made around a system that has had CQRS applied … just don’t confuse all of those architectural decisions with CQRS itself.

On a personal note it feels so weird to be writing to a blog again. Writing for a book is hard, writing for a blog is easy. Pages in the book average me an hour or more while this took me just over 30 minutes to write. The less formal style allows one to write in a more stream of consciousness manner. Anyways, hope everyone enjoys the blog posts that will be pouring in over the next short period.

Event Storage and Persistence Ignorance

2010-02-15T14:31:58Z

Someone brought up to me that they commonly hear concerns about persistence ignorance when dealing with a domain that uses event sourcing. The reasoning behind this is that the domain object knows about the events and rebuilds from them and is therefore not persistence ignorant. This is untrue.

One could store the events in a relational database.

One could store the events in a document database.

One could store the events in a file with binary serialization

One could store the events on pieces of paper that were typed into the system when they were needed (though this would not be very efficient for most systems).

the fact of the matter is that we could turn the logic of this statement around and say a typical system is not persistence ignorant because it can only be stored by representing its current state. A typical domain model cannot be stored as a series of events that are replayed to represent its current state so it is therefore not persistence ignorant.

To illustrate

One could store the snapshot of current state in a relational database.

One could store the snapshot of current state in a document database.

One could store the snapshot of current state in a file with binary serialization

One could store the snapshot of current state on pieces of paper that were typed into the system when they were needed (though this would not be very efficient for most systems).

Both can in fact be persistence ignorant, it is just that we have different kinds of methods of storage, snapshot current state verses a history. You cannot be persistence ignorant between the two different types because they are just different. Again and this is the second time I am writing this today, things are just different.

CQRS is more work because of the read model?

2010-02-15T13:57:13Z

I very commonly hear that applying CQRS is in people’s opinions more work that staying with the more classic CRUD based systems they are used to. The main reason for this objection is that the read layer is separated and goes directly back to the data model as opposed to through their domain model. They envision this as introducing more work. It could not be further from the truth, using CQRS even eat the simplest level where you only have a single data model is often though not always less work.

Let us consider for a moment the process of reading in a more classical architecture where we project our DTOs off of the domain model. We have our domain objects littered with getters and setters and our code maybe using a tool like Automapper will take these domain objects and project DTOs off of them. On the other side we will use the same process to map the DTOs back to the domain objects then save them. I will for the time being not even go into the fallacy of having a domain of property buckets or the fact that one has created an ubiquitous language with no verbs … can you use that in a sentence for me? Let’s just focus on the read / write behaviors.

In our system we are simply projecting which is not too bad if everything matches up 1-1 but does it? More often than not such systems have a DTO model that is screen based as it we gain perceived performance gains at the client due to lower numbers of round trips back and forth to the server –There are times where a canonical DTO model is preferred but that can be another post, for the sake of discussion we will imagine that we are dealing with a screen based DTO model—. The unfortunate thing about screen models is that they are tied to how our screens show data which may or may not have anything to do with our aggregate boundaries in our domain as the aggregate boundaries tend to be focused on consistency and invariants in order to perform operations. In other words they don’t line up.

The problem here is that it is very difficult to use our domain model, that was created for processing transactions, in an efficient way to provide reading behaviors. After optimization some general symptoms of this problem would be unneeded data being read, multiple round trips for building a DTO to the database, or my personal favorite gratuitous use of prefetch paths with the ORM of choice. Getting prefetch paths setup appropriately (and more importantly maintaining them over time) is hard and developers spend a lot of time trying to deal with the impedance mismatch between the domain model and the data model, especially over time as the two fall out sync with each other due to features being added etc.

Let us compare this with the simplest version of CQRS (a separated read model hitting the same datasource as the write model). Here the read layer talks directly to the data model. Query optimization becomes well … query optimization. We are not spending our time on the impedance mismatch between the object model and the data model. We instead are just writing queries against the model and mapping to DTO’s, which can be done using a tool like automapper just like in the first case. We are writing basic queries and mapping them, we can join in new and unusual ways, deal with roll ups, or even use an object model if we so chose! We can optimize queries easily (or even better let the DBA’s optimize them).

A similar issue exists when we start wanting things in a different way that the domain has things. Some examples of this would be when the screen DTO wants counts of something, or some rolled up view of data. It becomes extremely difficult to handle many of these situations in an efficient way when dealing with an object model.

Many people have run into issues like this, but instead of dealing with them they do something far worse. They just make their aggregate root boundaries the same as what their screens are (which may or may not have any correlation). Doing this completely breaks the concept of the aggregate root in the domain.

Some have brought up persistence ignorance on the read side, that building off the domain brings in persistence ignorance. There are a few options here. The first is to question, what is the actual risk of needing to support a completely different backing data model (note not a different database but a completely different data model), if it is low, how concerned should you really be? What would be the amount of work to change? Is the cost of losing this decision in the probability that you lose a good bet? The second option is to use a criteria api over the top of your object model, this adds much complexity but can isolate you from such a change.

Said differently, with CQRS you are not doing more work by separating the read service from the write. You are doing different work, it may be that its more, less, or the same. This is a common thread of thought I run into with CQRS style architectures, they are rarely more work but you are doing different work and the different work offers different architectural properties. More often than not people assume its more work because its different to them and they project it as being more complex when in fact it is just, different.

CQRS and Event Sourcing

2010-02-13T09:36:12Z

Somewhat recently --I have not been blogging much I know— Udi put up a good post (and long) about Command and Query Responsibility Segregation. One point that Udi brought up is that people have tied together Event Sourcing and Command and Query Separation. They are certainly two different patterns but I believe that it is a relatively rare case where two patterns share a symbiotic relationship.

Command and Query Responsibility Segregation enables the use of Event Sourcing on many systems that you would otherwise be unable to use Event Sourcing with. Martin Fowler does a good job of describing the downfall of Event Sourcing for most systems. The basic problem is that it is troublesome for querying data. There is no way you can query your event log for all of the customers with the first name of “Gregory”. This is a huge problem as most systems do need to issue large numbers of often times complex queries. This issue prevents most systems from being able to use Event Sourcing. Command and Query Responsibility Segregation however specifies a separation of the query model, this allows one to use an OLAP system on the read side and to deal with the write side separately. Because there are no longer queries on the write side the largest single downfall of event sourcing has been quietly removed.

I however stated that the two patterns have a symbiotic relationship. We have seen how CQRS enables Event Sourcing but is CQRS itself better off by using Event Sourcing on the data storage on the right side? The answer is a resounding yes.

Firstly when one uses Event Sourcing with CQRS one can avoid the need for 2pc between the data store and the message queue that you are publishing messages to. This is because the Event Store can be used as both an event store and a queue. Conceptually the Event Store is an infinitely appending file. One can just have a read location that gets updated (chasing the end of the file) and presto you have a queue as well. A side benefit from this is that you also end up with only one disk write for both your storage and your durable queue, this may not initially seem like a big win but when you are trying to build performant systems this can help a lot!

The second and I believe most important area where Event Sourcing really aids CQRS is in the only having a single model. If we were to say use a relational database, object database, or anything else that only keeps current state we would have a slight issue. The issue is that we have two different models that we cannot keep in sync with each other. Consider that we are publishing events to the read model/other integration points, we are also saving our current state with a tool like nhibernate. How can we rationalize that what nhibernate saved to the database is actually the same meaning as the events we published, what if they are not? When we use Event Sourcing we only use our events, they are the only model there is nothing to keep in sync. If there is a problem with the Event Stream that people use to integrate with us, like that we had a bug and never sent an event OUR state will be wrong as well.

Going with this if we fix broken data in our system either through an automatic or manual process the changes will automatically (by necessity) be shipped to anyone who may be integrating with us. This is not necessarily the case when one keeps an model representing current state as someone could feasibly want to change the data that’s in the store without going through the domain (and as such no events would be created).

Another benefit of this is that the data in the originating system will be screwed up as well. Generally speaking the closer you are to the people that are responsible for a set of data the more likely they are to notice that something is wrong. When dealing with integration scenarios it tends to workout that people have a level of trust of data from the other system. Since with a separate model the originating would be correct but the integrated systems would be wrong, there is a lower chance that people would notice that the data was in fact wrong.

I hope this goes a way to explain why Event Sourcing and CQRS really go hand in hand.

Delegate Mapper

2009-10-03T11:27:11Z

When I was in Holland Jelle Hissink pointed out a quick easy piece of code that is extremely useful in many scenarios I have seen in the past. It maps delegates from Base classes to Derived classes. Previously working with messaging I used the Handles<T> methodology which has since become popular in both MassTransit and nServiceBus. It allowed you to use typed message handlers as opposed to having to write handlers that used the base class then casted to the specific type. This code does the same but using delegates which is pretty slick and can be used in an aggregate root base class quite easily to handle the mapping of events to methods that apply them internally (you can also use it for event/command handlers if you don’t want to use an existing framework such as nServiceBus or MassTransit). Here is the code.

public class DelegateAdjuster
{
    public static Action<BaseT> CastArgument<BaseT, DerivedT>(Expression<Action<DerivedT>> source) where DerivedT : BaseT
    {
        if (typeof(DerivedT) == typeof(BaseT))
        {
            return (Action<BaseT>)((Delegate)source.Compile());

        }
        ParameterExpression sourceParameter = Expression.Parameter(typeof(BaseT), "source");
        var result = Expression.Lambda<Action<BaseT>>(
            Expression.Invoke(
                source,
                Expression.Convert(sourceParameter, typeof(DerivedT))),
            sourceParameter);
        return result.Compile();
    }
}

Simple, and slick.

DDD Step by Step

2009-08-31T03:22:47Z

I don’t know how many people already know about this but figured I would put up a post about it as a pointer for those who have not seen it.

I was surfing over at DDDStepByStep.com and there are *a lot* of new resources available there for people who are learning DDD. Casey ~~Charaltan~~ Charlton :) is the one who has put it together and there is really some valuable information there. As an example he has (recently?) put up this about 50 page PDF that is a short book on getting started with DDD including many code examples. It also seems to contain a pretty good blog roll that can be subscribed to for those that are interested in DDD.

DDD is a dense book

2009-08-13T17:42:00Z

But seriously there is some classic humour if you look hard enough. My friend Matthieu Guyonnet-Duluc sent me this scan today definitely worth a smile.

Command Query Separation?

2009-08-13T05:07:29Z

There has been a lot of talk lately about Command and Query Separation. One thing that has come up with many people in learning it is that they get confused between CQS [Meyer] and CQS [Architecture or DDD]. As such many have called for us to rename the latter to something different as although it is very similar to CQS [Meyer] they find it to be quite different. I however disagree with this let’s look at our definitions.

Meyer:
Separate command methods that change state from query methods that read state.

Current:
Separate command messages that change state from query messages that read state.

The reader should notice that these two are nearly identical, the principle has remained the same, only the definitions of what commands and queries are has changed. A really quick reader will also be quick to jump on me for not bringing in all of what CQS has come to be known as.

In CQS [Architecture/DDD/whatever] we also go one step further and we create two end points (or objects) by splitting the responsibilities of the original concept. We give one sub-concept the responsibility of processing all of the commands and another the responsibility of processing all of the queries. We do this because we have come to understand that there are very different non-functional requirements for each and that by separating the two we can better tailor solutions.

One question that has been hitting me is whether or not this is something that may work as a pattern at the object granularity as opposed to the endpoint as many of these types of patterns are relevant to both scopes.

This part is new and is not CQS [Meyer] I am going to throw out a quick pattern name for this and call it Command and Query Responsibility Segregation or CQRS as we are in fact simply taking what would usually be one object and splitting its responsibilities into two objects.

So the first step is that we apply CQS as a principle, and we then come through and apply the CQRS pattern in order to end up with what people are currently calling “CQS”. I would hope that this is a clearer explanation and manages to bring in the original intent of command and query separation while realizing that there is in fact an important difference.

Does this help clear things up or does it only make things more confusing?

Unshackle Your Domain

2009-07-15T20:21:26Z

Many people have asked me for links to this video. It has been up on infoq for a week or two now. http://www.infoq.com/presentations/greg-young-unshackle-qcon08 this is from last fall in SanFran for QCon.

The Anemic Domain Model Pattern

2009-07-15T20:10:39Z

OK so I assume that you have run into the Anemic Domain Model Anti-Pattern at some point or another likely here on the bliki.

To quote from the bliki

The basic symptom of an Anemic Domain Model is that at first blush it looks like the real thing. There are objects, many named after the nouns in the domain space, and these objects are connected with the rich relationships and structure that true domain models have. The catch comes when you look at the behavior, and you realize that there is hardly any behavior on these objects, making them little more than bags of getters and setters. Indeed often these models come with design rules that say that you are not to put any domain logic in the the domain objects. Instead there are a set of service objects which capture all the domain logic. These services live on top of the domain model and use the domain model for data.

What if I were to tell you that there are times and places where gasp an anemic domain model were in fact a best practice that would be highly recommended to a team? I really do vision the pitchforks and torches coming at me when I say this … Many including myself have lambasted unmentioned groups for pushing people towards anemic domain models so ….

Making the case

To start with let’s analyze why you were using that domain model in the first place. Ideally you were using it because it provided you the benefit of being able to effectively handle more complexity as opposed to a system like active record or transaction script. In other words you found your system was too complex to be modeled in a more procedural way. This complexity is often seen by analyzing the duplication of code in your more procedural transaction script based system.

Of course I find it rare to hear people cite this as why they are using a domain model. The reasons I hear tend to be more focused on the layering that necessarily comes with having a domain model. Instead I hear reasoning like testability, maintainability, abstraction from persistence mechanisms, strongly defined contracts etc.

Where am I going with all this what happened to the anemic domain model pattern? Well it depends why you wanted a domain model in the first place. If you are using a domain model in an object oriented way to help you in the managing of complexity it is absolutely an anti-pattern for you to be seeing an anemic domain model. What if you are in the IMHO much larger group that is mainly seeking the benefits that come with a domain model in layering?

The simplest in me would say that we would need to then compare the anemic domain model with other mechanisms to see if it would derive any benefit. We can pretty easily gain over a classical Active Record pattern because we can better express what is happening (especially in cases where we have an existing data model as we can map the data model if we want to). This leaves us with the more interesting case of transaction script, I deem the case “interesting” as both models are in fact transaction script we are comparing transaction script over an object model vs transaction script over say table module.

I would say that even in this close distinction that the anemic domain model can have some advantages. At the least single property level validation tends to be encapsulated within the object. This although really a slight step towards having a functional domain model can offer huge gains in terms of dealing with transaction script where all of the validation is distributed. There can however be other advantages such as the ability to more easily test in isolation and the long term maintainability of the system.

Defining a context

So the real question becomes can we define a context where purposefully creating an anemic domain model would be a good idea over our alternatives?

We want a layered architecture. We understand that our application due to non-functional requirements will need the benefits that a layered architecture provides. We understand the cost of creating a layered architecture and it has been justified by the stakeholders of our project.

Our domain is not extremely complex. This is a bit of a misnomer, but let’s say that our domain falls within the bottom 90% of systems in terms of domain complexity. There are likely some spots of higher complexity that we may focus more on (and perhaps even model in a more object oriented way).

Our team is not highly functional in object oriented design and analysis. When I say this I mean that team members must be highly functional in order to attempt creating a domain model (all other attempts will be doomed to failure, likely as an anemic domain model). When I say highly functional I would consider the AJM model (and people at the least being journeyman with maybe a single apprentice as the complexity of the system increases the number of masters needed increases). Said bluntly, 90% of teams in the Microsoft world should be looking at building anemic domain models as opposed to actual domain models for solely this reason. Using things that the development does not have a large understanding of is a certain recipe for failure.

We would otherwise consider a simpler model such as transaction script but feel that we can benefit from things such as further testability.

I believe that using these five viewpoints we can effectively create contexts within it makes perfect sense to be choosing to create an anemic domain model.