Greg Young [MVP]

DDDD 11 [Basic Consistency]

Greg — Tue, 06 May 2008 22:48:29 GMT

So I was bad last week and didn't get much written shame on me... hopefully I can get back on track. In DDDD 10 we looked at Command Query Seperation, now we are going to take that idea a little bit further as we really need to break it down a bit more than commands and queries.

Let's say that I have a domain with a Customer Aggregate Root. When I want to update this Customer I need to issue a query to get its current state so I can validate whether my current changes would make the object en up in an invalid state. Generally for these types of reads one would want consistent data. Why bother with all of my nice validation if it can be bypassed by inconsistency? One of the key concepts in domain driven design is that I know that certain things about my objects are true.

In that same domain I may also be able to search for customers with a first name of "Greg". In this type of situation it is no longer a technical requirement (may still be a business requirement) that my search is completely consistent (for future reference I call any read such as this a "Report" even though they may not meet your personal criteria of a report).

If my search does not need to be completely consistent I can do many very neat things with my data such as having it come from a different model than my transactional model (i.e. query a reporting database). I can also do other things like cache the data with expirations. These types of operations can allow me to become much more scalable and they are a key concept we will be later exploiting in moving towards distributing our systems.

Because of how important our gains are; let's make some rules.

Aggregate roots provide the boundary for where data is always consistent. Anytime you deal with data that belongs to an aggregate from outside of an aggregate you should make your new default to assume it to be eventually consistent.

Bounded Contexts are always considered to be eventually consistent. If you have Bounded Contexts where this is not true there is probably something wrong with your model.

When I say you should make it your new default; I mean that:

Unless it is explicitly stated otherwise and a business reason is provided with the story; it will be eventually consistent.

One problem that you will run into with this transition is managing expectations. Domain Experts like most non-technical (and most technical for that matter) people have been trained to think that everything should be instantaneous with computers ("You mean when I click save and it says it saved it takes 45 seconds to show up in the search?"). They have sucked from the evil tit of the RDBMS and its global consistency for far too long. The next post will be about getting over this hurdle.

I Want Spec#

Greg — Mon, 28 Apr 2008 22:23:00 GMT

Scott Hanselman has recently put up a new hanselminutes of an interview with the spec# team. This interview was done at altdotnet; one of the key points of discussion both in general and in the interview was "how can we get spec#?". The simple answer is we have to want spec# and Microsoft needs to know that we want it. The more people who want it the more likely we are to get it or to get it in pieces. The best way for them to know is for us to tell them!

Let's give Mike and the whole spec# team the best compliment possible on their research; asking for it to become integrated in products and/or to be available with a commercial license!

I want verifiable software...

I WANT SPEC#!

Here is a reusable bumper sticker for your blog to make it easier to express how you feel.

Drop a note here so I can try to keep track of people!

If you don't know about spec# now may be a great time to watch the recent presentation from altdotnet

Knuth: Wow

Greg — Mon, 28 Apr 2008 20:35:43 GMT

Donald Knuth is a mainstay of computer science from the last what 30 years? He recently did an interview with InformIT talking about some more modern concepts.. Here are some quotes I found quite interesting!

On the multi-core problem

Donald: I don’t want to duck your question entirely. I might as well flame a bit about my personal unhappiness with the current trend toward multicore architecture. To me, it looks more or less like the hardware designers have run out of ideas, and that they’re trying to pass the blame for the future demise of Moore’s Law to the software writers by giving us machines that work faster only on a few key benchmarks! I won’t be surprised at all if the whole multithreading idea turns out to be a flop, worse than the "Titanium" approach that was supposed to be so terrific—until it turned out that the wished-for compilers were basically impossible to write.

later ...

They think a magic bullet will come along to make multicores speed up my kind of work; I think it’s a pipe dream. (No—that’s the wrong metaphor! "Pipelines" actually work for me, but threads don’t. Maybe the word I want is "bubble.")

I agree; pipelines rock!

On unit testing and mocking

As to your real question, the idea of immediate compilation and "unit tests" appeals to me only rarely, when I’m feeling my way in a totally unknown environment and need feedback about what works and what doesn’t. Otherwise, lots of time is wasted on activities that I simply never need to perform or even think about. Nothing needs to be "mocked up."

On extreme programming

Still, I hate to duck your questions even though I also hate to offend other people’s sensibilities—given that software methodology has always been akin to religion. With the caveat that there’s no reason anybody should care about the opinions of a computer scientist/mathematician like me regarding software development, let me just say that almost everything I’ve ever heard associated with the term "extreme programming" sounds like exactly the wrong way to go...with one exception. The exception is the idea of working in teams and reading each other’s code. That idea is crucial, and it might even mask out all the terrible aspects of extreme programming that alarm me.

On reusable code

I also must confess to a strong bias against the fashion for reusable code. To me, "re-editable code" is much, much better than an untouchable black box or toolkit. I could go on and on about this. If you’re totally convinced that reusable code is wonderful, I probably won’t be able to sway you anyway, but you’ll never convince me that reusable code isn’t mostly a menace.

At the least these words should be stopped and thought of.

DDDD 10 [CQS]

Greg — Mon, 28 Apr 2008 07:55:18 GMT

Boring Sunday night so I figured I would get ahead on my posts for the week (I have not been good enough about hitting 5 but I am trying). This post is probably out of order but the concepts are simple, very important, and I will want to link to this later. Also this post and concept is valuable whether or not you are intending to use messaging.

At altdotnet one thing I talked a bunch about that most people had not really dealt with before was Command Query Separation( Martin Fowler also has a good post about it as well ). This is still a running theme in these posts; what I am talking about is rarely if ever new.

I know its a really big surprise that Greg is into things that are based in DbC ... but later in our systems we will care *a lot* about what is mutating the state of the system and the consistency that is required for various actions.

Basically all things that change mutable state are commands and anything that only reads data is a query. Personally I like to add some further distinctions to this like "Only reads immutable state" vs "Reads mutable state" as with the latter I need to worry about the consistency of those reads but command and query are the roots.

To help think about Commands and Queries let's look at them in terms of SQL as it has good separation (usually)

Command = Insert, Update, Delete
Query = Select (yeah surprisingly a query is a query :))

Not surprisingly commands and queries have quite different interactions in a SQL database (in fact most of what your transaction isolation levels (Read Commited, etc) talk about is how commands and queries interact with each other). This concept has largely been lost from our code and thought (largely because we rely on databases too much) though some have pushed greatly for this separation. Most systems allow too many things to become commands and queries at once leading to difficult to understand code... Patrick: this is a key metric ;-)

This may seem quite weird to many people at first but one key point on commands vs queries is that commands should not return a value. Only a query is allowed to return a value though a command can technically change state that is passed into it so ...

public void Foo(something s) {
//modifies state
s.somevalue = "greg";
}

is ok but ..

public string Foo() {
//modifies state
return "greg";
}

is not!! this is a funny distinction eh?

In general I would prefer you in your code to follow an even TIGHTER distinction and assume that things passed into you are *also* immutable (The reason for this is that handling byref data in anything that crosses serialization boundaries is troublesome, but there are some cases where this can be ok (like setting something in a context)). It may feel really weird and unnatural at first but eventually it will become second nature; I promise. The only thing you are allowed to do is mutate state in some way (or add to an observable state hint hint).

We will get more into this later but as a critical thought what does this do to the concept of request/response? Maybe tomorrow?

Side Effect Free Functions and CQS

In Domain Driven Design, Eric Evans pushes Side Effect Free functions which are great and should be in your tool bag. What side effect free functions essentially do is change a command into a query. Instead of changing mutable state they create immutable state that is only visible to the caller. Since it is only visible to the caller it can be seen as a query as it is understood to be in a transient state and not visible to other queries.

Altdotnet Spec# Session

Greg — Sat, 26 Apr 2008 20:13:00 GMT

Here is another video in the videos from altdotnet. We had Rustan and Mike of the spec# come out from their cave (j/k) at MSR to talk about spec#, boogie, and the future of compile time proving!

Enjoy it and as always feel free to leave feedback!

the video is not the best quality but hey ... it was taken on a phone!

DDDD 9 [Producers and Consumers]

Greg — Fri, 25 Apr 2008 05:58:36 GMT

Not surprisingly the Producer/Consumer relationship is the single most important concept in messaging. Let's take a look at those interfaces and how they work. The first we will look at is the interface that an object must meet in order to consume a message.

public interface IConsume<T>
{
     void Consume(T message);     
}

Next we will need something that can produce a message so we can hook up the consumer to a producer (generally used in a pipeline not for domain objects, domain objects tend to call a message publisher to act for them).

public interface IPublisher   
{
      void Publish<T>(T message);   
}

public interface IProduce<T>  
{
     void AttachConsumer(IConsume<T> consumer); 
}

One important fact to note is that most objects who are producers will only ever have one consumer. So AttachConsumer for most objects will throw an exception (or replace) any time that it already has a consumer and we connect another consumer to that producer (there are some notable exceptions to this rule in the framework that we will discuss later). The reason that we do this is separation of concerns. The producer does not need to know whether there is one or many consumers connected to it.

I think now is a great time to mention that in terms of bringing my domain to messaging these interfaces are the main leak of messaging into the domain. All of the infrastructure etc outside of my domain stays completely outside of my domain. This non-leakage of the infrastructure into our domain is a key concept as we will see shortly when we Remove Infrastructure Out Of Domain and rid ourselves of those pesky nouns inserted by our infrastructure.

One of the great parts of alt.net seattle was the chance to talk with many developers and bounce ideas off of them (ask Dave Laribee I am an "Idea Bouncer"). One of these idea bouncing sessions was with Chris Patterson and Dru Sellers who are working on (and doing a great job I should add) a light weight ESB they call MassTransit. I discussed with them my Producer/Consumer relationships that were quite different that what they were using.

Last night Chris came back to me with a question of whether it was explicit enough in the contract that in a request/response style message which request matched up with which response. While my "first step" in teaching someone messaging is to say "stop thinking request response" these messages can happen fairly often (though not in a synchronous fashion). What is even more likely to have happen is you have an object in a state who when it receives a message "Responds with" no a reply to the client but another routed message itself (see controller example in Mocks are a Code Smell) ... In going through this conversation I started thinking back to DDD and "Intent Revealing Interfaces".

What if I could create an interface that was essentially the same thing but did a better job at revealing my intent of using both of these messages in a "triggered" way.

    class RespondsTo<T>
    {
        public interface With<V> : IConsume<T>, IProduce<V>
        {
            
        }
    }

important note: I would NOT use this interface for request/response style messaging ... there are better ways of handling this... the domain object still dispatches!

This allows me as a user of this class to realize not only that it produces and consumes a message but that the two messages are in fact linked (when/if they are). Consider the following two examples.

class FooHandler : IConsume<Foo>, IProduce<Bar>

class FooHandler : RespondsTo<Foo>.With<Bar>

The second example here although it simply implements the same two interfaces gives me a lot more easily discoverable information about how that relationship interacts. A more "appropriate" example might be from the controller example

class ATMController : RespondsTo<CardInsertedMessage>.With<RequestAuthenticationMessage>

Take this new interface with a grain of salt as I have not spent a good deal of time playing with it yet (I prefer to re-dispatch generally and not have the producer interface on the object itself) but it would seem to be at the outset quite useful (as well as more intention revealing).

altdotnet seattle session video

Greg — Fri, 25 Apr 2008 05:32:11 GMT

Here is the video from the altdotnet talks... There are many good points brought up through out this fishbowl; the one that I really wish was answered better was the question from Adam on introducing stuff to a team who is not familiar with messaging... What are your thoughts?

DDDD 8 [Fluent Builders and Tests]

Greg — Tue, 22 Apr 2008 06:43:21 GMT

Ok this will not be the only post I wrote on this subject as its a pretty big one but I had some good discussions with Udi about this over the weekend. I find that my present way of doing this is pretty cool and of great value to people already using messaging systems without having a big discussion about all of the aspects of testing in messaging based systems (I imagine that would be about 50-60 pages for a cursory overview).

In commenting on Messages are Value Objects korkless hit the nail on the head as to why equality comparisons for messages should be overridden to operate with value semantics. When I am testing I can use my fluent interface to build up the message I expect to receive and compare the two of them in a single Assert. I should have been more clear when I said "I can't see why anyone would use this in their domain"; there is a use for it ... just not so much in your domain as in your tests.

Since I am fried still but code speaks 1000 words; I am taking a simple test from Mocks are a Code Smell and implementing it here with a fluent builder to illustrate.

public void Controller_ShouldRequestAuthentication_When_CardInsertedMessageReceived()  
{
     ATMController Controller = New.ATMController.InState(ATMController.States.AwaitingCard);
     CardInsertedMessage InsertedMessage = New.CardInserted.ForCard("1111111111111111").WithAccountHolderOf("Greg Young");
     controller.ConsumeMessage(InsertedMessage);
     List messages = new List(FakeMessageDispatcher.AllMessages());
     RequestAuthenticationMessage Expected = New.RequestForAuthentication.ByPinNumberOnly.From(Controller);
     Assert.AreEqual(Expected, messages[0]);
}

This allows me to write a very fluent test ... as we will see later, we can even refactor this test to be much smaller and more expressive : imagine ...

public void Controller_ShouldRequestAuthentication_When_CardInsertedMessageReceived()  
{
     When.A(New.ATMController.InState(ATMController.States.AwaitingCard))
	.Consumes(
		New.CardInserted.ForCard("1111111111111111").WithAccountHolderOf("Greg Young")
        )
        .Expect.AMessageOf(New.RequestForAuthentication.ByPinNumberOnly.From(Controller));
}

The fact that I am using a fluent builder to define my expectation helps to make my expectations a lot clearer and to avoid having multiple asserts. If later I decide to add a field to the message and the object sets that field away from its default I will also now get a test failure since I am allowing the message to do its own comparison.

This is an important pattern to keep in mind not only for messages but also for Value Objects within your domain since as I said before .... Messages are Value Objects.

One thing I have been playing a lot with lately is changing tests like this to a context based format ... just a passing thought.

DDDD 7 [Wow]

Greg — Tue, 22 Apr 2008 06:14:02 GMT

I have to say Alt.Net Seattle and the MVP summit were one of the best developer community experiences I have ever have. Aside from all the awesome side discussions over beers or just hanging out in the halls I was quite surprised to find this during the opening of the open spaces.

This was a huge surprise to me to say the least, not even that there was one ... but there were THREE on the same subject! At first I was a bit hesitant as there are still many things not completely cohesive in my head but I am really glad that this session happened. It was amazing to have not only Udi Dahan in the fish bowl but also Martin Fowler and quite a bit of Dru. Let's just say that the discussions were great!

I do wonder though ... who decided the first D was Distributed? :)

I have to say that I was a bit disappointed that many of those discussions focused primarily on infrastructure but it is an important part of the topic that I will go into quite a bit of later.

I think my camera got most of the session so I will try to post a video of this and the Next Generation Architectures fishbowl (I was expecting this one to be more focused on infrastructures)

There was a question asked in the DDDD fishbowl that was never answered very well that I hope to address in the near future:

My team has never used messaging before; what would be the best way for us to get up to speed and to learn to do things right.

If you are already doing messaging and have thoughts on this drop me an email offline at gregorzyzounzgz1@gzmail.com (remove the zs)

Aside from those two great talks the rest of altdotnet was also great. It is just crazy what happens when you get so many smart people into one place with high bandwith communications. Aside from conversations with Udi (which were great btw because they were mainly about the 5% of things we disagree on) I also had great conversations with many others including Chris and Dru who are working on MassTransit which is shaping up to be a useful and lightweight tool for enterprise integrations. I am looking forward to seeing this project progress.

In the Next Generation Architectures talk I got to meet Addy S for the first time as well as some other smart gents. If the guy with the beard who was in the fishbowl a bunch is reading this please leave a comment or drop me an email, I never got your name.

I am still in shell shock from all of last week and in need of sleep. Before tomorrow though I wanted to ask people reading this whether they would prefer the next cluster of posts to be implementing the easiest messaging scenario (in process infrastructure abstraction) or whether they would prefer to get more into the underlying infrastructure and concepts of a next generation default architecture ...

DDDD 6 [Fluent Builders, Alternate Ending]

Greg — Wed, 16 Apr 2008 22:22:32 GMT

Sitting with an awesome view of Seattle ... Last night I spent some time over at Eleutian with Aaron and during our discussions we talked a bit about Fluent Builders. He brought up an interesting idea to me of whether or not builders should be immutable and I think I agree with his mutable version so I am going to provide another way of implementing the Fluent Builders from DDDD 5 [Messages have Fluent Builders].

The basics of the discussion revolve around the methods used for setting fields in the builder. It is best seen in code comparing the builder in the last post of ...

	public class EquityQuoteMessageBuilder {
		internal readonly ExchangeGateways m_Gateway;
		internal readonly string m_Security;
		internal readonly DateTime m_Time;
		internal readonly decimal m_BidPrice;
		internal readonly decimal m_AskPrice;


		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder OnGateway(ExchangeGateways _Gateway) {
			return new InsertEquityQuoteCommandBuilder(_Gateway,m_Security,m_Time,m_BidPrice,m_AskPrice);
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder ForSecurity(string _Symbol) {
			return new InsertEquityQuoteCommandBuilder(m_Gateway,_Symbol,m_Time,m_BidPrice,m_AskPrice);
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder At(DateTime _Time) {
			return new InsertEquityQuoteCommandBuilder(m_Gateway,m_Security,_Time,m_BidPrice,m_AskPrice);
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder WithSpreadOf(decimal _BidPrice, decimal _AskPrice) {
			return new InsertEquityQuoteCommandBuilder(m_Gateway,m_Security,m_Time,_BidPrice,_AskPrice);
		}

		private EquityQuoteMessageBuilder(ExchangeGateways _Gateway,string _Symbol,DateTime _Time,decimal _BidPrice,decimal _AskPrice) {
			m_Gateway = _Gateway;
			m_Security = _Symbol;
			m_Time = _Time;
			m_BidPrice = _BidPrice;
			m_AskPrice = _AskPrice;
		}

        	public EquityQuoteMessageBuilder {}
		
        	public static implicit operator EquityQuoteMessage(EquityQuoteMessageBuilder _Builder) {
				return new InsertEquityQuoteCommand(_Builder.m_Gateway, _Builder.m_Security, _Builder.m_Time, _Builder.m_BidPrice, _Builder.m_AskPrice);
		}
	}

With the modified version being ....

	public class EquityQuoteMessageBuilder {
		internal ExchangeGateways m_Gateway;
		internal string m_Security;
		internal DateTime m_Time;
		internal decimal m_BidPrice;
		internal decimal m_AskPrice;


		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder OnGateway(ExchangeGateways _Gateway) {
			m_Gateway = _Gateway;
			return this;
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder ForSecurity(string _Symbol) {
			m_Symbol = _Symbol;
                        return this;
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder At(DateTime _Time) {
			m_Time = _Time;
			return this;
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder WithSpreadOf(decimal _BidPrice, decimal _AskPrice) {
			m_BidPrice = _BidPrice;
			m_AskPrice = _AskPrice;
			return this;
		}

        	public EquityQuoteMessageBuilder {}
		
        	public static implicit operator EquityQuoteMessage(EquityQuoteMessageBuilder _Builder) {
			return new InsertEquityQuoteCommand(_Builder.m_Gateway, _Builder.m_Security, _Builder.m_Time, _Builder.m_BidPrice, _Builder.m_AskPrice);
		}
	}

The main difference between the two implementations of the Fluent Builder being whether or not the builder itself is mutable. This version is much more terse but the question remains for me of ... Do I care? I am generating them ... The one place there is quite a bit of benefit is in terms of performance if this is being called often in production code as it will create many less objects and the JIT could be more intelligent about inlining.

My original reason for making the builders immutable was for some odd edge conditions like

InsertOrderMessageBuilder AlreadyHasSecurityAndBroker = New.InsertOrder.ForSecurity("RIM").FromBroker(0);

InsertOrderMessage FirstMessage = AlreadyHasSecurityAndBroker.ToBuy(1000).AtPrice(15.00).AcceptingOnlyCash;

InsertOrderMessage SecondMessage = AlreadyHasSecurityAndBroker.ToSell(1000).AtPrice(15.50);

But after thinking about this a bit I could do something similar using a mutable builder ... I could extract a method that returned me the AlreadyHasSecurityAndBroker version of the builder to something similar to an object mother. I will have to think a bit more about this but as of now I am considering moving my immutable builders to their mutable brethren.

Another difference that we found in implementation was my use of the DebuggerStepThrough attribute. For those who are not familiar with it; it tells the debugger to not step into this method ... so if I have a long line like ..

InsertOrderMessage message = New.InsertOrder.ForSecurity("RIM").FromBroker(12).ToBuy(2000).AtPrice(15.00);

and I set a break point on it ... then hit F11 to step into the code, it will bypass all of the short "setter" calls and take me directly to the implicit cast which is generally where my problem would be happening as the builder doesn't really care about its state. I really like the way this works! It is a good tool to remember for scenarios like this

A side discussion came up with someone else in regard to why I define the internal state of the builder as internal and not private. The reason why I do this is because I write (read: generate, the reason why I generate tests is that these classes are generated then manually edited from that point forward) tests for the builders and want to check the fields one by one when issuing a set ... As an example of this kind of test consider:

		[Test]
		public void Should_Set_Gateway() {
			EquityQuoteMessageBuilder builder = new EquityQuoteMessageBuilder().OnGateway(ExchangeGateways.CNQToronto);
			Assert.AreEqual(ExchangeGateways.CNQToronto,builder.m_Gateway);
		}

I could just as easily put properties on the builder to expose its internal state or could make the fields public but when a builder is being used outside of these tests I don't want code to be able to access the internal state of the builder (I like to consider them write only). The need to have these things working properly kind of fits in with the want to have the debugger step through the code (since the code is simple and I have tests to make sure the simple code is working, I can safely guess that I wouldn't need to be debugging the code).

DDDD 5 [Messages have Fluent Builders]

Greg — Wed, 16 Apr 2008 03:11:42 GMT

In DDDD 4 I was lucky enough to get some commentary from an astute reader Jimmy Bogard, he wrote in his comment:

I think I'd have to respectfully disagree here. Valid state can be enforced in more ways than an immutable object. Immutability leads to everything supplied in the constructor. If I have coarse-grained services, messages can get fairly large. Suppose instead of Email, you had something like Order. Order contains LineItems (flattened out), maybe Addresses, ShippingOptions, etc etc. This is quite a lot to satisfy in an immutable fashion. For simple message, I like to make them immutable to satisfy invariants. But if I'm using things like XML Serialization, it's much simpler to have a plain DTO and a Validate method. I leave it to my Factory to satisfy invariants. As for other Value Object semantics like equality, I never got much use out of those. I never compared DTOs, as they were strictly used at the service layer boundary. Nothing interesting was ever done with them, as they are immediately used with a Mapper to get back to real domain objects. When I tried this concept (DTOs are Value Objects) a couple of years back, it started out nicely but broke down in our real-world scenarios of coarse-grained services. I chalked it up to my misplaced desire to model everything in our system in DDD concepts (entity, value object, service). Perhaps you've had a different experience.

The builder helps alleviate a lot of this pain while at the same time making our code more readable. I have talked about fluent builders in the past in A Use for Extension Methods but I will briefly go through how they work again. The basic parts of the pattern are to create a class that is immutable and has internally the same members that will be required to call the constructor of the target object. Properties and methods are exposed with fluent names to help build the target object. The target object's constructor is eventually called through an implicit cast operation. As is always the case code speaks 1000 words.

Let's take a simple message.

InsertOrderCommand ioc = new InsertOrderCommand(null, MarketSide.Buy, "B12345678", 0.12m, "NT",
TestDate, 1000, false, 100, 1000, false, TestDate, OrderDuration.Day, ExchangeGateways.CNQToronto, false, false);

YUCK!

Let's try rewriting that with a builder.

InsertOrderCommand ioc = New.InsertDayOrder

.On(ExchangeGateways.CNQToronto)

.FromBroker(10)

.ToBuy(1000)

.OfSecurity("NT")

.At(0.12m)

.WithUniqueOrderNumberOf("B12345678");

note that the builder does not always match up 1-1 with the object ... in this example ToBuy(Volume) sets that the order type is of type buy, it sets another field, and it sets the volume.

Phew that's a whole lot easier to read! Let's look at how to have this happen. First let's look at a message.

    [Serializable]
    public class EquityQuoteMessage : ILevelOneCommand, IHasSecurityInformation
    {
        public readonly DateTime Time;
        public readonly string Symbol;
        public readonly decimal BidPrice;
        public readonly decimal AskPrice;
        public readonly ExchangeGateways Gateway;

        public EquityQuoteMessage(ExchangeGateways _Gateway, string _Symbol, DateTime _Time, decimal _BidPrice, decimal _AskPrice)
        {
            Gateway = _Gateway;
            Symbol = _Symbol;
            Time = _Time;
            BidPrice = _BidPrice;
            AskPrice = _AskPrice;
        }

        public string GetSummary()
        {
            return "InsertEquityQuote: " + Gateway.ToString() + " " + Symbol + "  " + BidPrice + "-" + AskPrice;
        }

        ExchangeId IHasSecurityInformation.ExchangeId
        {
            get { return ExchangeGatewayIdTranslator.TranslateGatewayToExchangeId(this.Gateway); }
        }

        string IHasSecurityInformation.Symbol
        {
            get { return this.Symbol; }
        }
    }

Now ... we need a builder for this message.

	public class EquityQuoteMessageBuilder {
		internal readonly ExchangeGateways m_Gateway;
		internal readonly string m_Security;
		internal readonly DateTime m_Time;
		internal readonly decimal m_BidPrice;
		internal readonly decimal m_AskPrice;


		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder OnGateway(ExchangeGateways _Gateway) {
			return new InsertEquityQuoteCommandBuilder(_Gateway,m_Security,m_Time,m_BidPrice,m_AskPrice);
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder ForSecurity(string _Symbol) {
			return new InsertEquityQuoteCommandBuilder(m_Gateway,_Symbol,m_Time,m_BidPrice,m_AskPrice);
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder At(DateTime _Time) {
			return new InsertEquityQuoteCommandBuilder(m_Gateway,m_Security,_Time,m_BidPrice,m_AskPrice);
		}

		[DebuggerStepThroughAttribute]
		public EquityQuoteMessageBuilder WithSpreadOf(decimal _BidPrice, decimal _AskPrice) {
			return new InsertEquityQuoteCommandBuilder(m_Gateway,m_Security,m_Time,_BidPrice,_AskPrice);
		}

		private EquityQuoteMessageBuilder(ExchangeGateways _Gateway,string _Symbol,DateTime _Time,decimal _BidPrice,decimal _AskPrice) {
			m_Gateway = _Gateway;
			m_Security = _Symbol;
			m_Time = _Time;
			m_BidPrice = _BidPrice;
			m_AskPrice = _AskPrice;
		}

        	public EquityQuoteMessageBuilder {}
		
        	public static implicit operator EquityQuoteMessage(EquityQuoteMessageBuilder _Builder) {
				return new InsertEquityQuoteCommand(_Builder.m_Gateway, _Builder.m_Security, _Builder.m_Time, _Builder.m_BidPrice, _Builder.m_AskPrice);
		}
	}

I have written a generator to make skeletons of these builder classes that I may post which simplifies this process greatly (especially if you use resharper ctrl+rr or f2 everything else is done) but I will only release it if people promise not to make fun of the code as it was written in an afternoon just to save time when writing a few and looks like it was written by a second year student :-)

Aaron over at elutian uses a structurally identical pattern (except for the additional use of extension methods in A Use for Extension Methods) in his post Fluent Fixtures the key difference between what I call a Fluent Builder and what he calls a Fluent Fixture is in the intent. Aaron uses his fixtures only for the initialization of setup data for tests which they are EXTREMELY valuable for (especially when dealing with messages as it produces a soluble method of expressing what is in the method) ... My fluent builders are used not only for the initialization of test data but also for the creation of real objects (I might even go so far as to say that the construtors for messages should be made internal and you should only expose builders though in some extremely high performance conditions you might want to actually use constructors).

Jimmy Bogard hit the nail on the head in that complex messages can be a nightmare to build immutably because they get really big. What we do to get around that is we use an mutable builder that we then cast to an immutable message. This gives us the best of both worlds although the builders can be annoying to type, since essentially we have made explicit a transient mutable version of the message and a fixed immutable version of the message.

To be more clear, let's try a bigger example like the one that Jimmy mentions and see how bad it is to create a bigger message. I will use a DTO example since they tend to be the offenders

OrderDTO order = New.OrderDTO
        .WithId(SomeGuid)
        .ShippingTo(New.Address .....)
        .AddLineItem(New.LineItem.For(12).Of("Product 1").At(15.99)
        .AddLineItem(New.LineItem.For(1).Of("Product 2").At(22.97)
        .AddLineItem(New.LineItem.For(15).Or("Product 3").At(15.22)

naturally you could use a loop etc when building the message (and sometimes in terms of test fixtures you may find your self actually referencing the Builder then later adding more items to it similar to how an object mother would work but this is another post)

Of course we can go a different route than this. You will notice that a natural break is occurring in the setup of our message between the root and the pieces under the root. What if instead of trying to create one really big message we created a few smaller ones just like how our actual setup is working. In other words what if instead of having an OrderDTO as one big message we introduced a container to hold smaller messages and relate them as being one something like:

SnapShot

CreateOrderMessage

AddLineItemMessage

EndSnapShot

This would help prevent us from having to deal with the larger items but each method has their own benefits. The main benefit of the first method is that its schema exists within the message. Especially when dealing with external clients or other types of integration points this can be extremely beneficial (there is no need of logic to figure out how to build the data). When combined with something like XML this can be extremely powerful.

The second method can also be extremely powerful in a closed environment for a few reasons. In many circumstances it is actually simpler to send over these smaller pieces, the big problem with this methodology is that you need to have domain logic existing on the client in order to know how to process these smaller messages to get the client to a state matching what the originator of the message intended. As we will see later on though, in some circumstances (especially when we want disconnected access) some of this logic will be distributed for other reasons anyways so it may become a viable option.

DDDD 4 [Messages are Value Objects]

Greg — Tue, 15 Apr 2008 00:51:19 GMT

This week I may take a break from these posts for a few days due to the Summit and Alt.Net but I should still push a few posts...

Messages as we will see are an interesting breed of objects as the same object will sometimes have differing connotations in different sections of code. In the domain messages are to be treated as value objects even though they may be treated differently later by certain types of infrastructure. There a a few aspects of being a value object that are imperative to a successful implementation of a messaging system within the domain. By looking at the general rules for value objects we can derive many of the rules for messages within our domain.

Messages ARE immutable
Messages should vary in equality based upon the data they hold
Messages should have fluent builders

Messages ARE immutable...

I don't think that I can stress this point enough. Mutable messages are an anti-pattern. They are the path to a system that is held together with duct tape and bubble gum. There are many reasons why this is the case but the simplest reason is that you suddenly end up with different semantics when you cross a serialization boundary.

Let's imagine for a moment the ridiculous case of a simple service that receives a message, alters a field in it then sends an ack that it completed its work (the caller keeping a copy of the message checks for the changed field when it receives the ack). Even worse it does something to the message that causes the message to say raise an event back to the caller. These sound like ridiculous cases that no one would ever do but I have seen both done. Either will work just fine so long as you are in the same process but will break once you introduce a serialization boundary.

Let's make an additional rule to get help this issue (please,never need this rule): You should never hold a reference to a message after you send it. As far as you are concerned, once you dispatch it, its gone. We will learn later that if at all possible we also want to limit our exposure to any shared state between the message sender and receiver as it is this shared state that limits scalability.

Another reason that mutable messages are evil is that you never actually know that the message is in a valid state. The use of immutable messages introduces a stronger contract for your messaging as you know that if a message exists that it is also in a valid state. You will never have a SendEmailMessage that is missing a To and From unless you specify that that is a valid way of sending an email in your system. Because you know that your messages are in a valid state you can in the case of using a shared library for messaging fail early on your requests i.e. instead of whatever processes the SendEmailMessage saying "Hey guy you need to actually send me someone who this email is to!", the thing sending the message fails to create a valid message. This is hugely beneficial because what if the thing processing the SendEmailMessage is actually on another system? Not only have we transported an invalid request across the wire but we now also have to route an error back to the sender!

What we have done by making our messages in an assumed valid state is allowed other code to operate upon them while making assumptions that the message is meeting its minimum contract. In other words, the thing processing the SendEmailMessage does not need to actually check that the fields in the SendEmailMessage are what it needs them to be unless it is requiring something that is above the specified contract of the message (let's say that it requires something the message defines as optional). The SendEmailMessage has been given the responsibility of insuring that it is always in a valid state.

A side benefit of this is that when dealing with messaging to infrastructure services the invariants you maintain to them are part of your messages which are part of your ubiquitous language as opposed to being implemented in the code handling the messages (infrastructure).

I like to think about using validated messages in this way in relation to design by contract. My code that sends the email doesn't just work on a message that defines data layouts, the message also defines invariants about my data! These can be quite simple (To is not null) or they can be extremely complex like To must be a valid email address that is not at gmail.com. This becomes extremely valuable later when we go to deal with the consumption of messages; one of the biggest problems with interfaces and testing with .NET is that you cannot specify enough information to narrow your contract. You should therefore to be precise test the entire surface of the interface for every implementation (yuck, these tests add up). It is far easier to introduce the message that defines the invariants and then assume them to be true in your code.

Quite quickly we end up thinking about a contract first method of development for these types of things. In other words that we create our message objects before we create the code that sends or processes them. I personally like this method of development a lot for messaging based systems.

Serialization can cause issues with your messaging if you deal with un-trusted clients (versionization problems can cause this as well) as if you use default serialization it can (and will) deserialize messages that are not valid according to say the logic you apply in their constructor! If you are running with untrusted clients you should implement ISerializable on your messages to take control of the serialization process for them so you can insure they are only created in a valid state. For a large system you may want to consider moving this behavior to a "message" layer super type or consider mixing in an interface/implementor pair for ISerializable if you are using compile time AOP

We will look later why but never use runtime AOP on your messages or on the handling functions, its slow, redundant, and way too complicated.

Messages should vary in equality based upon the data they hold

Why someone would feel the need to compare messages for equality in a domain context is completely beyond but .... If you do, do consider them to be varying based upon the data that they hold not upon their reference. Beyond that if you can come up with a reason to do it I would love to hear it :-)

Messages should have fluent builders

This is something I believe may be controversial and this post is getting a bit long so I will finish this part as DDDD 5 [Messages have FluentBuilders] on the bus to the summit today.

DDDD 3 [What/Why this]

Greg — Fri, 11 Apr 2008 22:00:25 GMT

I jumped right into DDDD 1 without really explaining what this is or why I am doing it. I am going to break a day to explain my reasons for posting this information and what I intend to go through.

Those who have had a beer with me at a summit/conference/etc may know that I have been dealing with the unusual combination of messaging and domain driven design together for probably about four years now though there are bits and pieces that are even older than that. Its a favorite discussion topic of mine over a beer... Over the time I have tried many different schemes of getting this working (many of which were drastic failures in my mind) that eventually became workable solutions, each with its own strengths and weaknesses.

I presented some of what has been come up with at QCon last year but to be honest I don't think it was fleshed out enough yet at that point. I wanted to drastically change my slides only a week before the presentation! I have been trying since QCon (about 6 months) to get all of these thoughts put together into a cohesive paper; the problem is that paper quickly becomes more like a book and I find myself rewriting it constantly in search of cohesion. I have probably typed up about 1000 pages of information yet only have about 45 pages to show for it and right now I am still extremely frustrated with those 45 pages.

To me a passing thought style delivery is much easier for me to deal with as I can reasonably maintain cohesion over a few pages of information. As such I have decided to just begin throwing up my thoughts / realizations on the subject into blog posts. Hopefully at the end I will have learned a lot by posting all of the material as I find trying to get things written down often forces me to distill them in my own mind to simpler versions. I warn you they may be scattered and/or difficult to follow and may not necessarily be in what "most" would be deemed a correct chronological order. I am unfortunately not a person who can easily sit down and write what is probably a books worth of information in any sort of cohesive manner, nor to I particularly want the stress of trying to do so.

Beyond that because much of what I am doing has been done in environments that I have been in I worry about the generality of some of the things I have come across with. These posts also serve as a great way for me to get feedback on those types of issues. I would imagine someone who is reading such a post set probably has at a minimum an exposure to DDD or to messaging; I encourage feedback if you think something that I said is 'full of it' because frankly; I may be!

When I am done with all of these thoughts I may attempt to seek someone to help me make them more coherent and linear to unify them into one piece of material that can be more easily consumed ...

That said I will do my best to keep cohesion, this is much easier to do when the code starts coming into play ... Although I do not have an exact outline for what everything will be here I will be releasing the associated code into a SVN repository that is built upon with each post and that by no way means I will be using vertical slices ... I am now looking more at growing that code slowly and organically by attempting to solve new situations brought in through these posts. At the end it should be at the least an interesting bit of code for exploration of the concepts.

Some have asked that I start pushing up large amounts of code and discussion early, things like how the messages get dispatched and processed. Over my experiences the largest thing I have learned is that how these things happen is very much dependent upon the needs of the application (especially things like levels/regions of trust, disconnected data access, networking boundaries, and the need for persistent subscriptions). Throughout these posts I will be giving all of the building blocks as well as guidance for how you may want to use them in different scenarios but there is no one size fits all solution (no matter what BizTalk people tell you ;-)) As I said last night to someone ... messaging is like legos, it only requires a bit of creativity to make something not in the picture.

As for what I intend to post, let me start by saying; there will be ALOT of posts, months worth. I would not be surprised to see DDDD with a three digit number next to it.

For the first month or so of posts I think most things will be conceptual but it will include the introduction of a simple in-process messaging framework. I want to look in particular at things like using messages and a fluent interface instead of using DI for domain/infrastructure services.

After that I want to continue into some other more advanced areas of distributed meshing, some things I want to cover include basic client server interactions, dealing with data warehouses, peer meshing, load balancing, distributed locking, consistency, versionization of clusters, pure pubsub environments, and much more. To be honest I have no idea how long these will take but I expect a pretty large number with frequent posts.

Thrown in throughout these will be other posts that pop into my mind like Application Sagas vs Application Services, Aggregates as Sentries, or my next post Messages are Value Objects. These tend to be more so on the DDD side where the messaging side is deliberately removed. One of our goals in moving our domain to using messages is to make it care about LESS infrastructure, while the messages themselves may be first class citizens in our domain the infrastructure supporting them is not (nor is it really referenced in the domain).

I apologize in advance for the lack of cohesion in these posts but hopefully after writing them all I will have a better view of how to make this all cohesive (I hear putting all of the titles on index cards and moving them around on a wall works pretty well but would love to hear any successes people have had with things like this).

DDDD 2 [Messaging and the Ubiquitous Language]

Greg — Fri, 11 Apr 2008 04:33:00 GMT

Yay!! 2 days of blogging at lunch (then at home, time crunch today so excuse the hurried format) in a row, keeping on track. In DDDD 1 we looked at boundaries where messaging is applicable / good. We will in the future look more at why we would want to message on those boundaries but first it is time to see how messaging may impact our ubiquitous language which is imho the most important part of Domain Driven Design.

The largest place that our ubiquitous language is changed is in dealing with infrastructure services within our domain. As has been written about Java, C# tends to be the land of the nouns. Unfortunately this creeps into our domain and we end often with many nouns in our language that only represent gateways to verbs. An example may make this more clear ...

Do you have an EmailGateway, IEmailGteway : EmailProvider, IEmailer, DomainContext.Current.EmailProvider.SendEmail(email), etc in your domain? Is it part of your ubiquitous language? How do you discuss the action of SENDING an email? Why is it that this verb has become an interaction with a noun?

I have seen some go much further than this as well under the guise of loose coupling (read testability). I have even seen people going to the point of injecting things like email providers directly into domain objects. This is a great example of technical details leaking into the domain (even with dependency injection).

Domain Driven Design is not all about nouns; it is more so about verbs (and some very important adjectives/adverbs ;-)). Verbs are the most important parts of our ubiquitous language as they define interactions which in turn defines our nouns, consider this in comparison with active record where the process tends to be reversed. The introduction of nouns such as EmailProvider to support our verbs such as SendEmail not only is artificial but it also distracts our attention from the domain verb of sending an email and focuses it on the noun that is doing it. There is no need for this noun to be in our ubiquitous language therefor there is also no reason for it to exist in our model.

By replacing the interaction with a Message we gain in terms of our Ubiquitous Language. Let's imagine a contrived example of printing a purchase order to a file at the completion of it being created. Instead of introducing a FileDocumentPrinter that has a method Print<PurchaseOrder> on it that we would then have to deal with the injecting of ... We would introduce a first class domain concept of a PrintPurchaseOrderDocumentMessage ...

One valid argument here is that PrintPurchaseOrderDocumentMessage is still a noun but in conversation it loses its suffix and becomes PrintPurchaseOrderDocument. We have quite a few of these in our domain InsertTrade, CancelOrder, MarkOrderAsKnownTrader. One could make the argument that they should be renamed without the suffix but I have found that it can be confusing for developers to not have the suffix. Another idea I have been working with is to instead give a prefix such as Do for these style messages or to use a fluent builder ala A use for extension methods and prefix it with Actions. then have the creations work with the verb.

By replacing with a message we have removed the nouns from our discussion, and at the same time lowered our coupling to the actual services involved. We can at runtime bind up our infrastructure and our domain (and ubiquitous language) are blissfully unaware of the mechanism that we use to hook it up.

Another interesting note here is a subtle difference between the Ubiquitous Language and the model implementation is the synchronous vs asynchronous behaviors of any method based approach. When do I consider it ok to continue in terms of the printing being done? Is it when the document is done or when I told it to do it? This can lead to bugs in the implementation as well as a much less fleshed out domain model. Messaging makes this interaction specific in the Ubiquitous Language, if it is not waiting on a reply the action is considered to be asynchronous.

Later ... I will come back to this topic in terms of Repositories and making the implicit explicit but there is alot to cover first.

DDDD 1 [When to Message]

Greg — Wed, 09 Apr 2008 21:49:00 GMT

Just as a bit of a preamble, today is the first of what will hopefully be many days of me knocking out my blog backlog by actually taking an hour lunch and bringing my laptop to write. Unfortunately I currently have a backlog of over 40 posts ... Hopefully in a few weeks that number will come down a bit. This is the first in many posts about DDDD (the D is for Distributed), oddly when talking about it here the extra D seems to make sense just about everywhere so I will leave it to you to figure out which one it is...

I recently posted about using messaging in the domain in Mocks are a Code Smell. One item that I left out of that post was when one should be using a message or a direct call. I have over time found boundaries that work very well for me at least ;-)

Services

DO use a message any time you call a service from any where.

Services are the most natural (and easiest implemented) boundary for messaging. Services generally have a clear interface that is easily changed to become messages instead. It is also quite easy to route these service calls as you can usually just route the messages based on the type of message. Ex:

When I have a SendEmailService I can route all SendEmailRequest messages to it, in terms of dispatching; this is extremely simple and has about the same overhead as using DI but I have greatly reduced my coupling. As has been seen in Mocks are a Code Smell it can also make my tests more explicit.

When moving to DDDD this allows you to load balance and/or distribute services in a transparent manner.

Repositories

DON'T message to your repositories ...

Not a lot of explanation here ... Just don't do it. I will go into more later (in another post) about how repositories and messaging can interact.

Aggregate Roots

DO use a message anytime you are in an aggregate root and make a call out of the aggregate root

DO use a message to access functionality in or under an aggregate root.

NEVER use a message when you are within the same aggregate

Aggregate boundaries are one of the most important messaging boundaries when introducing messaging to a domain. Aggregates should not be considered groups of related objects but in terms of messaging should be treated as a single functional unit (update: this also applies to little helper functions etc that the aggregate calls).

This becomes especially true when you move towards DDDD. Aggregates exist within the mesh in their entirety, server 4 will either have the entirety of the Customer aggregate for customer 17 or it will have none of it, it will never hold "parts", this does not mean that the entirety of the aggregate will be loaded (it may still support lazy loading). Not only does this greatly simplify the routing of messages within the mesh it also helps to prevent chatty interfaces.

All messages coming into an aggregate should be destined to the aggregate, never to a piece of the aggregate. The aggregate should be responsible for coordinating the messages it receives to its contained parts. By dispatching messages to aggregates we have less information to store for routing purposes, locate by aggregate. This also fits directly into the concept of an aggregate in DDD in that it should be controlling access to all objects within its boundary.

Bounded Contexts

DO use a message anytime you cross a context boundary

DO make messages between context boundaries a format that is specific to neither Bounded Context

Consider messaging between Bounded Contexts to be a key Integration point.

Context Boundaries are one of the places that messaging shines in any system although this could be included in Domain Service in many cases because interactions generally sits behind a service. Messaging introduces what is essentially a very thin and loosely coupled Shared Kernel (or perhaps its more of a Core Domain) between the Bounded Contexts. It is key that when you develop these messages you treat them as the global interface to the originating Bounded Context, any other Bounded Context will be accessing the originating Bounded Context through this interface.

Anti-Corruption

The anti-corruption layer when introducing messaging across Bounded Contexts is especially interesting, there are two common ways of implementation. In either implementation the originating Bounded Context translates its own concept of data to the shared messaging contract. For many cases with small amounts of interaction this is actually enough to keep the contexts decoupled and the receiving Bounded Context can deal directly with the messages.

If this becomes a bit burdensome it can be beneficial to have the receiving bounded context translate the shared messages it receives into its own concept of the messages.

Application Services

You may use messages for talking to your application services.

Since if you follow the rules above your application services will use messages to talk to the domain there may or may not be a gain in using messaging to talk to your application services. One place that it can be a gain is if you are sharing your API with others as your application services would generally match up better with a SOA type API.

There is another side of this discussion and that is you message TO your application services and ONLY to your application services (i.e. each instance of the application services is its own unique instance of the domain). This can work very well if you want your domain to be primarily Request/Response but it begins to fail very quickly if you want to do more pub/sub style interactions with the domain objects that live behind your application service facades.

I have actually been liking lately the idea of distributing application services to all clients and having those application services generate messages to the other pieces of the system. This can at times be chatty but later we will discuss ways to work around that.

to be continued ...