Dru Sellers

Sponsors

The Lounge

Wicked Cool Jobs

Advertisement

Why Messaging #4 - So hows it going?

Its been over a year since I went down the messaging path and how has it been? What have a learned? What has sucked? The following are some thoughts on my experiences with deploying MassTransit

Although explaining the benefits of a messaging based system has been easy, trying to apply some of the new paradigms that Udi and Greg Young have been trying to pound into my skull has not been so easy. This includes a slew of things like message idempotence, worrying about message timeout (and elevating its importance in the whole system), what to do when a message doesn't get where its supposed to go, figuring out how to rerun a messaging system, and others has been a bit of a mind warp. And figuring out how to test a messaging system...that took a while too. My team has pushed pretty hard in these areas and although it took a little while I think we are starting to see the seeds of our effort. I guess the take away here is that its just going to take time

I know that I am still in the early stages of use for this pattern / style of development and further more as a developer of the framework I can get a bit excited about using it, but at times I think I am treating the tool a bit like a 'hammer and that everything is a nail' but its hard to know when it makes sense and when it doesn't. My hope is that I continue to experience more and more use, that I can find that line sooner rather than later. Its like, oh that could be a useful event that the accounting system is closed. Then we build it up as an event and broadcast the account totals out to our systems that are interested. Right now we have just one system and my coworkers and I kinda stand back and say "Really?" that could have just been a file transfer. I know though as soon as we get that second system we will be very happy.

Another thing that seems to be happening is that I am writing an awful lot of windows services. One to broadcast the message and one to receive the message. Not really sure if this is good or bad or anything, but it did allow me to sit back and create Topshelf an OSS library for writing windows services (imagine that), and from what I have heard it seems to be working out quite well for people. But its something that is new to me, and I just don't have the experience writing that kind of code. But it seems to be going well, it just adds a lot of stuff that I am not used to watching. Thankfully log4net is quickly becoming a very near and dear friend.

As I imagine writing all of these services, so that I can integrate all of my organization I am also starting to wonder about how in the heck I am going to manage this multitude of services. Documentation will help alot, but I am lazy and don't want to write documentation. So how can I expose information about our systems with out having to keep it in sync with the word docs? This is where I have started to look into the concept of metadata. Basically, i want an MT service to tell me something about itself. Where is it installed, how many messages is it handling per minute, error count, etc. I feel that if I can get this baked into MT and exposed to us in some sort of dashboard, managing this beast will get a lot easier.

Well, I hope this was helpful. I will try to write another one when it seems to make sense.

-d


Posted Thu, May 14 2009 9:40 AM by drusellers

[Advertisement]

Comments

Stacy wrote re: Why Messaging #4 - So hows it going?
on Thu, May 14 2009 3:34 PM

Take a look at Sql Service Broker for your messaging needs. Works great. No need to re-invent the wheel! It does have a learning curve but it's worth it. I use it. MySpace uses it too on 60 db servers  as an alternative to the built-in replication.

Paco wrote re: Why Messaging #4 - So hows it going?
on Thu, May 14 2009 4:32 PM

@Stacy: I would recommend anyone to get the as far as possible from a that. Many of us have tried the database hell before...

drusellers wrote re: Why Messaging #4 - So hows it going?
on Thu, May 14 2009 5:18 PM

@Stacy: Thank you for the idea. Although I am not an expert on Sql Service Broker I am concerned about the testability of such a system. Can I run it through MbUnit? I am also somewhat concerned about the future of this system, as it is a child product to Sql Server itself will it stay around? Also, as I said before: I am a big ol' nerd and wanted to write something myself to better understand the problems we are trying to solve. :)

bradtwurst wrote re: Why Messaging #4 - So hows it going?
on Thu, May 14 2009 9:21 PM

Dru,

One of the projects that I am keeping an eye on is the Managed Services Engine (MSE) at servicesengine.codeplex.com

Don't know if it will be useful in itself, but there appears to be some interesting concepts.

Stacy wrote re: Why Messaging #4 - So hows it going?
on Thu, May 14 2009 10:20 PM

@dru I only do integration testing with SSB. Not sure about MbUnit. Yes, I'm concerned about it being orphaned too, but it sure saved me a ton of work in a pinch and upgraded in sql2008.

@Paco are you referring to sql2005 in general or specifically SSB? Complex messaging can be a lot of work to roll your own and maintain it. However, once I get a break, I plan to look into Erlang for my needs.

Joshua Flanagan wrote re: Why Messaging #4 - So hows it going?
on Fri, May 15 2009 9:09 AM

If you haven't already, I would look into performance counters and other WMI mechanisms for publishing data such as error count, messages/sec, etc. There are already existing tools that would be able to consume that data from many services scattered around a large organization.

dru sellers wrote re: Why Messaging #4 - So hows it going?
on Fri, May 15 2009 4:08 PM

@Josh: hook a brotha' up with some sweet sweet WMI code

Phil wrote re: Why Messaging #4 - So hows it going?
on Tue, May 19 2009 7:45 AM

You could set your services up as WCF clients and then have them register themselves with a WCF server hosted inside your central dashboard application.

Colin Jack wrote re: Why Messaging #4 - So hows it going?
on Mon, May 25 2009 8:40 AM

"And figuring out how to test a messaging system...that took a while too"

Be very interesting reading more about your preferred strategies.

"This is where I have started to look into the concept of metadata. Basically, i want an MT service to tell me something about itself. Where is it installed, how many messages is it handling per minute, error count, etc. I feel that if I can get this baked into MT and exposed to us in some sort of dashboard, managing this beast will get a lot easier."

If you haven't read it already I'd recommend the transparency section at the end of Releast It, also are you currently using performance counters or something else for some of this?

Qjqhnnbf wrote re: Why Messaging #4 - So hows it going?
on Tue, Jul 14 2009 7:56 PM

cD4U3K

Add a Comment

(required)  
(optional)
(required)  
Remember Me?
Devlicio.us