After reading the article my head hurts... from all of the nodding-in-agreement I found myself doing through out.

I'm dealing with this very issue right now, only difference being we're building a WinForms application, and it's causing us all kinds of pain. The extra hoops we have to jump through to expose EVERYTHING via the service are just ridiculous... and for what? A hypothetical 2nd consumer of the service that _might_ come around, maybe, some day?

Solid post Jeremy... keep fighting the good fight! ;)

You've been kicked (a good thing) - Trackback from DotNetKicks.com

Excellent post on something we all face at one point or another.

Nothing to add here, you got it nailed.

It sounds as if the environment you describe is more adult than the one I currently exist in, but the missing reason from our environment is "because all the other kids are doing it."  And before someone assails me about "Morts," I'll point out that your last line -- performance plan inducements for web service creation -- is the second verse of the same song, but sung by management:  we have to have web services "because everyone else is doing it."  Not a hint of ROI in the bunch.

Wow this sounds familiar...a service was created, contract agreed to months ahead of actual use, and lo and behold, when we actually tried to use the service, it didn't meet our current needs.  It met our imaginary needs of 6 months ago, but not the actual needs of today.

The service team attributed this to us "not having detailed requirements".  Too bad the service team is a 6 month long waterfall dinosaur that requires everything in BDUF requirements docs.  Scrum is hard to make work with dependencies on non-agile teams...

I like the Web Service option for just the reason I like MVP/MVC - it force you to have a cleaner design ...

For example, I don't have to worry about the existence of DB in the caller application ...

Reusablility is just a welcomed side-effect.

BTW, I've seen people use the same arguments to against MVP/MVC ....

This is an interesting decision to make, and I don't see a clear answer to either side. The more import thing is to do it right instead of do it one way or another ...

The only counter-argument I can think of (and it's only for the sake of argument because I agree with you wholeheartedly) is that such a practice discourages the use of web services where it actually does have some benefit. You mention that we could easily switch to a web service tomorrow when the next project comes along. But that assumes that people on that project know about the previous one and how it is architected.

The counter-counter to that is that the web services need to be effectively organized and published, which is a pretty tall order in and of itself.

Interesting post.

This is exactly the kind of architecture that we have developed here at work except we chose the 1st version and I will defend it as being the right one. The points that you have made are very valid although I don't see choice number 2 as less SOA then choice number 1. What you describe is a choice between an in-process library versus an out-process library as described in this very good book: astore.amazon.com/.../002-0126298-6964872

Architecturally they are both equally SOA compliant.

I also very much agree with you that the architecture is more complex and it's more difficult to test against since now you need to test the process layer and the UI separately.  I don't agree with you on the building part. If you have a good build tool then this becomes a non issue although obviously the time it takes to do the build will be longer. I also don't agree with you on the maintenance, by design the contract between the UI and the process layer is not loose coupled and is not meant to be reused. (The core business functionality is a different story. ) For each channel you will need to develop a facade in front of the core business services or the existing process services.  So whether you have this as an in process library versus an out process library if you need to change the contract the impact is the same. There is less impact on the deployment with option number 1 because if you want to reuse the web service then all you need is to redeploy the web service. For option number 2, all the applications that use the in process library need to be updated, although from your statement I believe that you would choose then to move to option number 1. There is another point that you failed to mention is that the development is very much related to the project at hand. If you have the budget and the time then why not save yourself the trouble of having to do this later on (especially since you know that you will not be allowed to do this in the future) and surprise your employer by giving them a quick win with a ready made service.

I fully agree with you that from an agile point of view option number 2 sounds more reasonable. I believe that maybe this discussion should be more about development methodologies and not architecture since they are both right taken in different contexts.

"OPPORTUNITY COST!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

For the simple reason that the extra time you put into the web service that adds NO BUSINESS VALUE could have been spent on other efforts that ADDED BUSINESS VALUE!!!!!!!!!!

You do NOT know that you'll need this later.  And why would it be cheaper to do it now?  Part of my point is that:

a.) Even if it's on the roadmap for later the roadmap could change and it would end up being unnecessary

b.) You'll know more about exactly what web service you need to build when you're at the point that you need the web service."

You are 100% correct in the points that you have made. My problem is that most of the time you don't get a chance to move to a web service after the project ends. When a new project comes along if you tell the project manager that you need to modify an existing project to enable it as a web service then they'll see it as something that's not related to their specific project. It is overkill to enable everything as a web service (and certainly that should not be the standard) but if you know that there is a chance (say 50%) that it will be used in the future then I gamble on the web service given the constraint I outlined above.  

"If you accept the SOA definition of loose coupling, you're right.  Using the older OO definition of loose coupling is a different story.  As long as I'm conforming to SRP and the Open/Closed Principle, I should be able to take the exact same business logic & persistence classes and use them *unchanged* to make up the new web service.  Sure, I'll have to build a brand new remote facade over the top of them, but that's additive code and I'd argue that that's not that big of a deal.  Adding new code is far, far less risky than changing old code.  A good design and good separation of concerns all the way down to the class level can make the code more reusable and get us to Open/Closed compliance.

You didn't say otherwise, but I would add that I would never try to expose the business objects in my domain directly via any sort of remoting or web service.  I would always have some sort of remove facade."

I think we miss understood each other on this, what I wanted to point out is that there is nothing wrong with changing the process contract as long as the business services consumed beneath don't change. The logic should not be tied to the contract, it's just an entry point. One of many. Also I agree with you that you should always have some sort of facade over your business services that maps to your business process instead of the underlaying system.  

"If I had to change later to build the web service, I'd probably just host the web service in the ASP.NET application to reduce configuration overhead.  For me, using IoC containers is effectively mandatory, so changing the ASP.NET application to switch to using a web service instead wouldn't hurt all that much."

And that's exactly what we have done as well. Except the process layer and the data binders are hosted as out of process services instead of in process. :)

Seems to me a service with only one consumer (and on the same box no less) is a fairly fragrant architecture smell.  Web services have a significantly higher maintenance cost in versioning, backwards compatibility, and higher cost of change than POCO's.

I have to deal with many of these "just in case" services as both a consumer and a provider, and it stinks on both ends.  Either you have to consume a contract that isn't what you need or you have to expose a contract you know doesn't work but costs too much to change.

This reminds me of the outsourcing arguments too.  Everyone jumps on the potential savings but completely ignore the significant overheads these approaches incur.  Sounds like McConnell's classic mistakes #33 and #34.  There's a middle ground between over- and under-engineering, it's called YAGNI.

Hi Jeremy,

I'm dealing with a similar approach at my workplace. The standard architecture insists on two (2!!!) web service layers between the business logic DLL and client. I disagree with most of their rationale (SOA, reusability etc) for the same reasons you mentioned in your post.

The rationale I have slightly more trouble arguing with is their security one. They run the web service layers with different identities (and separate boxes on different, firewalled network segments) and use integrated authentication to secure the calls between layers. The web services themselves are just proxies through to the business logic DLL.

The theory is that owning any but the final box (we host internal and external facing stuff) will still only expose a limited interface to the system. Now this approach made me choke for so many reasons its not funny -- but I can't argue that this approach does not provide additional security. And any suggestion that the additional security may be a tad on the side of overkill isn't taken particularly well.

Any ideas on saving me from developing a script to generate these web service layers and the deployment nightmares of setting up 3 webs per application over multiple boxes?

Jeremy, your argument for option 2 makes sense and is perfectly logical. However there is one serious flaw: it is not buzzword compliant. Haven't you heard? It has to be SOA, SOA, SOA, SOA anything and everything that moves. What are you going to say when the Architecture Board asks you if your architecture is using SOA? No??? bzzzzt. Wrong answer.

"To really be Agile in your design and delivery efforts, almost all production code needs to be built with solid design principles".

'solid design principles', is that a product? When is it coming out?

I'm with you and also not with you on this.  One of the benefits of having the web services or using .Net remoting is to allow for scalability also.  Although you can load balance and just through more and more hardware what you end up with is adding hardware for both the UI rendering and the business logic all at the same time.  By having the seperation you can allocate more expensive hardware to the piece that needs it saving money.  

That being said by simply building the libraries correctly you do buy the ability to migrate to any needed architecture down the road.  The key is always in how flexible the design really is.  In most of my scenarios there is always a need for communication between the system I'm designing and some third party system so it is easy to always pick Web Service architecture.  

One of the things I've always been interested in is performance vs. development costs.  I think I have my next blog.  I'll have to come up with a scenario to test perfomance of library objects versus the cost of web services.  Thanks for the post you've driven me back to thinking about architecture and anti-architecture again.

I can't agree more.

I dream about having you as consultant in my office

Good article, I agree 100%.

I like what you say about Team Composition as I believe that architects and programmers belong together.

dcadenas.blogspot.com/.../architects-and-programmers.html

Hi Jeremy,

I'm an Architect at a big Fortune 500 company, who happens to make a lot of money. Somehow I stumbled across your blog, and feel compelled to comment on your argument. The way I see it your argument essentially boils down to this: Option 2 provides greater ROI than Option 1. Disregarding your technical reasons (which few would listen to and even fewer would understand), this argument can neither be proved or disproved. The only way to 'prove' it would be to implement Option 1 and 2 in two identical companies in exactly the same circumstances.

Since it can't be proved, the default option will be 1, the SOA approach. Why? Because option 1 is backed by Microsoft, IBM, Gartner etc...while Option 2 is backed up by a bunch of bloggers such as yourself.

Now, technically your argument may be correct but we won't know that for a couple of years when the inevitable SOA backlash begins. This makes no difference to me since I would've then moved on to the next 'enterprisey' thing which helps corporations escape their 'SOA Integration Hell'. All I have to do is update my powerpoint presentations.

Keep fighting the good fight. I'll be rooting for you.

My choice is also option 2. But I prefer to build a thin service layer (not a web service) between the business logic and the website logic.

If another project ever needs the same functionality, its easy to build a web service on top of the service layer. And because this service layer is already refactored many times during the development process of the site, there is a good chance is it a good starting point for the interface of a webservice.

OK .. you definitely sound like your looking for an argument or at least a discussion rather than everyone just agreeing with you so here goes.

Your whole article is based on the theory that your architecture buddy is working in a sensible, grown up environment .. and on that premise there's nothing I can fault on the theory front. And I'm assuming that the article is aimed at being educational so preaching the theory is fine and good.

However I think it's pretty true to say that most software development environments are dysfunctional to a lesser or greater degree. In which case politics, lack of knowledge and just plain stupidity on the part of just about everyone can all come into play.

In a scenario such as this then the architect may be valid in pushing his technical requirement (to have a web service) as having equal importance to other requirements of the project. Why? Because although the development team  may well produce a system that offers an easy transition to a web service at a later date (because it adheres to separation of concerns etc etc) she may never get the business backing to actually do it .. not for any good reason just pure and simple politics.

I've worked at places where the technical requirements are just not given any kind of worth .. and "the business" decide on everything. Unfortunately a lot of the time "the business" don't know how to build software systems. In places like this you are always faced with some pretty grey choices.

So I have a question: is there any scenario where you would consider option 1 as the correct response?

Or to open it up even further .. would you ever deliberately break Agile guidelines (such as YANGI) for strategic reasons? Or do you believe Agile is always right .. at every level, every time?

Oh and the usual Agile cop out of "I'd never work there" is not allowed and neither is the observation that such an environment is not Agile and therefore doesn't count ;)

The problem is that most people that reads this blog, agrees with the agile/lean architecture.

You are preaching for the believers...not the infidels. ;-)

I have a system where we needed Social Security numbers from an old COM application (It was a remote proxy). At first we included the COM references into the project, with all the dll hell and tightly coupling problems when we tested.

So we decided to create a web service on top of the COM application, effectivly creating 2 systems (1 for the business domain) and a WS wrapper system.

We where then able to *STUB* out the WS by creating a FAKE WS system in our local development enviroment.

It worked, but it is very difficult to do testing an builds or even setting up a new developer.

In retrospect, I think it would have been much easier to create a true mock in system, in the long run it would have been a huge time saver.

"Simplicity--the art of maximizing the amount of work not done--is essential. "

Post of the year!  

Thank you so much Jeremy.  

Time for another weekly round-up of developer news that focuses on .NET, agile and general development

Hi,

how do you quote on agile projects?

I've worked with some people on projects and typically what happens is that either we get a brief or specifications. If we just got a brief then in some cases we get the client to pay for a speicification to get created. From the specification the project will get quoted on.

Clients typically want to know how much a project will cost and want a cost put down so that they know if they want to proceed with a project. So that's where the design up-front way of doing things comes into place.

If I told clients that I wanted to do the project without big design at the begining and doing it in a agile way they wouldn't be too comfortable.

I agree that 'big design' type usually wastes lots of money and time in that things either change while developing or the original spec didn't cut it.

Any tips?

Resumo da semana - 10/12/2007

Wow the the comments are immense on this topic.  I apologize if i'm repeating someone elses sentiments.

I'll bite on this one.  I haven't done any true SOA yet, but i view it this way.  When the tools to build Web Services and SOA are JUST AS EASY as building an application with a domain layer or MVC, then the rule may very well be "Always build with SOA" because the cost is negligible.  Microsoft's WCF is one step in this direction.   Sun and other vendors are also trying to make services easier to build.  The idea is that building a service should be just as easy as building a C# or Java class.  I took a WCF class from Juval Lowy and he seemed to be suggesting that the shift to SOA is analogous to the shift to OOP 10 years ago (although arguably many, many developers STILL haven't made the shift to OOP)

In the meantime, there is a lot of develoment overhead so for many applications, it may simply not be worth it.  

Your arugment doesn't hold up for me, in one section you say:

"I understand the rationale behind my old architect buddy's reasoning, but I'll put forth a couple reasons why I think the web service approach was dead wrong. It was more expensive..."

And then later say

"He wouldn't have bought my argument about the web service being inexpensive to add later..."

Now it sounds like to me that either you believe you are a better developer than the team that did the implementation or you haven't used any of the tools out there that allow you do web service contract design in a quick and agile manner.

IMO Web services should only ever be a thin facade around well designed\imlpemented business logic arranged as services. So to add a web service upfront or later is a no brainer and costs very little in time and money.

A good point made above is the one about companies NEVER EVER go back and refactor an existing working implementation just because another implementation comes alongs and wants the same business logic - they will duplicate the code (DLL's) in multiple projects so in the long term the YAGNI approach will fail the business from this point of view.

I did this project...

Lead architect gung ho on SOA, project manager as well, perhaps to show business value through integration at some point in the future. Me, well, pretty sick of it after a month. Yep, just getting a known environment was a project in itself with services you're working on locally and the main build on a dev server and config files to manage what points where and tools to configure the config files... Many hours spent there so everyone could just develop! And no other known clients but the web application (you're real tipoff that YAGNI). A year later, first release, and I looked back on it as 1) longest time I've ever been on a project and 2) what did I get out of these services (other than working knowledge)? How much quicker and easier would it have been with just a web app and business assemblies! And this project was mostly a rewrite of an existing, messy Classic ASP application.

When I get done with an application, I only want the absolute barest of necessity of design to be there. It should look like it was built exactly for it's purpose. Quicker to test and debug, and look at and change months later.

Nice article!

I've just started a blog series to talk about how the first causes of software development can help

Pingback from  Agile Hong Kong  » Blog Archive   » Agile at 10,000 feet

Last week I gave a talk at Agile Austin entitled “How does design get done on an Agile Project?” 

Last week I gave a talk at Agile Austin entitled “How does design get done on an Agile Project?” 

I haven’t been nearly as prolific a blogger as I used to be this year, so this one is going to be short

I haven’t been nearly as prolific a blogger as I used to be this year, so this one is going to be short

Pingback from  My ???Best Of??? for 2008 - taccato! trend tracker, cool hunting, new business ideas

My latest article is up on MSDN magazine for August, and this will be what finally gets me kicked out