Sponsored By Aspose - File Format APIs for .NET

Aspose are the market leader of .NET APIs for file business formats – natively work with DOCX, XLSX, PPT, PDF, MSG, MPP, images formats and many more!

A Microcosm of Agile Design

Last week at DevTeach I did a talk entitled “How does Design get done on an Agile Project?”  In the talk I tried to explain and defend the way that we design software on Agile/Lean projects.  Among the issues that I talked about were:



  1. Return on investment.  How to wrest the maximum business value out of the development effort spent in delivery.

  2. Doing design continuously is more responsible than big upfront design.

  3. Delaying technical complexity and decision making until the proper time.

  4. Enabling the future by creating maintainable code rather than trying to anticipate the future

  5. You aren’t gonna need it — YAGNI!  Why YAGNI is important and valid.  How to call YAGNI today without eliminating tomorrow’s efforts.

  6. How does continuous design as practiced by Agile teams affect the composition and roles of the development team? 

  7. How the desire for continuous design affects our design choices

In order to jump straight into these subjects I used the following scenario taken from real life at one of my former employers shortly after I left. 


A software team is building a new ASP.NET application with substantial business logic and processing. Do you:



  1. Create a web service for the business logic and make the web application call this web service – even if the web service is running in process with the web application.

  2. Build and deploy the business logic in process with the web application as a simple set of library calls.

The architect (a friend of mine) with oversight responsibilities was adamant that the team chose option #1.  I wasn’t there, but I would have vehemently argued for option #2 (partially just to argue with the architect truth be told).  I do have a pretty good idea what that architect was thinking and the rationale behind his decision making.  Let’s take a look at his reasoning first, then talk about why I think option #1 is wrong and option #2 is the more responsible choice.


Why the Web Service?


The web application was the one and only consumer of the business logic and processing exposed by the web service today, but what if some other project needed that functionality tomorrow?  This enterprise had any number of large scale enterprise projects get effectively derailed by the difficulty of integration with many of their legacy systems.  In particular, the inventory system was notoriously hard for integration.  Because the company was a large manufacturing company, almost every significant business automation flow had to touch the inventory system, so the difficulty with integration was a large drag across the enterprise.


The problem with the old inventory system was that all the business logic was tightly embedded into the user screens written in a proprietary language.  When we did projects that required automated interaction with the inventory system we had two basic choices: 



  1. Use the screen emulation software from the vendor to drive the UI automatically

  2. Rewrite the functionality of the screens in some other language, in this case PL/SQL stored procedures

Neither choice is very desirable.  You could say that everybody involved with creating enterprise solutions there had some scar tissue is an understatement.  To prevent this problem from happening with the new ASP.NET application, the architecture team was demanding that the business layer be exposed and called by web services that would be open to the rest of the world.


Some of the desire for web services was SOA utopia run amok.  The central architecture team was making a very large bet on SOA throughout the enterprise.  Their philosophy was to expose all business functionality and processing via SOAP services until they reached a critical mass where Business Processing Management and Business Activity Management would be possible.  The infamous “build it (services), and they will come” SOA strategy.  The SOA vision was compelling and set the architect’s heads a-spinning with visions of grandeur and colored their thinking.  But was it really paying off?  Would the new services really be useful down the road?*


 


Why I think my old Architect Buddy was wrong


I understand the rationale behind my old architect buddy’s reasoning, but I’ll put forth a couple reasons why I think the web service approach was dead wrong.


It was more expensive.  The simplest way to deliver the ASP.NET system was to develop the business logic as libraries that ran in process with the ASP.NET application.  Think about that for a minute.  If you built a web service you’d have:



  1. An extra set of build scripts for the second web application, and the build scripts would take longer to run.  That’s friction.

  2. Additional security infrastructure between the UI and the backend, especially if you want the web service to be publicly open

  3. Automated and manual testing between the UI and the web services would take longer to run than the corresponding in process solution.  Don’t discount how much drag that would introduce into development productivity.

  4. Extra code complexity with the web service contract and web service proxies

  5. More complicated deployment scenarios

In addition, I’d say that the usage of a web service would make continuous design harder.  If I’m developing both layers in process I can handily evolve the interaction between the UI and the backend by using automated refactoring’s with ReSharper.  If I have that kind of “reversibility,” I can shave off development time by making simpler assumptions now and add complexity if I absolutely have to later.  With the web service I have to spend more time upfront getting the web service contract right because making evolutionary changes to the web service is harder than using ReSharper on the in process model.  If the web services are really going to be publicly exposed for reuse, I have to spend even more time polishing the service contracts because once it’s published I can’t easily change my mind.  My advice is to err on the side of reversibility as much as possible.  In this particular case that means bypassing web services in favor of an in process model.


Reversibility is a real issue.  Because a mistake in the web service contract is so much more difficult and costly to change once it’s being used, the rightful tendency is to do more rigorous analysis upfront.  That analysis is going to cost us time.  If I know I can back out or modify a design later, I can do the simple thing to deliver value sooner.  In the case of the web service with only once consumer, an Extreme Programmer would “call YAGNI” to permanently table the construction of the web service until it was absolutely necessary.


 


The Economic Rationale


All arguments have to be grounded in some sort of economic rationale, and this case is no different.  Going with the in process model is more economically valuable for two reasons:



  1. Opportunity cost.  The extra mechanical cost of building the web service can be applied instead to creating other features that will provide business value now.  Delivering the web service now creates no immediate value.

  2. It potentially gets the business feature into production earlier.  There isn’t any return on investment until something is deployed.  By deploying some features earlier we also start the ROI meter earlier.

  3. The extra complexity would slow down feature delivery throughout the project

One of the ways that an Agile team strives to deliver a good return on the investment is by eliminating waste.  We don’t need a web service so we don’t build it.  As simple as that.  The way that I think about it is an analogy from manufacturing.  Exercise “pull” design and architecture instead of “push” architecture.  Don’t build infrastructure because you think it’ll be valuable later.  Build only the infrastructure that you need for the business functionality on the table.  When we try to get clever with upfront frameworks, abstractions, and infrastructure we can often miss.  We’re speculating on what’s going to be valuable instead of working off of hard facts.  Remember that no matter how good you are in predicting what you’ll need, you’re requirements can easily get changed underneath you.  Sometimes the speculation will pay off and other times it’ll make things worse.  In the formulation of Extreme Programming I think that Kent Beck made an implicit assumption that the percentages are against speculative design paying off most of the time, so you should practice simplicity every time until forced to do more complex things as a way to be more efficient.


 


Tomorrow comes


You did things the way that I said to and built the business logic as simple libraries running in process with the ASP.NET application.  Tomorrow is here and it turns out that some new initiative does indeed need access to our business logic and a web service seems to be the logical choice.  The fears of our enterprise architect have been fully realized – or have they?  Is the business logic hopelessly coupled to the ASP.NET application like he feared would happen, or is it perfectly economic to just expose a new web service on top of the existing business logic and use it as is?  It depends on how we built the system.


Agile design isn’t all dessert and no vegetables.  We saved time and effort on the first project by forgoing the web service we didn’t need then, but tomorrow always comes.  The key to working iteratively and incrementally is following good design principles all the time.  If we built the ASP.NET with a good separation of concerns so that the business logic is orthogonal to the user interface, we should be able to put a new headless web service right on top of the existing business logic.  In effect, we just need to build a new remote facade for the business logic and expose this to the outside world. 


 


But it’s cheaper to build it now!


As a consultant in decidedly non-Agile clients I’m constantly fighting off the notion that it’ll be cheaper to just “build it now while we’re in there.”  It’s a tempting thought and it’s partially caused by people with overwhelming memories of bad code and painful regression testing experiences.  I’m making the argument that the cost of building the web service should remain constant no matter when we choose to build it assuming that we’re following good design principles (separation of concerns, orthogonality, Single Responsibility Principle, etc.).


 


Team Composition


As I’ve tried to convey, my old architect buddy was acting quite rationally according to what he knew and thought.  He wouldn’t have bought my argument about the web service being inexpensive to add later because he flat out didn’t trust the development team to build the ASP.NET application with a good architecture.  He probably had the design chops to get it done himself, but he wasn’t on the project himself.  By forcing the business logic into a web service he thought that he could mandate to the development team some semblance of separation of concerns.  The actual development team needed some developers or architects dedicated to the project that had the ability to create a good design.  It’s not good enough to have a supervisory architect drop by for a couple hours at the beginning of the project.  Technical design strength should be applied at the point of attack.


 


It would have been wrong anyway


There’s one last reason not to build the web service upfront.  It would have been wrong anyway.  In the absence of any other concrete use cases for the web service we would have been either tailoring it to the ASP.NET client or making assumptions about how future callers would be using the web service.  In my opinion, the chances of getting it wrong are far too high to make building the web service upfront all that valuable.  The future projects that need to consume this business logic may need an entirely different usage than the original application.  We’re better off waiting until the actual needs arise in order to create the exact web service that we need.  Building the web service now is acting far in advance of the Last Responsible Moment.  Assuming a good design, we haven’t eliminated the possibility of building the web service later and we’ll know much more about the proper shape of the web service at a later date.  There’s no justifiable reason to build the web service now.


So how about we just do more analysis upfront so that we can make some educated guesses on the future usage?  Now we’re allowing ourselves to be vulnerable to analysis/paralysis.  That extra analysis is going to delay the delivery of the initial functionality.  And it’ll still be wrong.


 


One other last thought about the Web Service


Let’s say there was some perfectly valid reason for building the business logic into a web service from the very beginning.  We’ve got a reusable web service!  Cool!  Except it isn’t exactly what the next project needs.  Part of the functionality is usable, but other parts need to be different.  If the actual business logic is built granularly with the Single Responsibility Principle we can reuse the pieces that we need and write all new code for the new functionality.  If the business logic was created as one big chunk and tightly coupled to the service contract of the web service, we’re screwed.  My only point here is that the design of the innards of the web service needs to follow the same design principles as we would in endeavoring to create a solid application structure.  I think the SOA enthusiasts too often overlook this issue or somehow believe that a design is “loosely coupled” just because there’s a well defined service boundary in there somewhere.  To really be Agile in your design and delivery efforts, almost all production code needs to be built with solid design principles.


 


Anyway, that’s my take on the scenario.  What do you think?  What did I miss?  Wanna argue?


 


 


* Part of the web service craze was caused by the fact that all of us architect types had a provision in our yearly performance plan that we would be responsible for creating at least one new web service each year.

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Featured. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://Bryan.ReynoldsLive.Com Bryan Reynolds

    Nice article!

  • MikeH

    I did this project…
    Lead architect gung ho on SOA, project manager as well, perhaps to show business value through integration at some point in the future. Me, well, pretty sick of it after a month. Yep, just getting a known environment was a project in itself with services you’re working on locally and the main build on a dev server and config files to manage what points where and tools to configure the config files… Many hours spent there so everyone could just develop! And no other known clients but the web application (you’re real tipoff that YAGNI). A year later, first release, and I looked back on it as 1) longest time I’ve ever been on a project and 2) what did I get out of these services (other than working knowledge)? How much quicker and easier would it have been with just a web app and business assemblies! And this project was mostly a rewrite of an existing, messy Classic ASP application.

    When I get done with an application, I only want the absolute barest of necessity of design to be there. It should look like it was built exactly for it’s purpose. Quicker to test and debug, and look at and change months later.

  • Ollie Riches

    Your arugment doesn’t hold up for me, in one section you say:

    “I understand the rationale behind my old architect buddy’s reasoning, but I’ll put forth a couple reasons why I think the web service approach was dead wrong. It was more expensive…”

    And then later say

    “He wouldn’t have bought my argument about the web service being inexpensive to add later…”

    Now it sounds like to me that either you believe you are a better developer than the team that did the implementation or you haven’t used any of the tools out there that allow you do web service contract design in a quick and agile manner.

    IMO Web services should only ever be a thin facade around well designed\imlpemented business logic arranged as services. So to add a web service upfront or later is a no brainer and costs very little in time and money.

    A good point made above is the one about companies NEVER EVER go back and refactor an existing working implementation just because another implementation comes alongs and wants the same business logic – they will duplicate the code (DLL’s) in multiple projects so in the long term the YAGNI approach will fail the business from this point of view.

  • http://www.dotnettricks.com Craig Bowes

    Wow the the comments are immense on this topic. I apologize if i’m repeating someone elses sentiments.

    I’ll bite on this one. I haven’t done any true SOA yet, but i view it this way. When the tools to build Web Services and SOA are JUST AS EASY as building an application with a domain layer or MVC, then the rule may very well be “Always build with SOA” because the cost is negligible. Microsoft’s WCF is one step in this direction. Sun and other vendors are also trying to make services easier to build. The idea is that building a service should be just as easy as building a C# or Java class. I took a WCF class from Juval Lowy and he seemed to be suggesting that the shift to SOA is analogous to the shift to OOP 10 years ago (although arguably many, many developers STILL haven’t made the shift to OOP)

    In the meantime, there is a lot of develoment overhead so for many applications, it may simply not be worth it.

  • Richard Venter

    Hi,

    how do you quote on agile projects?

    I’ve worked with some people on projects and typically what happens is that either we get a brief or specifications. If we just got a brief then in some cases we get the client to pay for a speicification to get created. From the specification the project will get quoted on.

    Clients typically want to know how much a project will cost and want a cost put down so that they know if they want to proceed with a project. So that’s where the design up-front way of doing things comes into place.

    If I told clients that I wanted to do the project without big design at the begining and doing it in a agile way they wouldn’t be too comfortable.

    I agree that ‘big design’ type usually wastes lots of money and time in that things either change while developing or the original spec didn’t cut it.

    Any tips?

  • Bjorn Reppen

    Post of the year!

    Thank you so much Jeremy.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Hans,

    The service layer is an assumption that I made but didn’t call out.

    @Morten,

    Yes, this post is preaching to the choir because that’s the way that the blogging ecochamber works unfortunately. What I am trying to do is pre-can some arguments for myself and my readers. We’re all gonna get in situations where we’re going to need to argue with people from traditional backgrounds. I use the blog as a means to get ready for those arguments. Being asked to build something that nobody needs in the near future because it’s “cheaper to do it now” is a very common problem for me in my consulting gigs.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Dax:

    “So I have a question: is there any scenario where you would consider option 1 as the correct response?”

    The scenario about the technical architect pushing the requirement because he/she doesn’t think it would be possible to retrofit the service later is exactly the real scenario that this is based on. The traditional architect had SOA blinders anyway, but that aside, the decision to go ahead with the web service isn’t that far fetched for exactly the reasons you describe. I’m not sure that that team would have created orthogonal code to allow for switching to a web service later a feasible approach. I’m not even convinced that the architect himself would have been able to design the code along the ideals I”m promoting if he had actually been involved with the project.

    That’s nothing but real world compromise. There’s nothing good about the paragraph above. That organization was compromised by a lack of talent and ability. I do believe that you can create a team/shop that’s capable of working the way that I describe to seize business opportunity without slamming the door shut on the future. I have worked on teams that could do that, and will again. It’s not pure theory.

    And sorry, you do get the usual Agile copout, I did work at that shop and I will never, ever willingly work in that type of environment ever again.

    “I’ve worked at places where the technical requirements are just not given any kind of worth .. and “the business” decide on everything. Unfortunately a lot of the time “the business” don’t know how to build software systems. In places like this you are always faced with some pretty grey choices.”

    – Business value is the real goal, never SOA compliance or whatever else we’re using. SOA is only ever a means. Not that I would let the business design the software….

  • http://morten.lyhr.dk Morten Lyhr

    The problem is that most people that reads this blog, agrees with the agile/lean architecture.

    You are preaching for the believers…not the infidels. 😉

    I have a system where we needed Social Security numbers from an old COM application (It was a remote proxy). At first we included the COM references into the project, with all the dll hell and tightly coupling problems when we tested.

    So we decided to create a web service on top of the COM application, effectivly creating 2 systems (1 for the business domain) and a WS wrapper system.

    We where then able to *STUB* out the WS by creating a FAKE WS system in our local development enviroment.

    It worked, but it is very difficult to do testing an builds or even setting up a new developer.

    In retrospect, I think it would have been much easier to create a true mock in system, in the long run it would have been a huge time saver.

    “Simplicity–the art of maximizing the amount of work not done–is essential. “

  • http://blog.tieredsolutions.com Dax

    OK .. you definitely sound like your looking for an argument or at least a discussion rather than everyone just agreeing with you so here goes.

    Your whole article is based on the theory that your architecture buddy is working in a sensible, grown up environment .. and on that premise there’s nothing I can fault on the theory front. And I’m assuming that the article is aimed at being educational so preaching the theory is fine and good.

    However I think it’s pretty true to say that most software development environments are dysfunctional to a lesser or greater degree. In which case politics, lack of knowledge and just plain stupidity on the part of just about everyone can all come into play.

    In a scenario such as this then the architect may be valid in pushing his technical requirement (to have a web service) as having equal importance to other requirements of the project. Why? Because although the development team may well produce a system that offers an easy transition to a web service at a later date (because it adheres to separation of concerns etc etc) she may never get the business backing to actually do it .. not for any good reason just pure and simple politics.

    I’ve worked at places where the technical requirements are just not given any kind of worth .. and “the business” decide on everything. Unfortunately a lot of the time “the business” don’t know how to build software systems. In places like this you are always faced with some pretty grey choices.

    So I have a question: is there any scenario where you would consider option 1 as the correct response?

    Or to open it up even further .. would you ever deliberately break Agile guidelines (such as YANGI) for strategic reasons? Or do you believe Agile is always right .. at every level, every time?

    Oh and the usual Agile cop out of “I’d never work there” is not allowed and neither is the observation that such an environment is not Agile and therefore doesn’t count 😉

  • Hans van Dodewaard

    My choice is also option 2. But I prefer to build a thin service layer (not a web service) between the business logic and the website logic.

    If another project ever needs the same functionality, its easy to build a web service on top of the service layer. And because this service layer is already refactored many times during the development process of the site, there is a good chance is it a good starting point for the interface of a webservice.

  • El Architect

    Hi Jeremy,

    I’m an Architect at a big Fortune 500 company, who happens to make a lot of money. Somehow I stumbled across your blog, and feel compelled to comment on your argument. The way I see it your argument essentially boils down to this: Option 2 provides greater ROI than Option 1. Disregarding your technical reasons (which few would listen to and even fewer would understand), this argument can neither be proved or disproved. The only way to ‘prove’ it would be to implement Option 1 and 2 in two identical companies in exactly the same circumstances.

    Since it can’t be proved, the default option will be 1, the SOA approach. Why? Because option 1 is backed by Microsoft, IBM, Gartner etc…while Option 2 is backed up by a bunch of bloggers such as yourself.

    Now, technically your argument may be correct but we won’t know that for a couple of years when the inevitable SOA backlash begins. This makes no difference to me since I would’ve then moved on to the next ‘enterprisey’ thing which helps corporations escape their ‘SOA Integration Hell’. All I have to do is update my powerpoint presentations.

    Keep fighting the good fight. I’ll be rooting for you.

  • http://dcadenas.blogspot.com Daniel Cadenas

    Good article, I agree 100%.
    I like what you say about Team Composition as I believe that architects and programmers belong together.

    http://dcadenas.blogspot.com/2007/10/architects-and-programmers.html

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Object Bigot — I’d call you a cynic, but it might just be realism. SOA has more buzz appeal than “just write well designed code”

    @Curtis,

    I prefer to scale horizontally anyway with both UI and business logic on each server with a hardware IP router in front. One example I did think about was replacing your homegrown business logic with a third party engine at a later time. I still think you get away with the initial approach by simply creating an adapter for the third party product at a later time then swapping out the original business logic.

    “I think I have my next blog. I’ll have to come up with a scenario to test perfomance of library objects versus the cost of web services.”

    Remember the development time too. It’s not just the production performance that matters. It’s also the time it takes you to spin up the web service and UI every time you want to test out a new change.

    @nicogranelli:

    Thank you, but there’s plenty of days that I don’t feel like I get that much traction as a consultant

  • http://nicolasgranelli@gmail.com nicogranelli

    I can’t agree more.

    I dream about having you as consultant in my office

  • http://insomdev.com Curtis Gibeaut Jr

    I’m with you and also not with you on this. One of the benefits of having the web services or using .Net remoting is to allow for scalability also. Although you can load balance and just through more and more hardware what you end up with is adding hardware for both the UI rendering and the business logic all at the same time. By having the seperation you can allocate more expensive hardware to the piece that needs it saving money.

    That being said by simply building the libraries correctly you do buy the ability to migrate to any needed architecture down the road. The key is always in how flexible the design really is. In most of my scenarios there is always a need for communication between the system I’m designing and some third party system so it is easy to always pick Web Service architecture.

    One of the things I’ve always been interested in is performance vs. development costs. I think I have my next blog. I’ll have to come up with a scenario to test perfomance of library objects versus the cost of web services. Thanks for the post you’ve driven me back to thinking about architecture and anti-architecture again.

  • Object Bigot

    Jeremy, your argument for option 2 makes sense and is perfectly logical. However there is one serious flaw: it is not buzzword compliant. Haven’t you heard? It has to be SOA, SOA, SOA, SOA anything and everything that moves. What are you going to say when the Architecture Board asks you if your architecture is using SOA? No??? bzzzzt. Wrong answer.

    “To really be Agile in your design and delivery efforts, almost all production code needs to be built with solid design principles”.

    ‘solid design principles’, is that a product? When is it coming out?

  • http://davesquared.blogspot.com dave^2

    Hi Jeremy,
    I’m dealing with a similar approach at my workplace. The standard architecture insists on two (2!!!) web service layers between the business logic DLL and client. I disagree with most of their rationale (SOA, reusability etc) for the same reasons you mentioned in your post.

    The rationale I have slightly more trouble arguing with is their security one. They run the web service layers with different identities (and separate boxes on different, firewalled network segments) and use integrated authentication to secure the calls between layers. The web services themselves are just proxies through to the business logic DLL.

    The theory is that owning any but the final box (we host internal and external facing stuff) will still only expose a limited interface to the system. Now this approach made me choke for so many reasons its not funny — but I can’t argue that this approach does not provide additional security. And any suggestion that the additional security may be a tad on the side of overkill isn’t taken particularly well.

    Any ideas on saving me from developing a script to generate these web service layers and the deployment nightmares of setting up 3 webs per application over multiple boxes?

  • http://grabbagoft.blogspot.com/ Jimmy Bogard

    Seems to me a service with only one consumer (and on the same box no less) is a fairly fragrant architecture smell. Web services have a significantly higher maintenance cost in versioning, backwards compatibility, and higher cost of change than POCO’s.

    I have to deal with many of these “just in case” services as both a consumer and a provider, and it stinks on both ends. Either you have to consume a contract that isn’t what you need or you have to expose a contract you know doesn’t work but costs too much to change.

    This reminds me of the outsourcing arguments too. Everyone jumps on the potential savings but completely ignore the significant overheads these approaches incur. Sounds like McConnell’s classic mistakes #33 and #34. There’s a middle ground between over- and under-engineering, it’s called YAGNI.

  • Tomasz Kaszuba

    “OPPORTUNITY COST!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    For the simple reason that the extra time you put into the web service that adds NO BUSINESS VALUE could have been spent on other efforts that ADDED BUSINESS VALUE!!!!!!!!!!

    You do NOT know that you’ll need this later. And why would it be cheaper to do it now? Part of my point is that:

    a.) Even if it’s on the roadmap for later the roadmap could change and it would end up being unnecessary

    b.) You’ll know more about exactly what web service you need to build when you’re at the point that you need the web service.”

    You are 100% correct in the points that you have made. My problem is that most of the time you don’t get a chance to move to a web service after the project ends. When a new project comes along if you tell the project manager that you need to modify an existing project to enable it as a web service then they’ll see it as something that’s not related to their specific project. It is overkill to enable everything as a web service (and certainly that should not be the standard) but if you know that there is a chance (say 50%) that it will be used in the future then I gamble on the web service given the constraint I outlined above.

    “If you accept the SOA definition of loose coupling, you’re right. Using the older OO definition of loose coupling is a different story. As long as I’m conforming to SRP and the Open/Closed Principle, I should be able to take the exact same business logic & persistence classes and use them *unchanged* to make up the new web service. Sure, I’ll have to build a brand new remote facade over the top of them, but that’s additive code and I’d argue that that’s not that big of a deal. Adding new code is far, far less risky than changing old code. A good design and good separation of concerns all the way down to the class level can make the code more reusable and get us to Open/Closed compliance.

    You didn’t say otherwise, but I would add that I would never try to expose the business objects in my domain directly via any sort of remoting or web service. I would always have some sort of remove facade.”

    I think we miss understood each other on this, what I wanted to point out is that there is nothing wrong with changing the process contract as long as the business services consumed beneath don’t change. The logic should not be tied to the contract, it’s just an entry point. One of many. Also I agree with you that you should always have some sort of facade over your business services that maps to your business process instead of the underlaying system.

    “If I had to change later to build the web service, I’d probably just host the web service in the ASP.NET application to reduce configuration overhead. For me, using IoC containers is effectively mandatory, so changing the ASP.NET application to switch to using a web service instead wouldn’t hurt all that much.”

    And that’s exactly what we have done as well. Except the process layer and the data binders are hosted as out of process services instead of in process. :)

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Tomasz,

    Two more things;-)

    1.) Okay, the build might not be that more complex, but the build and deployment time will still be slower. Even if you automate everything, the development cycle time is going to take longer to push code to both places. That’s friction.

    2.) Debugging is far, far easier if everything is in the same AppDomain. Yeah, I insist on TDD everywhere and that cuts down on debugging, but there’s still edge cases.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Tomasz,

    Thank you very much for the comment.

    “There is another point that you failed to mention is that the development is very much related to the project at hand. If you have the budget and the time then why not save yourself the trouble of having to do this later on (especially since you know that you will not be allowed to do this in the future) and surprise your employer by giving them a quick win with a ready made service. ”

    OPPORTUNITY COST!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

    For the simple reason that the extra time you put into the web service that adds NO BUSINESS VALUE could have been spent on other efforts that ADDED BUSINESS VALUE!!!!!!!!!!

    You do NOT know that you’ll need this later. And why would it be cheaper to do it now? Part of my point is that:

    a.) Even if it’s on the roadmap for later the roadmap could change and it would end up being unnecessary
    b.) You’ll know more about exactly what web service you need to build when you’re at the point that you need the web service.

    “by design the contract between the UI and the process layer is not loose coupled and is not meant to be reused.”

    If you accept the SOA definition of loose coupling, you’re right. Using the older OO definition of loose coupling is a different story. As long as I’m conforming to SRP and the Open/Closed Principle, I should be able to take the exact same business logic & persistence classes and use them *unchanged* to make up the new web service. Sure, I’ll have to build a brand new remote facade over the top of them, but that’s additive code and I’d argue that that’s not that big of a deal. Adding new code is far, far less risky than changing old code. A good design and good separation of concerns all the way down to the class level can make the code more reusable and get us to Open/Closed compliance.

    You didn’t say otherwise, but I would add that I would never try to expose the business objects in my domain directly via any sort of remoting or web service. I would always have some sort of remove facade.

    “For option number 2, all the applications that use the in process library need to be updated, although from your statement I believe that you would choose then to move to option number 1.”

    If I had to change later to build the web service, I’d probably just host the web service in the ASP.NET application to reduce configuration overhead. For me, using IoC containers is effectively mandatory, so changing the ASP.NET application to switch to using a web service instead wouldn’t hurt all that much.

  • Tomasz Kaszuba

    Interesting post.

    This is exactly the kind of architecture that we have developed here at work except we chose the 1st version and I will defend it as being the right one. The points that you have made are very valid although I don’t see choice number 2 as less SOA then choice number 1. What you describe is a choice between an in-process library versus an out-process library as described in this very good book: http://astore.amazon.com/soa-books-20/detail/0131465759/002-0126298-6964872
    Architecturally they are both equally SOA compliant.

    I also very much agree with you that the architecture is more complex and it’s more difficult to test against since now you need to test the process layer and the UI separately. I don’t agree with you on the building part. If you have a good build tool then this becomes a non issue although obviously the time it takes to do the build will be longer. I also don’t agree with you on the maintenance, by design the contract between the UI and the process layer is not loose coupled and is not meant to be reused. (The core business functionality is a different story. ) For each channel you will need to develop a facade in front of the core business services or the existing process services. So whether you have this as an in process library versus an out process library if you need to change the contract the impact is the same. There is less impact on the deployment with option number 1 because if you want to reuse the web service then all you need is to redeploy the web service. For option number 2, all the applications that use the in process library need to be updated, although from your statement I believe that you would choose then to move to option number 1. There is another point that you failed to mention is that the development is very much related to the project at hand. If you have the budget and the time then why not save yourself the trouble of having to do this later on (especially since you know that you will not be allowed to do this in the future) and surprise your employer by giving them a quick win with a ready made service.

    I fully agree with you that from an agile point of view option number 2 sounds more reasonable. I believe that maybe this discussion should be more about development methodologies and not architecture since they are both right taken in different contexts.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Kyle,

    “You mention that we could easily switch to a web service tomorrow when the next project comes along. But that assumes that people on that project know about the previous one and how it is architected.”

    I almost put something in here about that very subject. I wanted to say that that’s a people issue, but I’m not sure how that really gets solved either way. The “delay the web service because we can build it later if we really need it” doesn’t help too much if the people who built the system are gone and the new folks don’t know the system. It might make a maintainable codebase that can be changed by other developers that much more important. I know that a lot of the pain that this company faced was due to bad code.

  • http://kyle.baley.org The Coding Hillbilly

    The only counter-argument I can think of (and it’s only for the sake of argument because I agree with you wholeheartedly) is that such a practice discourages the use of web services where it actually does have some benefit. You mention that we could easily switch to a web service tomorrow when the next project comes along. But that assumes that people on that project know about the previous one and how it is architected.

    The counter-counter to that is that the web services need to be effectively organized and published, which is a pretty tall order in and of itself.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Harry,

    Thank you for your comment, it’s about time someone started an argument.

    “I like the Web Service option for just the reason I like MVP/MVC – it force you to have a cleaner design … ”

    Survey says “bzzzzt.” You can have a clean design without a web service. That’s a bit like the TypeMock argument that says TypeMock will let you get away with a bad design. You can use TypeMock and have a good design.

    “Reusablility is just a welcomed side-effect. ”

    What makes you think it is really reusable? That’s a big part of my argument to say that you have no guarantee that the web service you built ahead of time is actually going to be reusable by something else. No reason whatsoever to back up that claim. Only speculation.

    “BTW, I’ve seen people use the same arguments to against MVP/MVC …. ”

    Fair enough, but my vote is on MVP/MVC because I do think MVP/MVC is cheaper for any nontrivial screen. It manages complexity and increases testability, both of which make me faster. That’s still an economics question.

  • Harry

    I like the Web Service option for just the reason I like MVP/MVC – it force you to have a cleaner design …
    For example, I don’t have to worry about the existence of DB in the caller application …
    Reusablility is just a welcomed side-effect.
    BTW, I’ve seen people use the same arguments to against MVP/MVC ….
    This is an interesting decision to make, and I don’t see a clear answer to either side. The more import thing is to do it right instead of do it one way or another …

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Jimmy,

    Wanna guess where the story comes from?

    “Scrum is hard to make work with dependencies on non-agile teams…”

    – It’s a constraint you just have to live with. I couldn’t handle the frustration I bet you’re feeling.

  • http://grabbagoft.blogspot.com/ Jimmy Bogard

    Wow this sounds familiar…a service was created, contract agreed to months ahead of actual use, and lo and behold, when we actually tried to use the service, it didn’t meet our current needs. It met our imaginary needs of 6 months ago, but not the actual needs of today.

    The service team attributed this to us “not having detailed requirements”. Too bad the service team is a 6 month long waterfall dinosaur that requires everything in BDUF requirements docs. Scrum is hard to make work with dependencies on non-agile teams…

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @George,

    Don’t feel too bad, my current client and previous client would have tried to build the web service upfront because it’s “cheaper.”

    My all time favorite horror story about stupid inducements is from the old SDMagazine.

    A manager goes to a seminar and internalizes the “if you can’t measure it, you can’t manage it” mantra. He promptly comes back and starts using the dreaded Lines of Code metric to measure developer performance. The first month a notorious “copy and paster” gets an award for productivity. Other developers get yelled at for apparent productivity dropoffs because they refactor away a lot of code. Sooner or later they figure out the manager is determining lines of code by counting the number of “;” characters. Every line of code is now ended with:

    int x = 0;;

    Eventually they start doing this:

    /*;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
    ; Comment block
    ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;*/

    The manager finally figures out what’s happening and gets mad. He measured the number of semicolons in the code, and quite happily optimized the number of semicolons being introduced into the code. Mission accomplished.

  • George

    It sounds as if the environment you describe is more adult than the one I currently exist in, but the missing reason from our environment is “because all the other kids are doing it.” And before someone assails me about “Morts,” I’ll point out that your last line — performance plan inducements for web service creation — is the second verse of the same song, but sung by management: we have to have web services “because everyone else is doing it.” Not a hint of ROI in the bunch.

  • http://blog.phatboyg.com/ Chris Patterson

    Excellent post on something we all face at one point or another.

    Nothing to add here, you got it nailed.

  • http://stevenharman.net Steven Harman

    After reading the article my head hurts… from all of the nodding-in-agreement I found myself doing through out.

    I’m dealing with this very issue right now, only difference being we’re building a WinForms application, and it’s causing us all kinds of pain. The extra hoops we have to jump through to expose EVERYTHING via the service are just ridiculous… and for what? A hypothetical 2nd consumer of the service that _might_ come around, maybe, some day?

    Solid post Jeremy… keep fighting the good fight! 😉