Trying to answer hard questions about Agile development

My immediate organization (formerly Finetix) largely uses XP/Scrum practices, but much of our larger parent organization is still new to Agile development.  Since more and more clients are asking for Agile project delivery, several of my coworkers and I were asked to participate in an Agile roundtable event.  The roundtable was asked a series of five questions one at a time and then went around the table.  To the best of my recollection, here is a distillation of my responses.


Yes, my answers are very biased, but I’ve come by these bias honestly.  I’ve had mostly positive experiences with XP/Scrum, and nothing but irritation from working with waterfall mentalities.  It’s probably true that many shops don’t really do a strict waterfall in practice, but in a way that’s even worse to give lip service to the waterfall while working adaptively on the side.  Those shops are living a lie.  They’re trying to work adaptively since that’s the only real way to succeed at complex projects, but their project lifecycle simply doesn’t support that adaptation.  Any decent usage of an Agile process is built around gathering feedback and making adaptations.  We ought to just start telling ourselves and management the truth and bring this adaptation out into the open sunshine where it’s easier to control.


What about Fixed Bids?


I am largely passing on a question about doing Agile on a fixed bid, fixed deliverable project.  I don’t have any first hand experience, and my cheeky response is that it works the exact same way as any other process.  You do your damndest to estimate the work by breaking it down into as fine grained tasks as possible then work stupidly long hours when the estimate turns out to be wrong.  Yeah, the extra upfront work to do the estimate isn’t necessarily Agile, and probably makes the whole of the work more expensive than it would be otherwise, but the situation is what it is. 


I don’t really think the Agile answer to a fixed bid is any different from any other process.  I do think that Agile practices and project management can give you far more control and feedback on the “Iron Triangle” of resources, time, and features.  Agile/RUP/CMMI/waterfall whatever, the iron triangle constraints still apply.  If you try to lock all three constraints you’re in for either pain, unhappiness, or protective sandbagging in your estimates.  I would still choose to use Agile delivery for fixed bid projects because I think that is the most efficient way to execute and allows for the ability to fail “softly” with some fraction of the features instead of total abject failure to deliver any features on time like a waterfall project.


How do you create agile requirements? Some teams define stories poorly and then discover later that implementing these stories takes much longer than estimated. Does a note card’s worth of information provide adequate detail for a good estimate? Is it a problem if estimates are highly inaccurate?


I’m going to be brief here because I have a “Jeremy length” essay on Agile requirements coming soon that hits this topic in detail.  The short answer is that user stories on note card’s are definitely not enough information for good estimates.  That’s okay though, because that isn’t all the information.  Story cards are primarily a project management tracking device plus a way of creating a common language for the team.  The note cards are simply an icon representing some finite amount of work.  The actual estimates are produced by the team with the developers tasking out the larger feature with plenty of help from the anaylysts, customers, and project manager, but that conversation absolutely has to happen.  In an Agile project you are very consciously trading in intermediate deliverables for far more face to face communication.  If I’m allowed to have my way, the detailed requirements will be captured in the form of acceptance tests and largely automated – but only close to the time that a given feature is actually put into development.  There’s no use in doing detailed analysis for some feature that ends up getting scrapped.  That’s just waste.


Check out this link from Jim Shore:  Beyond Story Cards: Agile Requirements Collaboration



How is Agile different from RUP? Each has development and release in an iterative process. Unlike waterfall, both can have late inclusion of requirements.

I had fun with this one.  I should probably say that while I have plenty of “book” knowledge about RUP, I’ve never used it in anger.  I’ve spoken to many people that very happily transitioned from RUP to XP or Scrum and refused to ever go back.  I did try to champion a conversion to RUP at a former employer before running away to join the XP circus.  This is a partial repeat of an earlier post, but who cares?



























Subject RUP XP/Scrum
Are they Iterative? RUP is supposed to be iterative and the founders of RUP will turn their faces blue saying this.  The problem for me is that RUP still includes a lot of waterfall mentality baggage in the form of project phases.  The iterations are much longer and seem to really amount to mini-waterfall projects in their own right.  The typical RUP iteration lengths I’ve heard ranged from 6 weeks to a couple months Extreme Programming uses 1 to 2 week iterations, Scrum teams originally worked in 30 day sprints, but the cross pollination with XP has led to shorter iterations.  Testing is engaged much earler in an Agile team. 
Ceremony RUP is commonly disparaged for its dizzying array of intermediate deliverables.  Most are optional and teams are meant to pick and choose which deliverables are appropriate for their project.  Some of the RUP deliverables may simply be an excuse to justify the purchase of the Rational lifecycle products. XP and Scrum used to be described as low ceremony, but that might be a bald faced lie.  The project management ”ceremonies” are simpler, but have to be followed closely.  XP in particular will have more impact on the minute by minute activity and behavior of the team than any other process.
Tools Rational is a software company that makes their living from selling their lifecycle tools. There are tools from vendors that support or aid with Agile processes and practices, but by and large, Agile teams use far more open source tooling than non-Agile teams. 
Origin I think that RUP is largely created by people with C++ experience.  Coding is nasty, brutish, and hard.  The only way to succeed is regimented discipline.  Quality gates and the creation of documents like the Software Architecture Document. Most of the early Agile leadership had a background in Smalltalk.  Coding can be productive and pleasant, but the extreme flexibility of the language requires an internalized discipline.  There might not be many intermediate deliverables in XP/Scrum, but XP/Scrum requires a very disciplined approach from the developers in the form of Test or Behavior Driven Developement, Continuous Integration, and Simple Design (much harder than that sounds).

Design

All UML all the time.  The Three Amigos were all architects and RUP has a strong emphasis on architecture. The Simplest Thing that could Possibly Work, the Last Responsible Moment, You Aren’t Gonna Need It (basically, don’t ever do anything more complicated than what you need *right now*).  Most Agile proponents eschew UML, but I’d write this off more to personal preference than a real prohibition.  I do think that CRC cards and Responsibility Driven Design is more effective inside rapid iterations than even informal UML.

 


You’re an Agile team, but inside of a Waterfall environment.  Is it possible to remain Agile?  What compromises must you make?


You will absolutely have to compromise.  Your team may be flexible in regards to your feature ordering and iteration planning, but the waterfall team isn’t.  If a waterfall team needs something from the Agile team, that feature simply needs to be made a priority and played in the next iteration.  The other way around is more tricky.  Waterfall teams generally work off of longer plans and don’t have that type of flexibility.  If you’re going to be dependent upon work from a waterfall team you have to treat that dependency as a constraint. 


Here’s where I think the bitter irony is, the waterfall teams purport to be more predictable because they have a linear project plan, but those plans are rarely accurate unless the plans are constantly adjusted in the face of feedback.  Because those plans are never truly accurate, we need to be able to adapt if it turns out the waterfall team is late with their work.  I think that the flexible delivery schedule of rapid iterations should give us more ability to simply switch to working on other features


As an Agilist I try to make all design decisions at the Last Responsible Moment.  In the case of a dependency on a waterfall team, my Last Responsible Moment comes much, much earlier than it would if that same feature was completely controlled by the Agile team.  Much of software design is being cognizant of what design decisions have to be made and the appropriate time to make that decision.  In other words, you need to decide when to decide.  In this particular case, I have to make decisions earlier than normal just to determine the concrete needs from the waterfall team early enough to get those needs into the waterfall teams project plan.


There is a very serious mismatch in terminology and vocabulary between teams using XP or Scrum and other teams doing more traditional waterfall work.  They think we’re out of control and we think they’re largely nuts.  You will have to invest some time with the other management to make them understand, or agree on a compromise for, how the Agile team is going to communicate progress.  There are no real intermediate deliverables on a typical Agile project.  I take the fairly common viewpoint that the only real measure of progress is features completed that are potentially able to be shipped.


Project staffing can be the killer in my experience.  To really do an iterative lifecycle we really need to include the analysts and testers as part of the holistic team — and leave them there!  Part of the enduring allure of nice, linear waterfall lifecycles is the belief that analysts can be rolled off of a project early and testers only need to be on the project at the end.  It’s a nice little myth, but I’ve seen nothing but severe pain from that type of project staffing in practice. 


 


 

About Jeremy Miller

Jeremy is the Chief Software Architect at Dovetail Software, the coolest ISV in Austin. Jeremy began his IT career writing "Shadow IT" applications to automate his engineering documentation, then wandered into software development because it looked like more fun. Jeremy is the author of the open source StructureMap tool for Dependency Injection with .Net, StoryTeller for supercharged acceptance testing in .Net, and one of the principal developers behind FubuMVC. Jeremy's thoughts on all things software can be found at The Shade Tree Developer at http://codebetter.com/jeremymiller.
This entry was posted in Featured. Bookmark the permalink. Follow any comments here with the RSS feed for this post.
  • http://www.jroller.com/page/nweber Neil Weber

    I concur with “Not an IBM Employee.”

    I find it interesting that Agile fans pretty much only compare Agile to some waterfall straw man that no one ever uses.

    Jeremy, your RUP vs Agile comparison was pretty weak. Frequently, the RUP and Agile columns aren’t even talking about the same thing. For example, under “Ceremony” you talk about RUP optional artifacts and under Agile you talk about ceremony. Under Tools, you don’t even talk about tools but rather where the tools come from. Under Iterative, at first you seem to be saying that RUP is not iterative, but then say it is and the iterations are like short eev-ill waterfall spurts. Aren’t Agile iterations like short eev-ill waterfall spurts too?

    I didn’t realize people still use/talk about XP because of the extreme demands it places upon the project and team members. On my last three projects, there is no way we could do XP for many reasons. My last project started out as Agile, but was spinning out of control. I stopped the spinning by adopting some simple artifacts (like written requirements and a written design spec). This small effort brought focus to the project and allowed us to move forward. I found that Marketing hated Agile because they didn’t have any feel about what we were building. QA also hated Agile.

  • Kevin Thex

    Funny, I didn’t see you at our Thursday stake holders meeting.

  • http://www.rgoarchitects.com/nblog Arnon Rotem-Gal-Oz

    Fixed and Agile is hard but as other said here – it isn’t that everything is dandy with Fixed and Waterfall.
    Nevertheless you can do a little better than work with agile and hope for the best such as working with the customer to break down the project, introducing exchange requests etc.
    I also wrote about it in my blog: http://www.rgoarchitects.com/nblog/2007/09/02/FixedBidsAndAgileProjects.aspx

  • http://www.e-Crescendo.com jdn

    I was hoping for something better than the CHAOS report (of course there are many). I don’t really think that supplies much of any really supportive evidence of the kind I was hoping for.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @jdn,

    Go for the CHAOS report. It doesn’t necessarily say “Agile is the one true answer,” but it picks up a definite project success ratio for iterative and flexible waterfall projects versus straight waterfall. One quick Google search will find all you could ever want to know. You might also look through Craig Larman’s book on iterative development, but go google some of the criticisms of the book as well.

  • http://www.e-Crescendo.com jdn

    “There’s a wealth of empirical data pointing to the superiority of iterative methods (of which Agile methods are a subset of) over the classical waterfall.”

    Can you point me to some examples that show this? It would be helpful in ‘marketing’ iterative methods to some skeptical audiences I know of.

  • http://www.evanhoff.com/ Evan

    @Mike D,

    It’s really not about fixed bid, it’s about fixed bid, fixed deliverable. When building the house, the blueprint is a fixed deliverable. Given a fixed deliverable, creating a fixed bid is fairly painless for them. The blueprint accurately describes the entire deliverable in total.

    Software (enterprise software) does not work that way. It’s primarily due to the nature of business and the cost (time and money) associated with creating a comprehensive (and completely accurate) set of blueprints that accurately and totally describe the software deliverable in total.

    Once we realize this fact, it’s up to us to come up with an alternative that works.

    And don’t read into this thinking I’m saying that you cannot do project estimates. ;-)

  • Mike D

    Jerry,

    I understand that you are saying the fixed bid is unrealistic. I respectfully disagree, even on an in house project you want a real estimate of costs before proceeding. I don’t think asking the client to change the way they contract work to match the way you do work, is a realistic expectation and I would argue that the estimating process is flawed. I see many benefits to the Agile process, it benefits everyone, but it’s not an all or nothing proposition.

  • http://blog.rdn-consulting.com Bob

    Here’s another hard question re: Agile:

    http://rdn-consulting.com/blog/2007/07/22/agile-development-in-a-fda-regulated-setting/

    When you have a regulatory structure with a long history of doing things a certain way compromise may not be an option.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Mike D,

    I think you’re indulging in some intellectual wishy washiness and shallow observation. There are some absolutes in the world. There’s a wealth of empirical data pointing to the superiority of iterative methods (of which Agile methods are a subset of) over the classical waterfall.

    The issue isn’t necessarily that waterfall & agile are both flawed so much as that the Fixed bid, fixed deliverable scenario is a flawed proposition for both the development team and the customers. If you’ll follow both the links to Udi and McConnell you’ll see the same.

    Comparing software development to house construction is a tired and deeply flawed analogy.

  • Mike D

    Agile verses Waterfall, In this world there is no Black and White only shades of Grey. The real point in this one is that neither can accurately estimate a project, and that is the issue.

    From this I conclude that both Agile and Waterfall are flawed and cannot conclude that one is better than the other they are only different.

    Perhaps the solution is to refine both the development methodology and the development tools, in their current state they are both flawed.

    I would argue that it is unrealistic to expect a client to enter in to an open ended agreement on a project, would you contract to have a house built this way, absolutely not.

    I see these arguments every day, A verses B, the answer is always somewhere in the middle, the problem is that the A and B advocates never agree on this point.

  • Not an IBM employee

    RUP instructs that iterations should be 2-6 weeks. Not several months. All documents and models are optional.

    There is really nothing waterfallish about RUP. If you think there is, you haven’t understood RUP and should read the books again.

    Further, any even moderately complex system requires some level of up-front design and thinking. It needs thoughtout architectural solutions which will be incrementally built and tested.

    Little or even no design up front leads to endless refactoring, increasing rework + technical debt, loss of velocity and a lot of other problems.

    It’s not about Waterfall or Extreme IID. It’s about finding the suitable middle ground for your project.

  • http://blog.phatboyg.com/ Chris Patterson

    Wow, awesome post and perfectly timed. I love being in an organization that speaks Waterfall yet ends up paying for it in the end with frequent iterations and ever changing requirements that break the schedule and expectations.

    A lot of what is said in here will make for good firepower to work with upper management to bend the minds and avoid spending months planning how we’re going to plan.

    Gracias!

  • http://www.evanhoff.com/ Evan

    Just to clarify my last comment..

    When we look at the realities of business, there is no such thing as a fixed deliverable project–any way you slice it. Either you need to help show them that, or (if they are just trying to make you squirm), you need to poke back and tell them you’ve never seen a fixed deliverable project (I know I havent). Models are often wrong and there are always unknowns.

    If that still doesnt work, you can just tell them they are forcing you to adding lots of extra padding to the triangle (at which point they shouldnt care about methodology at all).

  • http://www.evanhoff.com/ Evan

    There’s a few interesting observations on this post/comments…

    A lot of people freak out when they think about Agile and Fixed Bid, but does Waterfall and fixed bid work flawlessly either? That’s not the point/strength of a lightweight methodology.

    I think if you approached a typical business person (a project stakeholder) and explained that by “embracing change” you will be using shifting market conditions and mini-innovations as a competitive advantage to meet needs rather than nickel and diming him throughout the project, he would be very receptive. Next you explain that as part of the iron triangle, if he wants to fix the bid and the deadline, you need some wiggle room in scope. You need to break him free from the “engineering” mentality and help him see the value in a “production” mentality. And then you can tell him about your personal experiences with both (project success wise). At that point, I think the fixed bid obstacle becomes a minor thing. For many business projects, real success isn’t the efficient execution of a construction process, it’s the adaptive delivery of a solution who best meets the needs of its users–knowing full well that the users often time don’t know what that means up front.

    As a side note, you might get resistance on this from middle management. Let’s not forget that much of the Waterfall mentality is driven by Cover Your Ass and placing blame in case of failure. But then again, what can we expect from a guy stuck in the middle of the corporate ladder.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Joe,

    Your goal isn’t to win the bid, it’s to make money. That’s a disaster avoided, not a failure at all. That doesn’t sound like a great client anyway.

    I’ve got a similar story. When my father was very young, he submitted a losing bid for a new house. The customer yelled at him for bidding so much higher than the other guy. The other guy forgot to add in cabinetry in *his* bid. Oops.

  • http://www.e-Crescendo.com jdn

    @Joe

    That’s good to hear. Working in the financial industry as well, I can tell you that some SOX/SDLC implementations make it darn near impossible to do much beyond BUFD, except in limited cases.

  • http://www.lostechies.com/blogs/joe_ocampo Joe Ocampo

    @jdn,

    SoX compliance can be accomplished using an Agile approach. I am in the financial industry and what it really comes down to is refinement of your process in the terminology that the auditors understand and implementing certain control point that are automated throughout the process. Separation of responsibility is huge as well but that is easily overcome outside of the actual development effort. The point is it can be done and is being done.

    @jeremy

    “I can’t imagine any type of spec, much less an RFP, being detailed enough to do an accurate story point breakdown upfront. How do you accommodate that?”

    Absolutely! You start to compare and contrast different systems you have developed with the team that are similar. (Planning game at a project level) and then take into account the high level features that are mentioned in the proposal)

    I will say this, more that likely you won’t win the bid because you are being honest with your estimation. I remember 2 years ago and I estimated a certain CRM project would take a team 2 years to complete but that we would have working software by the 2nd month of the project. The client didn’t care and wanted the software in 8 months. Needless to say we lost the bid but the team that did win the bid took 2 years to deliver the FINAL product and both sides were completely frustrated with the end result.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    jdn,

    I was trying to be sarcastic. It’s absolutely no surprise that the Economist is mocking it.

    Ironically, one of the other CodeBetter guys was working last year on a SoX compliance tool, but building it with a textbook XP process.

  • http://www.e-Crescendo.com jdn

    I wouldn’t find it odd they want to scrap it. At all. Supposedly, Congress is going to revisit and loosen it up, but we’ll see.

    SOX can have a huge impact when it determines what your SDLC is going to look like. I think you can imagine.

  • http://codebetter.com/blogs/jeremy.miller Jeremy D. Miller

    @Joe,

    I just realized yesterday that my feed for lostechies isn’t working in RSS Bandit. I’ll certainly give it a look.

    Correct me if I’m wrong, but you’re just doing the Release/Planning Game before giving the bid right? I can’t imagine any type of spec, much less an RFP, being detailed enough to do an accurate story point breakdown upfront. How do you accomodate that?

    The problem I have with people saying that Agile can’t work for a fixed bid is how anything else is better. One way or another you’re estimating with incomplete facts. The McConnel link immediately goes into breaking the work into iterations to make the estimation easier and more accurate.

    @jdn,

    SoX is a totally different problem. I don’t see why SoX would have that much impact. You could rig up automatic logging and metrics to help satisfy the auditing requirements.

    There was an article in the Economist about a month back questioning the value vs. the cost of SoX. Oddly enough, they were for scrapping it.

  • http://www.lostechies.com/blogs/joe_ocampo Joe Ocampo

    The is perfect timing Jeremy, I just did a post that compliments what you are talking about here.

    http://www.lostechies.com/blogs/joe_ocampo/archive/2007/08/30/enterprise-agile-generations.aspx

    Fixed bids work like this in Agile. (Mind you I am using a combination of Agile, Lean, PMBOK and traditional wet finger in the air analysis here to come up with this, Not to mention I am assuming that the development team is an experienced Agile development team.)

    -Take RFP and Break down high level features. With entire Team. (Buisness Analyst, Development, Quality Assuarance)

    -Assign Story Points(fibs, units) to each feature -> have multiplier that converts the story points to person hours. (This is where the wet finger in the air comes in handy, SWAG)

    -Take your standard 5-6 week release cycle and figure, resource allocation based on your estimated velocity per iteration. Ex. 1 Release Cycle can accomplish 50 story points worth of work at a cost of 1200 person hours per release, including analysis, testing and development.

    -Last but not least, like a good project manager you place 2 release cycles in reserve to account for risk

    Estimated Project Timeline = (Feature Points/Release Velocity) + release cycle reserve.

    Project Cost = Total Releases Needed * Estimated Project Timeline

    Hope this helps.

  • http://blog.donnfelker.com Donn Felker

    I forgot to mention, I’m a huge Agile fanatic, so I’m not trying to say it CANT work. I’m trying to say it needs to be done very carefully on a fixed cost project.

  • http://blog.donnfelker.com Donn Felker

    Agile on Fixed bids? Oi vey… it hurts.

    The problem with this is that if you’re in the consulting business and the company your with sends out Consultant X to bid the project that consultant better be bidding with Agile in mind. Unfortunately thats NEVER the case. If so, its so rare that I’ve yet to track it once.

    Its estimating and the method of estimation that kills the agile methodology.

    I’ve done work on a few of these fix bid projects and then attempted to do agile and unfortunately it failed miserably because the estimate didnt acurrately reflect what actually needed to be done.

    Do I think its possible to do Agile on Fixed bids? Yes, but i needs to be estimated at a much finer grain of detail.

    Steve McConnell has a great post on bidding on fixed cost projects and the risks associated with them. Check it out here: http://forums.construx.com/blogs/stevemcc/archive/2007/06/06/estimating-in-an-outsourcing-context.aspx

  • http://www.e-Crescendo.com jdn

    I realize I’m probably repeating myself, but in a world of fixed bid projects and public corporate companies that have legislated mandates like SOX imposed upon them, it sounds to me like you are saying that Agile can’t really work in that world.

    Or at the least I’m drawing that inference. Not sure.