My immediate organization (formerly Finetix) largely uses XP/Scrum practices, but much of our larger parent organization is still new to Agile development. Since more and more clients are asking for Agile project delivery, several of my coworkers and I were asked to participate in an Agile roundtable event. The roundtable was asked a series of five questions one at a time and then went around the table. To the best of my recollection, here is a distillation of my responses.
Yes, my answers are very biased, but I’ve come by these bias honestly. I’ve had mostly positive experiences with XP/Scrum, and nothing but irritation from working with waterfall mentalities. It’s probably true that many shops don’t really do a strict waterfall in practice, but in a way that’s even worse to give lip service to the waterfall while working adaptively on the side. Those shops are living a lie. They’re trying to work adaptively since that’s the only real way to succeed at complex projects, but their project lifecycle simply doesn’t support that adaptation. Any decent usage of an Agile process is built around gathering feedback and making adaptations. We ought to just start telling ourselves and management the truth and bring this adaptation out into the open sunshine where it’s easier to control.
What about Fixed Bids?
I am largely passing on a question about doing Agile on a fixed bid, fixed deliverable project. I don’t have any first hand experience, and my cheeky response is that it works the exact same way as any other process. You do your damndest to estimate the work by breaking it down into as fine grained tasks as possible then work stupidly long hours when the estimate turns out to be wrong. Yeah, the extra upfront work to do the estimate isn’t necessarily Agile, and probably makes the whole of the work more expensive than it would be otherwise, but the situation is what it is.
I don’t really think the Agile answer to a fixed bid is any different from any other process. I do think that Agile practices and project management can give you far more control and feedback on the “Iron Triangle” of resources, time, and features. Agile/RUP/CMMI/waterfall whatever, the iron triangle constraints still apply. If you try to lock all three constraints you’re in for either pain, unhappiness, or protective sandbagging in your estimates. I would still choose to use Agile delivery for fixed bid projects because I think that is the most efficient way to execute and allows for the ability to fail “softly” with some fraction of the features instead of total abject failure to deliver any features on time like a waterfall project.
How do you create agile requirements? Some teams define stories poorly and then discover later that implementing these stories takes much longer than estimated. Does a note card’s worth of information provide adequate detail for a good estimate? Is it a problem if estimates are highly inaccurate?
I’m going to be brief here because I have a “Jeremy length” essay on Agile requirements coming soon that hits this topic in detail. The short answer is that user stories on note card’s are definitely not enough information for good estimates. That’s okay though, because that isn’t all the information. Story cards are primarily a project management tracking device plus a way of creating a common language for the team. The note cards are simply an icon representing some finite amount of work. The actual estimates are produced by the team with the developers tasking out the larger feature with plenty of help from the anaylysts, customers, and project manager, but that conversation absolutely has to happen. In an Agile project you are very consciously trading in intermediate deliverables for far more face to face communication. If I’m allowed to have my way, the detailed requirements will be captured in the form of acceptance tests and largely automated – but only close to the time that a given feature is actually put into development. There’s no use in doing detailed analysis for some feature that ends up getting scrapped. That’s just waste.
Check out this link from Jim Shore: Beyond Story Cards: Agile Requirements Collaboration
How is Agile different from RUP? Each has development and release in an iterative process. Unlike waterfall, both can have late inclusion of requirements.
I had fun with this one. I should probably say that while I have plenty of “book” knowledge about RUP, I’ve never used it in anger. I’ve spoken to many people that very happily transitioned from RUP to XP or Scrum and refused to ever go back. I did try to champion a conversion to RUP at a former employer before running away to join the XP circus. This is a partial repeat of an earlier post, but who cares?
|Are they Iterative?||RUP is supposed to be iterative and the founders of RUP will turn their faces blue saying this. The problem for me is that RUP still includes a lot of waterfall mentality baggage in the form of project phases. The iterations are much longer and seem to really amount to mini-waterfall projects in their own right. The typical RUP iteration lengths I’ve heard ranged from 6 weeks to a couple months||Extreme Programming uses 1 to 2 week iterations, Scrum teams originally worked in 30 day sprints, but the cross pollination with XP has led to shorter iterations. Testing is engaged much earler in an Agile team.|
|Ceremony||RUP is commonly disparaged for its dizzying array of intermediate deliverables. Most are optional and teams are meant to pick and choose which deliverables are appropriate for their project. Some of the RUP deliverables may simply be an excuse to justify the purchase of the Rational lifecycle products.||XP and Scrum used to be described as low ceremony, but that might be a bald faced lie. The project management “ceremonies” are simpler, but have to be followed closely. XP in particular will have more impact on the minute by minute activity and behavior of the team than any other process.|
|Tools||Rational is a software company that makes their living from selling their lifecycle tools.||There are tools from vendors that support or aid with Agile processes and practices, but by and large, Agile teams use far more open source tooling than non-Agile teams.|
|Origin||I think that RUP is largely created by people with C++ experience. Coding is nasty, brutish, and hard. The only way to succeed is regimented discipline. Quality gates and the creation of documents like the Software Architecture Document.||Most of the early Agile leadership had a background in Smalltalk. Coding can be productive and pleasant, but the extreme flexibility of the language requires an internalized discipline. There might not be many intermediate deliverables in XP/Scrum, but XP/Scrum requires a very disciplined approach from the developers in the form of Test or Behavior Driven Developement, Continuous Integration, and Simple Design (much harder than that sounds).|
|All UML all the time. The Three Amigos were all architects and RUP has a strong emphasis on architecture.||The Simplest Thing that could Possibly Work, the Last Responsible Moment, You Aren’t Gonna Need It (basically, don’t ever do anything more complicated than what you need *right now*). Most Agile proponents eschew UML, but I’d write this off more to personal preference than a real prohibition. I do think that CRC cards and Responsibility Driven Design is more effective inside rapid iterations than even informal UML.|
You’re an Agile team, but inside of a Waterfall environment. Is it possible to remain Agile? What compromises must you make?
You will absolutely have to compromise. Your team may be flexible in regards to your feature ordering and iteration planning, but the waterfall team isn’t. If a waterfall team needs something from the Agile team, that feature simply needs to be made a priority and played in the next iteration. The other way around is more tricky. Waterfall teams generally work off of longer plans and don’t have that type of flexibility. If you’re going to be dependent upon work from a waterfall team you have to treat that dependency as a constraint.
Here’s where I think the bitter irony is, the waterfall teams purport to be more predictable because they have a linear project plan, but those plans are rarely accurate unless the plans are constantly adjusted in the face of feedback. Because those plans are never truly accurate, we need to be able to adapt if it turns out the waterfall team is late with their work. I think that the flexible delivery schedule of rapid iterations should give us more ability to simply switch to working on other features
As an Agilist I try to make all design decisions at the Last Responsible Moment. In the case of a dependency on a waterfall team, my Last Responsible Moment comes much, much earlier than it would if that same feature was completely controlled by the Agile team. Much of software design is being cognizant of what design decisions have to be made and the appropriate time to make that decision. In other words, you need to decide when to decide. In this particular case, I have to make decisions earlier than normal just to determine the concrete needs from the waterfall team early enough to get those needs into the waterfall teams project plan.
There is a very serious mismatch in terminology and vocabulary between teams using XP or Scrum and other teams doing more traditional waterfall work. They think we’re out of control and we think they’re largely nuts. You will have to invest some time with the other management to make them understand, or agree on a compromise for, how the Agile team is going to communicate progress. There are no real intermediate deliverables on a typical Agile project. I take the fairly common viewpoint that the only real measure of progress is features completed that are potentially able to be shipped.
Project staffing can be the killer in my experience. To really do an iterative lifecycle we really need to include the analysts and testers as part of the holistic team — and leave them there! Part of the enduring allure of nice, linear waterfall lifecycles is the belief that analysts can be rolled off of a project early and testers only need to be on the project at the end. It’s a nice little myth, but I’ve seen nothing but severe pain from that type of project staffing in practice.