Overview
Extreme Programming gets its name from the fact that it takes industry best practices and “turns the knobs up to 10.” [1] It is a radical departure from the traditional, document-heavy waterfall processes in widespread use today.
The 12 original core practices of XP are:
- The Planning Game: Business and development cooperate to produce the maximum business value as rapidly as possible. The planning game happens at various scales, but the basic rules are always the same:
- Business comes up with a list of desired features for the system. Each feature is written out as a User Story, which gives the feature a name, and describes in broad strokes what is required. User stories are typically written on 4x6 cards.
- Development estimates how much effort each story will take, and how much effort the team can produce in a given time interval (the iteration).
- Business then decides which stories to implement in what order, as well as when and how often to produce production releases of the system.
- Small Releases: Start with the smallest useful feature set. Release early and often, adding a few features each time.
- System Metaphor: Each project has an organizing metaphor, which provides an easy to remember naming convention.
- Simple Design: Always use the simplest possible design that gets the job done. The requirements will change tomorrow, so only do what's needed to meet today's requirements.
- Continuous Testing: Before programmers add a feature, they write a test for it. When the suite runs, the job is done. Tests in XP come in two basic flavors.
- Unit Tests are automated tests written by the developers to test functionality as they write it. Each unit test typically tests only a single class, or a small cluster of classes. Unit tests are typically written using a unit testing framework, such as NUnit.
- Acceptance Tests (also known as Functional Tests) are specified by the customer to test that the overall system is functioning as specified. Acceptance tests typically test the entire system, or some large chunk of it. When all the acceptance tests pass for a given user story, that story is considered complete. At the very least, an acceptance test could consist of a script of user interface actions and expected results that a human can run. Ideally acceptance tests should be automated, either using the unit testing framework, or a separate acceptance testing framework.
- Refactoring: Refactor out any duplicate code generated in a coding session. You can do this with confidence that you didn't break anything because you have the tests.
- Pair Programming: All production code is written by two programmers sitting at one machine. Essentially, all code is reviewed as it is written.
- Collective Code Ownership: No single person "owns" a module. Any developer is expected to be able to work on any part of the code base at any time.
- Continuous Integration: All changes are integrated into the code base at least daily. The tests have to run 100% both before and after integration.
- 40-Hour Work Week: Programmers go home on time. In crunch mode, up to one week of overtime is allowed. But multiple consecutive weeks of overtime are treated as a sign that something is very wrong with the process.
- On-site Customer: Development team has continuous access to a real live customer, that is, someone who will actually be using the system. For commercial software with lots of customers, a customer proxy (usually the product manager) is used instead.
- Coding Standards: Everyone codes to the same standards. Ideally, you shouldn't be able to tell by looking at it which person on the team has touched a specific piece of code. [2]
XP practitioners claim that there is a synergistic effect when using all of these practices all the time. Others try to pick and choose which parts of XP will work in their environment or that they are comfortable with. Results for both groups have varied.
The Most Important XP Practices
Although they are meant to be used all at once, XP has several “cornerstone” practices. The most important ones are the planning game, iterations, test-driven development, pair programming, refactoring, and continuous integration. Some of these practices have been updated from the standard definition above to take into account several years of learning in the field.
The key to the Planning Game is the separation of business from technical decisions. Business people are not allowed to make technical decisions. That is, they cannot force a developer to reduce his or her estimate because it “makes things work better.” On the other side, developers cannot make business decisions. No features can be added nor can they be done in any order other than the priority specified by the customer.
Iterative development is also a cornerstone of XP, or any agile method for that matter. XP recommends the shortest iterations amongst all agile software development methodologies. This, along with other practices such as on-site customer and test-driven development, enable rapid feedback to correct the project’s course in-progress.
Test-driven development (TDD), made popular in large part by XP and one of its proponents, Kent Beck, is a major enabler of the other practices. TDD, with all of the automated unit tests, makes refactoring easy, collective code ownership possible, and simple design more probable. Without TDD, XP might not be possible. You could be doing agile software development, but not XP [1].
Pair programming is another new invention. Based on the fact that code reviews are one of the best ways to reduce defects, why not do code reviews all the time? So XP does it. More people object to this than any other XP practice. The reasons why are fully covered in the references, but suffice it to say that pair programming has been proven to be slightly less productive (10-15 percent) than two individuals working separately but has a far lower defect rate (40-50 percent) [3]. (Note, this contradicts my Agile Software Development by the Numbers post. Different sources account for the differences.)
Refactoring, first made popular by Martin Fowler’s book Refactoring, is very important in XP. With little upfront design, refactoring helps redesign and extend the existing code base based on the customer’s needs. Refactoring keeps the technical debt down so that the development team can move as fast as possible.
Finally, continuous integration trades a few long, painful integrations with many short, easy code integrations. Code is integrated several times per day, and the integrators are responsible for making sure all the unit tests pass [1].
You will notice that all of the core practices, except one, are software development-related. Management is not XP’s strength, but there are ways to deal with it which will be discussed later.
Other XP Practices
The other XP practices are system metaphor, simple design, 40-hour work week, on-site customer, and coding standards. Again, some of these practices have been updated from the standard definition above to take into account several years of learning in the field.
Roy Miller, in the XP Distilled article, says this about the metaphor practice:
The original metaphor practice was a dud. The idea wasn't bad: to have a controlling image that describes the system in terms everyone can understand and gives it a unifying theme that allows the team to know where new things fit. The problem was that the practice really distracted from the main goal: a shared understanding of the system. Everybody on the team needs it. Sometimes a great metaphor isn't available, or would take lots of time to figure out. Don't delay simply because you don't have a nice word picture of the system you're trying to build. Focus on having a shared vocabulary (what Ward Cunningham calls a "System of Names") that everyone can use. If a metaphor helps, use one. If you can't find a good metaphor, just agree on the shared vocabulary (which shows up in names for classes, methods, and so on). [4]
Instead of trying to force-fit a metaphor, a better idea is to use the language of the problem domain. Domain vocabulary serves as a common language, making communication between developers and customers easier. In fact, the domain should drive the design, as Eric Evans shows in his book Domain-Driven Design.
Simple design is another name for emergent design. Commonly referred to as YAGNI (you ain’t gonna need it) by XP practitioners, it is based on the principle that the easiest way to be responsive to future changes is to have the least amount of current functionality/code possible. According to Kent Beck, the simplest design
- Runs all the unit test (and functional tests, if you have them)
- Contains no duplicate code
- Clearly states the programmers' intent for all code
- Contains the fewest possible classes and methods [1]
The 40-hour work week has also come under fire since XP was released. The limitation to 40 hours is just too specific. Some suggest the term “sustainable pace,” indicating a lack of overtime. Ron Jeffries defines overtime as any time you are in the office when you don’t want to be there. If developers are starting to work overtime regularly, the project is off course.
One of the most difficult XP practices is the on-site customer. While there are various arguments on whether this is feasible or not, in the end it is up to the customer. However, it is a proven fact that there is a direct correlation between the amount of customer involvement and the chances of success for the project [4].
Finally, coding standards. Everyone has an opinion, and they are usually defended with religious fervor. The key, if a coding standard is to work, is to keep it simple. Complicated standards are never followed. The means of enforcement should also be simple. If pair programming is used, the developers can easily correct each other throughout the coding process. Without pair programming, an automated code formatter or lint-type code checker should be used.
What’s Wrong with XP?
From Chris Collins and Roy Miller, software developers at RoleModel Software, Inc. (contributor and author of Extreme Programming Applied, respectively) note:
Fortunately, there has been significant talk in the XP community, Internet newsgroups, and other forums about the "12 practices" of XP. People are recognizing that the first release of XP, so to speak, seemed to emphasize the programming aspects of software development to the detriment of other things, and that the business side wasn't quite as complete as it needed to be. The foundational principles are good, but there are some more practices that need to be included to complete the picture. [4]
XP focuses on the coding part of software development. Although the few management trappings may have worked for the early adopters, it will not suffice for the majority.
One thing that hurts XP in this area, in my opinion, is the use of “ideal days” for estimates. To translate this to schedule time, the estimate is multiplied by a “factor” that is supposed to take into account interruptions, communication difficulties, and the like. The problem with this is that it buries the bottlenecks and their associated buffers in the individual estimates. According to critical chain theory, this is the worst thing to do! [5] Another problem with this is that it effectively hides work inhibitors, rather than bringing them into full view of management to be dealt with on a daily basis (reference here is to the daily Scrum). And the worst thing about this is that it is not the language the business people speak. Business people want to know things in hours and days, not in craft-units or ideal days. Putting up artificial barriers to communication is contrary to XP’s value of communication.
Another practice that hurts XP in the management arena is the planning game. The new “release planning” practice proposed in XP Distilled is much improved, but still has a few flaws. Even Chris and Roy notice this:
The main idea behind this practice is to make a rough plan quickly and refine it as things become clearer. The artifacts of release planning are:
- A stack of index cards, each containing a customer story, which will drive the project's iterations
- A rough plan for the next release or two, as described in Planning Extreme Programming by Kent Beck and Martin Fowler [6]
The critical factor that lets this style of planning work is to let the customer make business decisions and allow the programmer team to make technical ones. Without that, the whole process falls apart.
Programmers determine:
- Estimates of how long it will take to develop a story
- Cost implications of using various technology options
- Team organization
- The "risk" of each story
- Order of story development within an iteration (doing risky items first can mitigate risk)
Customers determine:
- Scope (the stories for a release, and the stories for each iteration)
- Release dates
- Priority within each release (which features get developed first, based on business value) [4]
So what is wrong with the new “release planning” concept? First, there is a mandate to use index cards. This is unlikely to work with more conservative companies for a variety of reasons. They are the customer after all.
Second, the programmers should not determine the order of story development within an iteration. This assumes that all stories scheduled for an iteration are of equal priority. If every story is completed by the end of the iteration, it does not matter. If not, and the scope needs to be reduced (the scheduled end date should never slip), the customer may not be getting the stories completed that they had prioritized. Customers should be informed of the benefit of doing stories in a certain order, i.e., that it will take less time if one story is done before another. But it is the customer that makes the decision. Anything else and the programmers are making a business decision, which violates the separation of business and technical decisions rule.
If the “order stories are developed” responsibility is given to the customers, there is a nice symmetry in the two lists. Again, Scrum handles all of this nicely and in a more flexible manner.
Perhaps most importantly XP does not have a process for continuous improvement. This is The Goal for all empirical processes, especially processes managed on the edge of chaos [7]. Many XP practitioners have suggested that a retrospective become part of the XP process at the end of each iteration. While every 2 weeks is certainly better than never, it does not enable the same bottom-up process improvement inherent in Scrum.
Collins and Miller propose a system of modifications to XP that address most of the following concerns. There system still does not address continuous improvement in an adequate manner, so I still think Scrum is the better choice.
What to Do
Extreme Programming, then, is really only missing a management wrapper. Fortunately,
Scrum (see Scrum Review) provides all necessary management functions for a software development project or group of projects. It is scalable (which XP is not) and provides a process for continuous improvement. It also speaks the language of business by keeping all estimates in hours, days, or weeks. Problems are not buried under a “fudge factor” but brought to the forefront for immediate resolution. Scrum includes all the quantitative metrics that need to be tracked and is Big Visible Chart-friendly. Scrum also works with integration projects where software is only a part of the final delivered solution.
This is the basis for the XP@Scrum agile software development methodology. It is also very similar to XBreed (read “cross-breed”), but XBreed is more focused on multiple project integration with reusable components. XP@Scrum is eminently more usable and applicable to most software development efforts.
References
[1] – Extreme Programming Applied: Playing to Win. Ken Auer and Roy Miller. Available at Amazon.com.
[2] – Extreme Programming FAQ, by John Brewer of Jera Design.
[3] – The Costs and Benefits of Pair Programming. Laurie Williams and Alistair Cockburn.
[4] – XP Distilled. Chris Collins and Roy Miller, RoleModel Software Inc.
[5] – Critical Chain. Eli Goldratt. Available at Amazon.com.
[6] – Planning Extreme Programming. Kent Beck and Martin Fowler. Available at Amazon.com.
[7] – The Goal: A Process of Ongoing Improvement. Eli Goldratt. Available at Amazon.com.
Websites
Excellent Analysis of XP
General XP Information