Notes from the Software Estimation webcast.

“The gap between the best software engineering practice and the average practice is… perhaps wider than in any other engineering discipline.”
Fred Brooks

Only 32% of projects were less than 20 percent late (including on-time). Average software project is 100 percent late (takes twice as long as expected).
Standish Group

Three parts to project success:

• Good target setting – statement of real business need
• Effective control – good project management controls
• Accurate estimates – focus of the rest of the webcast

Understanding an Estimate’s Value

Why estimate?

• Provides info on how many resources and how long it will take
• Provides info on the risk/reward of a project
• Insight into kinds project risks

Estimate Value

• Is a function of the potential gain if correct and potential loss if incorrect
• Have to balance the investment you put into an estimate with the value you get back

Average companies treat estimation as a black art. Best companies treat estimation as a specialized skill.

Embracing Uncertainty

Software developers generally need precision.

“Perfect is the enemy of the good!”

Just because it’s not precise does not mean it’s not valuable.

Cone of Uncertainty

Average company practices:

• Give precise estimate/commitment
• Estimate once and never change it
• Spend too much time estimating early when little info is available

Best company practices:

• Provide estimates in ranges that reflect uncertainty
• Use low-cost, low-fidelity methods at wide end of cone
• Re-estimate as the cone narrows

Learn to Think in Size

How big is the project?

Estimation Workflow

• Size -> Effort -> Schedule
• Test of estimate: would you be willing to risk your job?

Size Measurements

• Lines of code
• Function points
• Screens, reports, tables
• Other (web pages, GUI components, classes)
• What works best for you?

SLOC (source lines of code) - has many faults, but can still be useful:

• Easy to collect
• Nearly all estimation tools are based on SLOC
• Effort per SLOC roughly constant across languages

Average companies:

• Think in terms of effort and schedule, not size
• Estimates based on gut feel than data
• Create estimates that are hard to defend

Best Companies:

• Use multiple size measurements
• Use tools to help think in size
• Able to defend estimates with data based on size measurements (i.e., XL modules = 10,000 LOC, which then takes x amount of time)

Collect and Use Historical Data

• The use of historical data is the biggest differentiating factor between successful and unsuccessful estimation.
• Use historical data to compute effort, then use historical data to compute schedule. An analytical process removes the emotion.
• Productivity is an organizational characteristic that cannot be changed significantly from one project to another. Past performance is the best indicator of future performance.

For each project, the following is the minimum:

• Collect size measurements (SLOC, FP, etc.)
• Effort (staff months)
• Time (calendar months)
• Defects
• The key is to be consistent across projects!

Can collect additional data that is more specific on each of the above.

Haven’t collected any data? Go back and “mine” it or estimate it.

Average companies:

• Never collect historical data
• Never learn from past experience
• Repeat the same estimation errors over and over

Best companies:

• Collect historical data from every project
• Analyze data to learn from it
• Improve ability to estimate over time, and prove it

Establish an Estimation Procedure

Benefits

• Encourage uniform results
• Allows retracing steps in case of failure
• Protects against biases
• Ensures proper use of historical data
• Aligns expectations between estimate producer and estimate customer

Elements

• Required and optional steps
• Description of each step’s imprecision
• Standard re-estimation points
• How to combine multiple approaches (look for convergence)
• Defines when an estimate becomes a commitment
• Changes approaches depending on location in cone of uncertainty

See presentation for a detailed description of a “best in class” example.

For More Info

• COCOMO – http://sunset.usc.edu/research/COCOMOII/
• Putnam SLIM
  • Construx Estimate – http://www.construx.com/estimate/
  • Putnam’s company – http://www.qsm.com/
• Measures for Excellence: Reliable Software on Time, Within Budget by Larry Putnam
• Software Cost Estimation with Cocomo II by Barry Boehm