Project Management Journal       June 1998 vol 29 Number 2    page 8

      [This extract excludes tables and images from original
      text.]
Cost Benefit Discussion for Knowledge-Based Estimation Tools

Charles Douglis, Software Productivity Research One New England Executive Park Burlington, Massachusetts 01803-5005 USA. chas@spr.com,chas@spr.com, http://www.spr.com


Supporting technology investments with a rational return on investment (ROI) calculation is both unusual and difficult to accomplish in the software industry. Software executives often make technology decisions based upon gut feelings, which may prove to be correct but are also susceptible to influences from vendor marketing and advertising campaigns.

We have prepared the following analysis to help software executives quantify the benefits associated with purchasing knowledge-based estimating tools with the functionality of SPR KnowledgePLAN. For this paper, knowledge-based estimation tools are defined as any tool using a credible internal or external knowledge base to develop a detailed task-level plan that considers project size, complexity classification, languages, and attributes (personnel, technology, process, and environment). Plans developed by these tools are seamlessly interfaced with standard project management tools.

Because we realize no single calculation captures the complexity of this issue, this examination presents several ways of calculating the cost savings (and correspondingly the ROI). The paper is limited to three of the many distinct ways of viewing the benefits or savings from software estimating tools. These approaches are not mutually exclusive and, in fact, should be viewed as additive.

Reduced Planning Time

This analysis is both the simplest and most tangible of the three approaches. It focuses on actual cost savings per project associated with using automated estimation and planning tools versus a manual approach. While this cost savings may be small on an individual project basis, it should be multiplied by the number of project managers within the organization responsible for planning and the total number of project plans developed within the target time frame. It assumes that planning includes the development of a detailed project plan at the task level.

The calculation in Table 1 takes into account two factors: the number of planning days currently required to develop a manual estimate (this response will vary by organization based upon organization size, project size, and software process discipline) and anticipated time savings based upon using knowledge-based estimating tools.

We don't anticipate that savings from this analysis alone will provide adequate economic justification for the purchase of knowledge-based estimation tools; however, in organizations requiring detailed manual project estimates, the savings will be surprisingly large. In other software environments, the cost benefit calculated in this analysis, when coupled with the savings in the analyses below, will result in a compelling ROI justification.

Improved Planning Results

The results from this analysis potentially represent the greatest savings to an organization. This analysis focuses on efficiency gains from better planning and savings associated with avoiding canceled projects. In fact, leading industry analysts view canceled projects as a major problem in the software industry, as demonstrated by the following quotes from the Gartner Group: 


    Absent significant changes in project management a 100-person
    applicaiton development group can expect to spend more than
    $10 million on cancled software projects over the next five
    years.

    Companies could reduce their project failures 80% by using
    standard procedures for estimating costs, tracking time and
    resources and document changes.
Efficiency Gains.

This analysis suggests that improved project planning through knowledge-based estimation tools will improve overall project efficiency. It is generally believed a realistic picture of the resources and technology required to successfully complete the project emerges when software organizations prepare compreher1sive project estimates based UpOtl either internal historical experience or a credible external knowledge base.

Therefore, project teams armed with this realistic project plan will allocate resources more ettcctively, gain adequate funding for the project, shorten project schedules, and imple ment the right blend of perso'1tlel, process, and technology (correspondingly these same project teams will minimize utilization of external consultants, personnel turnover due to excessive overtime, morale pr~oblctns, lengthy reaming curves, and ineffective technology ~Jecisions.

Instead of performing the extremely difficult task of quantifying each of the above-mentioned benefits or pitfalls, the analysis in Table 2 applies an overall efficiency gain percentage to the related department costs to capture the total benefit. Our recommended efficiency range of 2% to 7% is highlighted.

Efficiency gains realized within organizations may be treated in very different ways. In some software organizations, real project cost reductions will translate into overall cost reductions, while in other environments these same savings will translate into increased capacity (i.e., we can complete more work with less resources). However, in either case, the benefit is appropriately used in an ROI calculation.

Canceled Projects. This analysis is based upon the following assumption: The risk of canceled projects is increased without effective knowledge-based estimation/planning tools. If an organization deploys this class of tools, we believe risky projects either are not started because the risk is fully

8 Project Management Journal June 1998


understood stood prior to commencing development or are effectively implemented because the major risk factors are identified and managed. The likelihood of project cancellation is generally a function of size. According to industry expert Casper Jones: "Small software products are successful in the majority of instances, but the risk and hazards of cancellation or major delays rise quite rapidly in both number and severity as the overall application size goes up. Indeed, the development of large applications in excess of 5,000 function points (which is roughly 400,000 source code statements in procedural programming languages) is one of the most hazardous and risky business undertakings of the modern world."

The analysis in Tables 3 & 4 demonstrates the significant costs associated with canceled projects. The center columns represent percentage of projects canceled when organizations use manual estimation or planning methods. That percentage is not factored into the Cost Prior to Cancellation but indicates the correlation between project size and cancellation risk.

Some organizations may be large enough to absorb these losses, while others may experience catastrophic results. Obviously, if knowledge-based estimation tools could eliminate the cancellation of one large project over a three-year period, the investment would more than pay for itself

Long-Term Benefits

These benefits are potentially enormous but difficult to measure in the context of this paper. Effectively addressing this general area would require detailed information from other functional areas such as sales, marketing, software maintenance, and the user or customer community

Even though these benefits are not easily quantified, their existence is irrefutable and can only serve to support the overall cost benefit analysis.

Many experts view This category of benefits, while not specifically quantified in this discussion, as extremely important. Knowledge-based estimation/planning would yield the following benefits:

• Reduced time to market

• Improved user or customer satisfaction

• Elimination of sales and marketing costs due to delayed product release schedules

• Improved product quality resulting in lower support costs.


Conclusion

The approaches outlined above are not intended to pinpoint precise formulas for calculating the cost benefit of software estimating tools, but rather provide a framework from which software managers can develop their own ROI analysis.

Efficiency gains achieved through improved planning practices seem to provide the most fertile ground for devel oping ROI calculations. Most organizations can easily imag- ine improving productivity within the 2% to 7% range. At this level, defensible cost benefit analysis supporting soft- ware estimating (ool acquisition are quite compelling and calculations associated with reduced planning time and cancelled projects become icing on the cake.

Effective planning techniques has been difficult to initiate in software organizations, while many established indus- tries have accepted estimating as a necessary ingredient for success. For example, few building construction projects would commence without a thorough set of architectural drawings accompanied by detailed cost estimates. The software industry is quickly approaching the era when the same level of rigid will be required and industry trends, such as outsourcing, Year 2000 initiatives, and fixed-priced software development serve as catalysts for this transformation

Charles Douglis is president and chief operating officer of Software Productivity Research. Mr. Douglis served as vice presi- dent with Visage and audit manager for Coopers & Lybrand before starting his own consulting firm. Mr. Douglis received a B.S. in political science from St. I awrence University and a M.S. in finance/accounting from Northeastern University. He is also a graduate of the Harvard University program on Negotiation for Senior Executives.