An effective software estimate provides the information needed to design a workable software development plan. “Good estimates are key to project (and product) success. Estimates provide information to make decisions, define feasible performance, objectives and plans … bad estimates affect everyone associated with the project …” This chapter addresses the need for project metrics and the fundamental software estimation concepts and discusses the first three steps in the software estimation process.
When some managers insist that “you can’t manage what you can’t measure,”2 they are focusing on the ideal and ignoring the real. The fact is we manage things we can’t measure all the time. Pure research, product design, and manuscript development are managed without metrics or by using metrics that at best are inadequate and at worst are misleading. In reality, metrics used to manage intellectual and creative processes often do not provide the insights managers would like to have. They do not answer questions such as: How good are the processes we are using? What quality can we expect? When can we finish this effort? How much will it cost? As a result, managers tend to measure intellectual processes, including software engineering, qualitatively rather than quantitatively.
Alternate strategies to quantitatively measure the progress of intellectual and creative activities do exist, although they are best suited for measuring specific issues such as defect rates rather than broader issues such as quality. Indeed, monitoring intellectual activities such as software development by using metrics to evaluate broad issues such as project progress is often counterproductive. However, measuring specific progress against an identified quantitative constraint to determine the likelihood of meeting that constraint is a realistic goal. For example, quantitative measures can be used to monitor specific issues such as defect rates, the sufficiency of assigned resources, and whether enough money or time is left to complete the project based on the task completion rate.
Many organizations employ this type of issue-based measurement, which is a tested and mature process, and resources are available to help determine effective metrics. For example, Practical Software and Systems Measurement,3 a guide describing a useful process with supporting measures, and Sixteen Critical Software Practices, a program manager’s guide,4 provide issue-based metrics linked to specific project practices. The practical software measurement (PSM) process can be applied to all stages of a software project: planning, requirements analysis, design, implementation, and integration of hardware and software. It provides a means to collect and analyze project data — includes estimates, plans, changes to plans, and counts of actual activities, products, and expenditures — at a sufficient level of detail to identify and isolate problems.
These resources provide measurement structures that address the needs of software managers. They rely on reasonable projections of targets and they evaluate narrow project factors or attributes against established project goals and issues. Issue-based measurement helps you make statements such as, “Based on your current milestone completion rate you will (or will not) meet the September 7 deadline.” This measurement provides objective and quantitative information required to make informed decisions that affect project cost, schedule, and technical performance objectives.
Metrics should be set and decided on as part of a software development plan before development begins. By measuring specific project factors, project teams can gain the information needed to monitor key issues related to progress and quality and performance against the plan. With current objective information, managers can also answer critical questions and take corrective actions early enough to avoid or minimize problems before they get out of hand.
Crucial to the measurement process is the ability to distinguish a metric, a measure, and an indicator. A metric is a parameter that provides a quantitative standard of measurement of the degree to which a system, component, or process possesses a given attribute. Software lines of code (SLOC) is a metric used throughout this book. A measure is quantitative evidence of the extent, amount, dimensions, capacity, or size of a specific attribute (metric) of a product or process. For example, we would show the measure of software size (software size is the metric) by saying it is comprised of 150,000 source lines of code (the measure of the metric).
An indicator is a metric or combination of metrics that provides qualitative information describing the state of a process, a project, or the product itself, for example, number of defects per SLOC might be an indicator of project testing or quality. A software project benefits from an effective measurement process by acquiring the information needed to:
Keeping information current enables effective communication among stakeholders throughout all levels of an organization, reduces ambiguity, and enables supplier and acquirer organizations to accurately communicate status.
With accurate information, software managers can objectively assess the effects of their decisions, which enables them to evaluate viable trade-offs that will better support project objectives.
Quantitative information enables managers to answer questions such as: Will the project meet its schedule if we continue with the same productivity? Can we anticipate excessive rework if our productivity continues at the current level? Will we deliver a product with too many defects to meet user expectations? The answers to such questions allow managers to track progress toward project and organizational objectives.
Measurement provides the information software managers and project staff need to effectively identify and manage potential problems
before they become intractable.
Measurement processes are integral components of risk management, which is a core best practice of any software project (see Chapter 10). Many software project measurements are leading indicators, the analysis of which enables managers to forecast project conditions. (Trailing indicators, on the other hand, provide information about past performance.)
By analyzing leading indicators based on quantitative information, managers can identify risks while it is possible to mitigate them effectively and will have the information required to analyze a specific risk’s likelihood of occurrence and likely impact. In addition, the objective information that results from effective measurement enables managers to consistently monitor potential risks by setting realistic thresholds against which they can evaluate risks and monitor project performance against established metrics.
By measuring specific project factors, managers are provided the objective information regarding performance (i.e., current, historical, and trend) they need to make effective decisions regarding schedule, cost, product (or code) growth, quality, developer capability and process maturity, technology, and user satisfaction. Such information enables project man-agement, stakeholders and staff to accurately determine whether a project is meeting its goals and requirements.
As important as measurement is, it should be noted that not all indicators of a project’s success are measurable. Many indicators are subjective or result from qualitative factors. For example, no quantifiable
metric exists to objectively measure morale. Managers can get a feel for the state of staff morale, but any objective measure would for all practical purposes be meaningless. The effects of poor morale, however, can potentially be identified by tracking project performance through earned value, which measures output against effort. (For a detailed discussion on
earned value see Chapter 9.)
Ideally, the following attributes of a software project would be measured:
1. Cost
a. Staff effort
b. Phase effort
c. Total effort
2. Defects
a. Found or corrected
b. Effort required
c. Defect source and class
d. Defect density
3. Process characteristics
a. Development language
b. Process model
c. Technology
4. Project dynamics
a. Changes or growth in requirements or code
b. Schedule and schedule compression5
5. Project progress (earned value; see Chapter 9)
a. Development dates
b. Project size
c. Total effort
d. Budget performance
e. Schedule performance
f. Cost performance index (CPI)6
g. Schedule performance index (SPI)7
h. To-complete performance index (TCPI)8
6. Software structure
a. Size
b. Complexity
Project managers, stakeholders, and staff members can use software metrics to more accurately estimate progress toward project milestones, especially when historical (trailing) indicators or trend data are available.
Measurement enables project participants to plot weekly or monthly changes that can reveal trends, and in turn this information can enable prediction of problem areas such that action can be taken.
These metrics and others have their basis in the size projections, cost estimates, and schedules and work plans that are based on them. Size and cost estimates serve as basis for many project measures. They are the basic criteria used to evaluate project performance, progress against plans, product quality based on defect density, and other measures that are needed to accurately monitor schedule, cost, product (or code) growth, quality, and other key factors.
Size and cost estimates are not the same as targets, although estimates may be used as targets. In principle, estimates should be used to assess the feasibility of targets (i.e., budget or schedule constraints) and to confirm that the current status of a project indicates that final project targets are feasible.