Introduction and Chapter Goals

An effective software estimate provides the information needed to design a workable software development plan. “Good estimates are key to project (and product) success. Estimates provide information to make decisions, define feasible performance, objectives and plans … bad estimates affect everyone associated with the project …” This chapter addresses the need for project metrics and the fundamental software estimation concepts and discusses the first three steps in the software estimation process.

Need for Efficient Software Project Management Metrics

When  some  managers  insist  that  “you  can’t  manage  what  you  can’t measure,”2 they are focusing on the ideal and ignoring the real. The fact is we manage things we can’t measure all the time. Pure research, product design, and manuscript development are managed without metrics or by using metrics that at best are inadequate and at worst are misleading. In reality, metrics used to manage intellectual and creative processes often do  not  provide  the  insights  managers  would  like  to have.  They  do  not answer  questions  such  as:  How  good  are  the  processes  we  are  using? What quality can we expect? When can we finish this effort? How much will it cost? As a result, managers tend to measure intellectual processes, including software engineering, qualitatively rather than quantitatively.

Alternate strategies to quantitatively measure the progress of intellectual and creative activities do exist, although they are best suited for measuring specific  issues  such  as  defect  rates  rather  than  broader  issues  such  as quality. Indeed, monitoring intellectual activities such as software development by using metrics to evaluate broad issues such as project progress is often counterproductive. However, measuring specific progress against an identified quantitative constraint to determine the likelihood of meeting that constraint is a realistic goal. For example, quantitative measures can be used to monitor specific issues such as defect rates, the sufficiency of assigned resources, and whether enough money or time is left to complete the project based on the task completion rate.

 

Many organizations  employ  this  type  of  issue-based  measurement, which is a tested and mature process, and resources are available to help determine effective metrics. For example, Practical Software and Systems Measurement,3 a guide describing a useful process with supporting measures, and Sixteen Critical Software Practices, a program manager’s guide,4 provide issue-based metrics linked to specific project practices. The practical software measurement (PSM) process can be applied to all stages of a software project: planning, requirements analysis, design, implementation,  and  integration  of  hardware  and  software.  It provides  a  means  to collect and analyze project data — includes estimates, plans, changes to plans, and counts of actual activities, products, and expenditures — at a sufficient level of detail to identify and isolate problems.

 

These resources provide measurement structures that address the needs of software managers. They rely on reasonable projections of targets and they evaluate narrow project factors or attributes against established project goals  and  issues.  Issue-based  measurement  helps  you  make  statements such as, “Based on your current milestone completion rate you will (or will  not)  meet  the  September  7  deadline.”  This measurement provides objective and quantitative information required to make informed decisions  that  affect  project  cost,  schedule,  and technical  performance objectives.

 

Metrics should be set and decided on as part of a software development plan  before  development  begins.  By  measuring  specific  project  factors, project  teams  can  gain  the  information  needed  to  monitor  key  issues related  to  progress  and  quality  and  performance  against  the  plan.  With current objective information, managers can also answer critical questions and take corrective actions early enough to avoid or minimize problems before they get out of hand.

 

Crucial to the measurement process is the ability to distinguish a metric, a  measure,  and  an  indicator.  A  metric  is  a  parameter  that  provides  a quantitative  standard  of  measurement  of  the  degree  to  which  a  system, component, or process possesses a given attribute. Software lines of code (SLOC) is a metric used throughout this book. A measure is quantitative evidence of the extent, amount, dimensions, capacity, or size of a specific attribute (metric) of a product or process. For example, we would show the measure of software size (software size is the metric) by saying it is comprised of 150,000 source lines of code (the measure of the metric).

An indicator is a metric or combination of metrics that provides qualitative information  describing  the  state  of  a  process,  a  project,  or  the  product itself, for example, number of defects per SLOC might be an indicator of project  testing  or  quality.  A  software  project  benefits  from  an  effective measurement process by acquiring the information needed to:

 

Enable effective communication

Keeping information current enables  effective  communication  among  stakeholders throughout all levels of an organization, reduces ambiguity, and enables supplier and acquirer organizations to accurately communicate status.

Make  timely  trade-offs

With  accurate  information,  software managers can objectively assess the effects of their decisions, which enables them to evaluate viable trade-offs that will better support project objectives.

Monitor progress toward meeting specific project objectives

Quantitative  information  enables  managers  to  answer  questions such as: Will the project meet its schedule if we continue with the same  productivity?  Can  we  anticipate  excessive  rework  if  our productivity  continues  at  the  current  level?  Will  we  deliver  a product  with  too  many  defects  to  meet  user  expectations?  The answers to such questions allow managers to track progress toward project and organizational objectives.

Identify and correct problems and address risk early

Measurement provides the information software managers and project staff  need  to  effectively  identify  and  manage potential  problems

before they become intractable.

Manage,  control,  and  contain  risk

Measurement  processes are integral components of risk management, which is a core best practice of any software project (see Chapter 10). Many software project measurements are leading indicators, the analysis of which enables managers to forecast project conditions. (Trailing indicators, on the other hand, provide information about past performance.)

By analyzing leading indicators based on quantitative information, managers  can  identify  risks  while  it  is  possible  to  mitigate  them effectively  and  will  have  the  information  required  to  analyze  a specific risk’s likelihood of occurrence and likely impact. In addition, the objective information that results from effective measurement enables managers to consistently monitor potential risks by setting  realistic  thresholds  against  which  they  can  evaluate  risks and monitor project performance against established metrics.

Defend and justify decisions

By  measuring  specific  project factors, managers are provided the objective information regarding performance (i.e., current, historical, and trend) they need to make effective  decisions  regarding  schedule,  cost,  product  (or  code) growth, quality, developer capability and process maturity, technology, and user satisfaction. Such information enables project man-agement,  stakeholders  and  staff  to  accurately determine  whether a project is meeting its goals and requirements.

 

As  important  as  measurement  is,  it  should  be  noted  that  not  all indicators  of  a  project’s  success  are  measurable.  Many  indicators  are subjective or result from qualitative factors. For example, no quantifiable

metric exists to objectively measure morale. Managers can get a feel for the state of staff morale, but any objective measure would for all practical purposes  be  meaningless.  The  effects  of  poor  morale,  however,  can potentially be identified by tracking project performance through earned value, which measures output against effort. (For a detailed discussion on

earned value see Chapter 9.)

Core Metrics Categories

Ideally, the following attributes of a software project would be measured:

1.   Cost

  a.  Staff effort

  b.  Phase effort

  c.  Total effort

2.   Defects

  a.  Found or corrected

  b.  Effort required

  c.  Defect source and class

  d.  Defect density

3.   Process characteristics

  a.  Development language

  b.  Process model

  c.  Technology

4.   Project dynamics

  a.  Changes or growth in requirements or code

  b.  Schedule and schedule compression5

5.   Project progress (earned value; see Chapter 9)

  a.  Development dates

  b.  Project size

  c.  Total effort

  d.  Budget performance

  e.  Schedule performance

  f.   Cost performance index (CPI)6

  g.  Schedule performance index (SPI)7

  h.  To-complete performance index (TCPI)8

6.   Software structure

  a.  Size

  b.  Complexity

 

Project  managers,  stakeholders,  and  staff  members  can  use  software metrics  to  more  accurately estimate  progress  toward  project  milestones, especially when historical (trailing) indicators or trend data are available.

Measurement  enables  project  participants  to  plot  weekly  or  monthly changes that can reveal trends, and in turn this information can enable prediction of problem areas such that action can be taken.

These metrics and others have their basis in the size projections, cost estimates,  and  schedules  and  work  plans  that  are  based  on  them.  Size and cost estimates serve as basis for many project measures. They are the basic criteria used to evaluate project performance, progress against plans, product  quality  based  on  defect  density,  and  other  measures  that  are needed to accurately monitor schedule, cost, product (or code) growth, quality, and other key factors.

Size and cost estimates are not the same as targets, although estimates may be used as targets. In principle, estimates should be used to assess the feasibility of targets (i.e., budget or schedule constraints) and to confirm that the current status of a project indicates that final project targets are feasible.