Practical software metrics psm

When the set of all possible items in a population is very large it may be too costly or time consuming to do a comprehensive analysis of all of the items. For example, during an audit, there is just not enough time or resources to talk to every auditee, witness every process step or look at every quality record. If the customer base is large, it may be too costly to survey all the customers to determine their satisfaction level.

Evaluating or estimating attributes or characteristics of the entire system, process, product or project through a representative sample can be more efficient while still providing the required information. To legitimately be able to use a sample to extrapolate the results to the whole population requires the use of one of four statistical sampling methods.

This paper introduces the reader to a practical process for establishing and tailoring a software metrics program that focuses on goals and information needs. The process provides a practical, systematic, start-to-finish method of selecting, designing and implementing software metrics.

It outlines a cookbook method that the reader can use to simplify the journey from software metrics in concept to delivered information.

Utilize this template to document the design of your software metrics reports. This template can help you implement the 12 Steps to Useful Software Metrics. Software metrics don't solve problems — people solve problems.

What software metrics can do is provide information so you can make informed decisions and better choices.

In other words, you need decision criteria to obtain guidance that will help you interpret the measurement results. This paper shows you how to establish useful decision criteria for different types of metrics. In order to illustrate the power of the Process Measurement FrameworkSM, real examples from industry are used.

The Process Measurement FrameworkSM helps to ensure that all metrics are collected on a form, in a document, or in a database. This paper won the best paper award at the 13th International Conference on Software Quality, October Per Dr. Wilson, a testable requirement is one that is precisely and unambiguously defined, and one for which someone must be able to write a test case that would validate whether or not the requirement has or has not been implemented correctly.

The number of testable requirements may be very different from the number of test cases. There are a number of reasons for this: A testable requirement may require more than one test case to validate it.

Some test cases may be designed to validate more than one testable requirement. Testable requirements appear to have the granularity and flexibility to make earned value a practical tool for software developers. To redirect management focus on meeting requirements, a Systems Engineering SE process improvement team rewrote the SE procedures.

The new procedures mandated requirements traceability and the use of technical performance measures TPM. The time-phased plan for each project phase and each build should include milestones that objectively define the functional content to be achieved at that point. The milestones should be defined in terms of incremental functionality, both the number of testable requirements and the functional capabilities to be achieved.

An incremental milestone is normally defined as percent of the requirements needed to achieve a functional capability. However, for earned value purposes, it can also be targeted as a lesser percentage. The functionality targets should be documented as part of the criteria for completing the milestone and taking objective earned value.

The percentage target is normally related to the targeted quality, as measured by defects. Planning for Defects and Rework In planning for incremental builds, the Statement of Work for all builds subsequent to the first should include an estimate for rework of requirements or code to fix defects that were found in previous builds but will be fixed in subsequent builds. To ensure adequate budget and period of performance, the planning assumptions should include a planned rate or number of defects to be found in each build, and a plan to fix these defects within the rework Statement Of Work of each build.

Furthermore, rework should be planned in separate work packages. Failure to establish a baseline plan for rework and to accurately measure rework progress caused many projects to get out of control. The procedure requires that rework is planned in separate work packages from the initial development effort and that objective metrics for rework are used for earned value.

Selection and Use of Software Metrics For tracking progress against a plan using EVM, the most effective measures are those that address the issues, product size and stability, and schedule and progress. Three measurement categories are mapped to these issues: functional size and stability, work unit progress, and incremental capability. The specific measures to be discussed are requirements, requirements status, component status, test status, increment content-components, and increment content-functions.

Issue: Product Size and Stability. Category: Functional Size and Stability The requirements measure counts the number of requirements in the system or product specification. When incremental builds are planned, this measure is also the basic component of the measure, increment content-function, as discussed below.

Issue: Schedule and Progress. Category: Work Unit Progress. The recommended measures for Work Unit Progress are requirements status, component status, test status, increment content-components, and increment content-functions. Requirements Status : The requirements status measure counts the number of requirements that have been defined and allocated to software components, allocated to test cases, and successfully tested.

When used to measure test status, the measure is used to evaluate whether the required functionality has been successfully demonstrated against the specified requirements. Some requirements may not be testable until late in the testing process. Others are not directly testable or may be verified by inspection. This measure is ideal for EVM because it is objective. The budget allocated to requirements may be equally distributed or weighted according to the estimated effort for those requirements.

Since implementing requirements-based EVM, program progress has never been significantly overstated and the management control system has provided more reliable data and earlier warning of program problems see Table 1, page Tables 1 through 5 see page 28 are abstracts of measures that are fully described in PSM. Per PSM, there are three aggregation structures to accumulate measurement data. Tables 1 and 2 are component-based and functional-based aggregation structures.

Tables Abstracts of Measures Fully Described in PSM Component-based aggregation structures are derived from the relationship of the system components within a particular architecture or design. For projects that implement an incremental development approach, lower-level components such as units and configuration items are usually mapped to the incremental delivery products as part of the aggregation structure.

Functional-based aggregation structures define the functional decomposition of system requirements. They are often mapped to the system design components.

If they are mapped to design components, then measures of the requirements such as the number of requirements tested can be aggregated and evaluated for a particular function. The data collection level describes the lowest level at which data is collected to allow problems to be isolated and understood.

It can then be rolled up using the aggregation structure. Component Status : The component status measure counts the number of software components that complete a specific activity. An increase in the planned number of components may indicate unplanned growth and cost impacts. However, the number of components, although not constant, is the perpetual denominator for measuring percent complete. In the initial design phase for EVM, a unit of measurement should be selected based on the design standards and practices employed for each build.

This may be modules, packages, pages, or another appropriate component. Completion of components during the design and implementation phases should be based on component reviews, inspections, walkthroughs or specified tests, as appropriate see Table 2 page Test Status : The test status measures count the number of tests attempted, executed to completion, or completed successfully.

It can be applied for each unique test sequence, such as component, integration, system, and regression test and is a good basis for earned value see Table 3, page Therefore appropriate resources must be allocated for the measurement process to work-effectively.

The most important roles would be executive manager, project or technical manager, measurement analyst and project development team. The basic concern in this activity is indentifying the software issues that have the greatest potential impact on the project. The next tailoring activity is to define appropriate project-specific measures. The measures are selected by applying the PSM defined measurement tailoring mechanisms of common software issues, measurement categories and measures.

The main objective in this activity is to select measures most appropriate to the issues. For tailoring software measures, PMS defines tailoring activities in detail. Such as indentify and prioritize project issues, select and specify project measures, integrate measures into the software process and organization software measurement.

Data is collected and converted into the information that provides a basis for action by the project manager. During the measurement application, the particular measures are collected and analyzed to provide the feedback on the issue needed for effective decision-making. Practical Software Measurement PSM Process Collecting and understanding measurement data is the first activity in analyzing project issue. The key task in collecting and processing data are accessing the data, verifying the data and normalizing the data.

Second activity is analyzing issues. During the analysis activity measurement indicators are generated from the data collected activity as part of a systematic analysis process. The three types of analysis include estimation, feasibility and performance.

Finally the last activity is making decisions. This last activity in the PSM measurement application process encompasses three major tasks. Such as reporting information, selecting alternative course of action and taking appropriate action based on that information. However, measurement represents a significant change in an organization therefore, there are six key activities have describe by PSM.

Such as obtain organizational support, define measurement responsibilities, define measurement responsibilities and provide measurement resources, initiate the measurement process, make use of measurement result. For the measurement resources activity, we need to ensure that we have measurement tools for instance, database, graphing and reporting tools, software analysis and modeling tools, and measurement application tools etc. Measurement training required to the project personal for implantation of measurement process.

Therefore PSM process play key role in 8. PSM process describes an information-driven measurement process that will address the unique technical and business goals of a company. This guidance in PSM represents the best practices used by measurement professionals within the software and system acquisition and engineering communities.

By the help of Practical Software Measurement process activities, we would come to know how we can start the measurement process which is most difficult task in my option. We need to ensure that everyone in the organization understands both capabilities and limitations of the PSM measurement process.

We need to ensure that only the required measures are implemented based on the issues and objectives of the organization. We may use basic commercially available database, spreadsheet, and presentation graphics for the initial measurement process. More advance and sophisticated tools can be added as required.

Also all users must understand what the measurement data represents. At the end of PSM process we will get the measurement results. Those measurements should be made an integral part of the project or organization.

The project managers must be willing to understand all kind of feedback positive or negative result from the measurement analysis. PSM measurement process provides substantial benefit in addressing the original issues.

By using the PMS process we can clearly address the majority of the project issue and it can provide information to support project management decisions. Most importantly, the upper management can get the better understanding of the project. Cynthia Perkins Nov.