Training Measurement Program Implementation Process

This is one of my longer posts, but the subject of training in business is crucial.  Therefore, I thought I would include these thoughts.

Introduction

Format for Evaluation Guides

·    The Sought after Behavior
The “sought after behavior” is to document the actions that embody upon what the Measurement Program activities and tasks should be conducted.

The Clues
“Clues” for the Measurement Program are the artifacts, data, or evidence that the behavior is actually accomplished even without specific guidelines, published schedules, or actions.\

·    Evaluation Criteria
The rules and evaluation methods used to evaluate observed behavior, clues, or surrogates to determine if this Measurement Program criteria has been met.

·    Best Practice
A Best Practice is a behavior that is recommended because it meets many organizations needs to achieve the “sought after behavior” for the Measurement Program criteria. A Best Practice is a guideline recommendation and not an absolute. Different organizations may have specific needs that either eliminate the need, or otherwise meet the “sought after behavior” without the Best Practice documented.

1      Completeness

Does the Measurement Program have all six elements?

1)  Goal setting;

2)  Method definition;

3)  Metrics definition;

4)  Analysis;

5)  Determination of findings; and,

6)  A process for determining revisions to the goals?

 

The Evaluation Criteria focus on whether the established Measurement Program for Work Performance has all of these process and behavioral elements.

1.1      Goal setting

The Sought after Behavior

The Goal Setting behavior sought is to incorporate specific goals for the Measurement Program that show the state of business activity.

 

The Clues
“Clues” for Goal Setting include Management Objectives, Six-Sigma Initiatives, and Value Planning.

 

Evaluation Criteria
There must be written criteria documenting the goal setting behavior (defining resources, tasks, responsibilities, and schedule) that directly cover the Measurement Program and directly relate to performance measurement.

 

Best Practices
A Best Practices are for the goal setting behavior of the Measurement Program to be integrated into the development, decision making, and dissemination of corporate/organizational budgets and allocations.

 

 

1.2      Method Definition

The Sought after Behavior

Methods are the collections of work effort and measurement to get to a result. Metrics are the numbers and indices that result from measurements.

The Clues

Clues to good methods include organized efforts that produce results that can be verified, compared, and repeated.

Evaluation Criteria

There must be a list of measurement methods (for example, defined process points where activities or tangibles are counted; or, surrogates that can be repeat ably counted – such as entries in a CRM application) that are applied to Performance Measurement that are agreed upon by both those measured and those using the measurements.

Best Practices

A Best Practices are for the methods of activities for data collection, collation and reduction processing, and for work force management to be integrated into the operational systems supporting the effort (for example, automated data collection and reporting thru the CRM and SFA systems).

 

 

1.3      Metrics Definition

The Sought after Behavior

Metrics must be formally defined with rules, exception handling, and consistent behavior.

The Clues

Some metrics that affect territory choices, accounting cut-offs, and splitting compensation affects metrics.

Evaluation Criteria

For each metric quantity (each term in widgets sold per day, or service contracts per customer) there should be written rules defining the vocabulary, the counting rules, or the business definitions that make the metric quantity useful for comparisons, auditable, and repeatable).

Best Practices

The written rules should be widely available (not just as “shelf space holders”) as a document that is revised and managed actively.

 

 

1.4      Analysis

The Sought after Behavior

A good metrics program will have both the roles and responsibilities to analyze collected metrics and generate actions to change behavior in a desired path.

The Clues

A clue to a good program is someone assigned the regular operation work to analyze metrics on a regular basis. A surrogate function is someone watching the trends for metrics with the responsibility to break results

Evaluation Criteria

Document who has the role, and the responsibility to analyze, collect and process metrics on a regular basis (at least during the performance Measurement Program) to deliver analysis to management.

Best Practices

The analysis, in an organization with Best Practices, will have an internal, or combination internal and external, review of the analysis for quality control.

 

1.5      Determination of Findings

The Sought after Behavior

The analysis of metric data is incomplete without a person to translate the analysis work (with metrics) into series of corrective actions, countermeasures, or new analytical tasks.

The Clues

Clues to a regular process by managers are the operational reviews, staff reviews, or customer quality reviews that occur on a regular basis in many organizations.

Evaluation Criteria

For each scheduled period where analysis occurs, there must be a defined role (people resources), and schedule (calendar resources) where the analysis is reviewed and findings – determinations of conclusions and formulating action plans as the results from analysis of values from the metrics program.

Best Practices

The Best Practicess are to apply a consistent set of responses to similar (normalized) metric responses.

 

1.6      Revision Process

The Sought after Behavior

Regular revisions to the Measurement Program may include adding, subtracting, or acting on metric data.

The Clues

Business processes that lead to territory, commission reallocation, or staff level changes also are good fodder to look at Metric Program data.

Evaluation Criteria

There must be a revision process that takes in the business level of the Measurement Program (people, process elements, methodologies, and new corporate objectives) and makes revisions to the Measurement Program.

Best Practices

The Best Practices are to deploy a process that creates regular versions of the metrics data, methods, and implementation techniques as a part of a regular business process revision schedule.

 

2      Formal Process

2.1     Is there a formal and empirical plan for defining goals, and devising methods by which those goals may be achieved?

2.2     Once there is a method for defining the methods and goals, it is essential that there be a process for identifying the metrics associated with each metric. Is there a formal process for selecting the metrics?

2.3     Are these processes documented?

2.4     Are these processes supported by tools for the definition and selection?

 

The Evaluation Criteria focus on the formality of the methodology and process steps. ‘Formality’ isn’t the mathematical or work-flow definitions – ‘Formality’ here is the ability of different organizations to follow the defined steps and activities that reach the same level of quality goals and methods.

 

2.1      Formal and Empirical Plan

The Sought after Behavior

A formal process to select the goals (and matching methods) for the organization. Methods are expressed in action plans to produce a desired outcome. “Empirical” means that the methods are based on the numbers or systematic observation of business activities.

The Clues

“Strategic Plans”, goals tied to mission statements, or management initiatives goals are good places to start.

Evaluation Criteria

The measurement process being certified should incorporate a formal process with internal control steps to ensure application of the process to meet desired outcomes.

Best Practices

The Best Practices are to define the empirical plan with a formal definition of the relationships between the Measurement Program and the metric methods deployed to achieve the defined goals.

2.2      Defining Metrics for Methods

The Sought after Behavior

Measurement processes for work effort or actions are selected for their ability to produce meaningful metrics that can be analyzed.

The Clues

Metrics selected must have a direct relationship to the method being measured. i.e. an increase in expense dollars may not relate to the number of customers contacted.

Evaluation Criteria

The measurement process being evaluated should define metrics, methods, validation data, and quality attributes for the methods.

Best Practices

A Best Practices for metric definition provides both a formal definition in prose and implementation in an associated spreadsheet or computer program where learning or quality verification can be assured of repeatability and consistent rule applications of the methods counting rules and units.

2.3      Process Documentation

The Sought after Behavior

Measurement programs need the backing of management to be successful. A formal review, approval, and participation commitment process is important such that management has a stake in the construction, operation, and results of the Measurement Program.

The Clues

A less successful program will allow a champion, or single, manager to initiate the Measurement Program. Without a consensus that analysis of the metrics program and actions to follow findings the Measurement Program will not have organizational impact nor a high participation potential.

Evaluation Criteria

The evaluation criteria for process documentation should include a verification of participation by a senior management member (i.e. a ‘C’ person or Vice President) responsible for ‘owning’ the Measurement Program.

Best Practices

A Best Practices are for process documentation covering the Measurement Program to incorporate both ‘static’ process documentation (such as responsibility assignments, definitions, and historical backgrounds), but also ‘dynamic’ process documentation showing active roles, process event sequences, and related time and scheduling elements. Resource allocations for both static and dynamic process roles should also be documented to provide the succeeding participants to understand the resource relationships in the Measurement Program.

2.4      Process Tool Support

The Sought after Behavior

The efficient operation of a Measurement Program requires the use of process and data collection tools (typically, but not always) computer/infrastructure based processes. Some tools, such as meta-data from scanner operations, logistics, or compliance activities (such as third party confirmations) may also accommodate process tool implementation.

The Clues

Surrogate process tool supports may include metrics measuring activities in other groups linked to work (such as the volume and classes of work measured in an estimating or pricing workgroup). Clues to work measurements can also be found in post-work implementation resource demands (such as an increase in training and implementation consulting).

Evaluation Criteria

“Process tools” (in the broad sense) should be available to multiple levels of personnel involved in the measurement process. These process tools should embed the rules and definitions of the measurement process and assist in the consistent application of the process to business activities.

Best Practices

A Best Practices are to support the efficient operation of the Measurement Program with a variety of ‘tools’ (such as static paper forms for offline use, online web-based tools, and integrated methodology/workflow support tools) that can support the design, operation, collation, analysis, findings, and reporting elements of the measurement processes.

 

3      Menu of Metrics

3.1     Does the formal process include a suite of internally consistent, non-duplicated, discrete metrics?

3.2     Are the formulas for each metric defined?

3.3     Is there a method, or related documentation that maps the metrics to defined selling problems, opportunities and goals?

 

Evaluation Criteria for the Menu of Metrics are to examine the metrics (and process to produce metric values) to make sure that different organizations, or even different parts of the same organization, can apply the business processes that count, collate, and produce metric values in a way consistent from period to period and consistent throughout an organization.

 

3.1      Discrete Metrics

The Sought after Behavior

In order to compare metrics the internal consistency of metrics requires that the discrete metrics selected be compatible and have the same granularity, accuracy, timeliness, repeatability, and precision.

The Clues

Clues to metric selections that are not easily comparable include: metrics not from the same data collection mechanism, metrics not subject to audit, metrics not used by the same organizational processes.

Evaluation Criteria

The definitions of metrics, and measurement processes, should include sufficient measurement formalisms (such as granularity) to be repeatable and accurate when applied by different people.

Best Practices

A Best Practices are to review metric methods and processes to define the characteristics, trade-offs, and implementation limitations with respect to granularity, accuracy, timeliness, repeatability, and precision.

 

3.2      Metric Formulas Defined

The Sought after Behavior

Metrics that obey algebraic or statistically valid counting techniques are more easily

The Clues

Too precise choices – Can customer loyalty really be measured to four decimal places? – are as bad as wrong population selected.

Evaluation Criteria

The measurement process should include metric definitions that are appropriate for the activities and business operations being measured.

Best Practices

A Best Practices are to define metric methods with formal algebraic or computational algorithm that are not ambiguous.

 

3.3      Mapping Metrics to Selling Points

The Sought after Behavior

Selling problems, opportunities, and goals aren’t always easy, or inexpensive, to measure in a Measurement Program. Picking metrics to measure attributes of these work process components risks not really measuring what is wanted. The Measurement Program must be able to directly relate (even if it doesn’t directly measure) metrics acquired and findings made to these three work process elements.

The Clues

Measuring opportunities can be difficult. A trade show may generate a large number of leads, but very poor conversion to revenue. Surrogate measurement of opportunities – number of proposals to new customers, for example, might have limits (does tell you orders, but doesn’t tell you how many callers got lost telephoning).

Evaluation Criteria

The measurement process should provide guidance for the selection of metrics appropriate for consideration by customer and project personnel during the ‘selling process’ (whether internal or external).

Best Practices

A Best Practices includes the availability of work and project personnel training guides providing selection information and coaching in customer presentation.

 

4      Analysis

Is there a formal, documented method for analyzing and reporting the metrics collected?

The Sought after Behavior

Measurement programs require that resources (normally staff and infrastructure) be allocated to the analysis and reporting of collected metrics data. The resources allocated generally needs to be organic to the organizations that they are reporting to so that the appropriate context and findings can be produced with the analysis.

The Clues

The analysis and reduction of metric data is often found in staff positions where the analytical and organizational perspective will be available to formulate the findings and context for reporting metric-based conclusions.

Evaluation Criteria

The Evaluation Criteria for looking at analysis includes checking that the Measurement Process has checks that data collection, collation, and production of metrics is operational and has backup, cross-checks, and confidence measures that assure that the data going into the analytical process is timely, that analysis occurs on a timely basis, and that resources are allocated to do the work.

Best Practices

The Best Practices for resource allocation identifies multiple resources that can be trained and provided proper tooling to support metric data collection, collation, and analysis. Multiple resources are often needed because the skill set for a qualified analytical person (who prepared findings) are usually more active and in higher schedule demand.

 

5      Findings

5.1     Is there a formal review process of the findings?

5.2     Do the findings correlate to a set of baseline measures for each of the metrics?

5.3     Is there a formal process for defining and analyzing the variance between the baseline and the current findings?

5.4     Do the results provide guidance for determining revisions to the performance improvement goals and/or methods?

 

Evaluation Criteria for a Measurement Program’s process to generate and act on Findings are to reflect that, once timely analysis occurs, Findings (making decisions on recommendations produced from analysis) are processed and acted upon by management on a regular basis.

 

 

5.1      Formal Review of Findings

The Sought after Behavior

The investment in the infrastructure to perform the data collection, collation, analysis, and to build findings is only valuable if there is a formal process that allocate management time and resources to the review of findings.

The Clues

Presentation and reviews of findings should occur within the regular management processes on a regular basis. Findings will only result on a periodic basis in most organizations tied to the regular reporting cycle.

Evaluation Criteria

The measurement process should include the availability and application of regular reviews by managers on the results of the measurement process.

Best Practices

A Best Practices are for an organization to use extended ‘peer’ or other review/inspection processes to look with an independent viewpoint at findings to determine if they are appropriate to the context and analysis of metric data.

5.2      Baselines to Correlate Metrics

The Sought after Behavior

The comparative context for metric values are a baseline (and consistent) series of metric values across time and different business conditions. In a new Measurement Program other sectors or industrial experience can be applied (“industry norms”) to establish this baseline.

The Clues

For most metrics there are ranges of values that are normative in the industry or related businesses (for example, time to handle outbound work calls in a call center). Management chosen methods to reach goals are measured both against the metrics chosen in the Measurement Program, but often against the normative values set in an industry. A high variance (>20%) is almost always indicative of a local issue (either in measurement or method execution) such that countermeasures need to be in the findings.

Evaluation Criteria

The measurement process should define the availability of baselines and the ability to evaluate specific results against the baselines for reasonableness and validity.

Best Practices

A Best Practices are to establish a normal baseline of metric results that is provided during training to metric process participants so that they have a clear idea of what the likely results will be from a measured product or service engagement.

 

5.3      Formal Process for Variance Analysis

The Sought after Behavior

Baseline metric values set a basis for comparing metric values. Comparisons are against local objectives (target values), benchmarks (perceived external values), internal benchmarks (data collected by the same Measurement Program), or direct comparisons (previous periods for the same workgroup). A consistent comparison framework is sought to provide the ‘standard’ against the metric values were be judged. The comparisons provide a context where analysis of variance becomes a key technique to guide execution of the methods for business goals.

The Clues

Benchmark comparisons are usually across time and similarity boundaries. The variance for local values is important because it provides the highest correlation to determine if change is occurring.

Evaluation Criteria

The measurement process should have a variety of variance analysis techniques to both validate results and compare levels of success.

Best Practices

A Best Practices are create a central repository for all raw metric data collected, project descriptions, measurement process descriptions, and tools (or instruments) used to collect the data. Should analysis need repeating or a separate analysis performed to test measurement process characteristics then the repository should also keep a track of the availability of trained people to use/explain archived data.

 

5.4      Feedback for Goals and Methods

The Sought after Behavior

Changes to methods deployed to reach organizational goals need to be made on the basis of findings from Measurement Programs. Formalizing the process (cycle time, implementation process, etc) is an important part of the Measurement Program to realize desired goals.

The Clues

Informal “tinkering” with methods may change measured metric values, but the ease of analyzing the metric values and providing findings will be more difficult. In-process metric application of feedback to methods needs to be constrained so that the metric values aren’t distorted beyond the capability of an analyst to produce solid findings.

Evaluation Criteria

The measurement process should include both formal and informal means of incorporating feedback to make improvements.

Best Practices

A Best Practices for feedback is to establish both formal and role responsibilities to bring feedback into the management of the measurement process.

 

6      Revisions to Goals, Methods and Performance Improvement Program

6.1     Is there a formally defined review process?

6.2     Does the review process define specific roles and management participation?

6.3     Is there a formal process for documenting the outcome and mediating changes in goals and methods?

 

Evaluation Criteria are to verify that a formal process is a component of the Measurement Program to make active feedback and improvements to the program (and that are just as likely to add as to subtract more resources and efforts) are actively part of the Measurement Design.

 

 

6.1      Defined Review Process

The Sought after Behavior

A metrics program should have a regular (normally Calendar) scheduled review period. Two questions need to be answered: 1) is the metrics program delivering useful information? And 2) is the metrics program measuring problems, or are only good results coming up?

The Clues

If ‘stretch’ goals and good metrics are resulting, then the program may be measuring for issues and targets already met. The amount of data collected may be overkill or now unnecessary due to process changes. Alternatively the economic conditions or work force conditions that initiated the program may have resulted in changes sufficient that other issues or targets need to be established to further improve other performance.

Evaluation Criteria

The measurement process should be able to demonstrate a consistent review and problem resolution cycle.

Best Practices

A Best Practices are to establish business process steps where the measurement process is updated in concert with the refresh or updating of product and services descriptions.

 

 

6.2      Review Process Roles

The Sought after Behavior

In addition to specific calendar scheduling for a review of the metrics program there need to be specific participation by managers with both the responsibility and authority to effect changes, allocate resources, and alter policy directives. The review roles are possibly split into roles where the management receiving the metrics findings and approving actions determines if the metrics program is meeting their needs, or not, and, a group that is responsible for the performance the metrics are addressing that can allocate the capital and expense resources to make things happen. A review process should be cognizant of this split and provide multiple levels of review for action plans changing metric program if necessary.

The Clues

A good point to look at the metrics programs for work efforts is often tied to the ‘early’ or ‘accelerated gates’ for work performance plans that often fall into the 2nd or 3rd quarter of a financial year. Alterations for the metrics program (to become effective on a rolling basis or at the start of the next major period) can be made with enough advance analysis. Review analysis may be needed at the time that the formal work analysis review of 1st or 2nd period performance occurs (work reviews).

Evaluation Criteria

Evidence that the measurement process is receiving management attention, resources, and continuing process reviews should be accessible.

Best Practices

An established Measurement Program will have a Best Practices of updating the measurement process as evidenced by a succession of changes and updates to the measurement process definitions.

6.3      Formal Process for Changes

The Sought after Behavior

The formal process for changing metrics must be carefully performed to allow: 1) comparisons of periods ‘before and after’, 2) software or process changes to performance metric programs need to obey ‘Configuration Management’ rules so that changes in the field occur on a controlled basis, and 3) Training for analysts and consuming managers must occur early in the process to make the process for changing metrics data, reporting, and analysis useful.

The Clues

The formal process for effecting changes should include both operational and metrics program staff training as a ‘clue’. A surrogate measurement may be specific project management or implementation planning performed by IT or business unit analysts.

Evaluation Criteria

The measurement process should have configuration management rules and training on a continuing basis for Measurement Program practitioners/

Best Practices

A Best Practices level for formal measurement process changes is to have a versioned measurement process definition with regular training for practitioners (such as that incorporated into regular product or services updating).

7      Implementation Plan

 

7.1     Is there a formal, documented implementation plan for the performance Measurement Program?

7.2     Are there guidelines for selecting participants and their roles involved in the definition, selection, analysis, and reporting of the metrics?

7.3     Is there a process and documentation for educating the participants in the metrics program?

7.4     Are there guidelines and recommendations for collection of the baseline and continuous collection of metrics?

7.5     Are there formal guidelines for communicating the initiation of the process, and subsequent findings, to all involved parties – including the work personnel?

7.6     Is there a formal project management plan for launching and operating the Measurement Program?

 

Evaluation Criteria for Measurement Program Implementation Plans are to verify that the Measurement Design includes the tasks (resource allocation as an example) and activities (such as training) that are needed to successfully implement Measurement Program.

 

 

7.1      Project Plan for Performance Measurement

The Sought after Behavior

The sought after behavior is to make active Project Management a firm part of the implementation of a Management Program.

The Clues

Different Project monitoring, reporting, and visibility of the Management Program are possible, but some organizational recognition must be present.

Evaluation Criteria

The Measurement Program should be implemented by the organization with a defined project plan.

Best Practices

The Best Practices are to provide a consistent set of tasks and resource allocations in a project plan that is used to implement a Measurement Program.

 

 

7.2      Selecting Participants for Roles

The Sought after Behavior

The sought after behavior is to provide multiple roles where participants understand the tasks and actions for a specific Measurement Program engagement.

The Clues

A clue to the sought after behavior is a “roster” or project role assignment list where project participants are defined.

Evaluation Criteria

A measurement process that includes the definitions of roles, or the actions for different project participants, should be available.

Best Practices

A Best Practices has the measurement process embedded into a product or service project plan where the participants, tasks, and actions are defined when the project is initiated. The selection of participants should be based on their experience and training on the measurement process.

7.3      Educating Participants

The Sought after Behavior

The key behavior is that individuals are identified in the Measurement Design, not simply organizational roles.

The Clues

Training/education for participants in a Measurement Program can be performed many different ways, but a good clue is the presence of multiple kinds of training for different roles.

Evaluation Criteria

The measurement process should have specific training materials and processes.

Best Practices

A Best Practices are to support the measurement process with a certification for participants in the application within a project.

 

 

7.4      Metrics Collection

The Sought after Behavior

Changes to methods deployed to reach organizational goals need to be made on the basis of findings from Measurement Programs. Formalizing the process (cycle time, implementation process, etc) is an important part of the Measurement Program to realize desired goals and update the measurement process.

The Clues

The formal collection and archiving of measurement results is a frequent component of the metrics collection and comparison process. A surrogate process is the centralizing of quality reviews (for some or all projects) where the sought after behavior can be demonstrated.

Evaluation Criteria

The measurement process should include both formal and informal means of incorporating feedback and field experience to make improvements in the Measurement Program.

Best Practices

A Best Practices example is the periodic reporting of baseline results for metrics collected for current and past projects with a context to assist project participants using the measurement process.

 

7.5      Formal Communications Process

The Sought after Behavior

The sought after behavior is that Measurement Programs are supported with formal communications to participants, stakeholders, and managers.

The Clues

A surrogate is the high-frequency use of review/ops-reviews in an organization to show that the Measurement Program topic has high levels of visibility and that the important is communicated.

Evaluation Criteria

The measurement process should describe how formal communications occurs between different process participants.

Best Practices

The Best Practices level is to have a formal definition of participant roles, sequenced actions, and specific formal review contents and anticipated results.

 

 

7.6      Project Management Plan to Implement

The Sought after Behavior

The project management plan provides specific implementation event sequencing, resource allocations, and results as a part of the efforts to implement a measurement process.

The Clues

The work breakdown structure or organizational responsibilities should define the project resources and result responsibilities for elements of the project to implement the Measurement Program.

Evaluation Criteria

The Measurement Program project plan should incorporate sufficient resources to meet the requirements of the organization for measurement processes.

Best Practices

A Best Practices for organizations beginning to implement Measurement Programs across multiple products and services is to centralize the responsibilities for oversight and measurement process refinement.

 

 

8      Operating Plan and Support

8.1     Is there a formal ongoing support program?

8.2     Are there periodic reviews?

8.3     Is there a process for checking compliance on the collection of metrics?

 

The Evaluation Criteria for looking at how the rhythm and volume of a Measurement Program are supported by operational resources. The evaluation is to make sure that operational resources are available when needed, and that the Measurement Design has supports for both process components, and for human components.

 

 

8.1      Formal Support Program

The Sought after Behavior

A formal support program for the metrics program covers the needs of both program participants and those charged with operationally performing roles in the collection, analysis, and preparation of findings. The formal support program needs to provide pre-allocated resources, process, and lines of escalation to support the activities and tasks that make up the conduct of the Measurement Program.

The Clues

The formal support program for the Measurement Program will have both a responsible organizational party and specific individuals charged with the responsibility (operationally, process, and problem resolution) to make sure that the Measurement Program continues as planned. A surrogate process is to have this formal support program outsourced as part of the outsourcing of the Measurement Program. A clue to the conduct of the formal support program is the presence or absence in the regular reporting sequence of reports showing the utilization the formal support program resources.

Evaluation Criteria

The Measurement Program should have resources available on demand to support the needs of measurement process deliveries for customers.

Best Practices

A Best Practices for the formal support of a Measurement Program is to allocate staff, technological, and calendar resources to the support of measurement processes.

 

 

8.2      Period Program Reviews

The Sought after Behavior

The formal support program must also have a program review and feedback process built so that it stays current with enhancements and modifications to the Measurement Program. These reviews should coincide with the normal resource allocation processes for the organization.

The Clues

A surrogate review process is present if there is outsourcing of the conduct or operations of part of the Measurement Program. Normally, the contract reviews, payment reviews, or process change authorizations will provide a periodic trigger for a program review. The budgeting process for this outsourcing may also provide a periodic program review process.

Evaluation Criteria

The evaluation criteria is to review the documentation showing that a formal periodic review of the Measurement Program occurs and that resource allocation changes can be made in response to the priority and requirements of the Measurement Program.

Best Practices

A Best Practices are to have the resources assigned to the Measurement Program reviewed as part of the organizational resource allocation process (annually or more frequently).

 

 

8.3      Metric Compliance Process

The Sought after Behavior

The formal support program’s operation is a key support to ensure that data collection occurs on schedule, that analytical tasks are provided with sufficient information at the appropriate process point, and that people charged with generated findings have sufficient schedule and resources.

The Clues

The formal support program’s success or failure will usually mirror that of the Measurement Program. A key indicator of the sought after behavior is the availability of information about the formal support program. If individuals charged with operationally performing data collection, analysis, or the generation of findings are not aware of the program this is frequently sign of failure. Survey or evaluation of the availability of support information is a good surrogate to measure effectiveness of the formal support program. A clue to failure in the formal support program is a lack of training and ongoing involvement of formal support program personnel.

Evaluation Criteria

The evaluation criteria are to review the documentation showing that a formal operational support process exists to support the Measurement Program.

Best Practices

A Best Practices are to document and publicize the use, inside the measuring organization, of success stories and application of successful measurement process engagements.