This is one of my
longer posts, but the subject of training in business is crucial. Therefore, I thought I would include these
thoughts.
Introduction
Format for Evaluation Guides
· The Sought after Behavior
The
“sought after behavior” is to document the actions that embody upon what the
Measurement Program activities and tasks should be conducted.
The Clues
“Clues”
for the Measurement Program are the artifacts, data, or evidence that the
behavior is actually accomplished even without specific guidelines, published
schedules, or actions.\
· Evaluation Criteria
The
rules and evaluation methods used to evaluate observed behavior, clues, or
surrogates to determine if this Measurement Program criteria has been met.
· Best Practice
A Best
Practice is a behavior that is recommended because it meets many organizations
needs to achieve the “sought after behavior” for the Measurement Program
criteria. A Best Practice is a guideline recommendation and not an absolute.
Different organizations may have specific needs that either eliminate the need,
or otherwise meet the “sought after behavior” without the Best Practice
documented.
Does the Measurement Program have all six
elements?
1) Goal setting;
2) Method definition;
3) Metrics definition;
4) Analysis;
5) Determination of findings; and,
6) A process for determining revisions to the
goals?
The Evaluation Criteria
focus on whether the established Measurement Program for Work Performance has
all of these process and behavioral elements.
1.1 Goal setting
The Sought after Behavior
The Goal Setting behavior sought is to
incorporate specific goals for the Measurement Program that show the state of
business activity.
The Clues
“Clues”
for Goal Setting include Management Objectives, Six-Sigma Initiatives, and
Value Planning.
Evaluation Criteria
There
must be written criteria documenting the goal setting behavior (defining
resources, tasks, responsibilities, and schedule) that directly cover the
Measurement Program and directly relate to performance measurement.
Best Practices
A Best
Practices are for the goal setting behavior of the Measurement Program to be
integrated into the development, decision making, and dissemination of
corporate/organizational budgets and allocations.
1.2 Method Definition
The Sought after Behavior
Methods are the collections of work effort and
measurement to get to a result. Metrics are the numbers and indices that result
from measurements.
The Clues
Clues to good methods include organized efforts
that produce results that can be verified, compared, and repeated.
Evaluation Criteria
There must be a list of measurement methods (for
example, defined process points where activities or tangibles are counted; or,
surrogates that can be repeat ably counted – such as entries in a CRM
application) that are applied to Performance Measurement that are agreed upon
by both those measured and those using the measurements.
Best Practices
A Best Practices are for the methods of
activities for data collection, collation and reduction processing, and for work
force management to be integrated into the operational systems supporting the
effort (for example, automated data collection and reporting thru the CRM and
SFA systems).
1.3 Metrics Definition
The Sought after Behavior
Metrics must be formally defined with rules,
exception handling, and consistent behavior.
The Clues
Some metrics that affect territory choices,
accounting cut-offs, and splitting compensation affects metrics.
Evaluation Criteria
For each metric quantity (each term in widgets
sold per day, or service contracts per customer) there should be written rules
defining the vocabulary, the counting rules, or the business definitions that
make the metric quantity useful for comparisons, auditable, and repeatable).
Best Practices
The written rules should be widely available (not
just as “shelf space holders”) as a document that is revised and managed
actively.
1.4 Analysis
The Sought after Behavior
A good metrics program will have both the roles
and responsibilities to analyze collected metrics and generate actions to
change behavior in a desired path.
The Clues
A clue to a good program is someone assigned the
regular operation work to analyze metrics on a regular basis. A surrogate
function is someone watching the trends for metrics with the
responsibility to break results
Evaluation Criteria
Document who has the role, and the responsibility
to analyze, collect and process metrics on a regular basis (at least during the
performance Measurement Program) to deliver analysis to management.
Best Practices
The analysis, in an organization with Best
Practices, will have an internal, or combination internal and external, review
of the analysis for quality control.
1.5 Determination of Findings
The Sought after Behavior
The analysis of metric data is incomplete without a person to
translate the analysis work (with metrics) into series of corrective actions,
countermeasures, or new analytical tasks.
The Clues
Clues to a regular process by managers are the
operational reviews, staff reviews, or customer quality reviews that occur on a
regular basis in many organizations.
Evaluation Criteria
For each scheduled period where analysis occurs,
there must be a defined role (people resources), and schedule (calendar
resources) where the analysis is reviewed and findings – determinations of
conclusions and formulating action plans as the results from analysis of values
from the metrics program.
Best Practices
The Best Practicess are to apply a consistent set
of responses to similar (normalized) metric responses.
1.6 Revision Process
The Sought after Behavior
Regular revisions to the Measurement Program may
include adding, subtracting, or acting on metric data.
The Clues
Business processes that lead to territory,
commission reallocation, or staff level changes also are good fodder to look at
Metric Program data.
Evaluation Criteria
There must be a revision process that takes in
the business level of the Measurement Program (people, process elements,
methodologies, and new corporate objectives) and makes revisions to the
Measurement Program.
Best Practices
The Best Practices are to deploy a process that
creates regular versions of the metrics data, methods, and implementation
techniques as a part of a regular business process revision schedule.
2.1 Is there a formal
and empirical plan for defining goals, and devising methods by which those
goals may be achieved?
2.2 Once there is a
method for defining the methods and goals, it is essential that there be a
process for identifying the metrics associated with each metric. Is there a
formal process for selecting the metrics?
2.3 Are these processes
documented?
2.4 Are these processes
supported by tools for the definition and selection?
The Evaluation Criteria
focus on the formality of the methodology and process steps. ‘Formality’ isn’t
the mathematical or work-flow definitions – ‘Formality’ here is the ability of
different organizations to follow the defined steps and activities that reach
the same level of quality goals and methods.
2.1 Formal and Empirical Plan
The Sought after Behavior
A formal process to select the goals (and
matching methods) for the organization. Methods are expressed in action plans
to produce a desired outcome. “Empirical” means that the methods are based on
the numbers or systematic observation of business activities.
The Clues
“Strategic Plans”, goals tied to mission
statements, or management initiatives goals are good places to start.
Evaluation Criteria
The measurement process being certified should
incorporate a formal process with internal control steps to ensure application
of the process to meet desired outcomes.
Best Practices
The Best Practices are to define the empirical
plan with a formal definition of the relationships between the Measurement
Program and the metric methods deployed to achieve the defined goals.
2.2 Defining Metrics for Methods
The Sought after Behavior
Measurement processes for work effort or actions
are selected for their ability to produce meaningful metrics that can be
analyzed.
The Clues
Metrics selected must have a direct relationship
to the method being measured. i.e. an increase in expense dollars may not
relate to the number of customers contacted.
Evaluation Criteria
The measurement process being evaluated should
define metrics, methods, validation data, and quality attributes for the
methods.
Best Practices
A Best Practices for metric definition provides
both a formal definition in prose and implementation in an associated
spreadsheet or computer program where learning or quality verification can be
assured of repeatability and consistent rule applications of the methods
counting rules and units.
2.3 Process Documentation
The Sought after Behavior
Measurement programs need the backing of
management to be successful. A formal review, approval, and participation
commitment process is important such that management has a stake in the
construction, operation, and results of the Measurement Program.
The Clues
A less successful program will allow a champion,
or single, manager to initiate the Measurement Program. Without a consensus
that analysis of the metrics program and actions to follow findings the
Measurement Program will not have organizational impact nor a high
participation potential.
Evaluation Criteria
The evaluation criteria for process documentation
should include a verification of participation by a senior management member
(i.e. a ‘C’ person or Vice President) responsible for ‘owning’ the Measurement
Program.
Best Practices
A Best Practices are for process documentation
covering the Measurement Program to incorporate both ‘static’ process
documentation (such as responsibility assignments, definitions, and historical backgrounds),
but also ‘dynamic’ process documentation showing active roles, process event
sequences, and related time and scheduling elements. Resource allocations for
both static and dynamic process roles should also be documented to provide the
succeeding participants to understand the resource relationships in the
Measurement Program.
2.4 Process Tool Support
The Sought after Behavior
The efficient operation of a Measurement Program
requires the use of process and data collection tools (typically, but not
always) computer/infrastructure based processes. Some tools, such as meta-data
from scanner operations, logistics, or compliance activities (such as third
party confirmations) may also accommodate process tool implementation.
The Clues
Surrogate process tool supports may include
metrics measuring activities in other groups linked to work (such as the volume
and classes of work measured in an estimating or pricing workgroup). Clues to work
measurements can also be found in post-work implementation resource demands
(such as an increase in training and implementation consulting).
Evaluation Criteria
“Process tools” (in the broad sense) should be
available to multiple levels of personnel involved in the measurement process.
These process tools should embed the rules and definitions of the measurement
process and assist in the consistent application of the process to business
activities.
Best Practices
A Best Practices are to support the efficient
operation of the Measurement Program with a variety of ‘tools’ (such as static
paper forms for offline use, online web-based tools, and integrated
methodology/workflow support tools) that can support the design, operation,
collation, analysis, findings, and reporting elements of the measurement
processes.
3.1 Does the formal
process include a suite of internally consistent, non-duplicated, discrete
metrics?
3.2 Are the formulas
for each metric defined?
3.3 Is there a method,
or related documentation that maps the metrics to defined selling problems,
opportunities and goals?
Evaluation Criteria for
the Menu of Metrics are to examine the metrics (and process to produce metric
values) to make sure that different organizations, or even different parts of
the same organization, can apply the business processes that count, collate,
and produce metric values in a way consistent from period to period and
consistent throughout an organization.
3.1 Discrete Metrics
The Sought after Behavior
In order to compare metrics the internal
consistency of metrics requires that the discrete metrics selected be
compatible and have the same granularity, accuracy, timeliness, repeatability,
and precision.
The Clues
Clues to metric selections that are not easily
comparable include: metrics not from the same data collection mechanism,
metrics not subject to audit, metrics not used by the same organizational
processes.
Evaluation Criteria
The definitions of metrics, and measurement
processes, should include sufficient measurement formalisms (such as
granularity) to be repeatable and accurate when applied by different people.
Best Practices
A Best Practices are to review metric methods and
processes to define the characteristics, trade-offs, and implementation
limitations with respect to granularity, accuracy, timeliness, repeatability,
and precision.
3.2 Metric Formulas Defined
The Sought after Behavior
Metrics that obey algebraic or statistically
valid counting techniques are more easily
The Clues
Too precise choices – Can customer loyalty really
be measured to four decimal places? – are as bad as wrong population selected.
Evaluation Criteria
The measurement process should include metric
definitions that are appropriate for the activities and business operations
being measured.
Best Practices
A Best Practices are to define metric methods
with formal algebraic or computational algorithm that are not ambiguous.
3.3 Mapping Metrics to Selling Points
The Sought after Behavior
Selling problems, opportunities, and goals aren’t
always easy, or inexpensive, to measure in a Measurement Program. Picking
metrics to measure attributes of these work process components risks not really
measuring what is wanted. The Measurement Program must be able to directly
relate (even if it doesn’t directly measure) metrics acquired and findings made
to these three work process elements.
The Clues
Measuring opportunities can be difficult. A trade
show may generate a large number of leads, but very poor conversion to revenue.
Surrogate measurement of opportunities – number of proposals to new customers,
for example, might have limits (does tell you orders, but doesn’t tell you how
many callers got lost telephoning).
Evaluation Criteria
The measurement process should provide guidance
for the selection of metrics appropriate for consideration by customer and
project personnel during the ‘selling process’ (whether internal or external).
Best Practices
A Best Practices includes the availability of work
and project personnel training guides providing selection information and
coaching in customer presentation.
Is there a formal,
documented method for analyzing and reporting the metrics collected?
The Sought after Behavior
Measurement programs require that resources
(normally staff and infrastructure) be allocated to the analysis and reporting
of collected metrics data. The resources allocated generally needs to be
organic to the organizations that they are reporting to so that the appropriate
context and findings can be produced with the analysis.
The Clues
The analysis and reduction of metric data is
often found in staff positions where the analytical and organizational
perspective will be available to formulate the findings and context for
reporting metric-based conclusions.
Evaluation Criteria
The Evaluation Criteria for looking at analysis
includes checking that the Measurement Process has checks that data collection,
collation, and production of metrics is operational and has backup,
cross-checks, and confidence measures that assure that the data going into the
analytical process is timely, that analysis occurs on a timely basis, and that
resources are allocated to do the work.
Best Practices
The Best Practices for resource allocation
identifies multiple resources that can be trained and provided proper tooling
to support metric data collection, collation, and analysis. Multiple resources
are often needed because the skill set for a qualified analytical person (who
prepared findings) are usually more active and in higher schedule demand.
5.1 Is there a formal
review process of the findings?
5.2 Do the findings
correlate to a set of baseline measures for each of the metrics?
5.3 Is there a formal
process for defining and analyzing the variance between the baseline and the
current findings?
5.4 Do the results
provide guidance for determining revisions to the performance improvement goals
and/or methods?
Evaluation Criteria for a
Measurement Program’s process to generate and act on Findings are to reflect
that, once timely analysis occurs, Findings (making decisions on
recommendations produced from analysis) are processed and acted upon by
management on a regular basis.
5.1 Formal Review of Findings
The Sought after Behavior
The investment in the infrastructure to perform
the data collection, collation, analysis, and to build findings is only
valuable if there is a formal process that allocate management time and
resources to the review of findings.
The Clues
Presentation and reviews of findings should occur
within the regular management processes on a regular basis. Findings will only
result on a periodic basis in most organizations tied to the regular reporting
cycle.
Evaluation Criteria
The measurement process should include the
availability and application of regular reviews by managers on the results of
the measurement process.
Best Practices
A Best Practices are for an organization to use
extended ‘peer’ or other review/inspection processes to look with an
independent viewpoint at findings to determine if they are appropriate to the
context and analysis of metric data.
5.2 Baselines to Correlate Metrics
The Sought after Behavior
The comparative context for metric values are a
baseline (and consistent) series of metric values across time and different
business conditions. In a new Measurement Program other sectors or industrial
experience can be applied (“industry norms”) to establish this baseline.
The Clues
For most metrics there are ranges of values that
are normative in the industry or related businesses (for example, time to
handle outbound work calls in a call center). Management chosen methods to
reach goals are measured both against the metrics chosen in the Measurement
Program, but often against the normative values set in an industry. A high
variance (>20%) is almost always indicative of a local issue (either in
measurement or method execution) such that countermeasures need to be in the
findings.
Evaluation Criteria
The measurement process should define the
availability of baselines and the ability to evaluate specific results against
the baselines for reasonableness and validity.
Best Practices
A Best Practices are to establish a normal
baseline of metric results that is provided during training to metric process
participants so that they have a clear idea of what the likely results will be
from a measured product or service engagement.
5.3 Formal Process for Variance Analysis
The Sought after Behavior
Baseline metric values set a basis for comparing
metric values. Comparisons are against local objectives (target values),
benchmarks (perceived external values), internal benchmarks (data collected by
the same Measurement Program), or direct comparisons (previous periods for the
same workgroup). A consistent comparison framework is sought to provide the
‘standard’ against the metric values were be judged. The comparisons provide a
context where analysis of variance becomes a key technique to guide execution
of the methods for business goals.
The Clues
Benchmark comparisons are usually across time and
similarity boundaries. The variance for local values is important because it
provides the highest correlation to determine if change is occurring.
Evaluation Criteria
The measurement process should have a variety of
variance analysis techniques to both validate results and compare levels of
success.
Best Practices
A Best Practices are create a central repository
for all raw metric data collected, project descriptions, measurement process
descriptions, and tools (or instruments) used to collect the data. Should
analysis need repeating or a separate analysis performed to test measurement
process characteristics then the repository should also keep a track of the
availability of trained people to use/explain archived data.
5.4 Feedback for Goals and Methods
The Sought after Behavior
Changes to methods deployed to reach
organizational goals need to be made on the basis of findings from Measurement
Programs. Formalizing the process (cycle time, implementation process, etc) is
an important part of the Measurement Program to realize desired goals.
The Clues
Informal “tinkering” with methods may change
measured metric values, but the ease of analyzing the metric values and
providing findings will be more difficult. In-process metric application of
feedback to methods needs to be constrained so that the metric values aren’t
distorted beyond the capability of an analyst to produce solid findings.
Evaluation Criteria
The measurement process should include both
formal and informal means of incorporating feedback to make improvements.
Best Practices
A Best Practices for feedback is to establish
both formal and role responsibilities to bring feedback into the management of
the measurement process.
6 Revisions to Goals,
Methods and Performance Improvement Program
6.1 Is there a formally
defined review process?
6.2 Does the review
process define specific roles and management participation?
6.3 Is there a formal
process for documenting the outcome and mediating changes in goals and methods?
Evaluation Criteria are
to verify that a formal process is a component of the Measurement Program to
make active feedback and improvements to the program (and that are just as
likely to add as to subtract more resources and efforts) are actively part of
the Measurement Design.
6.1 Defined Review Process
The Sought after Behavior
A metrics program should have a regular (normally
Calendar) scheduled review period. Two questions need to be answered: 1) is the
metrics program delivering useful information? And 2) is the metrics program
measuring problems, or are only good results coming up?
The Clues
If ‘stretch’ goals and good metrics are
resulting, then the program may be measuring for issues and targets already
met. The amount of data collected may be overkill or now unnecessary due to
process changes. Alternatively the economic conditions or work force conditions
that initiated the program may have resulted in changes sufficient that other
issues or targets need to be established to further improve other performance.
Evaluation Criteria
The measurement process should be able to demonstrate
a consistent review and problem resolution cycle.
Best Practices
A Best Practices are to establish business
process steps where the measurement process is updated in concert with the
refresh or updating of product and services descriptions.
6.2 Review Process Roles
The Sought after Behavior
In addition to specific calendar scheduling for a
review of the metrics program there need to be specific participation by
managers with both the responsibility and authority to effect changes, allocate
resources, and alter policy directives. The review roles are possibly split
into roles where the management receiving the metrics findings and approving
actions determines if the metrics program is meeting their needs, or not, and,
a group that is responsible for the performance the metrics are addressing that
can allocate the capital and expense resources to make things happen. A review
process should be cognizant of this split and provide multiple levels of review
for action plans changing metric program if necessary.
The Clues
A good point to look at the metrics programs for work
efforts is often tied to the ‘early’ or ‘accelerated gates’ for work
performance plans that often fall into the 2nd or 3rd quarter of a financial
year. Alterations for the metrics program (to become effective on a rolling
basis or at the start of the next major period) can be made with enough advance
analysis. Review analysis may be needed at the time that the formal work
analysis review of 1st or 2nd period performance occurs (work reviews).
Evaluation Criteria
Evidence that the measurement process is
receiving management attention, resources, and continuing process reviews
should be accessible.
Best Practices
An established Measurement Program will have a
Best Practices of updating the measurement process as evidenced by a succession
of changes and updates to the measurement process definitions.
6.3 Formal Process for Changes
The Sought after Behavior
The formal process for changing metrics must be
carefully performed to allow: 1) comparisons of periods ‘before and after’, 2)
software or process changes to performance metric programs need to obey
‘Configuration Management’ rules so that changes in the field occur on a
controlled basis, and 3) Training for analysts and consuming managers must
occur early in the process to make the process for changing metrics data,
reporting, and analysis useful.
The Clues
The formal process for effecting changes should
include both operational and metrics program staff training as a ‘clue’. A
surrogate measurement may be specific project management or implementation
planning performed by IT or business unit analysts.
Evaluation Criteria
The measurement process should have configuration
management rules and training on a continuing basis for Measurement Program
practitioners/
Best Practices
A Best Practices level for formal measurement
process changes is to have a versioned measurement process definition with
regular training for practitioners (such as that incorporated into regular
product or services updating).
7.1 Is there a formal,
documented implementation plan for the performance Measurement Program?
7.2 Are there
guidelines for selecting participants and their roles involved in the
definition, selection, analysis, and reporting of the metrics?
7.3 Is there a process
and documentation for educating the participants in the metrics program?
7.4 Are there
guidelines and recommendations for collection of the baseline and continuous
collection of metrics?
7.5 Are there formal
guidelines for communicating the initiation of the process, and subsequent
findings, to all involved parties – including the work personnel?
7.6 Is there a formal
project management plan for launching and operating the Measurement Program?
Evaluation Criteria for
Measurement Program Implementation Plans are to verify that the Measurement
Design includes the tasks (resource allocation as an example) and activities
(such as training) that are needed to successfully implement Measurement
Program.
7.1 Project Plan for Performance Measurement
The Sought after Behavior
The sought after behavior is to make active
Project Management a firm part of the implementation of a Management Program.
The Clues
Different Project monitoring, reporting, and visibility
of the Management Program are possible, but some organizational recognition
must be present.
Evaluation Criteria
The Measurement Program should be implemented by
the organization with a defined project plan.
Best Practices
The Best Practices are to provide a consistent
set of tasks and resource allocations in a project plan that is used to
implement a Measurement Program.
7.2 Selecting Participants for Roles
The Sought after Behavior
The sought after behavior is to provide multiple
roles where participants understand the tasks and actions for a specific
Measurement Program engagement.
The Clues
A clue to the sought after behavior is a “roster”
or project role assignment list where project participants are defined.
Evaluation Criteria
A measurement process that includes the
definitions of roles, or the actions for different project participants, should
be available.
Best Practices
A Best Practices has the measurement process
embedded into a product or service project plan where the participants, tasks,
and actions are defined when the project is initiated. The selection of
participants should be based on their experience and training on the
measurement process.
7.3 Educating Participants
The Sought after Behavior
The key behavior is that individuals are
identified in the Measurement Design, not simply organizational roles.
The Clues
Training/education for participants in a
Measurement Program can be performed many different ways, but a good clue is
the presence of multiple kinds of training for different roles.
Evaluation Criteria
The measurement process should have specific
training materials and processes.
Best Practices
A Best Practices are to support the measurement
process with a certification for participants in the application within a project.
7.4 Metrics Collection
The Sought after Behavior
Changes to methods deployed to reach
organizational goals need to be made on the basis of findings from Measurement
Programs. Formalizing the process (cycle time, implementation process, etc) is an
important part of the Measurement Program to realize desired goals and update
the measurement process.
The Clues
The formal collection and archiving of
measurement results is a frequent component of the metrics collection and
comparison process. A surrogate process is the centralizing of quality reviews
(for some or all projects) where the sought after behavior can be demonstrated.
Evaluation Criteria
The measurement process should include both
formal and informal means of incorporating feedback and field experience to
make improvements in the Measurement Program.
Best Practices
A Best Practices example is the periodic
reporting of baseline results for metrics collected for current and past
projects with a context to assist project participants using the measurement
process.
7.5 Formal Communications Process
The Sought after Behavior
The sought after behavior is that Measurement
Programs are supported with formal communications to participants,
stakeholders, and managers.
The Clues
A surrogate is the high-frequency use of
review/ops-reviews in an organization to show that the Measurement Program
topic has high levels of visibility and that the important is communicated.
Evaluation Criteria
The measurement process should describe how
formal communications occurs between different process participants.
Best Practices
The Best Practices level is to have a formal
definition of participant roles, sequenced actions, and specific formal review
contents and anticipated results.
7.6 Project Management Plan to Implement
The Sought after Behavior
The project management plan provides specific
implementation event sequencing, resource allocations, and results as a part of
the efforts to implement a measurement process.
The Clues
The work breakdown structure or organizational
responsibilities should define the project resources and result
responsibilities for elements of the project to implement the Measurement
Program.
Evaluation Criteria
The Measurement Program project plan should
incorporate sufficient resources to meet the requirements of the organization
for measurement processes.
Best Practices
A Best Practices for organizations beginning to
implement Measurement Programs across multiple products and services is to
centralize the responsibilities for oversight and measurement process
refinement.
8 Operating Plan and Support
8.1 Is there a formal
ongoing support program?
8.2 Are there periodic
reviews?
8.3 Is there a process
for checking compliance on the collection of metrics?
The Evaluation Criteria for looking at how the
rhythm and volume of a Measurement Program are supported by operational
resources. The evaluation is to make sure that operational resources are
available when needed, and that the Measurement Design has supports for both
process components, and for human components.
8.1 Formal Support Program
The Sought after Behavior
A formal support program for the metrics program
covers the needs of both program participants and those charged with
operationally performing roles in the collection, analysis, and preparation of
findings. The formal support program needs to provide pre-allocated resources,
process, and lines of escalation to support the activities and tasks that make
up the conduct of the Measurement Program.
The Clues
The formal support program for the Measurement
Program will have both a responsible organizational party and specific
individuals charged with the responsibility (operationally, process, and
problem resolution) to make sure that the Measurement Program continues as
planned. A surrogate process is to have this formal support program outsourced
as part of the outsourcing of the Measurement Program. A clue to the conduct of
the formal support program is the presence or absence in the regular reporting
sequence of reports showing the utilization the formal support program
resources.
Evaluation Criteria
The Measurement Program should have resources
available on demand to support the needs of measurement process deliveries for
customers.
Best Practices
A Best Practices for the formal support of a
Measurement Program is to allocate staff, technological, and calendar resources
to the support of measurement processes.
8.2 Period Program Reviews
The Sought after Behavior
The formal support program must also have a
program review and feedback process built so that it stays current with
enhancements and modifications to the Measurement Program. These reviews should
coincide with the normal resource allocation processes for the organization.
The Clues
A surrogate review process is present if there is
outsourcing of the conduct or operations of part of the Measurement Program.
Normally, the contract reviews, payment reviews, or process change
authorizations will provide a periodic trigger for a program review. The
budgeting process for this outsourcing may also provide a periodic program
review process.
Evaluation Criteria
The evaluation criteria is to review the
documentation showing that a formal periodic review of the Measurement Program
occurs and that resource allocation changes can be made in response to the
priority and requirements of the Measurement Program.
Best Practices
A Best Practices are to have the resources
assigned to the Measurement Program reviewed as part of the organizational
resource allocation process (annually or more frequently).
8.3 Metric Compliance Process
The Sought after Behavior
The formal support program’s operation is a key
support to ensure that data collection occurs on schedule, that analytical
tasks are provided with sufficient information at the appropriate process
point, and that people charged with generated findings have sufficient schedule
and resources.
The Clues
The formal support program’s success or failure
will usually mirror that of the Measurement Program. A key indicator of the
sought after behavior is the availability of information about the formal
support program. If individuals charged with operationally performing data
collection, analysis, or the generation of findings are not aware of the
program this is frequently sign of failure. Survey or evaluation of the
availability of support information is a good surrogate to measure
effectiveness of the formal support program. A clue to failure in the formal
support program is a lack of training and ongoing involvement of formal support
program personnel.
Evaluation Criteria
The evaluation criteria are to review the
documentation showing that a formal operational support process exists to
support the Measurement Program.
Best Practices
A Best Practices are to document and publicize
the use, inside the measuring organization, of success stories and application
of successful measurement process engagements.
