If it is determined that there is a knowledge or skill deficiency, then a decision should be made about whether refresher
training or retraining is appropriate. When the evaluation determines that the operator lacks physical strength, required
skill, or mental capability to complete the task correctly, then retraining or refresher training wastes time and money and
raises false expectations. In that case, either change the nature of the task or redirect the operator to another, more suitable
position. If all else fails, you may need to explore an exit package (Table 3).
Trainers
Are trainers qualified as subject-matter experts? Find out what is the pace of their delivery. Is it too slow, too fast, too
technical, or too full of jargon for initial training? Review course evaluations, if applicable. Are the standards of acceptable
performance being communicated and demonstrated? Are trainers providing enough practice time with constructive feedback during
training? Determine if trainers are following a leader's guide or a lesson plan that includes a delivery checklist or on-the-job
training checklist to ensure complete and consistent training. The analysis may need to include interviews with random operators
to collect information on actual training practices and compare those practices to written training procedures. For example,
in one instance it was discovered that one trainer taught his group to do things one way, while another trainer instructed
her group differently.
Supervisors and Management
Supervisors and managers play a critical role in the effectiveness of training programs, and in the performance of operator
and trainee. To evaluate the potential role of supervisors and managers in root causes, explore the post-training reinforcement
that management provides. Mary Broad and John Newstrom, authors of Transfer of Training, concluded in their field research that the number one barrier of transfer of training is the absence of reinforcement from
management back on the job.3 Are managers scheduling adequate time for training? Are they providing sufficiently frequent feedback? Include second- and
third-shift training practices, in which there is frequently less managerial presence. Is the feedback constructive and specific?
Find out if, how, when, and how often feedback is delivered. Is it delivered only once a year during performance reviews,
or only during disciplinary sessions? Feedback delivered at these times comes too late—it's after the fact. For behavior and
performance to improve, managers must provide timely and specific feedback about what task, action, or behavior needs attention;
how the improvement is to occur; and what is the acceptable level of quality.
SECOND M, MACHINE: TOOLS
Tools can be anything from a physical tool itself, such as a computer, to a form or a checklist. What kinds of job aids are
available besides the standard operating procedures (SOP)? Are instructions for completing forms inadequate or confusing?
Is the physical equipment reliable and qualified, or is it new and not yet part of a routine work practice? Are there assessment
tools, such as knowledge checks and follow-up monitoring, for the knowledge transfer? Determine if these tools are available,
valid, and reliable. Do the conditions under which the operators are trained match the conditions under which they are expected
to perform? Unless the tool matches what is to be measured, you will not have substantial data to validate the effectiveness
of training. By matching the performance conditions called for in the assessment or knowledge check with the conditions of
the learning objectives, you will ensure that the objectives were in fact met, thus providing evaluation data of the effectiveness.4 If the problem involves a lack of tools, investigate whether reduced budgets have prevented restocking. If there is a lack
of, or poorly stocked and stored, equipment and tools, investigate whether there are improper housekeeping practices.2
THIRD M, METHODS: PROCEDURES
Figure 1
|
The first set of procedures to review is the training program or system procedures. Do they describe how to develop, deliver,
and assess training, or are they inadequate and incomplete?5 Consider the date trained versus the date performed. Investigate how much time elapsed before the operator put the skills
to use. Training loses its effectiveness unless it's used immediately. Also consider how the training was delivered: via hands-on
or read-and-understood-only training? Via group or classroom training as opposed to one-on-one? The bulk of today's training
involves procedures, and effective procedure training is critical. At the conclusion, you want the operator to perform the
procedure—not memorize it—so move quickly to the application by providing demonstrations and exercises that require operators
to perform the activity described in the procedure.6 Allowing trainees to practice new skills and procedures engages them in the learning process; it not only increases their
retention, but it promotes faster transfer of the learned skill back on the job.7 What if the root cause is something else that is causing non-compliance with a procedure or work instruction? The analysis
should explore barriers to performance. If the barrier is a lack of authority, then determine if there are unclear roles and
responsibilities. If it's a lack of time, look at scheduling conflicts.
But what if the situation is a case of "We've always done it this way?" Then the analysis should determine whether operators
were ever able to perform the approved procedure. Explore whether the procedure is accurate. Is the SOP author a subject matter
expert (SME)? Determine if the procedure was written with support from a lead operator. Confirm if the draft procedure was
challenged using a dry run or a field test with those responsible for execution. Verify if proposed changes are stalled in
change control awaiting effective release long after training is completed.
FOURTH M, MATERIALS: TRAINING MATERIALS
The analysis should evaluate the quality of training materials. Start with the objectives. Are they measurable? Assess the
training design and confirm if delivery includes activities, exercises, and demonstrations. Evaluate if content is up-to-date,
accurate, and approved by QA when required. Is the sequence logical and easy-to-follow? Determine if sufficient time is allocated
to each training item, and if there are opportunities to practice the new skills. Are assessments or knowledge checks required
and included in the leader's guide? Confirm if course evaluations are being completed.
KEY ELEMENT OF CAPA SYSTEM
Identifying the root cause is a key element of a CAPA system, so that appropriate action is taken to eliminate the cause or
source of the problem and to prevent further recurrence. By examining the problem using the 4 Ms and a fishbone diagram, you
avoid the temptation of selecting the first apparent cause as the root cause. The 4 Ms and the fishbone diagram force you
to generate all potential causes, and they foster a thorough analysis of the contributing factors, not just the obvious ones;
this ensures that the corrective and preventive actions are not just a quick fix.
SUMMARY
Thoroughly addressing persistent operator-error performance problems through thoughtful analysis allows companies to identify
the real root cause of a problem; it also enables them to take the proper corrective action to maintain and improve compliance
performance and manage risk effectively. The alternative is explaining to an FDA investigator, or to your management, why
repeat training is the appropriate corrective action even when numerous deviations indicate it is not fixing the problem,
or preventing it from recurring in the future.
Vivian Bringslimark is a senior consultant with PAREXEL Consulting, 19 Cobblers Mill Road, Sandy Hook, CT 06482, 203.470.8363, fax: 203.270.6519,
vivian.bringslimark@PAREXEL.com
|
