How to Apply Risk Assessment Techniques to Outsourcing

ABSTRCT

Over the last decade, as outsourcing has evolved into the rule rather than the exception in the pharmaceutical industry, many lessons have been gleaned by those involved in the day-to-day business of contracted operations. Experience has shown that it is imperative to assess and qualify a contract organization to ensure it is capable and compliant, but this practice alone rarely ensures that contracted operations will go as smoothly as hoped. Just as we have used risk assessment and risk management techniques in identifying the critical quality parameters of our products and processes, we can likewise use them in our efforts to identify those factors essential to monitoring and managing outsourced activities. The application of risk assessment principles to these supporting fundamentals will help us to realize the full benefit of these partnerships, while at the same time ensuring appropriate and adequate oversight of outsourced activities.

Between 2001 and 2008, many companies were reporting annual double-digit increases in dollars on outsourced activities. Even in the current economic climate, as spending is being curbed across the industry in general, the trend toward becoming more externally vested is still evident. A 2009 industry survey encompassing all categories of pharmaceutical companies reported that nearly two-thirds of those surveyed expected the amount of outsourcing spending at their company to increase or remain the same in the coming year.

BioConvergence, LLC

Experience has shown that it is imperative to assess and qualify a contract organization to ensure it is capable and compliant, but this practice alone rarely guarantees that contracted operations will go smoothly. These exercises provide valuable information and assurance concerning a site's or an organization's state of control, but they often have failed to demonstrate any consistent correlation with successful execution. We should accept initial or re-assessment activities for what they are—a snapshot in time providing an overview of a site's systems, prefaced with all the caveats and limitations accompanying that understanding.

If assessment and qualification practices cannot secure successful execution, what other practices should an organization adopt to help ensure that contracted activities meet user group requirements and expectations? There may be a host of technical causal factors behind outsourcing disaster stories, but, ultimately, most of them can be traced back to a breakdown in systems or a disregard of certain fundamentals. As with so many other elements within a thriving quality system, risk assessment can play a critical role, and support the three fundamentals of successful engagement listed below.

1. Clear delineation of roles, responsibilities, and expectations.

2. Providing appropriate oversight of contracted activities.

3. Monitoring performance.

FUNDAMENTAL #1: CLEAR DELINEATION OF ROLES, RESPONSIBILITIES, AND EXPECTATIONS

It is alarming how often the root cause of an issue in outsourced services can be traced back to a lack of understanding of the other party's expectations. The contract giver should, first, carefully evaluate and make full account of what its expectations are. Things that might be taken as understood in one environment may not be implicit in another setting. Complete disclosure of precise expectations should be made up front, not revealed as a project progresses (or fails to progress). After all, if the contract receiver is unaware or uncertain of what is expected, how can it possibly succeed? This fact also underscores the importance of tailored, understood, and comprehensive agreements and contracts, the scope and granularity of which should be commensurate with the complexity and risk associated with the tasks to be performed.

A risk assessment exercise encompassing the full list of activities to be performed by the contract organization should be conducted to identify all potential failure modes. Those activities receiving higher risk ratings should be afforded a greater deal of attention in the quality agreement or contract. Notification requirements must be clearly spelled out and terms that might be subject to interpretation should be avoided. For example, circumstances or events that would warrant notification should be explained, rather than simply including a requirement for notification in the event of a "significant departure." Even the phrase "having potential to impact safety, integrity, strength, purity, and quality (SISPQ)" could be open to subjective interpretation.

Having a quality agreement in place does little good if the people doing the actual work are unaware of its content and requirements. Depending on the criticality of the contracted activities, a mechanism must be put into place to draw attention to and ensure compliance with quality agreement or contractual requirements that depart from a site's standard operating practices. In very high-risk operations, this mechanism may translate into having a member of a client staff in the plant to directly oversee and verify that operations are in compliance with quality and contractual requirements. Alternatively, quality agreement or contract (e.g., master service agreement) awareness training could be conducted.

In the clinical arena, site initiation activities routinely include some sort of training by the sponsor on the specifics of the clinical investigation protocol, but orientation activities such as this often are completely dismissed in contract manufacturing or GMP services settings. Additionally, a clear definition of communication channels, with designation of primary contacts for various matters or concerns, can facilitate effective communication between parties and prevent the receipt of contradictory instructions or information. One effective means for delineating these channels would be including a primary contacts matrix as an attachment to the quality, contract, or service agreement.

FUNDAMENTAL #2: DETERMINING OVERSIGHT STRATEGY FOR CONTRACTED ACTIVITIES

Numerous recent 483 observations and deficiencies cited in warning letters have emphasized the current FDA expectation that outsourced activities must have the appropriate controls in place, with adequate oversight provided by the contract giver. However, one of the primary benefits of outsourcing work is that it frees internal resources for other purposes. If significant internal resources must be used to verify and double-check everything that happens at a contract site, much of the benefit of outsourcing will be negated. Tactical determination of adequate oversight for the outsourced activities becomes vital. Risk assessment exercises may be used effectively to determine and prioritize appropriate and adequate oversight strategies for the contracted activities. Levels of oversight should be commensurate with the risk associated with a given activity, and additional oversight resources should be directed toward activities with higher risk ratings.

When performing a risk assessment for contract operations or services, the process should take into account (or weigh) factors such as the level of experience with the contract organization, audit, or inspection histories (types of audit findings, status of audit findings, or occurrence of repeat findings), and personnel turnover rates, in addition to the nature and criticality of the contracted activities themselves. The customary elements of visibility or detectability of an event or non-conformance and severity, should be thoughtfully weighted when using risk assessment tools for determining oversight requirements. One simple example of a risk assessment tool is detailed in Table 1. Although not an exhaustive list of all the possible failure modes that might be associated with the subject activity, Table 1 illustrates the process. The severity ratings used may be based on several types of potential impact; regulatory, compliance or quality, patient safety, and development time lines. The site- or organization-specific considerations mentioned previously are accounted for and weighted within the critical inputs factor.

Table 1. Example of a risk assessment tool for outsourced activities (detectability x severity = overall risk factor)

The example in Table 1 is just one of endless possibilities. Each organization must make its own decisions regarding what risk factors it will assess, how it will weight risk factors, and the implications associated with a given risk rating or level. The point is to be able to demonstrate that all risk factors have been taken into consideration when evaluating oversight requirements. Another important benefit of performing a risk assessment as part of the oversight strategy determination is that it provides documented rationale supporting the application of more moderate levels of oversight for lower risk activities, as well as highlighting those activities requiring increased levels. In other words, it guides application of control measures used to appropriately mitigate and manage the identified risks.

FUNDAMENTAL #3: MONITORING PERFORMANCE

The risk assessment exercises that have been used in determining quality agreement, notification, and oversight requirements also can provide insight into the most meaningful performance metrics to be monitored for a contract organization. For example, a manufacturing activity has been deemed to be high risk given the criticality of the material, degree of detectability of material non-conformance, limited experience with the contract manufacturer, and outstanding audit issues. Given these particular risks, oversight activities should focus on verifying that the batch record has been executed properly, and that issues have been captured and addressed. Oversight requirements for this CMO or activity, at present, have therefore been determined to include a complete review of the batch record and analytical release data before disposition of the batch by the contracting party's quality unit. With these oversight requirements, performance metrics to be monitored for this contractor may include tracking issues noted during batch documentation review, such as missed deviations or any data integrity issues. The focus should be on monitoring things that would indicate whether the systems and controls used by the contract organization are working effectively.

Ideally, the metrics used should be developed and agreed on by both sides. Potential actions resulting from noted trends in the metrics also must be defined. For example, if execution errors are noted, "man-in-plant," oversight may ensue. Conversely, if documentation is of high quality, reviews may be reduced. As work progresses, additional relevant indicators of performance may become evident. A learning curve is to be expected, and adjustments to monitoring tactics should be made.

Monitoring performance metrics serves no purpose if they are not reviewed by the involved parties at some interval. Risk assessment results can help in determining an appropriate review frequency. As with the elements of the first two fundamentals, the frequency of performance reviews should be in balance with the risk associated with the outsourced activities. Given the high risk and an anticipated volume of work of five to six campaigns a year, the site may decide a biannual or quarterly review frequency is appropriate. The outcome of the review process should be shared with the contract organization, whether directly involving the contract organization and personnel, or performed chiefly as an internal exercise. In some cases, requests for formal corrective action may be made. Regardless, the information should be shared so that the contract organization can use it to address any problems. This ideally should be a collaborative exercise so that the contract organization feels free to provide feedback on how the customer might facilitate successful execution of the contracted activities.

The performance metrics reviewed, actions taken, and any other outcome of the review exercise also should feed back into the risk assessment process. The risk assessment process should not be thought of as a one-time exercise to be performed initially. It should be used as a tool, to be repeated at some frequency, because the factors influencing risk continually change. As new information is received regarding elements composing the risk profile for a contract organization, the risk assessment should be revisited. Based on changing risk factors, adjustments in contractual requirements, oversight strategy, review or monitoring frequency, or even qualified status may become necessary.

SUMMARY

In recent years, adopting the principles of Quality by Design has shown us that the more we know about the factors that are essential to our process and, therefore, truly impact product quality, the more manageable and flexible control of our process will become. Just as we have used risk assessment and risk management techniques in identifying the critical quality parameters of our products and processes, we can likewise use them in our efforts to identify those factors essential to monitoring and managing outsourced activities. The application of risk assessment principles to these supporting fundamentals will help us realize full benefit of these partnerships, while at the same time ensuring appropriate and adequate oversight of outsourced activities.

Dawn Schofield is a compliance advisor at Safis Solutions, LLC, Indianapolis, IN, 317.777.6200 (ext. 113), dschofield@safis-solutions.com.