Assessing a Risk Management Program

Since the introduction of the International Conference on Harmonisation (ICH) Q9 Quality Risk Management guidelines in November 2005, the biopharmaceutical industry has been engrossed in risk assessments, risk management, risk analysis tools, risk-based decision making, and a host of risk-associated activities. Companies have designed and implemented risk management programs and use them to make quality-related decisions daily. From the design phases of new projects to implementing corrective actions and preventive actions (CAPAs), risk management is now intertwined with nearly all GxP activities and is such a daily occurrence that one can barely remember a time when risk assessments were not done.

Now, nearly 10 years on, companies and regulators are asking: What have we achieved? Has risk management lived up to the promises of 10 years ago? Are patients truly better protected now that risk management is behind our decision-making process?

ICH Q9 is not a regulatory requirement. Although companies have no obligation to adopt it, address it, or even acknowledge its existence, risk management is firmly established within the biopharma industry and is here to stay. The advantage of having a guidance document such as ICH Q9 is that engineers, scientist, regulators, senior management, and other industry experts can assess problematic issues from the same point of view, with the same terminology, and intent.

The industry should review what has worked and what has not, and must answer the question: How has ICH Q9 been embraced or, perhaps, misinterpreted? This article will set forth a methodology of evaluating the effectiveness and consistency of risk management programs.

Step one: The quality risk management program and environment
In ICH Q9, the first two chapters address responsibilities and how to initiate a quality risk management process (1). When assessing a quality risk management (QRM) program, it is simpler to consider these steps together.

To have an effective and robust QRM program, an organization must ensure that the relevance of the risk management program is evident and consistent at all levels within the organization. This step evaluates whether the concept of risk management is consistent from the senior level managers to the shop floor.

There are three areas in which to investigate the risk environment: processes and procedures, risk management oversight, and culture.

Processes and procedures
In assessing the effectiveness of a program, policies and procedures should be evaluated to ensure that they meet the overriding principles of ICH Q9:

• Evaluate the risk to quality based upon science and link to the protection of the patient.
• Is the effort of formality and documentation commensurate with the level of risks?

Although many companies start with these two overriding principles as the cornerstone of a risk management program, over time, revisions to procedures and changes in personnel can divert from these original intentions. Many organizations have realized that revisions to standard operating procedures (SOPs) generally add checks and balances and rarely eliminate steps.

Risk management procedures have evolved from a slim, streamlined, effective, and efficient process to multiple burdensome, complex, and redundant processes.

QRM programs that bolt the risk assessment process on to existing quality systems are far more likely to suffer from these ailments. Many organizations have simply added quality risk management procedures to the list of other quality systems. Change-control procedures that require an evaluation of and provide mitigating actions against risks, for example, are far more efficient than change control processes that simply point the requestor to perform a separate risk assessment and mitigating actions. Change control is a means of risk management.

Risk management oversight
Is there a clearly defined process owner, person, or entity that is responsible for the entire program? How does this individual or entity report to senior management and down to the shop floor?

There is a clear correlation between inconsistent, ineffective, and inefficient QRM programs and those lacking some level of reporting structure. A QRM program that is implemented without some type of governance will be uncontrollable, and the outputs of such a program will be variable and inconsistent. Alternatively, a program that monitors its performance will be able to ensure faster delivery of a high-quality risk management process.

Oversight activities should include the key performance indicators, cycle times, periodic reviews, and senior management responsibilities. This information needs to be timely and widely advertised within the organization. The program assessment requires finding where and when this information is communicated.

Culture
Is risk management viewed and demonstrated with a consistent level of importance throughout the organization? Checking to see where QRM programs are discussed, reported, and participated in, is a good indication if there is a unified culture of QRM. Further, listing the sources of risk assessments will indicate areas using QRM programs and those that do not. This listing can show which departments, managers, or project teams are actually involved in risk management. The lack of a uniform distribution of departments conducting risk assessments may be an indication that there are areas within an organization that are not utilizing risk management.

Step Two: Risk assessment
Risk assessment consists of three components:

• Risk identification—recognizing risk and defining the scope of the assessment
• Risk analysis—quantifying or qualifying the level of risks
• Risk evaluation—determining if the risk level is acceptable or if mitigation must occur.

For each of these components, the QRM program must consider three fundamental questions of risk: What might go wrong? What is the probability it will go wrong? What are the consequences (severity)?

The accepted analysis of risk generally includes establishing a level of severity, likelihood, and detectability. Some QRM procedures call for a qualitative assessment, rating the different levels high, medium, or low. Other programs require a quantitative ranking, such as a numerical scale of 1 to 10.

Either way, precise and descriptive definitions of each level is necessary. When assessing this portion of the QRM program, one must determine if the tools that are being used are being applied with consistency and openness.

For example, one risk assessment ranked the likelihood to be ‘low’ because there were downstream quality checks (e.g., stability and microbiologic testing). Another risk assessment on the same process ranked it ‘high’, as downstream testing should not be relied on to control risks. The use of downstream checks was not discussed or defined in the risk management program; therefore, it was open to interruption and yielded varying results.

Assessing a QRM program requires comparing the analyses over a period of time and across different disciplines. Are the risk assessments evaluating the risk levels conducted in the same way in a quality-control laboratory and an engineering department? Is there a variance in the risk evaluations done during the summer months, before and after audits, and during the holiday season?

When assessing this portion of the QRM program, one must determine if the tools that are being used are being applied with consistency and openness. This task requires reading through several risk assessments from various authors, teams, and sources. In some cases, it will be clear that the author began the risk assessment with conclusions already in mind. It can be evident in the description of the risk. Language such as “making a small change to the system,” “an insignificant amount of material,” “has been done elsewhere with no significant implications” for example, can tip off the reader to risk assessments done disingenuously.

Also, the risk team itself can influence the outcome of a risk assessment. Participants who are adamant that there is no risk based on a ‘gut feel’ are dangerous to a scientifically sound risk assessment. Likewise, members who are risk adverse can alter the risk assessment in such a way that all risks are intolerable.

Step three: Risk controls
In this step, one must evaluate the decision-making process that leads an organization to either accept the risks or mitigate against them. Again, a comparison between points in time and between disciplines is warranted.

Risk controls provide insight into what the risk evaluation is yielding. For example, a department of engineers performed 86 risk assessments in the course of a year, documented in hundreds of pages of quality risk assessments as required by the governing QRM program. In all of the assessments and all of the effort to produce the documents, however, they only found two “intolerable risks” that required remediation.

Determining the cause of a risk assessment that yields so few risks requiring mitigating actions is imperative to improve the QRM program. The team or author that wrote the assessment may have predetermined the output of the assessment prior to performing it, there could be weaknesses in the risk assessment tools or training, or a lack of the culture discussed in step one.

Step four: Risk communication
When a risk assessment identifies risks that require mitigation, how are the risks shared with other stakeholders that may be unaware that this risk exists? Within a production plant, other departments or business units may be susceptible to the same risks. How can these departments learn about the risks identified in an assessment?

Are there other sister sites, contract manufacturers, or suppliers that need to be informed? How are mitigating actions from these areas tracked? A strong, well-defined governance, as discussed in step one, can easily remedy these challenges.

A robust QRM program also will have defined mechanisms to evaluate whether the risks identified require notification to regulatory authorities, the individual in the organaization who is responsible for deciding this, and the procedure to report such a risk.

Step five: Risk review
Many companies produce an annual risk-management summary report as a risk review. Essential topics for a thorough review are an analysis of the sources of risk assessments, the number of risks and the associated levels, mitigation actions, an analysis of success, and areas for continuous improvements.

Assessing a QRM program ensures such a review is taking place, that the review is effective, and suggests definitive actions for continuous improvements. Like feedback to any process, risk reviews should prompt positive, unambiguous measures to eliminate inefficiencies and risks. Finally, the assessment needs to determine that such actions produce the improvements they sought.

Conclusion
Although still optional, establishing a robust QRM program is crucial to remaining compliant with regulatory requirements and competitive in the market. QRM is the language that industry and regulators speak now and will continue to do so in the near future. Establishing a quality risk management program is just the start. Assessing the program is imperative to make it effective and efficient.

Reference
1. ICH, Q9 Quality Risk Management, Step 4 version (2005).

About the author
Michael J. Shea is the director of engineering at GxP Systems, Cork, Republic of Ireland. Email: michaelj.shea@gxpsystems.com.