A risk-based approach to the validation and qualification of computerized systems used in regulated environments must focus on systems and components that have a potentially high impact (in other words, represent a high risk) on product quality and consumer safety. This is generally the case for systems and system components that form the computer network infrastructure. Hence, the role of network monitoring hardware and software for the qualification of networks, and maintaining the qualification status of the network infrastructure will be increasingly important.

In August 2002, FDA announced an initiative to merge scientific risk management and an integrated quality systems approach.

 This risk-based approach will help industry, suppliers, and regulatory agencies focus resources on critical issues for public health and consumer safety, while adopting innovations made in pharmaceutical engineering.

Practical guidelines for conducting risk assessments have been published in appendix M3 of the GAMP4 guide.

 However, it is important to note that assessing risk is very different from risk management. The goal of risk assessment is the analysis of risks viewed from a specific angle, such as risk for the consumer or commercial risk to a business. Risk assessments result in a risk register and the classification of particular risks. The classification typically assigns a risk severity based on its impact and its probability.

The task of risk management is to define how identified risks can be controlled, minimized, or compensated. Risk management typically asks the following questions:

1. What risks exist, how do they affect us, and how can we manage them?

Risk Triggers: What is the trigger for us to change the risk severity classification? What is the trigger for us to address the risk as a real problem?

Risk Mitigation: What are we doing now to avoid or reduce the risk?

2. What will we do if the risk (the potential problem) becomes a real problem?

Risk Contingency Plan: What actions will we take if the risk is triggered?

3. The risky situation occurred! how do we deal with it? 

What actions do (did) we take? What is the impact of the risky situation so far?

For more information, refer to the recently published guidebook on the development of risk management master plans.

In December 2002, the ISPE submitted a whitepaper to FDA on a risk-based approach to computer system validation.

 The paper was based on the concepts emphasized by FDA's new cGMP initiative. This whitepaper appears to have contributed to the new guidance on 21 CFR Part 11.

 The paper concluded that internal system information not identified by any predicate rule was likely to be of low impact. Therefore, it is acceptable not to have additional Part 11 controls for these records provided that adequate procedures are in place and the required paper records are kept.

The authors of the whitepaper opposed the existing interpretation that software be considered GxP electronic records subject to Part 11, mainly because industry had already developed, in collaboration with FDA, "approaches for dealing with hardware and software in the GxP environment based on validation of systems, configuration management, change control, and adequate procedures and plans for maintaining the validated state. These approaches have been widely adopted and very successful in meeting GxP requirements. Considering software as GxP electronic records has little practical benefit, as well as discouraging firms from adopting innovative technological solutions."

The publication of the Part 11 guidance and FDA's statements about enforcement discretion for certain requirements led some to conclude that it was acceptable to revert back to paper records, (for instance, by defining the printed analysis report with the analysis chromatogram as the raw data and subsequently deleting the electronic record from the computer's hard disk). However, whether an electronic record is subject to Part 11 requirements depends on the predicate rules and whether the established business practices of the firm rely on the electronic version record to perform FDA-regulated activities.

In other words, it is not acceptable to delete the electronic record and just keep the paper record for the FDA auditor. FDA clearly states that it may take business practices into account in order to determine whether an electronic record is used instead of the paper record. It is therefore recommended to determine and document in advance whether the electronic record or the paper record will be used to perform regulated activities. Networked or chromatography data systems, laboratory information management systems (LIMS), and enterprise resource planning (ERP) systems manage critical decision-support data and continue to be a focus of GxP enforcement. The trustworthiness and reliability of the data managed by these systems is highly dependent on efficient technical controls that ensure access security, data integrity, and traceability. 

Client/server data systems are proliferating in regulated laboratories and manage large amounts of critical data. It is obvious that the operation and qualification of the network infrastructure must be an integral part of a company's validation strategy. By their nature, networks are heterogeneous and comprise a variety of hardware components employing diverse communication protocols. A change to a network component can affect other components and applications. Also, FDA is taking a closer look at networks and has been citing companies for violations (see 

Figure 1: Network topology of a distributed, networked chromatography data system

It has become clear that FDA is aware of CDS issues when operating within a network. A warning letter from FDA deals with network programs with functions of a laboratory management system. The letter states:

"The network program lacked adequate validation and/or documentation controls. For example:

System design documentation has not been maintained or updated throughout the software dating back to 1985 despite significant changes and modification that have taken place.  These include program code, functional/structural design, diagrams, specifications, and text descriptions of other programs that interfere with [this program].

Validation documentation failed to include complete and updated design documentation, and complete wiring/network diagrams to identify all computers and devices connected to the system.

The Quality Control Unit failed to ensure that adequate procedures were put in place to define and control computerized production operations, equipment qualifications, document review and laboratory operations.

The software validation documentation failed to adequately define, update, and control significant elements customized to configure the system for specific needs of operation."

It is clear from letters like this that compliance issues extend not only to the CDS itself but also to the network infrastructure within which it operates. Many IT departments have applications to monitor the configuration, health, and status of the network from a network operations center (NOC). These applications may extend from local workgroups throughout the entire enterprise. Many of these applications provide graphical presentations of the network and are able to store configurations from a point in time so that changes to the network may be monitored and documented as part of an overall network validation plan.

Additionally, personnel who are not GxP trained may have access to the network as part of their normal business responsibilities. It is a paradox that the network infrastructure must be compliant, but many components (cabling, utilities, and other devices) do not have validation plans. Networks require frequent changes, additions, and repairs, but can never be taken out of service. A risk assessment, in combination with a sound risk management plan, can help address these problems.

A striking example of a computer network infrastructure failure made headlines in April 2003. A recently installed laboratory computer system in a medical center became overloaded, resulting in a severe backlog of blood-testing samples.

In such situations, several questions have to be asked:

Did formal requirements include specifications for the anticipated load of the system?

Was the system installed according to the supplier's specifications?

Did it pass the test suite defined for the installation qualification (IQ) and the operational qualification (OQ)?

Did performance qualification tests simulate the anticipated load of the networked system in terms of number of samples and number of concurrent users in the context of the hospital's office and laboratory network?

What measures were in place for the prevention and early detection of severe failures and performance bottlenecks?

Could the bottleneck have been prevented through the use of a network monitoring system?

But where to start and, more importantly, where to stop? How much qualification, documentation, and testing is enough? The following points provide a good guide:

"It is not possible to test every possible branch point in operating, network, and application software used in a typical business system.

However, we can determine the level of quality of a subset of the software very accurately, by thoroughly testing the subset.

If rigorous and consistent development standards and methods are used, it has been observed that the quality level of the subset is representative of the quality level of the entire software system."

However, unexpected side effects frequently occur as components of a complex environment are changed:

"Validating networked systems not only requires qualifying individual networked components (for example, applications running on each computer), but it also means qualifying authorized access to the networked system and qualifying the data transfer between related computers, as in qualifying the interfaces of components at both sites.  The whole system (i.e., including the network) is validated by running typical day-by-day applications under normal and high load conditions and verifying correct functions and performance with a previously specified criteria."

Proper network administration and operation is an area that is subject to questioning by regulatory organizations. With this in mind, it is important to capture a snapshot of the network during validation. Whenever a change is made, this snapshot can be compared to the current configuration to insure proper communication among the various nodes on the network. In addition, a retrospective document can be maintained that tracks these changes over time. Network qualification is the next frontier in computer systems validation. The Part 11 guidance helps focus qualification activities by basing them on documented risk assessment. Qualification of network infrastructure should focus on the following tasks:

Design qualification (DQ): evidence that the network is suitable for the applications (the design is fit for the intended purpose)

Installation qualification (IQ): verification and documentation of the static network topology and inventory (evidence that the implementation matches the design)

Operation qualification (OQ): dynamic topology verification and capacity testing (evidence that the system operates properly according to the vendor specifications)

Performance qualification (PQ): maintain the qualification status, ensure continuous performance through ongoing monitoring during use and measurement of performance over time, and minimize the risk of failure during operation.

As clients, servers, and instruments are connected to a network, the available bandwidth may become dramatically reduced. This is especially true if thenetwork is not well segregated, leading to unnecessary broadcast network traffic (for example, between an analytical laboratory network and the office network). This decrease in bandwidth may result in decreased performance affecting real time processes (and could even result in data loss as the hospital network example cited above shows).

Modern analytical equipment and the networks within which they operate may be monitored by network analyzer software along with the clients and servers that control them. This software not only helps operators monitor the health of their networks but also aids in the qualification of the networks through which the instrument data flows. Network monitoring applications are commercially available from companies including Agilent Technologies, Computer Associates, Hewlett-Packard, IBM, and others. While software applications provide excellent monitoring capabilities, network qualification may also require powerful network measurement and test hardware to capture and document network connections, communication activities, available and consumed capacity, and control data. For regulated lab operations, the challenge is to make network measurements meaningful from a systems validation perspective. In the meantime, the first examples of metrology-based network assessment and qualification services designed specifically for laboratory networks have been developed.

Validation and qualification activities need to consider network infrastructure. The role of network monitoring hardware and software for the qualification ofnetworks and for maintaining the qualification status of the network infrastructure will continue to increase.

The authors thank Bob Giuffre, a senior network data system consultant with Agilent Technologies based in New Jersey, for input to this manuscript and sharing analysis data derived from network monitoring measurements using the Agilent Advisor/Distributed Network Analyzer and Agilent FrameScope 350 in combination with an Agilent Cerity for Pharmaceutical QA/QC networked data system and Agilent 1100 HPLC instruments directly connected to the local area network.

1.	FDA. Pharmaceutical cGMPs for the 21st century: a risk-based approach. Available at URL: 

The good automated manufacturing practices (GAMP) guide for validation of automated systems in pharmaceutical manufacture, GAMP 4.

3. Huber L. Risk management master plan and best practices series. Available at URL: 

4.	ISPE. Risk-based approach to 21 CFR Part 11. Available from URL: 

www.21cfrpart11.com/pages/library/index.htm

5.	Code of Federal Regulations, Title 21, Part 11; electronic records; electronic signatures; final rule. 

6.	FDA. Guidance for industry: Part 11, electronic records; electronic signatures and scope and application. (draft February 2003, final version August 2003). Available at URL: 

7.	FDA. Warning letter. File No. 320-01-08. Available at URL: 

8.	Associated Press. L.A. hospital computer system breaks down. 22 Apr 2003.

9.	Fiorito T, Quinn T. Qualifying peer-served network infrastructures. Presented at the 37th Drug Information Association Annual Meeting, 2001 Jul 8-12; Denver, Co. Available at URL: 

www.hollisgroup.com/downloads/DIA%20-%20C3Q%2001.pdf

Validation of computerized analytical and networked systems.

 Boca Raton (FL): Interpharm Press; 2002.

Qualification services for laboratory networks.

 Agilent publication 5988-9656EN. Palo Alto (CA): Agilent, 2003.

Articles

21 CFR Part 11

Networks are part of the compliance picture. Recent FDA warning letters show the agency considers network monitoring and qualification a necessary part of maintaining the security and integrity of electronic records.

Qualifying Network Infrastructure - A Risk-Based Approach

In 1997, FDA issued 21 CFR Part 11, which provides criteria for FDA acceptance of electronic records, electronic signatures, and handwritten signatures.

 In response to requests from industry, the regulation allows electronic records to be treated as equivalent to paper records and handwritten signatures. By providing faster and more productive access to documentation and accelerating the approval process, electronic records are expected to be more cost effective for industry and FDA. 

The rule applies to FDA-regulated industry segments that must follow Good Laboratory Practice (GLP), Good Clinical Practice (GCP), and current Good Manufacturing Practice (cGMP) requirements. 

Analytical development and quality control laboratories that regularly use computers for instrument control, data acquisition, data evaluation, data management, data transfer, and archiving must comply. Part 11 applies whenever computer systems are used for regulated activities, whether they are used as part of an automated analysis system, as part of a network, or as stand-alone machines (for example, for spreadsheet applications or word processing).

The primary requirements of Part 11 include:

secure retention of electronic records allowing instant reconstruction of analyses

user-independent, computer-generated, time-stamped audit trails

system and data security, data integrity, and confidentiality through system access control

use of digital signatures for open systems.

This article describes the rule's interpretation and enforcement as of January 2004, but discussions are ongoing. Updates are important and can be found at FDA's website (

Table 1: Records Subject to Part 11All computer systems used to generate, maintain, and archive electronic records must be validated to ensure accuracy, reliability, consistent independent performance, and the ability to discern invalid or altered records.

System validation is nothing new for laboratories using computers in a regulated environment. Validating computer systems has been described thoroughly, and most companies have developed strategies for implementation. System validation applies to both new and existing systems, and problems can arise with older systems. These require a formal evaluation and statement of their validation status. If an older system cannot be validated, it should not be used under 21 CFR Part 11. Information on validating software and computer systems is available from several sources.

 Procedures should be in place to generate accurate and complete copies of records in both human readable and electronic form suitable for inspection, review, and copying by the agency. Records must be protected to enable their accurate and ready retrieval throughout the records retention period.

FDA expects that final results be kept together with the original data and the procedures for processing the data (metadata). The agency wants to be able to trace final results back to the raw data using the same tools the user had when the data were generated. This is probably one of the most difficult requirements of Part 11, as some records must be kept for ten or more years, and computer hardware and software have a much shorter lifespan.

A second problem lies in deciding exactly which records should be logged and retained. These decisions can be complex, as in quantitative chromatographic analyses. Typically in chromatography data acquisition, preprogrammed methods perform evaluation and printout automatically. Occasionally the preprogrammed integration method proves inappropriate, and analysts must work with the raw data and adjust parameters to generate more appropriate measurements of peak integrations. This is a manual iterative process that is frequently subjective, varying from user to user. Should only the final results with the final acceptable parameters and chromatogram printouts be archived or should all intermediate data be archived as well?

A third problem is maintaining the availability of records throughout the retention period. The challenge lies not with the durability of storage devices (such as CD-ROMs) but with the longevity of computer hardware, operating systems, and application software required to reconstruct the analysis. One approach is to migrate existing data as new systems are adopted.

Procedures should be in place to limit the access to authorized users. Limited access must be ensured through physical and logical security mechanisms. Most companies already have similar procedures in place. Typically, users log onto a system with a user ID and password. However, problems have been reported in analytical laboratories when computer controlled systems collect data over time and users are unable to monitor the system the entire time. To prevent unauthorized access, a screen saver with password protection should be activated.

Further details on system security are discussed in a later article.

Procedures should be available to use secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes cannot obscure previously recorded information. Such audit trail documentation should be retained for a period at least as long as that required for the subject electronic records and also should be available for agency review and copying.

This requirement has been the subject of many questions and discussions, especially regarding which details must be recorded. For example, what should be recorded when generating a calibration table? Should each typing error be recorded when entering a compound name, should each line be recorded when the return key is pressed, or should entries only be recorded when the session is closed? Too many confirmation steps will affect productivity.

Another concern is that, because everything analysts do can be reconstructed, their creativity could be negatively influenced, especially in the development of new processes.

In order to deter record and signature falsification, written policies holding individuals accountable and responsible for actions initiated under their electronic signatures are necessary. This requirement necessitates not only the development of new procedures but also behavioral changes in the use of logon IDs and passwords. The taboo against sharing a password with a colleague is usually much lower than teaching somebody how to abuse a handwritten signature, but under Part 11 both have the same consequence. 

Persons who use open systems to create, modify, maintain, or transmit electronic records must ensure the authenticity, integrity, and, if necessary, the confidentiality of electronic records from the point of creation to the point of receipt. Such procedures and controls include those identified for closed systems, as appropriate, and additional measures such as document encryption and digital signatures.

Implementing digital signatures requires software for document encryption and may also require hardware and software for generating digital signatures. Typically, computer systems used in analytical laboratories are closed systems, which do not need digital signatures. However, record encryption and digital signatures are required in open systems, such as when analytical data generated by a contract laboratory are transmitted to the sponsor over the Internet. Technology and applications for this purpose have been described elsewhere.

Although Part 11 has been in place for six years now (and enforced for four years), there is still confusion in industry over how to implement it. 21 CFR Part 11, as well as early draft guidance documents and FDA staff interpretations, does not distinguish between record types or criticality. Part 11 compliance was requested for all records that passed through any computer, and FDA could audit any such records at inspections. Under this very broad interpretation, full implementation turned out to be very expensive and, for some applications, impractical. In some cases, companies decided against the use of new technology due to the anticipated additional complexity and cost of implementing Part 11. However, this contradicts the original intent and spirit of the rule, which was issued to enable the use of new technology while protecting and furthering public health.

With the release of a draft guidance on the scope and applications of Part 11 in February 2003,

 FDA initiated a new, narrower approach. This approach became official with the release of the final guidance on September 3, 2003 and FDA's announcement that it would re-examine Part 11 and initiate a new rule-making process. The new approach likely will be in effect for the next few years.

The new guidance states that Part 11 applies only when the record is required by a predicate rule and when one or both of the following conditions apply:

Electronic records are used instead of paper.

Persons make printouts but still rely on the electronic records to perform regulated activities.

Figure 1: Deciding where Part 11 AppliesFigure 1 illustrates the process described by FDA's Part 11 experts for deciding where Part 11 applies. Table 1 gives examples of records subject to Part 11.

First, check if the record is required by a predicate rule (or if the record must be submitted to FDA). Next, determine if the record falls under the new, narrower scope. Is the record maintained in electronic format in place of paper, or is it maintained as both electronic and paper records, but the electronic record is used to perform regulated activities? Finally, conduct a risk assessment of the criticality of the Part 11 records and document the result. Part 11 controls are implemented based on the outcome of this process.

There is no question that certain laboratory records must be retained according to predicate rules. These include raw data, instrument control, processing parameters (metadata), and results. There is also no question that Part 11 applies when we use electronic records instead of paper for these records. Typically, analyses and evaluation of the results of the final product is the last check before the product is released to the market. Laboratory systems that perform such analyses are considered high risk because errors that are not identified at this stage cannot be recovered. Therefore, all Part 11 controls discussed in this paper should be implemented, the most important being:

Access to the system and data should be limited and controlled.

The computer should have a built-in electronic audit trail that records all changes to records made by operators (for example, manual reprocessing of chromatograms).

Original data (like chromatographic raw data) should be kept in electronic form together with metadata and final results.

Handwritten signatures and electronic signatures should be linked to electronic records.

Systems must be validated according to the requirements of predicate (GxP) rules.

Other systems, like word processors used to write validation reports, are not so critical, and not all Part 11 controls need to be implemented. Validation efforts for such systems should include a well documented installation, but there is no need for extensive testing.

The final decision on whether records fall under the scope of Part 11 and which controls should be implemented should be based on a risk assessment and on the business practices applied in specific laboratories. 

More details of the new scope of Part 11 are available in a previously published article.

1.		FDA. Code of Federal Regulations, Title 21, Part 11 electronic records; electronic signatures; final rule. 

4.		Huber L, Winter W. Implementing 21 CFR Part 11 — electronic signatures and records in analytical laboratories, part 4 — long term archiving and ready retrieval. 

5.		Huber L. Case study: web applications. In: Wingate G, editor. 

 Boca Raton, FL: Interpharm/CRC Press; 2003.

6.		FDA. Guidance for industry: Part 11, electronic records; electronic signatures — scope and application. (Draft February 2003, Final version August 2003). Available at URL: 

7.		Winter W, Huber L. Part 11 is not going away: the new electronic records draft guidance. 

The FDA rule on electronic signatures and electronic records was issued in 1997, but the details of implementation are still being debated. The 2003 FDA guidance redefines the scope of 21 CFR Part 11. Understanding which records now fall under the scope of the rule can help you begin implementing your compliance plan.

21 CFR Part 11 - Requirements and New Scope

FDA issued the 21 CFR Part 11 Scope and Applicability Guidance Document partly due to industry concern that the breadth of applicability and the cost of Part 11 compliance have hindered the use of new technology. The guidance states that records must still be maintained in compliance to the underlying predicate rules, but FDA will take a "risk-based" approach to enforcing some of the technical controls for Part 11, such as validation, audit trails, record retention, and record copying. FDA will also include Part 11 in its formal review of current Good Manufacturing Practice (cGMP) regulations and follow a more subjective course in taking regulatory action. FDA's intent is to return emphasis to the predicate rules that govern Good Clinical Practice (GCP), Good Laboratory Practice (GLP), and cGMP, together referred to as GxP.

Due to these predicate rules, a risk-based approach to protecting product quality and public safety is not new for drug manufacturers. For example, FDA expects a firm subject to GxP to develop a risk evaluation of its product and to mitigate the identified risks. Identified risks may be effectively eliminated, or their likelihood of occurrence or severity of consequences may be reduced to acceptable levels. There may be risks for which there are no technical fixes. Such risks may be addressed by including warnings in product labeling. Some risks may be so minimal that no specific action is required to maintain them at acceptable levels.

Today, a risk analysis must be included in a firm's Part 11 remediation plan. It should address how systems that generate regulated electronic records can potentially affect consumer safety. A useful definition of risk is available in the 1999 ISO/IEC Guide 51: a combination of the probability of occurrence of harm and the severity of that harm. Whether applied to Part 11, or to other safety-related aspects of FDA-regulated products, risks to product quality or public safety are central. Such products obviously include foods and cosmetics, blood products and drugs, medical devices, and any other regulated products that are ingested or consumed by or applied to a living creature. When a system generates electronic records that can greatly impact product safety and quality or the integrity of other regulated records, it is considered a "high-risk" system, and the technical controls of Part 11 that protect electronic record integrity apply. Otherwise, the system is considered to be "low-risk," and the agency will simply enforce the GxP requirements for record integrity instead of the more stringent Part 11 controls.

Table 1: What is risk?Table 1 compares some of the high vs. low risk systems that generate GxP records according to a recent ISPE White Paper submitted to FDA.

In fact, applying a risk-based approach to Part 11 compliance should be nothing new for regulated firms. Quality System Regulation (QSR) requires a firm to perform a risk analysis of the various systems that generate or maintain electronic records and implement electronic signatures. This analysis allows companies to determine which records have high impact on consumer safety. The firm can then rank the identified risks according to their criticality.

For example, quality data from a Part 11-compliant database may be included in a Corrective and Preventative Action (CAPA) report, typically generated by a spreadsheet program. The spreadsheet formulae should be validated according to GxP. However, the overall relative risk to public safety is low, and therefore the typical Part 11 technical controls (for example, audit trails) are not required to protect the integrity of the spreadsheet.

On the other hand, adverse event reporting and clinical trial data that fall under GCP regulation potentially can have a high impact on public safety. Programs that analyze and visualize clinical data subsequently have an impact on record integrity. Such systems would be considered high risk and therefore should incorporate the technical controls for Part 11 compliance as well as maintaining compliance with predicate rules.

In summary, Part 11 remediation has not changed for high-risk GCP-related systems such as adverse event and case report form (CRF) data management systems, SAS analysis software, web trial systems, electronic patient diaries, patient randomization, and trial supply labeling systems. Both GCP and Part 11 definitely apply to these high-risk systems. (In addition, FDA's Guidance to Industry for Computerized Systems Used in Clinical Trials remains in effect and applies to these systems as well.)

Keep in mind that while Part 11 is an enforceable law, an FDA guidance document is not a law. Guidance documents present FDA's current thinking on a subject and are only a recommendation on how to address a law's requirements. Guidance documents are not binding on either industry or the agency.

There are many risk assessment protocols or methodologies available. What follows is a discussion of the most common.

 (FTA) is a deductive, top-down method of analyzing system design and performance. It involves specifying a "top event" to analyze, followed by identifying all of the associated elements in that system that could cause that top event to occur. Fault trees provide a convenient symbolic representation of the combination of events resulting in the occurrence of the top event. Events and "gates" that allow events to occur are represented by graphic symbols. Sometimes certain events must occur together in order for the top event to occur. In this case, these events are arranged under an "AND" gate. If the individual events each can trigger the top event alone, they are grouped under an "OR" gate. The entire system, including human interactions, is analyzed when performing a fault tree analysis.

 (FMEA) categorizes and ranks potential process failures, or critical issues, and targets their prevention. When the analysis quantifies the probability and criticality of potential failures, it is referred to as failure mode effects and criticality analysis (FMECA). FMEA prioritizes potential failures according to their risks and then implements actions to eliminate or reduce the likelihood of their occurrence. FMEA is a tool that should identify product and process failures before they occur, identify appropriate risk mitigation measures to prevent or otherwise control the failure, and ultimately improve product and process design. FMEA assumes that all product and process failures (and the actions required to control these failures) are predictable and preventable. Surprisingly, organizations still frequently experience predictable and preventable failures with costly consequences, and FMEA can help address these. Such failures can lead to product recalls, death or injury, poor quality, and unanticipated cost. Although the aerospace and defense industry have used FMEA for decades, this methodology recently has been making significant inroads into the biomedical device industry.

Hazard Analysis and Critical Control Points

 (HACCP) is embraced by FDA. HACCP was first used in food production in the 1960s. In 1996, the US Food Safety and Inspection Services Task Force (FSIS) developed a HACCP-based regulatory proposal that became the Pathogen Reduction/Hazard Analysis and Critical Control Point Systems (HACCP) Rule. With this rule, FSIS determined to reduce the risk of food borne illnesses from meat and poultry products by ensuring that appropriate and feasible measures were taken at each step in the food-production process where hazards are present and where procedures and technologies exist (or can be developed) to prevent or reduce the likelihood of these hazards.

HACCP has seven basic steps (see "The Seven Basic Steps of HACCP" sidebar). It is understood that implementation of HACCP does not mean the absolute elimination of risks, but rather, the reduction of hazards to reduce risk to an acceptable level.

It is necessary to analyze computer systems and information-handling processes to assess not only risk but also the cost of converting paper-based information to an electronic format. A good place to start is to perform a system assessment. Plot your various systems and processes on a simple X-Y matrix that measures (from low to high) the risk to security of the data (X-axis) and the cost of remediation (Y-axis). Then you can prioritize the systems and processes needing upgrades or replacement based on their position in the matrix. Systems that fall in the "high data security risk, low conversion cost" area of the matrix could be targeted first for compliance validation.

Because they had to address Y2K issues, many organizations have already generated an inventory of all their computer systems and hopefully evaluated the risks associated with computer error or failure. Companies with cost considerations and many non-compliant computer systems must, of course, prioritize which systems to remediate first. 

The following is a list of systems to be reviewed in a typical criticality assessment (high to low) for nonclinical laboratory systems.

systems for quality processes and standard operating procedures

lab spreadsheets and databases for data collection 

central database for inventory management

systems for customer relationship management

21 CFR Part 11 is not going away and the FDA intends to enforce it. What has recently changed is the adoption of a narrower scope for the rule, a new understanding of agency enforcement discretion, and the application of a risk-based approach to compliance. When choosing a risk assessment protocol or methodology for Part 11 remediation, it is important to use basic common sense. Identify the greatest potentials for risk to product quality (and ultimately to public safety) and implement measures to mitigate those risks. Finally, document the entire endeavor. Whether you choose to adopt a standard risk assessment methodology or develop your own, remember that FDA will show enforcement discretion if you have a well-documented plan in place and if you are making true progress toward implementing your plan.

The Seven Basic Steps of Hazard Analysis and Critical Control Points

Conduct on-site verification of flow diagram

2. Determine the critical control points (CCPs) using a decision tree. CCPs are the  points where hazards must be eliminated or minimized.

3. Establish critical limits that must be met to ensure CCPs are under control.

4. Establish a system for monitoring the control at the CCPs.

5. Establish the corrective actions to be taken when monitoring indicates that a particular CCP is not under control.

6. Establish procedures for verification to confirm that the HACCP system is working correctly.

7. Establish documentation for all procedures and records.

1.	ISPE. Risk-based approach to 21 CFR Part 11. Available from URL: 

www.21cfrpart11.com/files/library/compliance/isp_riskbasedapproach21cfr.pdf

2.	Clarkston Consulting. 21 CFR Part 11 compliance: an enterprise issue, not a point solution. Available from URL: 

www.clarkstonconsulting.com/WhitePaper/Part11Compliance.pdf

The FDA?s risk-based approach to pharmaceutical cGMPs applies to 21 CFR Part 11 enforcement as well. Understanding different methodologies for assessing and managing risk will help you develop and begin to implement a compliance plan.

21 CFR Part 11: Choosing a Risk Assessment Methodology

Ensuring data integrity by protecting original data from accidental or intentional modification, falsification, or even deletion is the key for reliable and trustworthy records that will withstand scrutiny during regulatory inspections. Many assessments and action plans stop at the system security level discussed in the "Access Security" article in this series. However, merely controlling and securing access to a data system does not address the real challenge for today's laboratories  ensuring data integrity.

Data integrity means that data records are complete, intact, and maintained within their original context, including their relationship to other data records. To use an analogy from the paper world, a contract is valid only if all the pages of the document are complete, legible, and if it contains the required authentic signatures and properly states the terms and conditions. In this sense, integrity denotes validity.

In the case of a chromatography data system (CDS), data integrity gives a high degree of certainty that a given record (such as a calculated chromatographic result) has not been modified, manipulated, or otherwise corrupted after its initial generation.

In the context of CDSs, data integrity requires automatic change management of metadata (for example, storage of method setpoints), including "revisioncontrol" for reanalyzed data. Data integrity also means that data cannot be entered out of context. Operational checks enforced by a computerized system should check user permissions and enforce a certain sequence of permitted steps according to a defined workflow.

Another technical challenge for data management systems is to ensure referential integrity, that is, the integrity of the record's relationships (dependencies). A database record is traceable, reliable, and trustworthy only if the complete set of related (dependent) records is available.

Think of the following scenario: Sample XYZ needs to be analyzed using Method A, revision 4 (the current revision). Due to a shortage of solvent during the analysis, chromatogram 1 of sample XYZ is invalid, and the sample must be re-injected. The system stores revision 2 of the binary chromatogram without deleting or overwriting the original. (Check whether your data system can really do this!) Chromatogram 2 is now processed, generating the result XYZ.2-A4-1. One of the points on the calibration curve is subsequently marked invalid because the reviewing analyst found a previously undetected sample preparation error. Chromatogram 2 must be reprocessed a second time, generating result revision XYZ.2-A4-2. The results are reviewed, approved, and archived. Over the course of the following months, method A is updated to revision 5 due to a specification change. In the course of an FDA audit, the results for sample XYZ  are revisited.

A system with appropriate measures for maintaining referential integrity will retrieve the requested revisions of those results, including the correct references to the revisions of the raw data (XYZ.2) and processing methods (A4). Many current systems will allow retrieving the final result and the original raw data. But will they show the correct version (A4 instead of A5) of the processing method used at the time? Will they really show the history of revisions? If your system does not do this, you should develop appropriate procedures to capture at least a paper trail of the iterative changes.

After security, traceability is one of the first prerequisites for the trustworthiness of a record. The computerized audit trail of a laboratory's data system holds the evidence of who did what to a record and when. According to McDowall, "audit trail is a software utility that monitors changes to selected data sets within the main application."

Section 11.10 (e) of 21 CFR Part 11 requires an audit trail for "actions that create, modify, or delete electronic records" and that it be "secure, computer-generated, time-stamped."

 It is neither new nor surprising that previous entries in the audit trail must not be obscured, a practice well known to the keepers of paper records in a cGMP environment.

During FDA inspections, auditors typically refer to laboratory logs for the sequence of analysis and manufacturing steps. Similarly, audit trails help to manage, control, and also inspect the history of changes made to raw data and intermediate results that are used to calculate final results. However, the audit trail is only a subset of the change management of electronic records. Change management for electronic records requires both an audit trail (frequently called logbooks) and revision control of records. Logbooks merely describe what happened and when, but keeping the record under revision control establishes the exact details (for example, the chromatographic result before and after the change). When implemented properly, change management therefore can be used to answer the following questions:

Did any instrument or processing errors occur during the analysis of a certain sample that could have caused the result to be invalid? A recent FDA warning letter specifically addressed the lack of required instrument maintenance records: "Failure to maintain records of the inspections of automatic, mechanical or electronic equipment, including computers or related systems. [211.68(a)]. For example, the firm failed to maintain any background data to verify that testing of laboratory HPLCs (b) had been performed or produced acceptable results. Also, written and approved protocols for testing of these HPLCs were not maintained."

What particular changes were made to the integration parameters of a certain injection within the sequence?

Why was the analysis result reintegrated and subsequently reprocessed?

When was the analysis result reviewed and by whom?

Why was a particular analysis result rejected and excluded from the result calculation? A recent FDA warning letter specifically addressed missing records associated with out-of-specification investigations: "Out-of-specification (OOS) results were invalidated, without a thorough investigation, supporting data, documentation, or justification. For example: Confirmed OOS results (...) were invalidated by Quality Assurance, that concluded that the chromatographs were incorrectly integrated. The Chromatographs were reprocessed with adjusted baseline parameters, yielding acceptable results, and the lots were released for distribution. However, the laboratory investigation concluded that the results could not be invalidated and that no problems were observed during the chromatographic run."

Were there any "on-the-fly" changes to the processing parameters (for example, integration) that were subsequently discarded? A recent FDA warning letter specifically addressed the lack of traceability and doubt about the integrity of required laboratory records: " Laboratory records do not include complete data derived from all tests including a record of all calculations performed in connection with the test [21 CFR 211.194(a)(5)]."

Obviously, the capability of attaching audit comments to an electronic record helps the originator as well as the reviewer in documenting an action and justifying why it was done. Part 11 does not explicitly require entering a reason for a change, but some predicate rules do (for example, Good Laboratory Practice regulations). Some modern data systems therefore offer a function for fixed or user-definable audit comments. The data system can record, for example, that acertain method parameter was changed from value X to value Y, and in the comment section the analyst may state that this was because of a revised SOP.

Finally, in addition to operational controls that enforce the sequence of permitted steps systemically, audit trails also play a role in preventing "pencil whipping," that is, "the entry of data before an action occurs or at the end of the day, as an afterthought."

Metadata is central to the trustworthiness of records and compliance with Part 11. Without metadata, the traceability of a record is extremely limited. The complete and uncorrupted package of raw data, metadata, and results represents a trustworthy and reliable set of information that helps generate knowledge that results, production processes, and product quality are under control. Without metadata, it is not possible to "replay" the original result using the original input parameters. Even though "instant replay" is subject to enforcement discretion according to the 2003 Part 11 guidance, it is important for data migration when replacing legacy systems. Theoretically, firms can get away with not carrying legacy data forward, especially since this is a tough technical challenge for the regulated industries and their suppliers. Practically, however, it is an unacceptable waste of resources if there is no electronic data transfer between the original system and its replacement. Can a pharmaceutical development or quality-control laboratory afford to manually re-enter hundreds or even thousands of analytical methods? How efficiently can they investigate a complaint if there is only a paper archive that is not keyword searchable? How do they judge a small unspecified impurity if only a paper printout of the original chromatogram is available with no possibilities for zooming, reintegrating, or inspecting the spectral data?

Electronic records generated by an analytical instrument can be regarded as trustworthy and reliable if there is evidence that the communication between the instrument and system controller is trustworthy and reliable.

In many cases, firms must rely on electronic raw data to perform regulated activities such as QA/QC testing of finished drug products for batch release or when investigating a suspected out-of-specification (OOS) result. For example, a regulatory agency may ask for documentation of instrument conditions to support the laboratory's conclusion that a certain result was not OOS due to a technical failure of the apparatus. It may be difficult to show evidence that a given measurement was in fact performed according to the defined procedure, unless there is detailed documentation of the metadata. Examples of such metadata are instrument setpoints used during the analysis and setpoints used for re-integration (including documentation of the previous setpoints and previous results). Without hard evidence like this, the regulatory agency may suspect attempts to test the results into compliance!

Level-4 instrument control employs techniques such as automatic tracking of instrument identification and configuration, early maintenance feedback (EMF), self diagnostics, real-time data acquisition and synchronization independent of the computer, and bi-directional handshake protocols between devices and controllers for reliable and traceable instrument communication. We discuss level-4 instrument control further in the Instrument Control article in this series.

Object Database Management Systems (ODBMS) are specifically designed to manage and store complex objects and their complex relationships. Applications based on ODBMS are commercially available and suppliers have implemented data management systems based on ODBMS for several years.

 According to Loomis, one of the main benefits, apart from referential integrity and ease of system administration, is that "the storage of objects as objects, rather than fields of tables, not only maintains the inherent nature of the object, but can also eliminate 30-70% of a project's total code, which is typically used to map objects to tables."

For analytical data management systems, a significant percentage of the entire data volume consists of instrument raw data. Due to the size of the raw data, it is typically stored in an efficient binary format. Binary objects in a database system are called binary large objects (BLOBs). Modern systems such as Oracle 9i allow BLOBs to be managed efficiently within the database. In contrast, older systems implemented their own hybrid data management structure: database schemas used a combination of relational tables managed in the database and binary objects managed in a flat file system. 

Managing tables and objects within the database management system simplifies system maintenance significantly, as standard IT procedures for disaster recovery (backup) and data archiving can be used instead of specialized, two-fold procedures that have to synchronize processes on the database with processes on the file system. 

Identify the systems and records that are critical from a product quality and business perspective.

Analytical data systems used in industries subject to 21 CFR Part 11 need to be evaluated in terms of data security, data integrity, and audit trails to comply with current regulations and guidelines. Database systems generally help but are not a guarantee of data integrity and security. 

Assess the risks that affect data security and data integrity.

When assessing the risks in an existing system or when selecting a new one, consider not only raw data and results, but also metadata. If your system does not provide referential integrity for interdependent records, you may need a specific, paper-based operating procedure to collect the required evidence that your process and your data are intact.

When evaluating a data management system, verify that the system implements a tight link between results, raw data, and metadata that cannot becorrupted by ordinary means. Tight revision control of each data object is mandatory to achieve this goal.

Consider the administration and maintenance of the data management system. If the system manages critical records, you are responsible for appropriate maintenance and disaster recovery procedures. Ask your suppliers for help if you do not have the technical expertise in-house.

Standardization is a good way to manage complexity. Your user requirement specification should list the technical standards that you consider critical (operating systems, security policies, backup procedures, maintenance procedures).

1.		McDowall RD. Operational measures to ensure the continued validation of computerised systems in regulated or accredited laboratories. 

Laboratory Automation and Information Management 

2.		FDA. Code of Federal Regulations, Title 21, Part 11; electronic records; electronic signatures; final rule. 

 1997; 62(54):13429-13466. (See sections 11.10(b) and 11.30.)

3.		FDA. cGMP warning letter, File No.: 04-NWJ-02. Available at URL: 

4.		FDA. cGMP warning letter, File No. 2004-NOL-03. Available at URL: 

5.		FDA. cGMP warning letter, File No. 04-NWJ-01. Available at URL: 

7.		Loomis TP. The best of LIMS and object and relational DBMS can be combined. 

8.		Guzenda L. 7 signs that you need an object database. 

Protecting the integrity of data is a challenge of 21 CFR Part 11 compliance. Integrity requires records to be complete, intact, and maintained in their original context ? associated with the procedures which were used to create the data.

BioPharm International-02-15-2004

Qualifying Network Infrastructure - A Risk-Based Approach

21 CFR Part 11 - Requirements and New Scope

21 CFR Part 11: Choosing a Risk Assessment Methodology

Data Integrity for Electronic Records According to 21 CFR Part 11

Getting a Handle on Access Security for 21 CFR Part 11

Level-4 Instrument Control - Why It's Relevant for Part 11