National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×

Executive Summary

The goal of improving health through the use of new medicines and medical devices cannot be achieved without public confidence and participation in the clinical trials process. During clinical development, investigational therapies are tested with human subjects, yielding essential information for assessments of their safety and effectiveness. The importance of informed consent and human subject protection to the integrity of the clinical trials process has been widely discussed over the last 50 years. It is well accepted that human subjects should not be needlessly exposed to risks in trials that fail to yield valid data. Assuring the quality and therefore the usefulness of the data from human clinical trials, however, has received less attention, but it is also a crucial topic because public confidence in the value of clinical trials is ensured when the public knows that the data are of high quality and useful. In addition, regulatory and medical decisions about approval or use of new or novel therapies are dependent upon the reliability of data from clinical trials. For these reasons, there is widespread agreement that data from such trials should be of high quality.

Despite this fundamental agreement, many challenges remain. The current processes for assuring data quality were developed individually in response to various problems or crises rather than in a comprehensive quality management framework. Although the current system is successful, it is relatively expensive and time-intensive, may limit the overall investment in clinical trials, and may not provide the best-attainable quality for the degree of investment. Additionally, there is no consensus definition of ''quality'' as it applies to data from clinical trials. Finally, many changes that have the potential to affect data quality are occurring in the areas of clinical practice and clinical trials, including widespread computerization of data entry and handling, use of contract research organizations (CROs) to perform or organize clinical trials, the increased frequency of multinational trials, and changes in the health care delivery system.

Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×

The Roundtable on Research and Development of Drugs, Biologics, and Medical Devices convened a workshop on April 14 and 15, 1998, to discuss these and related topics. Representatives of major parties with vested interests in new medical product development were included. The workshop was successful in broadening the dialogue among the Food and Drug Administration (FDA), industry, and the public on the subject of data quality and validity in clinical trials for the regulatory decision-making process. The workshop participants described the components of the current system, and debated challenges and opportunities for improvement.

Assuring Clinical Trial Data Validity: The Current Process

Data quality efforts begin during the planning stages of a clinical trial and continue throughout FDA review of a marketing application. Product developers are responsible for planning and conducting trials, assembling and analyzing data, and preparing accurate regulatory submissions. FDA recently published the Guidance for Industry on Good Clinical Practices. This document, which represents a harmonized position among the regulatory bodies of the United States, the European Union, and Japan, provides advice on record keeping and procedures for many aspects of the conduct of clinical trials. After receiving an application, FDA evaluates the quality of the data that have been submitted, and also performs audits at clinical trial sites to further assess data quality. The following sections describe the major components of these processes.

Design of Protocol, Case Report Forms, and Data Collection Systems

The industry, government sponsor, or a CRO prepares the clinical protocol, including the forms used for collection of clinical data (often with input from FDA). The organization also develops computer systems for creation of a database and analysis of the information. The protocol and the case report form design, particularly the complexity of the design and the amount of data collected, have important influences on data quality. Some companies are now using remote data entry systems, whereby some or all trial data are entered directly into a computer or a centralized database. The designs of such systems present challenges to regulators.

Clinical Investigator and Study Personnel Training

The sponsor or a CRO trains physicians who will be conducting the study and trains study coordinators and other personnel who organize the study at each

Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×

site. This training is critical to ensuring that the protocol is followed correctly and that case report forms are properly completed.

Clinical Site Monitoring

The sponsor or a CRO periodically sends trained personnel (monitors) to each study site to check on the progress and quality of performance of the study. Monitors review case report forms and other study records to ensure that the documentation is complete. The pharmaceutical industry estimates that the monitoring of drug trials can consume 15 to 30 percent of overall trial costs.

Industry Data Quality Assurance Procedures

This step involves assembly of all the data from the trial, entry of the information into databases, and evaluation of the data for quality. If case report forms are paper-based, double data entry into the database is usually performed to minimize transcription errors. The data are then subjected to extensive quality assurance procedures involving follow-up activities on missing or potentially inaccurate datum points. Often, audits of clinical sites will be performed by the sponsor or a CRO as part of the industry quality assurance program.

FDA Data Analysis

FDA clinical reviewers and statisticians evaluate the data submitted in the application. FDA data analysis often includes checking and verification of data from important analyses submitted by the sponsor, as well as performance of exploratory analyses to answer questions that emerge from the review.

FDA Data Quality Assurance Evaluation

FDA clinical reviewers evaluate the quality of the data in the application using techniques such as auditing of case report forms to verify the accuracy of tabulated data, evaluation of follow-ups on reported adverse events, and verification of the primary outcome measure at the case report form level. An overall assessment of data quality is developed. The overall assessment is a factor in determining whether the application merits approval. If serious questions regarding overall data integrity are not resolved, FDA will not approve the application.

Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×

FDA Clinical Study Audit Program

During application review, FDA selects clinical sites that generated data for the application for auditing purposes. A thorough on-site review of these sites is conducted by trained FDA inspectors. Record keeping, adherence to the protocol, informed-consent procedures, and other aspects of the study are assessed. If objectionable conditions are found, a report (referred to as FDA Form 483) is provided to the principal investigator at the conclusion of the audit.

FDA Enforcement Activities

If an investigator is found to have serious or repeated problems in performing clinical studies, FDA will take steps to debar the individual from performing trials for regulatory purposes. In cases of fraud, criminal prosecution may be pursued.

Opportunities for Improving the Process

During the workshop participants identified a number of opportunities for significant improvements in the overall process. Some of the goals included the need for an increase in the dialogue among the involved parties, maintenance or improvement of quality with a concomitant lowering of costs, increases in the levels of public knowledge and confidence in the process, a means of dealing with emerging changes such as widespread computerization and delivery of new therapies to the market, and protection of the public from risk. Overall, participants identified a need for systemic collaborative improvements, such as improving processes in the context of the entire system with the input of all parties involved in the system. The following sections describe some of the targeted and overall strategies for improving the process.

Targeted Strategies

Broader Involvement of Patient and Consumer Groups
  • Representatives of patient and consumer groups indicated that extensive involvement of their constituencies was essential to ensure that their needs and concerns are accounted for and adequately addressed. The constituencies' confidence and participation in the clinical trials process is essential to successful product development.
  • Additionally, particular concerns of patient representatives included issues of conflict of interest and investigator bias. Reimbursement of investigators for recruitment and retention of trial subjects was an issue; this practice could be
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
  • a disincentive to maintaining high quality in clinical trials. The interrelated aspects of financial reimbursement and incentives, as well as scientific conduct in clinical trials, deserve further discussion.
Greater Emphasis on Building Quality into the Process

In the current system, both industry and FDA expend considerable effort auditing and correcting data once the data are collected. Many participants suggested that higher quality could be achieved by designing quality during the planning stages of clinical programs.

Overall Strategies

The following sections describe some of the opportunities for improving the overall process of assuring data quality and validity.

More Extensive Training of Clinical Investigators and Study Personnel

Since many quality problems result from failure to follow study designs, protocols, or collect data properly, development of a larger cadre of well-trained and experienced investigators and study personnel has the potential to improve data quality across studies. Many organizations are now interested in providing or participating in such training.

Data Standardization

Standardization has been successfully used in many industries to improve quality. In the clinical trial arena, terminologies, forms or computer screens used to collect data, and the tools and systems used to analyze the data are candidates for standardization. Entries for medical events and for concomitant medications were noted as areas of particular complexity. FDA is evaluating ways to further standardize the presentation of safety data.

Data Simplification

Although millions of data points are collected during the average clinical trial, some of these data are not used in the decision-making process. One important strategy for improving quality is to simplify data collection by distinguishing critical data prospectively. Further work is needed to provide a better understanding of what data sets are being submitted and how they might be simplified.

Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Improvements In Quality Control and Quality Assurance Methods

Benchmark standards are developed for data quality in the clinical trial setting. A significant amount of effort goes into detecting and correcting problems, such as missing data points and transcription errors, which are very different from uncertainty issues. Participants agreed that there is a "hierarchy of errors." Because a clinical trial often generates millions of data points, ensuring 100 percent completeness, for example, is often not possible; however, it is also not necessary. High-quality data may be defined as data strong enough to support conclusions and interpretations equivalent to those derived from error-free data. Certain data points are more important to interpreting the outcome of a study than others, and these should receive the greatest effort and focus. Implementation of this definition would require agreement on data standards.

Evaluations of monitoring and auditing techniques are also needed. Industry uses a variety of monitoring and auditing techniques to ensure high-quality data. These have not been compared or tested to see which are the most effective. Additional discussions and evaluation of monitoring and auditing techniques should be undertaken.

Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Page 1
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Page 2
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Page 3
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Page 4
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Page 5
Suggested Citation:"Executive Summary." Institute of Medicine. 1999. Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report. Washington, DC: The National Academies Press. doi: 10.17226/9623.
×
Page 6
Next: Introduction »
Assuring Data Quality and Validity in Clinical Trials for Regulatory Decision Making: Workshop Report Get This Book
×
Buy Paperback | $47.00 Buy Ebook | $37.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In an effort to increase knowledge and understanding of the process of assuring data quality and validity in clinical trials, the IOM hosted a workshop to open a dialogue on the process to identify and discuss issues of mutual concern among industry, regulators, payers, and consumers. The presenters and panelists together developed strategies that could be used to address the issues that were identified. This IOM report of the workshop summarizes the present status and highlights possible strategies for making improvements to the education of interested and affected parties as well as facilitating future planning.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!