Executive Summary
In November 1997, investigators from Johns Hopkins University and Makerere University in Uganda began the HIVNET 012 clinical trial to evaluate the efficacy and safety of single-dose nevirapine (NVP) and short-course zidovudine (ZDV) regimens for preventing mother-to-child transmission of HIV infection. The trial was initially designed as a randomized, double-blind, placebo-controlled trial, but the placebo arms were dropped after results from a study in Thailand found short-course zidovudine to be effective in reducing mother-to-child transmission of HIV. Enrollment in the trial was concluded in April 1999. The trial was sponsored by the Division of AIDS (DAIDS) at the National Institute of Allergy and Infectious Diseases (NIAID) of the National Institutes of Health (NIH).
Preliminary trial results were published in the journal Lancet in 1999 (Guay et al., 1999). Given the encouraging evidence that single-dose nevirapine reduced the risk of mother-to-child transmission of HIV, Boehringer Ingelheim (BI), the manufacturer of nevirapine, decided to pursue a U.S. Food and Drug Administration (FDA) labeling change for the drug using HIVNET 012 as a registrational trial for its application in the United States. The decision to use the HIVNET 012 study as support for the labeling change made the trial subject to reviews that were conducted in a manner that was far more in-depth than would ordinarily occur for a clinical trial that, like HIVNET 012, was not originally intended to generate data to support a submission to FDA for approval of a new drug or new indication for an old drug.
Following BI’s own review of the HIVNET 012 study, DAIDS con-
tracted with the Westat Corporation to conduct a pre-FDA inspection audit in February 2002 (Chamberlin et al., 2002). Westat’s report cited some deficiencies in the conduct of the trial, which in turn prompted a comprehensive and lengthy remonitoring effort by DAIDS. BI subsequently withdrew its application to the FDA for a supplemental indication for the use of NVP in preventing mother-to-child transmission, stating that the “NIAID and Boehringer Ingelheim review could not be completed within the remaining timeline for FDA action for the supplement” (Boehringer Ingelheim, 2002). The investigators, the contract research organization monitoring the study (Family Health International, FHI), and staff of the HIVNET statistical center (SCHARP) responded to the Westat Site Visit Report, providing additional information that explained or resolved some negative audit findings (FHI, 2002; HIVNET 012 Investigators, 2002; SCHARP, 2002). The investigators and FHI also took steps to strengthen study procedures in response to findings with which the investigators concurred (HIVNET 012 Investigators, 2003).
Despite the series of evaluations of and subsequent correspondence about HIVNET 012, no definitive document in the public domain critically and objectively evaluates the study’s design, conduct, results, and validity. This has led to uncertainty among public health and medical professionals—as well as those in political circles and the broader HIV-1-affected communities—about whether single-dose NVP is efficacious and safe as a regimen for prevention of mother-to-child transmission of HIV (Cohen, 2004).
In August 2004, in response to a request from NIH, the Institute of Medicine (IOM) convened a committee to review the HIVNET 012 trial and provide an independent assessment of the validity of the study’s results. The committee’s charge was as follows:
“The IOM committee will address methodological and data interpretation questions related to protocol design, data collection, record keeping, quality control, and analysis. The committee will assess the impact of these issues on the validity of the overall findings and conclusions of the trial. The IOM committee is charged with addressing the following questions related to HIVNET 012:
-
Was the protocol design appropriate?
-
Does the fact that, in many cases, there were no informed-consent forms from the fathers cause enough significant concern to invalidate the conclusions?
-
Are there results available (published or unpublished) of assays of drug levels and should consideration be given to what, if any, impact they might have on the conclusions?
-
Was the protocol followed sufficiently to conclude that the data are sustainable?
-
Was the quality control sufficient to uphold the conclusions?
-
A certain number of documents were destroyed by a natural disaster. Is this a significant deterrent to drawing conclusions?
-
Can the integrity of the data be sustained in view of the deficiencies of the data collection, and the consistency of its recording?
-
Are the conclusions supportable by the data?
-
Is there any reason to suggest the need to retract the publications or to revise the conclusions?”
The IOM Committee on Reviewing the HIVNET 012 Perinatal HIV Prevention Study met three times and held numerous meetings by conference call between September 2004 and March 2005, and its work led to the present report. Early in its deliberations, the committee concluded that the validity of the trial’s findings ultimately rested on the following elements: (1) the integrity of the study design, treatment assignment, and treatment adherence, addressed in Chapter 3; (2) the completeness and accuracy of efficacy and safety data, addressed in Chapter 4; and (3) the study’s adherence to ethical principles for conducting clinical research, addressed in Chapter 5.
The committee reviewed relevant materials provided by NIH, by the investigators, previous auditors, and from a variety of other sources. In addition, the committee obtained copies of a subset of primary source documents from Uganda, as well as information from the study database maintained for HIVNET 012 by the Statistical Center for HIV/AIDS Research and Prevention (SCHARP). After its review of these materials, the committee reached its findings about the key aspects of study design and implementation. In its work, the committee was aware of one important aspect of the broader context in which the HIVNET 012 study occurred and was audited: the procedural guidelines and standards for clinical research are always evolving, and the Westat site visit occurred more than 4 years after the initiation of the HIVNET 012 trial. This report describing the committee’s analysis and findings contains six chapters and three appendixes.
In Chapter 1, the report provides background information about prevention of mother-to-child transmission of HIV in Africa, a concise summary of the study milestones, and the committee’s approach to its charge. Chapter 2 provides an overview of the published findings of the HIVNET 012 trial.
Chapter 3 discusses the committee’s assessment of the appropriateness of the HIVNET 012 study design (choice of site and treatment regimens, randomization, and statistical methods) and specific aspects of its imple-
mentation (randomization, drug management, and participant adherence to study regimens). Overall the committee found that the protocol designs for HIVNET 012—both before and after the discontinuation of the placebo arms—were appropriate. The committee also made the following specific findings:
The committee finds that Uganda was a reasonable setting for evaluation of short-course regimens for preventing mother-to-child transmission of HIV-1. Moreover, the regimens chosen for study were reasonable in the context of knowledge about prevention of mother-to-child transmission at that time. The decision to stop the placebo arms of the trial and continue with the originally designed active arms was reviewed and approved appropriately.1
The committee finds that the inclusion and exclusion criteria employed in HIVNET 012 were reasonable.
The committee finds that the designs of the original and modified randomization procedures in HIVNET 012 were scientifically sound and appropriate for the research setting.
The committee finds that the HIVNET 012 randomization procedures were implemented with a high level of accuracy, achieving the scientific goal of creating two comparable treatment groups.
The committee finds that the original and revised sample size targets for the HIVNET 012 trial were sufficient to achieve the study goals.
The committe finds that the statistical methods employed in HIVNET 012 and described in the publications were appropriate. The results obtained from the analyses of HIV infection and HIV-1 free survival2 were properly interpreted by the study authors. Additional analyses of efficacy in the Results and Discussion sections of Guay et al. (1999) and Jackson et al. (2003) were presented in a balanced manner and with appropriate qualifications.
The committee finds that the HIVNET 012 investigators used appropriate practices for packaging and distributing study drugs, so that the assigned drug was consistently provided to the appropriate mothers and their infants. Evidence from cord blood specimens indicates that
participants achieved a high level of adherence to the NVP regimen. Though no direct evidence is available on blood levels of ZDV, the maternal reports of high levels of adherence to the treatment regimen, the fact that hospital personnel administered a substantial fraction of the ZDV regimen, and the absence of detectable levels of NVP in the blood of participants in the ZDV arm suggest that high levels of adherence were also achieved in the ZDV arm. The high level of adherence to study regimens indicates that the treatment arms formed an appropriate basis for assessing the efficacy and safety of the study regimens.
In Chapter 4, the committee provides its assessment of the validity of study data related to the safety and efficacy endpoints. The committee reviewed the definitions of the safety and efficacy endpoints and their implementation. The committee also obtained copies of primary source documents and case report forms from the study site in Uganda for a sample of mother/infant pairs and compared those documents with HIVNET 012 database information provided by SCHARP. Using these data, the committee conducted its own evaluation of the accuracy, completeness, and timeliness in reporting of adverse events and serious adverse events, survival status, and HIV infection status of infants in the sample. Based on its detailed examination of study data, the committee found no evidence of misrepresentation of the study results. Finally, the committee reviewed other aspects of safety such as incidence of hyperbilirubinemia. The committee’s findings include:
The committee finds that the testing schedule and assays used in HIVNET 012 to diagnose HIV-1 infection in infants were appropriate. Use of HIV-1 positivity and HIV-1-free survival at 6–8 weeks, 14–16 weeks, and 18 months of age as the primary efficacy endpoints also was appropriate.
The committee finds that the definitions of adverse events (AEs) and serious adverse events (SAEs) specified in the protocol were reasonable. The committee finds that the follow-up periods and schedule of evaluations established for mothers and infants participating in HIVNET 012 were reasonable and were sufficient to capture relevant information about adverse events.
The committee finds no issues of concern regarding the reliability and validity of laboratory test results obtained in HIVNET 012, or the completeness and accuracy of study laboratory records.
The committee finds that the HIVNET 012 investigators interpreted definitions contained in the 1996 and 1997 Code of Federal Regulations and the protocol so as to use hospitalization as the primary, but
not sole, determinant of seriousness for capture of serious adverse events. Although this well may have been a practical and appropriate interpretation of the definition of serious adverse events, it means that the safety results, while meaningful in a Ugandan context and other similar settings, may not be entirely generalizable to settings in which the definition of seriousness is interpreted differently and where thresholds for hospitalization vary.
The committee finds that participation of HIVNET 012 infants in the vitamin A study had no impact on the HIVNET 012 efficacy endpoints or AEs, and finds no evidence that such participation might have biased the comparative SAE rates in HIVNET 012 in favor of NVP.
The committee finds that the record keeping system implemented in HIVNET 012 was reasonable and appropriate. While there were some documentation and procedural deficiencies reported by auditors, none appeared to have affected the results of the study. There is no evidence that flooding or any other natural phenomenon significantly impacted the completeness of study records.
In its review of HIVNET 012 records, the committee finds no evidence of and only a very limited opportunity for either unreported deaths or erroneous reports of deaths.
The committee finds that source document information regarding survival status was accurately transferred to the SCHARP database in a timely manner.
The committee finds that in the subset of 49 infants whose charts it reviewed, HIV-1 RNA polymerase chain reaction (PCR) and HIV-1 enzyme immunoassay (EIA) information in the source documents used to assess HIV-1 infection status was accurately transferred to the SCHARP database, and done so in a timely manner so that all results available at the time of the data freeze for study publications were included in the analyses.
The committee finds that infant deaths, hospitalizations and visits where an infant experienced an SAE were accurately reported to the SCHARP database, although, in some instances, not all concomitant SAEs were reported. The committee also finds that some (non-serious) AEs noted in the source documents were not reported on the case report forms. The underreporting of some (non-serious) AEs and some concomitant SAEs that accompanied a reported SAE may limit the generalizability of absolute AE rates and counts to other settings. However, the committee finds no reason to believe that the rates of unreported adverse events varied by treatment group, suggesting that
the comparative safety analyses reported by the HIVNET 012 investigators are valid.
The committee concurs with the HIVNET 012 investigators’ determination that 1.2 mg/dL, as suggested in the April 8, 2003, Investigational New Drug (IND) Safety Report, was not an appropriate upper limit of normal value for bilirubin in newborns, whose bilirubin levels change rapidly over the first few days after birth and are normally substantially higher than those in adults. The committee also concurs with DAIDS’ decision to withdraw its initial IND safety report finding of excess hyperbilirubinemia because it was derived from the application of an incorrect criterion to study data.
The committee finds no evidence in HIVNET 012 of an increased risk of clinically significant hyperbilirubinemia in the infants who received NVP compared to the infants who received ZDV.
In Chapter 5, the committee assesses the design and conduct of the study from the perspective of protection of human subjects, including HIVNET 012 compliance with requirements for independent Institutional Review Board (IRB) oversight, the inclusion of placebo arms in the original trial design, the circumstances that made the placebo control no longer appropriate, and the informed-consent process. The Westat Site Visit Report stated that HIVNET 012 investigators were found to lack training in and awareness of Good Clinical Practice Guidelines (Chamberlin et al., 2002). In this chapter, the committee explains that the HIVNET 012 trial was not subject to these “GCP” Guidelines, which are a voluntary set of international guidelines that inform but do not constitute the FDA regulations to which HIVNET 012 was subject (ICH, 1996). Further, the committee explains that the FDA regulations that did apply to HIVNET 012 were in some important respects more stringent than the international GCP Guidelines. Finally, the committee distinguishes between compliance with the voluntary international GCP Guidelines and actual good clinical practice, which refers to good clinical management and medical care in the course of a trial. The committee also noted that conducting good, ethical clinical research does not solely consist of following procedures, but rather consists of ensuring independent oversight, a reasonable balance between risks and benefits, assurance that subjects give free and informed consent, and that subjects are protected. The committee’s findings include:
The committee finds that HIVNET 012 was conducted under an IND as a matter of DAIDS policy, and that the study was not originally intended to provide data for later submission to FDA to support a labeling change for NVP, an already approved drug. The decision by
Boehringer Ingelheim to use the findings to support such a submission led to evaluating the documentation of regulatory compliance by the trial in light of a standard that did not apply when the trial began.
The committee finds that the HIVNET 012 investigators met their ethical obligation to design and conduct the study in accordance with international standards for the ethical conduct of research and ethical management of patient care. The HIVNET 012 investigators also complied with their legal obligation to design and conduct the study in accordance with FDA regulations and under the oversight of IRBs in both Uganda and the United States. The HIVNET 012 trial was not required to comply with specific procedural rules outlined in the voluntary GCP Guidelines published by the International Conference on Harmonisation, and an ethical evaluation of HIVNET 012 should not rest directly or indirectly on the degree to which it conformed to GCP Guidelines, but rather on the degree to which it conformed to FDA, IRB, and general medical ethics standards to which it was subject. The validity of the study’s findings is sustained by the fact that the trial was conducted in accordance with FDA requirements and met international standards for the ethical management of clinical trials.
The committee finds no evidence that the definitions used for adverse events and serious adverse events in HIVNET 012 placed human subjects at increased risk.
The committee finds no evidence that the failures identified by the Office for Human Research Protections (OHRP) with respect to ARC’s3 continuing review procedures resulted in a loss of information that would, had it been obtained at the time, have altered the risk-benefit balance in a way that would have triggered either a change in the protocol or a change in the information given to human subjects.
The committee finds that the initial design of the HIVNET 012 trial, which incorporated two placebo arms, was properly reviewed and approved by the relevant Johns Hopkins University and Ugandan IRBs, and that justifications for the use of placebo arms were adequately presented.
The committee finds that the HIVNET 012 trial was promptly and properly reevaluated and the placebo arms discontinued when new data emerged from other studies.
The committee finds that the initial study design incorporated all relevant protections relating to the need for voluntary informed consent, the acceptability of placebo control, the discontinuation of placebo control, and overall compliance with IRB reviews.
The committee finds that the investigators correctly identified appropriate guardians to consent to extended 5-year follow-up in situations where the original consenting parent had died, but that the investigators failed to do this in situations where the consenting parent died while the child was still enrolled in the original, 18-month follow-up.
The committee finds that requesting additional consent from the fathers before enrolling the pregnant women or their infants in the study was not necessarily required by U.S. federal regulations but was required per DAIDS policy and was therefore incorporated into the IRB-approved protocol.
The committee finds that the failure to obtain such additional paternal consent was based on the practical unavailability of the fathers and the ethical constraints that prevented the research staff from contacting fathers in the absence of the mother’s support and consent.
The committee finds that while auditors reported procedural lapses by the Ugandan IRB, there was evidence of rapid and appropriate response by the IRB in approving modification of the design of HIVNET 012 and discontinuation of placebo arms. There was also no evidence that a participant signed the wrong4 version of the consent form.
Despite some lapses in documentation, the committee finds no evidence that study subjects failed to give voluntary informed consent.
The committee finds that HIVNET 012 met the substantive standards for ethical conduct of research and was implemented in substantial compliance with regulations governing protection of human subjects, especially independent review of risks and benefits to them.
The committee finds that there is no reason based in ethical concerns about the design or implementation of the study that would justify excluding its findings from use in scientific and policy deliberations.
Chapter 6 is a concluding narrative that provides a discussion of the committee’s response to each item of the charge, based on the findings of earlier chapters.
Based on its review, the committee finds no reason to retract the publications or alter the conclusions of the HIVNET 012 study. The committee concludes that data and findings presented in Guay et al. (1999) and Jackson et al. (2003) are sound, presented in a balanced manner, and can be relied upon for scientific and policy-making purposes.
REFERENCES
Boehringer Ingelheim. 2002. Boehringer Ingelheim Comments on HIVNET 012 Trial. [Online] Available at http://www.boehringer-ingelheim.com/corporate/asp/news/ndetail.asp?ID+224. Accessed June 24, 2004.
Chamberlin J, Gustavson SA, Hensley M, Lander S. 2002. Site Visit Report, Kampala, Uganda, February 18-28, 2002. Westat Corporation, MD.
Cohen J. 2004. HIV transmission. Allegations raise fears of backlash against AIDS prevention strategy. Science 306(5705):2168-2169.
FHI (Family Health International). 2002. Response to DAIDS by FHI on Westat Inspection Report for HIVNET 012-Kampala, Uganda-Conducted February 18-28, 2002.
Guay LA, Musoke P, Fleming T, Bagenda D, Allen M, Nakabiito C, Sherman J, Bakaki P, Ducar C, Deseyve M, Emel L, Mirochnick M, Fowler MG, Mofenson L, Miotti P, Dransfield K, Bray D, Mmiro F, Jackson JB. 1999. Intrapartum and neonatal single-dose nevirapine compared with zidovudine for prevention of mother-to-child transmission of HIV-1 in Kampala, Uganda: HIVNET 012 randomised trial. Lancet 354(9181):795-802.
HIVNET 012 Investigators. 2003. Makerere University–Johns Hopkins University Quality Management Plan.
HIVNET 012 Investigators. 2002. Summary Sheet of Major Inaccuracies in the Westat Report.
ICH (International Conference on Harmonisation). 1996. Guideline for Good Clinical Practice. [Online] Available at http://www.ich.org/MediaServer.jser?@_ID=482&@_MODE=GLB. Accessed March 14, 2005.
Jackson JB, Musoke P, Fleming T, Guay LA, Bagenda D, Allen M, Nakabiito C, Sherman J, Bakaki P, Owor M, Ducar C, Deseyve M, Mwatha A, Emel L, Duefield C, Mirochnick M, Fowler MG, Mofenson L, Miotti P, Gigliotti M, Bray D, Mmiro F. 2003. Intrapartum and neonatal single-dose nevirapine compared with zidovudine for prevention of mother-to-child transmission of HIV-1 in Kampala, Uganda: 18-month follow-up of the HIVNET 012 randomised trial. Lancet 362(9387):859-868.
SCHARP (Statistical Center for HIV/AIDS Research and Prevention). 2002. Response to Westat Site Visit Report. Questions Responding to Major Concerns.
