4

Evaluation of Omics-Based Tests for Clinical Utility and Use

This chapter explains the recommended steps for assessing a validated omics-based test for clinical utility and use. In particular, this chapter will explore elements recommended by the committee to evaluate a test for clinical use in trials in which the omics-based test will be used to guide patient management decisions. It should be noted again that although the Institute of Medicine (IOM) committee was charged with recommending an evaluation process for omics-based tests, much of the proposed process, and the material presented in this chapter in particular, applies broadly to the development of biomarker-based tests. As described in Chapter 1, omics-based tests can be considered a complex form of a biomarker, using a defined set of measurements combined with fully defined computational procedures as a clinical test.

In parallel to the recommendations presented in Chapters 2 and 3, Figure 4-1 highlights the recommended steps in the evaluation for clinical utility and use of tests that are intended to guide patient management in a clinical care setting, the third component of the committee’s recommended development and evaluation process for omics-based test. The end product of the test development phase described in Chapter 3 is a fully defined and locked-down clinical test that has undergone analytical and clinical/ biological validation.

BACKGROUND

As discussed in Chapter 1, the committee adopted a modified version of the terminology for biomarker validation developed by the Evaluation of Genomic Applications in Practice and Prevention Working Group. Clinical utility is defined as “evidence of improved measurable clinical outcomes, and [a test’s] usefulness and added value to patient management decision-making compared with current management without [omics] testing” (Teutsch et al., 2009, p. 11). The Food and Drug Administration (FDA) does not require evidence of clinical utility for its evaluation of a clinical test; indeed, it can take many years after a test is introduced in the marketplace to demonstrate clinical utility of the test. Likewise, the lack of FDA review for a laboratory-developed test (LDT) does not mean the test does not have clinical utility.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 79
4 Evaluation of Omics-Based Tests for Clinical Utility and Use This chapter explains the recommended steps for assessing a validated omics-based test for clinical utility and use. In particular, this chapter will explore elements recommended by the committee to evaluate a test for clinical use in trials in which the omics-based test will be used to guide patient management decisions. It should be noted again that although the Institute of Medicine (IOM) committee was charged with recommending an evaluation process for omics-based tests, much of the proposed process, and the material presented in this chapter in particular, applies broadly to the development of biomarker-based tests. As described in Chapter 1, omics-based tests can be considered a complex form of a biomarker, using a defined set of measurements combined with fully defined computational procedures as a clinical test. In parallel to the recommendations presented in Chapters 2 and 3, Figure 4-1 highlights the recommended steps in the evaluation for clinical utility and use of tests that are intended to guide patient management in a clinical care setting, the third component of the committee’s recommended development and evaluation process for omics-based test. The end product of the test development phase described in Chapter 3 is a fully defined and locked-down clinical test that has undergone analytical and clinical/ biological validation. BACKGROUND As discussed in Chapter 1, the committee adopted a modified version of the terminology for biomarker validation developed by the Evalua- 79

OCR for page 79
80 Discovery and Test Validation Stage Evaluation for Clinical Utility and Use Stage Three Potential Pathways (IRB Approval and FDA Consultation) Discovery Phase Test Validation Phase B R Prospective/ Prospective Prospective I Retrospective Clinical Trial; Clinical Trial; Analytical and G Test Does NOT Test Directs Study with Product is a Clinical/Biological H Direct Patient Patient Archived fully speci ed Validation T Management Management Specimens candidate omics-based Product is a fully speci ed IDE Needed? test with and locked-down locked-down omics-based clinical test. No No Yes computational procedures. L FDA Approval/Clearance or LDT Process for Clinical Test I Additional High-Quality Evidence to Evaluate Clinical Utility of the Test N E See Chapter 2 See Chapter 3 Practice Guidelines and Reimbursement Clinical Use FIGURE 4-1 Omics-based test development process, highlighting the evaluation for clinical utility and use stage. The bright line signifies the point in test development where a fully defined, validated, and locked-down clinical test is necessary. Changes to the test after the bright line is crossed require going back to the test validation phase, approval by the IRB, and possibly consultation with FDA. In this stage of test development and evaluation, the fully defined and validated omics-based test undergoes evaluation for clinical use. Determination of clinical utility is a process that often occurs over a longer time frame. NOTE: FDA = Food and Drug Administration, IDE = investigational device exemption, IRB = Institutional Review Board, LDT = laboratory-developed test.

OCR for page 79
81 EVALUATION OF OMICS-BASED TESTS tion of Genomic Applications in Practice and Prevention Working Group. Clinical utility is defined as “evidence of improved measurable clinical outcomes, and [a test’s] usefulness and added value to patient manage- ment decision-making compared with current management without [omics] testing” (Teutsch et al., 2009, p. 11). The Food and Drug Administration (FDA) does not require evidence of clinical utility for its evaluation of a clinical test; indeed, it can take many years after a test is introduced in the marketplace to demonstrate clinical utility of the test. Likewise, the lack of FDA review for a laboratory-developed test (LDT) does not mean the test does not have clinical utility. Various methods have been proposed over the past two decades to sys- tematically establish the clinical utility of biomarkers in oncology, similar to those used for new therapeutics (Altman and Lyman, 1998; Altman and Riley, 2005; Altman and Royston, 2000; Hayes et al., 1996; Simon, 2005a, 2005b; Simon and Altman, 1994; Simon et al., 2009). Demonstration of clinical utility for an omics-based test is fundamentally similar to that of biomarker tests in general, although attention to the challenges unique to omics-based test development is necessary in the discovery and test valida- tion phases (see Chapters 2 and 3). Traditionally, devices such as biomarker tests have been subject to a regulatory process different from that used for new drugs. However, just as an ineffective drug can be harmful to patients, an inaccurate test has the potential to harm patients by directing them to ineffective, toxic treat- ments or by directing them away from potentially life-saving drugs. As the use of biomarker tests, including omics-based tests as discussed in this report, become more closely tied to patient management decisions such as choice of therapies, it is worth drawing a comparison between the two processes. Just as it is important to fully standardize and understand the components, dose, and schedule of a new therapeutic before taking it into clinical trials to assess clinical efficacy, it is essential that analytical validity for an omics-based test be demonstrated (Chapters 2 and 3). Likewise, in a manner similar to the use of a Phase II clinical trial to demonstrate that a new therapeutic appears to have some level of activity that would justify the expense and costs of a large-scale definitive Phase III trial, it is essential to demonstrate clinical/biological validity of a new omics-based test prior to assessing its clinical utility. Without a demonstration of a test’s “ability to accurately and reliably predict the clinically defined disorder or pheno- type of interest” (Teutsch et al., 2009, p. 10), the test is likely to fail, and a clinical trial to evaluate potential clinical utility would not be justified. For therapeutics, the goal of such a clinical trial is to demonstrate that the strat- egy is either superior to the current standard of care, or that it is equivalent to standard of care with some other advantage (lower cost, lower toxicity,

OCR for page 79
82 EVOLUTION OF TRANSLATIONAL OMICS more convenient administration, etc.), a concept that can be extended to biomarker tests, including omics-based tests. In addition to regulatory oversight, the threat of legal liability for a defective product should also deter organizations from moving a defective omics-based test into a clinical trial or clinical use prematurely. Under tort law, a company developing and/or marketing a test is responsible for paying damages to an individual who has experienced a harm or loss caused by a defective product. Omics-based tests that go through the FDA approval process are protected against certain torts.1 However, all test develop- ers are liable for losses and harms caused by negligence (i.e., the failure to exercise the care that a reasonably prudent person would exercise in like circumstances). The consequences of a products liability lawsuit may differ depending on the size of the test developer. For example, a small startup company has the net asset value of the firm to compensate for dam- ages from defective products. Large academic medical centers have more resources to lose, and hence are more at risk from defective products. EVALUATION FOR CLINICAL UTILITY AND USE STAGE Ultimate achievement of clinical utility, which is often assessed by organizations such as the U.S. Preventive Services Task Force (USPSTF), the BlueCross BlueShield Association Technology Evaluation Center, the National Cancer Center Network, and the American Society of Clinical Oncology, for example, is a process that often continues long after a test is first introduced into clinical practice. Recent assessments by USPSTF of mammography and prostate-specific antigen screening highlight the length and dynamic nature of this process (USPSTF, 2009, 2011). Nonetheless, the process of gathering evidence pertaining to clinical utility begins before a test is introduced into clinical practice, and the evidence necessary prior to clinical use of a test is the subject of much scholarly work (AHRQ, 2011; Simon, 2010; Simon et al., 2009: Teutsch et al., 2009). The purpose of this chapter is to briefly outline the types of clinical studies and clinical trials used to gather evidence prior to introducing tests into clinical practice and for assessing the clinical utility of tests, but only inasmuch as needed to explain the committee’s recommendations related to the Evaluation for Clinical Utility and Use Stage of omics-based test development. As shown in Figure 4-1, investigators might follow one of three pathways to generate sufficient evidence for this evaluation stage. The design of the clinical study or clinical trial used to assess clinical utility depends on the intended use of the omics-based test as well as the avail- 1 Murphree v. Pacesetter, No. CT-005429-00-3 (Cir. Ct. Tenn., 13th Judicial Dist.), and Horn v. Thoratec Corp., 376 F.3d 163 (3d Cir. 2004).

OCR for page 79
83 EVALUATION OF OMICS-BASED TESTS ability of appropriate archived specimens. Regardless of the pathway cho- sen, investigators are strongly urged to communicate with FDA, especially regarding the potential need for an investigational device exemption (IDE) if the test is to be used in a prospective clinical trial. Following the clini- cal studies or clinical trials, the test either must achieve FDA clearance or approval or must follow the LDT process for introducing a test into clinical practice, with subsequent evaluation and perhaps incorporation of the test into clinical practice guidelines and reimbursement by payers. These steps are shown for completeness, but are beyond the scope of this study and are not discussed in this chapter. FDA Regulation Introduction of an omics-based test into standard clinical practice can occur in two ways: by seeking FDA approval or clearance using the pre- market approval or 510(k) process, respectively, or by developing an LDT within a specific laboratory. Any clinical laboratory that reports tests for clinical management of patients falls under the purview of the Clinical Laboratory Improvement Amendments of 1988 (CLIA). LDTs developed by a CLIA-certified laboratory do not require FDA approval or clearance. Although FDA has the authority to regulate these tests, it has chosen to exercise enforcement discretion in most cases. The question of whether FDA should exercise greater authority over the regulation of LDTs has been the subject of much debate. The Secretary’s Advisory Committee on Genetics, Health, and Society recently recommended that FDA should provide over- sight of all LDT’s, with priority given to higher-risk tests (SACGHS, 2008). One challenge in implementing this approach is the need for additional resources and expertise at FDA to provide this level of oversight. Of note, FDA review of a biomarker test has been focused principally on analytical and clinical/biological validity, but not on demonstration of clinical utility, as defined in this report. Therefore, FDA approval or clear- ance does not necessarily imply that the test improves clinical outcomes or should be used for patient management. LDTs performed in CLIA-certified laboratories also do not require evidence of clinical utility; only analytical and clinical validity of the test must be demonstrated prior to clinical use.2 This situation has led guidelines committees and third-party technical assessment panels to perform opinion or evidence-based analyses of avail- able data to make recommendations regarding whether selected tumor 2 For clinical laboratories within New York State and for any clinical laboratory perform- ing testing on specimens from New York State patients, the New York State Department of Health must also review the validation and test performance characteristics of all LDTs prior to clinical use.

OCR for page 79
84 EVOLUTION OF TRANSLATIONAL OMICS biomarker tests should be used clinically, and whether they should be supported by reimbursement, independent of FDA approval or clearance (AHRQ, 2011; Allegra et al., 2009; Harris et al., 2007; Locker et al., 2006; NCCN, 2010; TEC, 2011; Teutsch et al., 2009). In general, these com- mittees and panels have used similar definitions of analytical and clinical/ biological validity and clinical utility adopted for this report. Nonetheless, to address the fundamental charge of this Committee (determining when an omics-based test is fit for use as a basis for a clinical trial design), it is important to ensure that a test used to direct care in a prospective clinical trial is vetted very carefully. As in the case of therapeu- tics, the potential for conflicts of interest and the overall benefit of external, objective review highlight the importance of involving experts not associ- ated with the design or conduct of the clinical trial in this vetting process. The committee believes that FDA is the most competent and appropriate body for this role. Therefore, as discussed in Chapters 2, 3, and 6, the com- mittee recommends that investigators should communicate early with FDA regarding the IDE process and validation requirements (Recommendation 3a). When a test will be used to direct patient management in a clinical trial, an IDE is required according to current regulatory standards. This corresponds with the third pathway to clinical utility shown in Figure 4-1. For the first two pathways, while IDEs are often not necessary, consulta- tion with FDA at an early stage of test development still is recommended because of the complexity of the development process and regulations for eventual approval of the test for clinical use. In a discussion with the Committee, Dr. Alberto Gutierrez, director of the Office of In Vitro Diagnostics in the Center for Devices and Radiological Health, acknowledged that FDA has not clearly defined what constitutes an LDT or when enforcement discretion would be applied, and noted that FDA has received fewer IDE applications for in vitro diagnostic test devices than expected from academic centers (Gutierrez, 2011). He added that while academic medical centers are subject to the same regulations and policies as commercial test developers, FDA may not expect academic medical cen- ters to comply with some quality systems regulations, such as good clinical practices (GCPs) and good laboratory practices (GLPs) (Gutierrez, 2011). Instead, a sufficient demonstration of analytical validation of the test would be required. Consistent with the committee’s recommendation, he also advised that test developers should consult with FDA to determine whether their devices qualify as LDTs or whether IDEs are needed (Gutierrez, 2011). FDA’s Center for Drug Evaluation and Research (CDER) has also rec- ognized the importance of biomarkers, including omics technologies, for improving drug development (FDA, 2004, 2006). In order to encourage submission of genomic and other exploratory omics data, FDA developed a nonregulatory voluntary data submission process, along with guidance

OCR for page 79
85 EVALUATION OF OMICS-BASED TESTS for industry (FDA, 2005; Orr, 2007). However, it is important to note that the primary regulatory focus of CDER has been and continues to be on drugs. If an omics-based test is considered a medical device, CDRH will take primary responsibility for regulatory oversight of the product. Design of Clinical Studies and Clinical Trials The objective of this IOM review was not to determine the criteria for clinical utility of an omics-based test. Rather, it was to determine the criteria for considering use of an omics-based test to direct management of patient care in a clinical trial designed to assess the clinical utility of the test. However, a discussion of how to generate high-quality evidence in a clinical trial to assess clinical utility is essential as a preamble for determining when such a test should be used to direct patient management in a clinical trial. To continue the comparison between the regulatory processes for approval of a new therapeutic and the regulatory process for clearance of a new omics-based test, a new therapeutic must undergo rigorous defini- tive testing in one or more properly designed and conducted prospective clinical trials before it is accepted for routine clinical use. The same is true for omics-based tests that might be used to direct clinical management. In order for such a test to be incorporated into routine clinical care, investi- gators should understand and have articulated the precise intended use of the test, as described in Appendix C. It is also important that an estimate of the magnitude of difference in clinical outcomes associated with differ- ent results of the omics-based test be sufficiently large to warrant different patient management (Henry and Hayes, 2006). For example, a test might have clinical/biological validity by virtue of distinguishing the outcome of two subgroups within a population with a high degree of statistical significance, but if this difference is not large, then these two groups of patients might not be treated differently. Such a test has clinical/biological validity, but not clinical utility. Ideally, an omics-based test would identify one group of patients for whom the benefits from a specific therapy are likely to outweigh the risks, and another group for whom the risks of the therapeutic strategy under consideration are likely to outweigh the benefits. A test with this character- istic would have utility as a clinically useful effect modifier. Importantly, this estimate of the magnitude of difference in outcomes must be reliable. As discussed in Chapter 3, the committee’s recommenda- tions to establish analytical and clinical/biological validity of an omics- based test are critical, and the decision regarding whether to move forward into a clinical trial requires compelling clinical/biological validity. While test validation studies may support the clinical/biological validity of a new test, they do not demonstrate clinical utility.

OCR for page 79
86 EVOLUTION OF TRANSLATIONAL OMICS Determination of clinical utility requires generation of high-quality evidence that supports the intended use of the test. In this regard, once the test performance characteristics are fully defined and the test is validated in a clinical laboratory, the investigator then needs to determine whether the test can and should be used to direct management of patient care. As illustrated in Figure 4-1, there are three pathways to generate such evidence: • Prospective–retrospective studies using archived specimens from previously conducted clinical trials that address the intended use of the omics-based test, or • Prospective clinical trials that directly address the utility of the omics-based test, where either — The test does not direct patient management, or — The test does direct patient management. Each of these pathways will be discussed in the next sections of the chapter, and examples of trial designs are summarized in Table 4-1. The trial designs discussed in these sections are examples of different approaches investiga- tors might take to “test the test”; different trial designs address different questions, so it is important to understand whether the selected trial design will provide the information needed to assess a given test for its intended use. More recently, adaptive trial designs, which have been used in thera- peutic clinical trials (IOM, 2010), also have been suggested for trials that address the clinical utility of a biomarker test. Before a decision has been made to initiate a clinical study or clinical trial to assess the clinical utility of a new omics-based test, the test should be fully defined and validated as described in Chapter 3, and should not change during the clinical study or clinical trial. The committee recom- mends that omics-based tests should not be changed during a clinical trial without a protocol amendment and discussion with FDA. A substantive change to the omics-based test may require that the study be restarted (Recommendation 3b). Both the Common Rule3 and FDA regulations4 require investigators to notify the IRB when a change to a trial protocol is made, and patient recruitment is paused until a decision can be made about whether the changes pose minimal risks to patients. FDA should also be notified if the test is modified. A modest change, such as simplification of the assay method, that does not alter the performance or results of the test would likely not affect the outcome of the trial, but if the change could alter the accuracy or repeatability of the test results, the trial results may not be reliable. 3 45 CFR part 46 §46.103(4) (2009). 4 21 CFR part 56 §56.108(a) (2011).

OCR for page 79
87 EVALUATION OF OMICS-BASED TESTS Prospective–Retrospective Studies Ideally, determination of clinical utility is derived from a prospective clinical trial. However, as stated by Simon et al., “In the case of tumor markers, practice guidelines and the availability of other diagnostic pro- cedures can sometimes make it very difficult to perform new clinical trials because such trials may involve withholding of therapy that is considered standard of care. Even when they are considered ethical, such trials usu- ally require many years to conduct and are quite expensive” (Simon et al., 2009, p. 6). As with any randomized clinical trial, for randomization to be ethically acceptable, there must be equipoise between the two arms, as determined by appropriate review committees (IRB and scientific peer review). As noted by Freidlin et al. (2010), monitoring such trials can also be quite complex because there may be multiple potentially overlapping patient subgroups and/or multiple hypotheses under consideration. “To protect patient interests, it may be necessary to stop the trial (or some of its components) before all of the study objectives are definitively addressed. Conventional monitoring rules that are based on the observed treatment effect in the overall randomized population may often not be sensitive enough for timely stopping based on biomarker subgroup–specific trends in treatment effect” (Freidlin et al., 2010, p. 157). Nonetheless, such trials have been and are being conducted to test new genomic assays (such as the TailoRX, MINDACT, and RxPONDER trials in breast cancer). However, given the difficulty of conducting prospective trials, it has been proposed that high-quality5 evidence to assess the clinical utility of a new omics-based test may be obtained by conducting prospective– retrospective studies using archived specimens from previously conducted prospective clinical trials or cohort studies that addressed the intended clinical use of the test (Pepe et al., 2008; Simon et al., 2009). However, even in this case, the investigators should have a prospective written pro- tocol describing their objectives, methods, and analytical plan, and the test should be “locked down”—fully defined, validated, and not changed during the study as described in Chapter 3. The requirements to achieve high-quality evidence to assess the clini- cal utility of a new test using the prospective–retrospective pathway illus- trated in the far left panel of Figure 4-1 are described elsewhere (Simon et al., 2009). Simon and colleagues explain the use of the prospective– retrospective study: 5 Simon et al. (2009) define levels of evidence to assess tumor biomarkers. However, the approach to systematic reviews and guidelines development currently is moving away from “levels of evidence” rating systems and toward a more comprehensive approach to synthesiz- ing the quality of a body of evidence (IOM, 2011).

OCR for page 79
88 EVOLUTION OF TRANSLATIONAL OMICS TABLE 4-1 Examples of Clinical Study Designs for Test Assessment Type of Trial Questions Study Design Design Description Can Answer Retrospective The test is studied using archived Generally retrospective studies studies specimens that were collected in should be viewed as exploratory, the past, have been stored, and and at best they can provide happen to be available. preliminary evidence of clinical validity by showing associations between the test result and patient outcome. Prospective– • ses archived specimens from U • hat is the best treatment in W retrospective a previously conducted clinical each test-defined subgroup? studies trial with treatment(s) that are • hat is the best treatment in the W relevant to the intended overall study population? clinical use of the test. • s test-directed treatment better I • his design requires a T than standard of care in the prospective written protocol overall study population? describing study objectives, (indirect assessment only) methods, and analytic plan, • s the test prognostic? I and the test must be locked • s the test a treatment effect I down and analytically modifier? validated. (Note that the ability to answer the above questions depends on whether the required treatment arms are represented and statistical power is sufficient.)

OCR for page 79
89 EVALUATION OF OMICS-BASED TESTS Advantages Limitations Availability of specimens and • ot collected as part of a clinical trial, so treatment N varying types of data, potentially is not randomized and choices could have been including clinical and pathologic confounded with markers underlying the test and variables and clinical outcome. other factors related to outcome. • atients may not be representative of the intended P use population for the test. • tandard of care may have evolved over time. S • rchived specimens may not be available, might A not have been collected, processed, or stored in the manner required, or might have degraded over time, rendering the test results unreliable. • linical and pathologic data may not reflect current C standard definitions and may have missing values and inaccuracies. • tudy design may have S • rchived specimens may not be available, might A evidentiary value close to a not have been collected, processed, or stored in the prospective study under certain manner required, or might have degraded over conditions (Simon et al., 2009). time, rendering the test results unreliable. • ess resource- and time- L • tandard of care may have evolved from the time S intensive than a prospective of the clinical trial. clinical trial because specimens • t least two prospective–retrospective studies A have already been collected and showing consistent promising results are required. follow-up for observation of • rospective clinical studies may still be needed to P clinical endpoint has already fully assess clinical utility. occurred. • tatistical inference concerns relating to multiple S • possible study design if a A testing, because the primary analysis of the original prospective trial is not feasible trial likely did not pre-specify a plan to study for ethical or other reasons. treatment effect in the test-defined subgroups. continued

OCR for page 79
94 EVOLUTION OF TRANSLATIONAL OMICS TABLE 4-1 Continued Type of Trial Questions Study Design Design Description Can Answer Test-guided • atients are randomly assigned P • s test-directed treatment better I strategy versus to the test-guided arm or the than standard of care in the non-guided non-guided arm. overall study population? (direct strategy with • atients randomized to the P assessment) randomization test-guided arm are directed to • s test-directed treatment better I (Figure 4-5) therapy as dictated by the test than the new therapy in the (test-positive to new therapy, overall study population? (direct test-negative to standard of assessment) care). • hat is the best treatment in W • n alternate rule must be A each test-defined subgroup? pre-specified to handle cases (Version 1 only) for which a test result is • s the test prognostic? I unavailable for a patient (Version 1 only) assigned to the test-guided arm. • s the test a treatment effect I • atients randomized to the P modifier? (Version 1 only) non-guided arm undergo a second randomization to receive either new therapy or standard of care. — Version 1: The test is applied to all patients (preferably at the same time in both arms). — Version 2: The test is applied only to patients who are randomized to the test-guided arm. SOURCES: Freidlin et al., 2010; McShane, 2011; Sargent et al., 2005; Simon et al., 2009. Adapted from Freidlin et al., 2010. Many biomarker studies are conducted with convenience samples of speci- mens, which just happen to be available and are assayed for the marker, with no prospectively determined subject eligibility, power calculations, marker cut-point specification, or analytical plans. Such studies are very likely to result in highly biased conclusions and truly deserve to be pejora- tively labeled as retrospective. However, if a retrospective study is designed to use archived specimens from a previously conducted prospective trial, and if certain conditions are prospectively delineated in a written protocol before the marker study is performed, we argue that it might be considered a prospective–retrospective study. Such a study should carry considerably more weight toward determination of clinical utility of the marker than a simple study of convenience, in which specimens and an assay happen to be available. Having multiple studies of different candidate biomarkers

OCR for page 79
95 EVALUATION OF OMICS-BASED TESTS Advantages Limitations • equired number of tests to be R • nefficient, in terms of number of patients required, I performed using Version 2 will relative to the test-stratified design. be half that required for • ersion 2 has the potential to produce biased results. V Version 1 (assuming equal If the non-availability of a test result has prognostic randomization), so it may be importance or is related to outcome on the new useful in settings of limited therapy, then the rule specified for how to treat resources. patients on the test-guided arm who do not have a • an be used to evaluate C test result will determine the nature of the bias. complex test-directed treatment strategies using a large number of treatment options or test categories. • equired number of tests to be R performed using Version 2 might be fewer than the completely randomized design or test-stratified design, depending on total sample size used in each design. However, not all questions can be addressed with Version 2 of this design. based on archived tissues from the same prospective trial would, how- ever, present a greater opportunity for false-positive conclusions than a single fully prospective trial focused on a specific biomarker. Consequently, independent confirmation of findings for specific biomarkers in multiple prospective–retrospective studies is important. (Simon et al., 2009, p. 3) If appropriate archived specimens from previously conducted clinical trials are not available, then investigators need to assess the clinical utility of an omics-based test in prospective clinical trials, as illustrated in the right side of Figure 4-1. Full descriptions of tumor biomarker trial designs have been published previously (Freidlin et al., 2010; Sargent et al., 2005; Simon, 2010) and are applicable to the evaluation of a new omics-based test. Prospective

OCR for page 79
96 EVOLUTION OF TRANSLATIONAL OMICS New therapeu c strategy a. All pa ents R Control New therapeu c strategy Marker + R Control b. All pa ents Omics-based test New therapeu c strategy Marker - R Control FIGURE 4-2 Two clinical trial designs in which the test is not used to direct therapy, but the primary objective of the clinical trial is to assess the clinical useful- ness of the test, given its intended use. (a) Completely randomized design, where the omics-based test is used on all patients, but where the results are not used for randomization. (b) Test-stratified design, where the omics-based test is used preran- domization, and patients are stratified based on the results and then randomized to a new treatment strategy or standard of care. NOTE: R = randomization. SOURCE: Adapted from McShane (2011). clinical trials in which an omics-based test either is or is not used for patient management decisions in the clinical trial are discussed in the next sections. Prospective Clinical Trials Where the Omics-Based Test Is Not Used for Patient Management To establish clinical utility, the study must be properly designed and powered to address the intended clinical use of the omics-based test. One approach is to perform clinical trials in which the test is not used to direct therapy, but the primary objective of the clinical trial is to assess the clinical usefulness of the test for its intended use. Two designs can be used (Figure 4-2): • Completely randomized design; or • Test stratified design (randomized marker-block design). In the completely randomized design, the test result is not used in the randomization nor is patient accrual stratified according to the test results. In this case, test results can be generated in real-time or at the completion of the trial accrual. However, for such a trial to adequately assess clinical util- ity, investigators should follow the committee’s recommendations regarding test development, analytical validation, and clinical/biological validation,

OCR for page 79
97 EVALUATION OF OMICS-BASED TESTS and ensure design and conduct of a properly powered study to address the specific intended use of the test. In contrast, in the test stratified design, the test result needs to be avail- able at the time of screening patients for accrual, and the result is used to stratify the randomization of patients to arms of the trial. Of these two designs, the approach that stratifies the randomization would be preferred when the test result is strongly prognostic and there would be interest in reducing the risk of confounding effects in the overall analysis that does not adjust for the test result. However, stratification of the randomization is not necessary in settings where the numbers of test- positive and test-negative patients is large because in this situation, the randomization should be sufficient to achieve approximate balance. In such a design, the test can be evaluated as a prognostic factor in the arm that does not receive the experimental therapy, and as an effect modifier when comparing outcomes in the arm that did receive the experimental therapy versus the standard of care (control). Treatment effect modification is assessed by the evidence that the mag- nitude of treatment effect is dependent on the results of the omics-based test. More precisely, the clinical utility of treatment effect modifier bio- markers is established by evidence that an intervention provides a clinically meaningful improvement in the benefit-to-risk profile of an intervention only in certain subgroups identified by the test result. While such analy- ses need to be prespecified in the statistical analysis plan, stratifying the randomization has little impact on the power of such analyses. Hence, the two designs in Figure 4-2 are equally well suited to assess whether the biomarker is an effect modifier of treatment. Prospective Clinical Trials Where the Omics-Based Test Is Used for Patient Management The design that most directly assesses the clinical utility of a new omics- based test is a prospective clinical trial in which the omics-based test under study is used to guide patient management decisions. However, this design also poses the highest potential risk to patients because of the possibility that the test could be wrong. Use of a new test to direct patient management in a clinical trial requires the use of a fully defined and validated omics- based test performed in a CLIA-certified clinical laboratory (see Chapter 3). Many prospective clinical trial designs have been proposed to evalu- ate potential treatment effect modifiers to guide the selection of patients to receive therapeutic interventions. In some instances, these effect modifier tests are studied in parallel with the development of a new therapeutic, an approach designated “co-development” by FDA (Woosley and Cossman, 2007). In such settings, the version of the treatment effect modifier test that

OCR for page 79
98 EVOLUTION OF TRANSLATIONAL OMICS would be used in clinical practice ideally should be assessed in the same clinical trial designed to evaluate the new therapeutic. Draft guidance on the development of a therapeutic product that depends on an in vitro “com- panion” diagnostic device was recently distributed by FDA (FDA, 2011b). Several designs for evaluating treatment effect modifier tests have been proposed (Freidlin et al., 2010; Sargent et al., 2005). The optimal design depends on whether the test will be used to guide decisions about treat- ment selection for patients in the clinical trial (see Freidlin et al., 2010; specific advantages and disadvantages of these study designs discussed are provided in Table 1 of this reference). Because a control group is needed to determine whether a test is a prognostic factor and whether it is a treat- ment effect modifier, designs of randomized trials rather than single-arm trials are discussed.6 Enrichment design In some cases, investigators might assume that the utility of one of the test-designated categories is established, or likely to be, while the others are uncertain and require prospective evaluation in a clinical trial. For example, investigators might determine that the clinical utility of the prognostic role of a biomarker test is established, but that the treatment effect modifier role for subsequent therapy in those patients who fall into the “poor” prognostic group still is investigational. In this case, patient groups designated as having a very favorable prognosis should not receive treatment, and therefore they are not eligible for a subsequent trial that addresses the benefit in those patients who are presumed to have a worse prognosis. The case study of Oncotype DX, and the TailoRX study designed to assess its clinical utility, illustrates such a case (see Appendix A). Likewise, investigators might conclude from prior preclinical or clinical studies that the negative predictive value of an effect modifier is established, but that the benefit of a targeted therapeutic agent still is investigational in those patients who have positive test results. In this case, patients who fall into a “negative” effect modifier subgroup might be considered to be so unlikely to respond that they are not eligible for a subsequent trial that addresses the utility of the investigational agent, and only patients with positive test results are enrolled. An enrichment trial design (Freidlin et al., 2010; Sargent et al., 2005) is illustrated in Figure 4-3. With this approach, the only patients entered into 6 Note that single-arm trials are sometimes used for FDA decisions. For example, in the case of codevelopment of crizotinib with ALK FISH, the drug received accelerated approval on the basis of results from two single arm trials (FDA, 2011a). While a Phase III study is under way, FDA approved the drug because it concluded that the clinical data were reasonably likely to predict a clinical benefit to patients. The objective response rate was 50-60 percent, with median response duration of 42-48 weeks. Approximately 3-5 percent of lung adeno- carcinomas contain ALK gene rearrangements.

OCR for page 79
99 EVALUATION OF OMICS-BASED TESTS New therapeutic strategy Test + R Standard of Care All patients Omics-based test Test - OFF study Or followed prospectively in a registry FIGURE 4-3 Enrichment design, where only test-positive patients are randomized and treated. Figure 4-3 New, replacewment 3/13 NOTE: R = randomization. Landscape SOURCE: Adapted from McShane (2011). the clinical trial are those who test positive for the biomarker at screening. If there is reliable evidence that the new drug would not be effective in patients with negative test results, then this approach would be both effi- cient and ethical by limiting the assessment of efficacy of the new drug to the target population with an anticipated favorable benefit-to-risk profile. Investigators should be aware that if there are uncertainties about whether the new drug might be effective in patients with negative test results, then the enrichment design is deficient because it provides no insight to address such uncertainties. Patients with negative test results could benefit from the new drug if the test does not reliably identify the target population, for example, by having the wrong threshold for defined positivity, or if the new drug has important mechanisms of action, not cap- tured by the test, that result in favorable efficacy in patients with both posi- tive and negative results. The development of trastuzumab (Herceptin), a humanized monoclonal antibody against HER2, for breast cancer provides such an example (see case study in Appendix A). Preclinical data strongly suggested that trastuzumab would only be effective against cancers that greatly overexpress HER2, and nearly all subsequent clinical trials have only enrolled patients with very high levels of HER2 protein and/or with HER2 gene amplification (Wolff et al., 2007). However, post-hoc analysis of HER2 test results in a central laboratory for patients entered into two of the largest prospective randomized trials of adjuvant trastuzumab identi- fied a small group of women whose breast cancers appear not to be HER2 positive by classic criteria, yet the hazard ratio for recurrence is equally favorable in these patients as it is for women with highly overexpressed or amplified HER2 (Paik et al., 2008; Perez et al., 2010). These results are the basis of an ongoing clinical trial testing the benefits of trastuzumab in

OCR for page 79
100 EVOLUTION OF TRANSLATIONAL OMICS patients who have lesser, but still detectable, HER2 expression (NSABP clinical trial B47). Test-guided strategy with control designs Frequently, investigators cannot assume that certain test-designated categories are established, and in these cases an enrichment design is not appropriate. Rather, a trial is designed to “test the test,” using appropriate control arms. There are at least two such trial designs. Figure 4-4 presents the test-guided treatment strategy where patients are randomized between test-guided use of a new drug versus assignment to a standard of care control regimen without regard to the test. Figure 4-5 presents an alternative test-guided treatment strategy in which patients are randomized to an arm in which the test is used to guide selection of an investigational or standard of care management strategy (upper arm of Figure 4-5) versus an arm in which patients are randomly assigned to investigational therapy versus control standard of care management without knowledge of the test results. For example, this was the design used in the prospective Duke trial of preoperative chemo- therapy for breast cancer included in the committee’s statement of task (NCT00636441; see Appendix B). In contrast to the design in Figure 4-4, this design is less efficient if the control regimen were to remain as the standard of care, but would be useful if the new drug could emerge as the standard of care regimen for all subjects in this clinical setting. However, Freidlin and colleagues argue that both of these test-guided treatment strat- egies are inherently less efficient than either of the designs in Figure 4-2 (Freidlin et al., 2010), which estimate the treatment effect in all relevant biomarker populations. New therapeutic strategy Test + Omics-based test determines treatment Test - Standard of Care All patients R Standard of Care FIGURE 4-4 Example of a test-guided strategy versus standard of care, where patients are randomly assigned to the test-guided arm or the non-guided arm. NOTE: R = randomization. SOURCE: Adapted from McShane (2011). Figure 4-4 new

OCR for page 79
101 EVALUATION OF OMICS-BASED TESTS New therapeutic strategy Test + Omics-based test determines treatment Test - Standard of Care All patients R New therapeutic strategy Randomize treatment R blinded to omics-based test result Standard of Care FIGURE 4-5 Example of a test-guided strategy versus non-guided strategy with ran- domized treatment design. Patients are randomly assigned to use of the omics-based test to determine treatment versus randomized treatment selection (i.e., patients are Figure 4-5 new tested but treatment is randomly assigned without consideration of the test result). NOTE: R = randomization. SOURCE: Adapted from McShane (2011). SUMMARY AND RECOMMENDATION After analytical and clinical/biological validation of the candidate omics-based test as described in Chapter 3, the test is ready for use in a clinical trial to assess clinical utility and use of the test in patient care. Figure 4-1 shows several options for clinical trial designs to assess clinical utility. Depending on the choice of trial design, investigators may need to obtain an IDE from FDA. Regardless of choice, however, the committee strongly recommends consulting with FDA prior to initiation of clinical trials. For a trial in which patient management will be influenced by the omics-based test findings, obtaining an IDE from FDA is a legal require- ment. In other cases, the committee recommends consultation with FDA because the requirement for an IDE based on the trial design is not always clear. In addition, if the test will later need clearance or approval from FDA before marketing for clinical use, the study design and analysis will be subject to FDA review, and a pre-IDE consultation can assist both the test developer and FDA in coming to an agreement on the data necessary for FDA clearance or approval. Critical considerations for moving a candidate omics-based test into clinical trials for assessing clinical utility are outlined in Figure 4-1 and Recommendation 3, below. RECOMMENDATION 3: Evaluation for clinical utility and use stage. For investigators conducting a clinical trial to assess the clinical utility

OCR for page 79
102 EVOLUTION OF TRANSLATIONAL OMICS and use of an omics-based test that has been confirmed and validated as described in Chapters 2 and 3, the committee recommends that a. Investigators should communicate early with FDA regarding the investigational device exemption (IDE) process and validation requirements. b. Omics-based tests should not be changed during a clinical trial without a protocol amendment and discussion with FDA. A sub- stantive change to the omics-based test may require that the study be restarted. REFERENCES AHRQ (Agency for Healthcare Research and Quality). 2011. Technology Assessments. http:// www.ahrq.gov/clinic/techix.htm (accessed October 20, 2011). Allegra, C. J., J. M. Jessup, M. R. Somerfield, S. R. Hamilton, E. H. Hammond, D. F. Hayes, P. K. McAllister, R. F. Morton, and R. L. Schilsky. 2009. American Society of Clinical Oncology provisional clinical opinion: Testing for KRAS gene mutations in patients with metastatic colorectal carcinoma to predict response to anti-epidermal growth factor receptor monoclonal antibody therapy. Journal of Clinical Oncology 27(12):2091-2096. Altman, D. G., and G. H. Lyman. 1998. Methodological challenges in the evaluation of prog- nostic factors in breast cancer. Breast Cancer Research and Treatment 52(1-3):289-303. Altman, D. G., and R. D. Riley. 2005. Primer: An evidence-based approach to prognostic markers. Nature Clinical Practice Oncology 2(9):466-472. Altman, D. G., and P. Royston. 2000. What do we mean by validating a prognostic model? Statistics in Medicine 19(4):453-473. FDA (Food and Drug Administration). 2004. Challenge and Opportunity on the Critical Path to New Medical Products. http://www.fda.gov/downloads/ScienceResearch/Special Topics/CriticalPathInitiative/CriticalPathOpportunitiesReports/ucm113411.pdf. FDA. 2005. Guidance for Industry Pharmacogenomic Data Submissions. http://www.fda.gov/ downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM079849. pdf. For updates, see also http://www.fda.gov/scienceresearch/bioinformaticstools/ arraytrack/ucm089695.htm. FDA, 2006. Critical Path Opportunities List. http://www.fda.gov/downloads/ScienceResearch/ SpecialTopics/CriticalPathInitiative/CriticalPathOpportunitiesReports/UCM077258.pdf. FDA. 2011a. News release: FDA approves Xalkori with companion diagnostic for a type of late-stage lung cancer. http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ ucm269856.htm. FDA. 2011b. D raft Guidance for Industry and Food and Drug Administration Staff— In Vitro Companion Diagnostic Devices. h ttp://www.fda.gov/MedicalDevices/ D eviceRegulationandGuidance/GuidanceDocuments/ucm262292.htm. Freidlin, B., L. M. McShane, and E. L. Korn. 2010. Randomized clinical trials with bio- markers: Design issues. Journal of the National Cancer Institute 102(3):152-160. Gutierrez, A. 2011. Discussion with the IOM Committee on the Review of Omics-Based Tests for Predicting Patient Outcomes in Clinical Trials, Washington, DC, August 19. Harris, L., H. Fritsche, R. Mennel, L. Norton, P. Ravdin, S. Taube, M. R. Somerfield, D. F. Hayes, and R. C. Bast, Jr. 2007. American Society of Clinical Oncology 2007 update of recommendations for the use of tumor markers in breast cancer. Journal of Clinical Oncology 25(33):5287-5312.

OCR for page 79
103 EVALUATION OF OMICS-BASED TESTS Hayes, D. F., R. C. Bast, C. E. Desch, H. Fritsche Jr, N. E. Kemeny, J. M. Jessup, G. Y. Locker, J. S. Macdonald, R. G. Mennel, L. Norton, P. Ravdin, S. Taube, and R. J. Winn. 1996. Tumor marker utility grading system: A framework to evaluate clinical utility of tumor markers. Journal of the National Cancer Institute 88(20):1456-1466. Henry, N. L., and D. F. Hayes. 2006. Uses and abuses of tumor markers in the diagnosis, moni- toring, and treatment of primary and metastatic breast cancer. Oncologist 11:541-552. IOM (Institute of Medicine). 2010. A National Cancer Clinical Trials System for the 21st Century: Reinvigorating the NCI Cooperative Group Program. Washington, DC: The National Academies Press. IOM. 2011. Finding What Works in Health Care: Standards for Systematic Reviews. Wash- ington, DC: The National Academies Press. Locker, G. Y., S. Hamilton, J. Harris, J. M. Jessup, N. Kemeny, J. S. Macdonald, M. R. Somerfield, D. F. Hayes, and R. C. Bast, Jr. 2006. ASCO 2006 update of recommen- dations for the use of tumor markers in gastrointestinal cancer. Journal of Clinical Oncology 24(33):5313-5327. McShane, L. 2011. Challenges in the Development and Validation of Biomarker-Based Tests for Personalized Therapeutic Decision Making in Oncology. Presented at Accelerating Anticancer Agent Development and Validation Workshop, Bethesda, MD. May 19. NCCN (National Comprehensive Cancer Network). 2010. NCCN Clinical Practice Guidelines in Oncology. http://www.nccn.org. Orr, M. S., F. Goodsaid, S. Amur, A. Rudman, and F. W. Frueh. The experience with voluntary genomic data submissions at the FDA and a vision for the future of the voluntary submis- sion program. 2007. Clinical Pharmacology & Therapeutics 81(2):294-297. Paik, S., C. Kim, and N. Wolmark. 2008. HER2 status and benefit from adjuvant trastuzumab in breast cancer. New England Journal of Medicine 358(13):1409-1411. Pepe, M. S., Ziding F., Janes H., Bossuyt P. M., and Potter J. D. 2008. Pivotal evaluation of the accuracy of a biomarker used for classification or prediction: Standards for study design. Journal of the National Cancer Institute 100(20):1432-1438. Perez, E. A., M. M. Reinholz, D. W. Hillman, K. S. Tenner, M. J. Schroeder, N. E. Davidson, S. Martino, G. W. Sledge, L. N. Harris, J. R. Gralow, A. C. Dueck, R. P. Ketterling, J. N. Ingle, W. L. Lingle, P. A. Kaufman, D. W. Visscher, and R. B. Jenkins. 2010. HER2 and chromosome 17 effect on patient outcome in the N9831 adjuvant trastuzumab trial. Journal of Clinical Oncology 28(28):4307-4315. SACGHS (Secretary’s Advisory Committee on Genetics, Health, and Society). 2008. U.S. System of Oversight of Genetic Testing: A Response to the Charge of the Secretary of Health and Human Services; A Report of the Secretary’s Advisory Committee on Genetics, Health, and Society. http://oba.od.nih.gov/oba/SACGHS/reports/SACGHS_oversight_report.pdf. Sargent, D. J., B. A. Conley, C. Allegra, and L. Collette. 2005. Clinical trial designs for predictive marker validation in cancer treatment trials. Journal of Clinical Oncology 23(9):2020-2027. Simon, R. 2005a. Development and validation of therapeutically relevant multi-gene bio- marker classifiers. Journal of the National Cancer Institute 97(12):866-867. Simon, R. 2005b. Roadmap for developing and validating therapeutically relevant genomic classifiers. Journal of Clinical Oncology 23(29):7332-7341. Simon, R. 2010. Clinical trial designs for evaluating the medical utility of prognostic and predictive biomarkers in oncology. Personalized Medicine 7(1):33-47. Simon, R., and D. G. Altman. 1994. Statistical aspects of prognostic factor studies in oncology. British Journal of Cancer 69(6):979-985. Simon, R. M., S. Paik, and D. F. Hayes. 2009. Use of archived specimens in evaluation of prognostic and predictive biomarkers. Journal of the National Cancer Institute 101(21):1446-1452.

OCR for page 79
104 EVOLUTION OF TRANSLATIONAL OMICS TEC (Technology Evaluation Center). 2011. Technology Evaluation Center Assessment Pro- cess. http://www.bcbs.com/blueresources/tec/tec-assessment-process.html (accessed Oc- tober 20, 2011). Teutsch, S. M., L. A. Bradley, G. E. Palomaki, J. E. Haddow, M. Piper, N. Calonge, W. D. Dotson, M. P. Douglas, and A. O. Berg. 2009. The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Initiative: Methods of the EGAPP Working Group. Genetics in Medicine 11(1):3-14. USPSTF (U.S. Preventive Services Task Force). 2009. Screening for breast cancer: U.S. Pre- ventive Services Task Force recommendation statement. Annuals of Internal Medicine. 151(10):716-726, W-236. USPSTF. 2011. Screening for Prostate Cancer: U.S. Preventive Services Task Force Draft Recommendation Statement. http://www.uspreventiveservicestaskforce.org/draftrec3.htm. (accessed December 5, 2011). Wolff, A. C., M. E. Hammond, J. N. Schwartz, K. L. Hagerty, D. C. Allred, R. J. Cote, M. Dowsett, P. L. Fitzgibbons, W. M. Hanna, A. Langer, L. M. McShane, S. Paik, M. D. Pegram, E. A. Perez, M. F. Press, A. Rhodes, C. Sturgeon, S. E. Taube, R. Tubbs, G. H. Vance, M. van de Vijver, T. M. Wheeler, and D. F. Hayes. 2007. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. Journal of Clinical Oncology 25(1):118-145. Woosley, R. L., and J. Cossman. 2007. Drug development and the FDA’s Critical Path Initia- tive. Clinical Pharmacology & Therapeutics 81(1):129-133.