The magnitude of therapeutic effects plays an important role in US Food and Drug Administration (FDA) approval of treatments based on nonrandomized clinical trials (RCTs), according to a study published in JAMA Network Open.
RCTs are widely used to evaluate the safety and efficacy of new clinical interventions and medical devices. However, RCTs are often limited by logistical or feasibility issues. As a result, alternative methods of treatment evaluation are growing in popularity. The objective of this study was to determine how often the FDA authorizes new interventions based on non-RCTs, and whether the magnitude of therapeutic effects is associated with the requirement of additional RCTs by the FDA.
In this study, researchers downloaded 606 drug applications for treatments within the Breakthrough Therapy designation from the FDA website. Data were downloaded in January 2017 and August 2018, and included all drug applications since the inception of the Breakthrough Therapy designation in January 2012. Researchers also downloaded 71 medical device applications in August 2017 for the Humanitarian Device Exemption since its inception in June 1996. Only consecutive non-RCTs were included; exclusions were RCTs, studies with nonextractable data, and studies for which applications were denied.
Continue Reading
The following data were extracted independently by 2 researchers, using a standardized data extraction form: study design, primary outcome, disease characteristics, interventions, and comparators. Primary outcomes were disease, laboratory, disease response, and patient survival.
The final sample consisted of 68 interventions (36 drugs and 32 medical devices). Of these samples, 17 required further testing in RCTs and 51 did not. Overall, the FDA authorized 36 drugs based on results from non-RCTs. Researchers found that most approvals (n=32) were based on data from single groups. Of the 36 comparators, 5 were based on no active treatment and 31 were active treatments. Of the active treatments, 14 included reference to standard treatments. The most commonly used outcome was overall response rate.
Results revealed that the magnitude of estimated therapeutic effects was higher in the group that did not require RCTs compared with the group that did (drug-only group: mean natural log [ln] odds ratio [OR], 2.23 [OR, 9.34; standard deviation {SD}, 1.83] vs mean ln [OR] 1.12 [OR, 3.06; SD, 1.32]; P =.02; combined group: mean ln [OR], 2.18 [OR 8.82; SD 1.98] vs mean ln [OR] 1.12 [OR, 3.06; SD, 1.32]; P =.03). The magnitude of effect observed for medical devices was similar to the treatment effects noted for drugs that required no further testing in RCTs, and had a mean ln (OR) of 2.14 (SD, 2.09) with a mean OR of 8.49 (SD, 8.10). The meta-analysis also showed a statistically greater magnitude of estimated effects associated with the group that did not require RCTs than for the group that did (Breakthrough Therapy designation drugs: OR, 6.79 [95% CI, 3.90-11.80] vs OR, 2.46 [95% CI, 1.70-3.56]; P =.003; combined group: OR, 6.30 [95% CI 4.38-9.06] vs OR, 2.46 [95% CI, 1.70-3.56]; P <.001).
Researchers did not identify any clear threshold beyond which the FDA would not require additional RCTs. Overall, 9 of the 677 total applications approved on the basis of non-RCTs had relative risks of 10 or greater, and 12 had relative risks of 5 or greater.
Limitations to the study were as follows: first, the sample size was small. Second, there was inadequate availability of data on comparators, and as a result, some of the treatment effect estimates were approximations.
The study researchers concluded that the FDA is less likely to request additional RCTs when a treatment shows a greater magnitude of estimated treatment effects.
Disclosure: This clinical trial was supported by GlaxoSmithKline. Please see the original reference for a full list of authors’ disclosures.
Reference
Razavi M, Glasziou P, Klocksieben FA, Ioannidis JPA, Chalmers I, Djulbegovic B. US Food and Drug Administration approvals of drugs and devices based on nonrandomized clinical trials: a systematic review and meta-analysis. JAMA Netw Open. 2019;2(9):e1911111.
