Author + information
- Douglas L. Mann, MD, FACC∗ ()
- Center for Cardiovascular Research, Department of Medicine, Washington University School of Medicine, St. Louis, Missouri
- ↵∗Address for correspondence:
Dr. Douglas L. Mann, Editor-in-Chief JACC: Basic to Translational Science, American College of Cardiology, Heart House, 2400 North Street NW, Washington, DC 20037.
Translational research serves as the critical bridge between target identification and early-phase clinical studies that provide critical go/no-go decisions for larger and more expensive phase III clinical trials. Insofar as translational clinical studies are smaller, they are inherently fragile by their very nature. In contrast to the later phases of clinical development, which are focused on safety and efficacy, understanding the therapeutic mechanism of action is far more important during the early stages of drug and device development, because the mechanism of action guides the appropriate choice of clinical endpoints that provide “proof of principle.” Given that these early-phase studies are underpowered to study “hard” clinical endpoints, such as cardiovascular death or hospitalization, they often employ a variety of “softer” surrogate endpoints, such as changes in biomarkers or changes in left ventricular function, which may or may not translate into clinically meaningful endpoints in subsequent phase III trials. Because phase Ib and II clinical trials lack formal guidelines for defining what a successful clinical outcome looks like, or how it should be measured, there is a frequent temptation to cast a wide net and measure a myriad of secondary endpoints. Not infrequently, when the chosen primary endpoints of the early-phase studies fail to yield the anticipated statistical results, secondary endpoints are used as a window to understand alternative mechanisms of action. However, when investigators employ secondary endpoints to explain mechanistic inconsistencies in early-phase translational studies, they run the risk of engaging in the scientific equivalent of an ancient literary device known as “deus ex machina.”
Deus ex machina is a Latin phrase that describes an ancient plot device used by Greek and Roman tragedy playwrights to help resolve seemingly hopeless plot situations. The phrase can be loosely translated as “god from the machine,” which refers to how the deus ex machina was performed in the ancient theater. To help resolve an unresolvable plot scenario, playwrights would have an actor playing a god or goddess lowered onto the stage by a crane-like device referred to as a “mechane.” In contemporary literature, the phrase “deus ex machina” is used to describe any situation where something unexpected or implausible is brought into the storyline to resolve difficult situations or disentangle a plot. The use of this phrase has also been expanded to include any plot resolutions that are not drawn directly or logically from the preceding plot and/or require the suspension of disbelief. Thus, the more modern usage of deus ex machina allows authors to conclude their stories with improbable, but nonetheless acceptable conclusions.
When translational investigators use encouraging signals in clinical studies to help refine/redefine the mechanism of action of a novel therapeutic as a work around to explain the inability to meet the primary endpoint of early-phase clinical trials, they are employing a similar plot twist used by ancient Greek and Roman writers, who lowered a god/goddess onto the stage to resolve a complicated plot situation. Even if extremely well-intentioned, translational mechanistic plot twists are unlikely to replicate in larger phase III trials. One major reason is that introducing a novel mechanism or mechanisms of action based on a post-hoc analysis of secondary endpoints runs the risk of inadvertently introducing bias into the experimental paradigm. John Ioannidis, an epidemiologist at Stanford University, defined bias as the “combination of various design, data, analysis, and presentation factors that tend to produce research findings when they should not be produced” (1). Bias differs importantly from the statistical “play of chance” that results in some findings being false by chance even though the study design, data, analysis, and presentation are perfect. The second reason that retrofitting the mechanism of action to secondary endpoints is fraught with peril relates to the seductive nature of “statistical significance,” which is often used to identify relevant clinical endpoints and/or generate secondary hypothesis regarding alternative mechanisms of action. When p values were introduced in the 1920s, they were developed to summarize the probability of a given observation being observed by random chance; p values were never intended to “work backward” and provide novel information regarding biological mechanism of action. Rather, the probability of a finding being true depends on: 1) the prior probability of the finding being true (i.e., Bayes rule); 2) the statistical power of the study; and 3) the level of statistical significance (1).
How Can Translational Investigators Avoid Falling Into the Translational Deus Ex Machina Trap?
There is likely not a “1 size fits all” answer to this question. Nonetheless, based on the author’s observations over an extended period of years, there are several self-evident truths that are worth discussing. First, having a clear well-defined specified mechanism of action based on preclinical studies prior to engaging in phase I/II clinical trials is important. Although it is certainly acceptable to modify the proposed mechanism(s) of action based on new information gleaned from both ongoing clinical and preclinical studies, the mechanism of action should be clearly understood before engaging in subsequent clinical trials. The difficulties surrounding the development of therapeutics that target high-density lipoprotein and the use of stem cells to treat patients with heart failure are 2 contemporary examples that come to mind that underscore the importance of understanding mechanism(s) of action before launching into clinical trials. Second, as noted above, unlike phase III trials wherein there are formal rules for how endpoints should be analyzed, there are no guiding principles for how the results of phase I/II clinical trial results should be analyzed. Given the small numbers of patients in early-phase clinical trials, it would be counterproductive to propose rigid data analysis plans for these early-phase clinical studies. Nonetheless, 1 suggestion to help safeguard against overanalyzing clinical translational data is to invoke Bayes rule, which posits that the post-study likelihood that the results of clinical trials will be replicated in subsequent trials depends on the pre-test likelihood that the new therapeutic directly affects the disease-causing pathway. The more implausible the finding, the greater the likelihood that the finding will not replicate in subsequent and/or larger clinical trials. This highlights, again, the importance of interpreting the results of translational clinical trials in light of pre-specified mechanism(s) of action. A third suggestion would be to employ a similar degree of scientific rigor employed in phase III clinical trials (e.g., control groups, block randomization, and clinical blinding where appropriate and possible) to the conduct of translational clinical trials. As noted by Ioannidis, “The greater the flexibility in designs, definitions outcomes, and analytical modes….the less likely the research findings are to be true. Flexibility increases the potential for transforming what would be negative results into positive results” (1). Although it is certainly the case that not all phase I/II clinical trials readily lend themselves to randomization and/or blinding, we as a cardiovascular scientific community need to remain circumspect, and to openly discuss the provisional nature of translational studies when they are published in scientific journals. To this end, the editors of JACC: Basic to Translational Science remain committed to discussing the strengths and weaknesses of new translational discoveries in an open and transparent manner, through the use of a rigorous review process, careful editing, and balanced editorials. As always, we welcome your thoughts, and would like to hear whether you think that mechanism matters in translational studies, either through social media (#JACCBTS) or by e-mail ( ).
- 2017 The Author