Research Pearls
Editorial Commentary: The Power of Interpretation: Utilizing the P Value as a Spectrum, in Addition to Effect Size, Will Lead to Accurate Presentation of Results
Statistics have helped develop evidence-based medicine. Comparing groups and rejecting (or not) a null hypothesis is a main principle of the scientific method. Many studies have demonstrated that drawing conclusions based on the statistical result of a dichotomic P value instead of a spectrum can mislead us to conclude that there is “no difference” between two groups, or two treatments. In addition to the P value, the utilization of effect size (magnitude of difference between studied groups), may help us obtain a better global understanding of the statement “no effect”.Authors Dichotomize Medical Research Findings as Significant Versus Not Significant, Creating a False Sense of Certainty, and Report Outcomes on Patients Whose Results Have Been Previously Reported Without Proper Disclosure
Statistical significance dichotomizes research findings into significant versus not significant, creating a false sense of certainty. It is insufficient to mindlessly report results as significant versus not significant without providing a quantitative estimate of the uncertainty of the data. Authors could provide a confidence interval, draw a P value function graph, or run a Bayesian analysis. Authors could calculate and report a Surprise or S value. Most importantly, authors could thoughtfully consider how the uncertainty within their research data informs the results of their study.Understanding Network Meta-analysis (NMA) Conclusions Requires Scrutiny of Methods and Results: Introduction to NMA and the Geometry of Evidence
Synthesis of medical literature to determine the best treatment for a given problem is challenging, particularly when multiple options exist. Network meta-analysis (NMA) allows the comparison of different treatment approaches in a single, systematic review including treatments that have never been compared head-to-head. A key to understanding NMA is to focus on the network geometry showing the number of included studies and their relationships: different treatment options are illustrated as nodes.Nearly One-Third of Published Systematic Reviews and Meta-analyses Yield Inconclusive Conclusions: A Systematic Review
To perform a systematic review that determines the percentage of published orthopedic surgery and sports medicine systematic reviews and meta-analyses that have a conclusive conclusion.Misinterpretation of P Values and Statistical Power Creates a False Sense of Certainty: Statistical Significance, Lack of Significance, and the Uncertainty Challenge
Despite great advances in our understanding of statistics, a focus on statistical significance and P values, or lack of significance and power, persists. Unfortunately, this dichotomizes research findings comparing differences between groups or treatments as either significant or not significant. This creates a false and incorrect sense of certainty. Statistics provide us a measure of the degree of uncertainty or random error in our data. To improve the way in which we communicate and understand our results, we must include in reporting a probability, or estimate, of our degree of certainty (or uncertainty).Health Research Titles and Guidelines Improve Impact and Article Acceptance Rates
The peer-reviewed publication of Arthroscopy, Arthroscopy Techniques, and Arthroscopy, Sports Medicine, and Rehabilitation is the result of a team effort. Our assistant editors-in-chief are a notable part of this team who are specifically credited for, among other contributions, leadership in organizing, creating, and publishing a series of pearls, templates, and checklists for health research authors as well as reviewers, editors, and readers. These articles and tools are available under drop-down menus on the Arthroscopy journal home page.Pain Management Research
Over the past decade, increased awareness by the medical profession of the devastating consequences of opioid addiction has resulted in substantial efforts to limit the number of opioid prescriptions for both perioperative pain management and chronic pain. While these efforts have had some success, opioid misuse remains a crisis, which we in the orthopaedic community have a particular opportunity to address. It is the belief of the undersigned that progress depends on improved research methods and reporting to further the understanding of pain experience and response to management, with the end goal of identifying more effective, nonnarcotic pain control measures for our orthopaedic patients.Strategies to Standardize Opioid Research and Improve Clinical Pain Management
Opioid abuse results in poor pain control, poor outcomes, and addiction. Clinical recommendations to manage pain include identifying the problem, considering multimodal anesthesia, avoiding overprescribing, acknowledging that minimizing opioid use is not equivalent to undertreating pain, minimizing preoperative opioid use, managing patient expectations, and continuing to investigate outcomes of pain management while limiting opioid prescriptions or forgoing opioids altogether. Authors are directed to new Recommendations for Pain Management Research to highlight critical research parameters and standardize outcome reporting.The Patient-Reported Outcomes Measurement Information System (PROMIS): Can We Finally Compare Apples to Oranges?
Legacy patient-reported outcome mea`sures lack standardization, resulting in difficulty comparing the results of diverse clinical outcome studies: “You can’t compare apples to oranges.” To address this concern, the National Institutes of Health initiated the Patient-Reported Outcomes Measurement Information System (PROMIS) to assess common dimensions of a wide range of diseases. PROMIS uses computer adaptive testing: A fluid questionnaire chooses subsequent questions based on the responses to previous questions to efficiently characterize outcomes using only 4 to 6 questions.Do Some Patients Count More Than Others? Reporting Outcomes of the Same Patient in More Than One Study Requires Disclosure
Clinical databases allow researchers to test multiple hypotheses. This could result in including outcomes on the same patient or patients in more than 1 study. When this occurs, it is vital for authors to clarify multiple reporting in their study methods to avoid having patients counted more than once in future systematic reviews or meta-analyses. As a caveat, primary authors should consider whether publication of multiple studies is important or whether they are simply generating “least publishable units” (LPUs, also known as salami slicing).Editorial Commentary: Defining Improvement After Arthroscopic Meniscal Surgery—How Much of a Difference Does a Difference Make?
Patient-centered metrics including the minimal clinically important difference, substantial clinical benefit, and patient acceptable symptom state have been proposed to determine the clinical significance of patient-reported outcome scores. These values allow clinically meaningfully interpretation of changes in scores such that the degree of improvement (minimal clinically important difference and substantial clinical benefit) and satisfaction (patient acceptable symptom state) can be determined.Methods to Improve Arthroscopic and Orthopaedic Biomechanical Investigations: A Few of Our Favorite Things
Controlled biomechanical studies, generally in vitro (and often ex vivo), may represent a first step in evaluation of a new arthroscopic or orthopedic implant or technique. The purpose and methods of biomechanical studies must be thoughtfully considered to achieve results translatable to a clinically relevant conclusion. A limitation is that with the exception of animal studies or rare human investigations, most biomechanical studies actually only investigate mechanics and do not study biological healing.Editorial Commentary: Network Geometry of Nonoperative Management of Patellar Tendinopathy—Can the Shape of the Evidence Inform Practice?
Recent research has examined the comparative effectiveness of nonoperative treatments for patellar tendinopathy using a network meta-analysis method. This method allows analysis of a network of clinical trials individually studying different treatment options in comparison to an eccentric exercise control; however, most treatments have not been compared head to head. Although leukocyte-rich platelet-rich plasma is statistically ranked as the treatment with the highest improvements in pain and function, concerns over the assumption of transitivity (on which network meta-analysis is based) and the lack of connection or comparisons among treatments suggest that future studies comparing treatments head to head are needed.Our Measure of Medical Research Should Be Appreciable Benefit to the Patient
The clinical relevance of research is much more important than statistical significance. Patient-reported outcome measures (PROM) are the strongest determinants of satisfaction as the result of an intervention or treatment. Outcomes can be measured in terms of the minimal clinically important difference (MCID) detectable by a patient, bearing in mind that “detectable” includes worsening as well as improvement. Patient-acceptable symptomatic state (PASS) and substantial clinical benefit (SCB) ultimately correlate with whether patients are happy or willing to undergo an intervention again.Reviews Pooling Heterogeneous, Low-Evidence, High-Bias Data Result in Incorrect Conclusions: But Heterogeneity is an Opportunity to Explore
Systematic Review submissions to our journal commonly pool heterogeneous studies of low levels of evidence and a high risk of bias. Pooling, or quantitative synthesis, of such study data regularly results in incorrect conclusions. We reject these submissions without peer-review (desk rejection), and typically invite authors to submit a new, subjective synthesis without pooling and to report ranges of the results of included studies rather than pooled values. Generally, quantitative synthesis, or meta-analysis, should restrict included studies to randomized controlled trials.Tools to Improve Scientific Research
The mission of Arthroscopy is to provide authoritative, current, peer-reviewed clinical and basic science information regarding arthroscopic and related surgery. In addition, with a goal of improving the quality of the scientific research published in our journal and others, we develop and publish research pearls, statistical guides, article checklists and templates, and related tools. In sum, this effort allows our cadre of editors, reviewers, authors, and readers to strive to improve in our ability to create and critically analyze medical literature of the greatest merit.Big Data: Progress or a Big Headache?
Large administrative database, or “big data” research studies can include an immense number of patients. Strengths of research based on big data include generalizability resulting from diverse patients, diverse providers, and diverse clinical settings. Limitations of research based on large administrative databases may include indeterminate quality and obscure purpose of data entry, lack of information regarding confounding variables, and suboptimal clinical outcome measures. Thus, research conclusions based on big data must be scrutinized in a discerning and critical manner.Expert Opinion Is Necessary: Delphi Panel Methodology Facilitates a Scientific Approach to Consensus
Our current trend and focus on evidence-based medicine is biased in favor of randomized controlled trials, which are ranked highest in the hierarchy of evidence while devaluing expert opinion, which is ranked lowest in the hierarchy. However, randomized controlled trials have weaknesses as well as strengths, and no research method is flawless. Moreover, stringent application of scientific research techniques, such as the Delphi Panel methodology, allows survey of experts in a high quality and scientific manner.Getting Back to Basics: The “Primary” Outcome Measure Determines a Study's Conclusion
In this month's issue, Hollman, Wolterbeek, Zijl, van Egeraat, and Wessel from Nieuwegein, The Netherlands, investigate an essential and most basic concern: how does an abduction brace compare to a simple sling after arthroscopic rotator cuff repair? A criticism of the study, well noted in Editorial Commentary by Kevin Plancher, is that Hollman et al. selected postoperative pain (rather than long-term clinical outcome) as their “primary” outcome measure. The primary outcome measure determines the conclusion of a study.Put Systems in Place to Better Measure Surgical Outcomes
When it comes to interventional medicine, we must put systems in place to better measure outcomes. If surgeons detect poor outcomes, surgeons study and improve. The challenge is detection. Outcome measurement is paramount, and can mitigate against low volume or limited experience.Research Pearls: The Significance of Statistics and Perils of Pooling
A series of articles on statistics are intended for an audience of clinicians, as well as statisticians and authors. Statistical significance is different than clinical significance. Understanding of clinical outcomes, value, quality, or generalizability requires critical analysis of medical research literature to ensure that statistical analyses have been properly applied and interpreted.Making the Right Treatment Decision Requires Consideration of Utility and Reconsideration of Value
To achieve a good clinical outcome, arthroscopic and related surgeons must choose the proper treatment, and the basis of this choice is accurate diagnosis. Generally, our clinical focus is on outcome, but outcome is achieved after the fact. While this seems obvious, arthroscopic and related surgeons—and our patients who participate in shared decision making—evaluate the utility, or usefulness, of potential treatments based on desired and expected benefits versus potential risks. Today, cost is frequently considered as a determinant of value in medicine and may be applied to the decision analysis, but if an individual patient perceives health to be priceless, cost becomes irrelevant.Pearls of Publishing: Advice for Increasing Your Acceptance Odds
Pearls of wisdom can be a convenient and efficient strategy to improve performance. As Editors, we employ pearls to standardize the review and editorial process, and we offer our own pearls to you to help facilitate acceptance of submitted research manuscripts with the ultimate goal of strengthening scientific conclusions that can affect patient care, and ultimately, improve outcome.
