Advertisement

Author Reply to “Placebo Trials in Orthopaedic Surgery” and “Review of Randomized Placebo-Controlled Trials”

      We thank Drs. Harris, Poolman, and Buchbinder
      • Harris I.A.
      • Poolman R.W.
      • Buchbinder R.
      Placebo trials in orthopaedic surgery.
      and Drs. Reito, Karjalainen, and Louhiala
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      for their interest in and commentary on “Sham Surgery Studies in Orthopaedic Surgery May Just Be A Sham: A Systematic Review of Randomized Placebo-Controlled Trials.”
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      We appreciate the opportunity to respond to their comments. As international value-based health care initiatives continue to be rapidly developed and used based on high-quality evidence, we are excited to begin a productive discussion on how to optimize guidelines and policy from sham surgery–controlled investigations in orthopaedic surgery in our reply. We agree with both groups of authors that high-quality sham surgery–controlled randomized studies do have immense value in evidence-based orthopaedic surgery. The intent of our systematic review and the intent of this response are to respectfully characterize the limitations of these studies so that future research may avoid them.
      In the quest to achieve the highest-quality evidence-based medicine in orthopaedic surgery, it is fairly safe to assume that we can all agree “the perfect study” does not exist. Despite best efforts of investigators, every study is limited in some way in design, conduct, or reporting. Even the top 10 most-cited randomized trials of all time have significant biases—“all trials inevitably produce bias.”
      • Krauss A.
      Why all randomised controlled trials produce biased results.
      Even the seminal placebo surgery–controlled trial in orthopaedic surgery
      • Moseley J.B.
      • O'Malley K.
      • Petersen N.J.
      • et al.
      A controlled trial of arthroscopic surgery for osteoarthritis of the knee.
      had dozens of significant limitations that “prompted significant criticism from the orthopaedic discipline at large.”
      • Sihvonen R.
      • Paavola M.
      • Malmivaara A.
      • Järvinen T.L.
      Finnish Degenerative Meniscal Lesion Study (FIDELITY): A protocol for a randomised, placebo surgery controlled trial on the efficacy of arthroscopic partial meniscectomy for patients with degenerative meniscus injury with a novel 'RCT within-a-cohort' study design.
      If we continue to identify and rectify methodologic limitations and bias, the increasing quality can improve the fidelity of the research that we translate to the office and operating room clinical settings. The sham surgery–controlled investigation attempts to mitigate the effect of the surgical placebo response and truly identify whether the actual intervention works (i.e., is it the actual specific effect of the arthroscopic meniscectomy that works, or is it just the placebo effect [and contextual effects] of the knee arthroscopy?). We agree with both the letter to the editor by Harris et al.
      • Harris I.A.
      • Poolman R.W.
      • Buchbinder R.
      Placebo trials in orthopaedic surgery.
      and the excellent text Surgery, The Ultimate Placebo by Harris
      • Harris I.
      The placebo effect.
      that there are several reasons that an individual in the placebo group may improve after a treatment or surgical procedure that are not a result of the placebo effect: self-limiting natural history of the condition (Voltaire’s “amusing the patient while nature cures the disease”), regression to the mean, concomitant treatment(s), and potentially, other unknown reasons. It is perhaps for these reasons—and not an “actual placebo effect” (which contradicts its definition)—that patients improve after a sham or placebo operation.
      • Harris I.
      Surgery has a placebo effect, so what?.
      ,
      • Enck P.
      A matter of perspective: Sham surgery as effective as surgery, or surgery as uneffective as sham?.
      It is for these points that we do owe Harris et al. a debt of gratitude for their efforts in understanding and teaching the surgical and medical communities all there is to know regarding placebo surgery–controlled randomized trials.
      The randomized controlled trial (RCT) has always been the ideal investigation to identify whether an intervention works: Is an experimental surgical technique (e.g., knee arthroscopy) “better” when compared with a control group (i.e., superior, non-inferior, or equivalent) in ideal (efficacy) or real-world generalizable (effectiveness) settings? In theory, randomization may prove causation. However, in practice, randomization may fail.
      • Goldberg M.
      How often does random assignment fail? Estimates and recommendations.
      Causal inference experts continue to insist on a healthy dose of skepticism in interpreting the conclusions of randomized trials—“considering the practical difficulties of conducting an ideal RCT, observational studies have a definite advantage: they interrogate populations at their natural habitats, not in artificial environments choreographed by experimental protocols.”
      • Pearl J.
      Challenging the hegemony of randomized controlled trials: A commentary on Deaton and Cartwright.
      Relevant to the latter and more pertinent to randomized sham surgery–controlled trials is the issue of the “no treatment” control group, given that in practicality, in patients with significant pain and dysfunction, it is unlikely they would be willing to just “wait and see” how well they do while the surgery and sham groups evolve through trial completion. It is even possible that eligible trial participants who are told about surgical treatment during the informed consent process but are then allocated to nonsurgical treatment (or even a no-treatment natural-history control group) may then equate any or all symptoms to insufficient care, and this may drive them to drop out or cross over to surgical treatment. Nevertheless, despite these known issues, the practicality of the no-treatment control is outweighed by its necessity to obtain the best answer or outcome possible.
      Clearly, we humbly recognize that the issue of randomization is not only imperfect but also incredibly controversial. In 2018, Deaton and Cartwright
      • Deaton A.
      • Cartwright N.
      Understanding and misunderstanding randomized controlled trials.
      presented a well-written approach to understanding RCTs. On the basis of their article, 18 letters to the editor or commentaries have already been written by some of the world’s leading experts on study design about randomization, stating the following: “often flawed, mostly useless, clearly indispensable,”
      • Ioannidis J.P.A.
      Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright.
      “a good portion of current RCTs [are] . . . small, biased, uninformative trials representative of largely wasted efforts,”
      • Ioannidis J.P.A.
      Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright.
      ,
      • Hopewell S.
      • Dutton S.
      • Yu L.M.
      • Chan A.W.
      • Altman D.G.
      The quality of reports of randomised trials in 2000 and 2006: Comparative study of articles indexed in PubMed.
      “all randomized trials produce biased results,”
      • Krauss A.
      Why all randomised controlled trials produce biased results.
      “26 assumptions that have to be met if single random assignment experiments are to warrant ‘gold-standard’ status,”
      • Cook T.D.
      Twenty-six assumptions that have to be met if single random assignment experiments are to warrant "gold standard" status: A commentary on Deaton and Cartwright.
      and “if our conception of causal effects had anything to do with randomized experiments, the latter would have been invented 500 years before Fisher.”
      • Pearl J.
      Challenging the hegemony of randomized controlled trials: A commentary on Deaton and Cartwright.
      Deaton and Cartwright
      • Deaton A.
      • Cartwright N.
      Reflections on randomized control trials.
      offered a thoughtful response to these letters and commentaries. Although beyond the scope of our letter, we direct readers to both publications by Deaton and Cartwright
      • Deaton A.
      • Cartwright N.
      Understanding and misunderstanding randomized controlled trials.
      ,
      • Deaton A.
      • Cartwright N.
      Reflections on randomized control trials.
      and an authoritative causal inference text relevant to randomized trials by Pearl and Mackenzie
      • Pearl J.
      • Mackenzie D.
      The book of why: The new science of cause and effect.
      for a comprehensive analysis of the randomized trial and its limitations relevant to interventional causation.
      Randomized groups may differ because of chance (smaller sample sizes have greater risk, larger sizes mitigate risk) or post-randomization experiences (e.g., intention to treat vs per protocol).
      • Ioannidis J.P.A.
      Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright.
      Chance may cause imbalanced groups that asymmetrically distribute risk in which the differences on outcome may be quite substantial.
      • Ioannidis J.P.
      • Lau J.
      The impact of high-risk patients on the results of clinical trials.
      In a review of the top 10 most-cited randomized trials worldwide, the most common limitation was that participants’ background traits that affect outcomes were poorly distributed between trial groups.
      • Krauss A.
      Why all randomised controlled trials produce biased results.
      These confounder influences can be imbalanced in randomized trials with sample sizes of up to as many as 1,500 participants. In a smaller trial, like all 7 trials analyzed in our review, if the trial is replicated over and over again, each set of participants can have widely different confounders (known and unknown), some of which can cause the trial to show no difference and some, to show a significant difference. Relevant to our systematic review
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      and the issue of group asymmetry was the issue of genetic analysis. Although Reito et al.
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      are correct that “randomization is the best tool to avoid such bias since randomization will distribute hidden covariates efficiently,” they fail to account for the very real possibility that, without measurement, they may actually be exceedingly different because of chance owing to small group sizes. In some cases, the effective sample size may be up to 100 times smaller than the apparent sample size.
      • Ioannidis J.P.
      • Lau J.
      Heterogeneity of the baseline risk within patient populations of clinical trials: A proposed evaluation algorithm.
      Furthermore, although our systematic review revealed that only 4 of the 7 studies were adequately powered,
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      it has been shown that the median number of subjects in over 1,000 PubMed-indexed randomized trials is only 80 participants (median of 122 in our review).
      • Hopewell S.
      • Dutton S.
      • Yu L.M.
      • Chan A.W.
      • Altman D.G.
      The quality of reports of randomised trials in 2000 and 2006: Comparative study of articles indexed in PubMed.
      In an investigation of the fragility index of 48 randomized trials in sports medicine and arthroscopic surgery, the median sample size was 64 (interquartile range, 48.5-89.5) and the median fragility index was 2, meaning that changing only 2 patients from a non-event to an event in the treatment arm changed the study’s conclusions to a statistically nonsignificant result.
      • Khan M.
      • Evaniew N.
      • Gichuru M.
      • et al.
      The fragility of statistically significant findings from randomized trials in sports surgery: A systematic survey.
      In our systematic review, no analyzed study even evaluated the fragility index.
      In addition to genetics and randomization, the letters by Harris et al.
      • Harris I.A.
      • Poolman R.W.
      • Buchbinder R.
      Placebo trials in orthopaedic surgery.
      and Reito et al.
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      both critique our preference for clarity in treatment allocation. Both groups of authors prefer intention-to-treat analyses because they retain the benefits of randomization and avoid bias associated with nonrandom loss of participants. Although we understand why they have this preference, per the CONSORT (Consolidated Standards of Reporting Trials) guidelines, there are 2 reasons that a strict intention-to-treat analysis is hard to achieve: missing outcomes for participants and non-adherence to protocol. Thus, the CONSORT guidelines do not require an intention-to-treat analysis but rather favor a clear description of exactly which patients were included in each analysis.

      12a. Statistical methods. Box 6: Intention-to-treat analysis. http://www.consort-statement.org/checklists/view/32--consort-2010/96-statistical-methods. Published 2020. Accessed July 19, 2020.

      Missing outcomes can be included if imputed, but imputation requires strong assumptions, with decreasing fidelity with more assumptions made. Last-observation-carried-forward imputation is simple but prone to bias. As soon as a single randomized and allocated study participant is excluded, there is no longer an intention to treat. These “per protocol” subjects are frequently the result of crossover (either before or after participant and/or investigator unblinding). For example, in a seminal randomized trial comparing hip arthroscopy with physical therapy using an intention-to-treat analysis, a 70% crossover rate was observed; in plain terms, this means that the comparison consisted of a group of participants in which 95% underwent surgery versus a group in which 70% underwent surgery.
      • Mansell N.S.
      • Rhon D.I.
      • Meyer J.
      • Slevin J.M.
      • Marchant B.G.
      Arthroscopic surgery or physical therapy for patients with femoroacetabular impingement syndrome: A randomized controlled trial with 2-year follow-up.
      This greatly affects the study’s conclusions.
      • Faucett S.C.
      • Nepple J.J.
      • Andrade T.
      • et al.
      Randomized controlled trial of hip arthroscopy surgery vs physical therapy: Letter to the editor.
      Clearly, intention-to-treat analyses are far from infallible because of increasing numbers of crossover. In the context of sham surgery studies, this is important. Although no crossover threshold exists to define “acceptable,” it is accepted that greater proportions of crossover from control to treatment likely indicate patient dissatisfaction with the control “intervention.” Similarly, although no exact threshold of the crossover percentage that would invalidate a trial exists, with a 0% crossover rate being theoretically ideal and a 100% crossover rate completely invalidating a trial, the Centre for Evidence-based Medicine suggests “a crossover rate of greater than 20% is a rough guide to the number that may invalidate the final results.”
      • Faucett S.C.
      • Nepple J.J.
      • Andrade T.
      • et al.
      Randomized controlled trial of hip arthroscopy surgery vs physical therapy: Letter to the editor.
      Of the 7 studies analyzed in our review, 5 reported crossover rates, at 8%, 12%, 13%, 36%, and 36%.
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      As we reported in the systematic review, these study conclusions with greater than 20% crossover rates are largely invalidated. We take this last sentence seriously with use of the term “invalidated” because we recognize the amount of time, effort, money, and teamwork required to perform studies of this caliber. Both Harris et al. and Reito et al. question this term given its finality and how it essentially negates any and all utility of the study (or studies). Nevertheless, unfortunately, because the conclusions can be used to make such widespread critically important policy decisions, we find the validity of their conclusions incongruous (by the terms of Reito et al) with making those decisions.
      To tie hip arthroscopy into the ethics of sham surgery–controlled randomized studies is highly relevant at the current time and merits discussion. A terrific review of the ethics of sham surgery–controlled investigations can be found in the seminal report outlining ethical issues for study design and conduct,
      • Miller F.G.
      Sham surgery: An ethical analysis.
      as well as a recent high-quality review that created the ASPIRE (Applying Surgical Placebo in Randomized Evaluations) guidelines.
      • Beard D.J.
      • Campbell M.K.
      • Blazeby J.M.
      • et al.
      Considerations and methods for placebo controls in surgical trials (ASPIRE guidelines).
      Considering these ethics, we contend that hip arthroscopy is a technically challenging surgical procedure with multiple different types and severities of major and minor complications in which a sham-controlled randomized trial is ethically controversial.
      • Weber A.E.
      • Harris J.D.
      • Nho S.J.
      Complications in hip arthroscopy: A systematic review and strategies for prevention.
      ,
      • Sochacki K.R.
      • Jack II, R.A.
      • Safran M.R.
      • Nho S.J.
      • Harris J.D.
      There is a significant discrepancy between "big data" database and original research publications on hip arthroscopy outcomes: A systematic review.
      Although a placebo pill is inert, without an inherent risk in and of itself, a sham surgical procedure is an invasive placebo that carries many of the same risks as the actual surgical procedure and the associated anesthesia.
      • Miller F.G.
      Sham surgery: An ethical analysis.
      Proponents of sham-controlled trials submit that this type of trial is necessary because if the use of a surgical procedure is quite rapidly growing across the world, all these patients are subjected to the risks of that procedure. In turn, the greater the risk of the procedure, the greater need for a high-quality sham-controlled trial. Opponents of sham-controlled trials submit that exposing patients to a sham surgical procedure with significant temporary or permanent complications is unethical—“it is inappropriate to sacrifice one or more individuals for the benefits of society.”
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      Both ongoing sham hip arthroscopy–controlled trials have a unique set of risks in the sham group that merit discussion (distraction, portal placement, joint entry, unrepaired capsulotomy) and cannot be avoided during an informed consent discussion.
      • Risberg M.A.
      • Ageberg E.
      • Nilstad A.
      • et al.
      Arthroscopic surgical procedures versus sham surgery for patients with femoroacetabular impingement and/or labral tears: Study protocol for a randomized controlled trial (HIPARTI) and a prospective cohort study (HARP).
      ,
      • Simunovic N.
      • Heels-Ansdell D.
      • Thabane L.
      • Ayeni O.R.
      Femoroacetabular Impingement Randomised controlled Trial (FIRST)—A multi-centre randomized controlled trial comparing arthroscopic lavage and arthroscopic osteochondroplasty on patient important outcomes and quality of life in the treatment of young adult (18-50 years) femoroacetabular impingement: A statistical analysis plan.
      In both studies’ sham groups, patients are undergoing surgery with a perineal post–assisted distraction technique with risks of temporary and/or permanent pudendal neuralgia; scrotal, labial, and/or vaginal necrosis; genitalia injury; sex organ damage; sexual dysfunction; and urologic dysfunction at rates of up to 25%.
      • Mei-Dan O.
      • Kraeutler M.J.
      • Garabekyan T.
      • Goodrich J.A.
      • Young D.A.
      Hip distraction without a perineal post: A prospective study of 1000 hip arthroscopy cases.
      • Reda B.
      • Wong I.
      Postoperative numbness: A survey of patients after hip arthroscopic surgery.
      • Pailhe R.
      • Chiron P.
      • Reina N.
      • Cavaignac E.
      • Lafontan V.
      • Laffosse J.M.
      Pudendal nerve neuralgia after hip arthroscopy: Retrospective study and literature review.
      • Kocaoglu H.
      • Basarir K.
      • Akmese R.
      • et al.
      The effect of traction force and hip abduction angle on pudendal nerve compression in hip arthroscopy: A cadaveric model.
      Portal placement (lateral femoral cutaneous nerve injury risk of up to 15%) and joint entry alone may cause iatrogenic injury (irreversible injury to articular cartilage and/or labrum).
      • Compton J.
      • Slattery M.
      • Coleman M.
      • Westermann R.
      Iatrogenic articular cartilage injury in arthroscopic hip and knee videos and the potential for cartilage cell death when simulated in a bovine model.
      ,
      • Harris J.D.
      • McCormick F.M.
      • Abrams G.D.
      • et al.
      Complications and reoperations during and after hip arthroscopy: A systematic review of 92 studies and more than 6,000 patients.
      In the sham group in one of the aforementioned studies, an interportal capsulotomy is performed but not repaired, leaving the hip at risk of postoperative instability and pain, as well as yielding a significantly increased risk of revision owing to microinstability (or even subluxation or dislocation).
      • Wylie J.D.
      • Beckmann J.T.
      • Maak T.G.
      • Aoki S.K.
      Arthroscopic capsular repair for symptomatic hip instability after previous hip arthroscopic surgery.
      ,
      • Duplantier N.L.
      • McCulloch P.C.
      • Nho S.J.
      • Mather III, R.C.
      • Lewis B.D.
      • Harris J.D.
      Hip dislocation or subluxation after hip arthroscopy: A systematic review.
      Outside the orthopaedic surgery world, sham-controlled studies are also at risk of serious complications in the sham group—the Objective Randomised Blinded Investigation With Optimal Medical Therapy of Angioplasty in Stable Angina trial sustained at least 7 serious adverse events in the sham group versus 2 in the percutaneous coronary intervention group.
      • Al-Lamee R.
      • Thompson D.
      • Dehbi H.M.
      • et al.
      Percutaneous coronary intervention in stable angina (ORBITA): A double-blind, randomised controlled trial.
      These numbers beg the question of whether patients are truly “informed” during the informed consent process. This may explain why in a separate trial on hip arthroscopy, 75.2% of eligible subjects declined inclusion.
      • Ayeni O.R.
      • Belzile E.L.
      • Musahl V.
      • et al.
      Results of the Perception of Femoroacetabular Impingement by Surgeons Survey (PROCESS).
      This, in and of itself, merits consideration and analysis of patient characteristics, including mental health, that may impact participation in a trial with those risks and their potential outcome “for the good of society.”
      In their letters, both Harris et al.
      • Harris I.A.
      • Poolman R.W.
      • Buchbinder R.
      Placebo trials in orthopaedic surgery.
      and Reito et al.
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      describe a problem with the discussion of sisu and hygge in patients from Finland and patients from Norway and Denmark, respectively. Of the 7 studies in our systematic review, 4 were from 1 of these 3 countries.
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      There is a plethora of evidence in orthopaedic surgery that discusses the impact of mental health, depression, anxiety, resilience, grit, and intolerance to uncertainty in interpreting the outcomes of both surgical and nonsurgical interventions. Thus, if geography entails any significant participant confounding bias, measured or unmeasured, it must be recognized as potentially rendering a comparison of like versus unlike, or apples versus oranges. Although Harris et al. were insightful enough to contend that the patients apparently were not resilient enough to avoid complaining about pain and seeking treatment, this does not mean the patients were not resilient—it could simply mean that their disease course (e.g., worse meniscal tear with or without more advanced degenerative knee arthritis; rotator cuff tendon pathology with larger tear size, as well as more atrophy and degeneration; and shoulder labral pathology) was more advanced at the time of presentation, making it more challenging to achieve a clinically meaningful outcome improvement with the investigated treatment.
      We agree with Reito et al.
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      that further improvements in methodology should be embraced; it is unfortunate that they contradict themselves on this point within their letter. They refute the further study of genetic analysis because it is “in its infancy” and is “basic research,” without “clinical studies in any field of medicine.” Well, “a journey of a thousand miles begins with a single step.”
      • Tzu L.
      Tao Te Ching. Chapter 64.
      Population-based randomized trials are assumption laden and not necessarily the most appropriate method to assess an intervention in many settings.
      • Deaton A.
      • Cartwright N.
      Understanding and misunderstanding randomized controlled trials.
      ,
      • Schork N.J.
      Randomized clinical trials and personalized medicine: A commentary on Deaton and Cartwright.
      The reason for the latter is the emergence of “personalized” medicine, in which interventions (orthopaedic surgical procedures included) are selected for individual patients based on their genetic profiles,
      • Schork N.J.
      Randomized clinical trials and personalized medicine: A commentary on Deaton and Cartwright.
      as offered in our systematic review, certainly not “demanded,” as suggested by Reito et al. In addition, because the doubling time of medical knowledge is now 73 days
      • Densen P.
      Challenges and opportunities facing medical education.
      and the use of machine learning and artificial intelligence in medicine is already highly prevalent in everyday clinical practice,
      • Helm J.M.
      • Swiergosz A.M.
      • Haeberle H.S.
      • et al.
      Machine learning and artificial intelligence: Definitions, applications, and future directions.
      it would be overly naive, not a “gross overstatement” (as made by Reito et al.), to believe that genetic analysis will not be a routine part of our clinical practices in the near future. Rather than dismiss genetic analysis altogether, it should be embraced with high-quality research for the potential to provide better patient care. Simply put, a significant goal of our systematic review was to ask all researchers, humbly and respectfully, ourselves included, to “aim higher,” as suggested by Harris et al.
      • Harris I.A.
      • Poolman R.W.
      • Buchbinder R.
      Placebo trials in orthopaedic surgery.
      Just because evidence is “the best available we can deliver to patients” does not mean it is “the best we can do.”
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      It does not mean we should exclusively rely on it to make decisions on guidelines and policy, derived from average treatment effects, but intended for individual patients in the office and operating room. We reiterate that the limitations described in the 7 sham surgery–controlled investigations analyzed in our systematic review preclude the fidelity of their conclusions.
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.U.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
      We respectfully disagree with the letters of Harris et al.
      • Harris I.A.
      • Poolman R.W.
      • Buchbinder R.
      Placebo trials in orthopaedic surgery.
      and Reito et al.
      • Reito A.
      • Karjalainen T.
      • Louhiala P.
      Review of randomized placebo-controlled trials.
      regarding the validity of our systematic review’s conclusions. We further reiterate that the purpose of our systematic review was not to conclude that placebo-controlled investigations should no longer be performed but rather to identify necessary components of trial design and conduct that should be in any future sham surgery studies—or, as Harris et al. say, “aim higher,” and as Reito et al. say, “improvements in methodology should be embraced.”

      Supplementary Data

      References

        • Harris I.A.
        • Poolman R.W.
        • Buchbinder R.
        Placebo trials in orthopaedic surgery.
        Arthroscopy. 2020; 36: 2777-2778
        • Reito A.
        • Karjalainen T.
        • Louhiala P.
        Review of randomized placebo-controlled trials.
        Arthroscopy. 2020; 36: 2778-2779
        • Sochacki K.R.
        • Mather R.C.
        • Nwachukwu B.U.
        • et al.
        Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials..
        Arthroscopy. 2020; 36: 2750-2762
        • Krauss A.
        Why all randomised controlled trials produce biased results.
        Ann Med. 2018; 50: 312-322
        • Moseley J.B.
        • O'Malley K.
        • Petersen N.J.
        • et al.
        A controlled trial of arthroscopic surgery for osteoarthritis of the knee.
        N Engl J Med. 2002; 347: 81-88
        • Sihvonen R.
        • Paavola M.
        • Malmivaara A.
        • Järvinen T.L.
        Finnish Degenerative Meniscal Lesion Study (FIDELITY): A protocol for a randomised, placebo surgery controlled trial on the efficacy of arthroscopic partial meniscectomy for patients with degenerative meniscus injury with a novel 'RCT within-a-cohort' study design.
        BMJ Open. 2013; 3e002510
        • Harris I.
        The placebo effect.
        in: Harris I. Surgery, the ultimate placebo. NewSouth Publishing, Sydney, Australia2016: 22-28
        • Harris I.
        Surgery has a placebo effect, so what?.
        in: Harris I. Surgery, the ultimate placebo. NewSouth Publishing, Sydney, Australia2016: 236-238
        • Enck P.
        A matter of perspective: Sham surgery as effective as surgery, or surgery as uneffective as sham?.
        Pain Med. 2019; 20: 200-201
        • Goldberg M.
        How often does random assignment fail? Estimates and recommendations.
        J Environ Psychol. 2019; 66: 101351
        • Pearl J.
        Challenging the hegemony of randomized controlled trials: A commentary on Deaton and Cartwright.
        Soc Sci Med. 2018; 210: 60-62
        • Deaton A.
        • Cartwright N.
        Understanding and misunderstanding randomized controlled trials.
        Soc Sci Med. 2018; 210: 2-21
        • Ioannidis J.P.A.
        Randomized controlled trials: Often flawed, mostly useless, clearly indispensable: A commentary on Deaton and Cartwright.
        Soc Sci Med. 2018; 210: 53-56
        • Hopewell S.
        • Dutton S.
        • Yu L.M.
        • Chan A.W.
        • Altman D.G.
        The quality of reports of randomised trials in 2000 and 2006: Comparative study of articles indexed in PubMed.
        BMJ. 2010; 340: c723
        • Cook T.D.
        Twenty-six assumptions that have to be met if single random assignment experiments are to warrant "gold standard" status: A commentary on Deaton and Cartwright.
        Soc Sci Med. 2018; 210: 37-40
        • Deaton A.
        • Cartwright N.
        Reflections on randomized control trials.
        Soc Sci Med. 2018; 210: 86-90
        • Pearl J.
        • Mackenzie D.
        The book of why: The new science of cause and effect.
        Basic Books, New York2018
        • Ioannidis J.P.
        • Lau J.
        The impact of high-risk patients on the results of clinical trials.
        J Clin Epidemiol. 1997; 50: 1089-1098
        • Ioannidis J.P.
        • Lau J.
        Heterogeneity of the baseline risk within patient populations of clinical trials: A proposed evaluation algorithm.
        Am J Epidemiol. 1998; 148: 1117-1126
        • Khan M.
        • Evaniew N.
        • Gichuru M.
        • et al.
        The fragility of statistically significant findings from randomized trials in sports surgery: A systematic survey.
        Am J Sports Med. 2017; 45: 2164-2170
      1. 12a. Statistical methods. Box 6: Intention-to-treat analysis. http://www.consort-statement.org/checklists/view/32--consort-2010/96-statistical-methods. Published 2020. Accessed July 19, 2020.

        • Mansell N.S.
        • Rhon D.I.
        • Meyer J.
        • Slevin J.M.
        • Marchant B.G.
        Arthroscopic surgery or physical therapy for patients with femoroacetabular impingement syndrome: A randomized controlled trial with 2-year follow-up.
        Am J Sports Med. 2018; 46: 1306-1314
        • Faucett S.C.
        • Nepple J.J.
        • Andrade T.
        • et al.
        Randomized controlled trial of hip arthroscopy surgery vs physical therapy: Letter to the editor.
        Am J Sports Med. 2018; 46 ([letter]): NP35-NP38
        • Miller F.G.
        Sham surgery: An ethical analysis.
        Sci Eng Ethics. 2004; 10: 157-166
        • Beard D.J.
        • Campbell M.K.
        • Blazeby J.M.
        • et al.
        Considerations and methods for placebo controls in surgical trials (ASPIRE guidelines).
        Lancet. 2020; 395: 828-838
        • Weber A.E.
        • Harris J.D.
        • Nho S.J.
        Complications in hip arthroscopy: A systematic review and strategies for prevention.
        Sports Med Arthrosc. 2015; 23: 187-193
        • Sochacki K.R.
        • Jack II, R.A.
        • Safran M.R.
        • Nho S.J.
        • Harris J.D.
        There is a significant discrepancy between "big data" database and original research publications on hip arthroscopy outcomes: A systematic review.
        Arthroscopy. 2018; 34: 1998-2004
        • Risberg M.A.
        • Ageberg E.
        • Nilstad A.
        • et al.
        Arthroscopic surgical procedures versus sham surgery for patients with femoroacetabular impingement and/or labral tears: Study protocol for a randomized controlled trial (HIPARTI) and a prospective cohort study (HARP).
        J Orthop Sports Phys Ther. 2018; 48: 325-335
        • Simunovic N.
        • Heels-Ansdell D.
        • Thabane L.
        • Ayeni O.R.
        Femoroacetabular Impingement Randomised controlled Trial (FIRST)—A multi-centre randomized controlled trial comparing arthroscopic lavage and arthroscopic osteochondroplasty on patient important outcomes and quality of life in the treatment of young adult (18-50 years) femoroacetabular impingement: A statistical analysis plan.
        Trials. 2018; 19: 588
        • Mei-Dan O.
        • Kraeutler M.J.
        • Garabekyan T.
        • Goodrich J.A.
        • Young D.A.
        Hip distraction without a perineal post: A prospective study of 1000 hip arthroscopy cases.
        Am J Sports Med. 2018; 46: 632-641
        • Reda B.
        • Wong I.
        Postoperative numbness: A survey of patients after hip arthroscopic surgery.
        Orthop J Sports Med. 2018; 6 (2325967118771535)
        • Pailhe R.
        • Chiron P.
        • Reina N.
        • Cavaignac E.
        • Lafontan V.
        • Laffosse J.M.
        Pudendal nerve neuralgia after hip arthroscopy: Retrospective study and literature review.
        Orthop Traumatol Surg Res. 2013; 99: 785-790
        • Kocaoglu H.
        • Basarir K.
        • Akmese R.
        • et al.
        The effect of traction force and hip abduction angle on pudendal nerve compression in hip arthroscopy: A cadaveric model.
        Arthroscopy. 2015; 31: 1974-1980.e6
        • Compton J.
        • Slattery M.
        • Coleman M.
        • Westermann R.
        Iatrogenic articular cartilage injury in arthroscopic hip and knee videos and the potential for cartilage cell death when simulated in a bovine model.
        Arthroscopy. 2020; 36: 2114-2121
        • Harris J.D.
        • McCormick F.M.
        • Abrams G.D.
        • et al.
        Complications and reoperations during and after hip arthroscopy: A systematic review of 92 studies and more than 6,000 patients.
        Arthroscopy. 2013; 29: 589-595
        • Wylie J.D.
        • Beckmann J.T.
        • Maak T.G.
        • Aoki S.K.
        Arthroscopic capsular repair for symptomatic hip instability after previous hip arthroscopic surgery.
        Am J Sports Med. 2016; 44: 39-45
        • Duplantier N.L.
        • McCulloch P.C.
        • Nho S.J.
        • Mather III, R.C.
        • Lewis B.D.
        • Harris J.D.
        Hip dislocation or subluxation after hip arthroscopy: A systematic review.
        Arthroscopy. 2016; 32: 1428-1434
        • Al-Lamee R.
        • Thompson D.
        • Dehbi H.M.
        • et al.
        Percutaneous coronary intervention in stable angina (ORBITA): A double-blind, randomised controlled trial.
        Lancet. 2018; 391: 31-40
        • Ayeni O.R.
        • Belzile E.L.
        • Musahl V.
        • et al.
        Results of the Perception of Femoroacetabular Impingement by Surgeons Survey (PROCESS).
        Knee Surg Sports Traumatol Arthrosc. 2014; 22: 906-910
        • Tzu L.
        Tao Te Ching. Chapter 64.
        Penguin Classics, London1964: 77
        • Schork N.J.
        Randomized clinical trials and personalized medicine: A commentary on Deaton and Cartwright.
        Soc Sci Med. 2018; 210: 71-73
        • Densen P.
        Challenges and opportunities facing medical education.
        Trans Am Clin Climatol Assoc. 2011; 122: 48-58
        • Helm J.M.
        • Swiergosz A.M.
        • Haeberle H.S.
        • et al.
        Machine learning and artificial intelligence: Definitions, applications, and future directions.
        Curr Rev Musculoskelet Med. 2020; 13: 69-76