Advertisement

Author Reply to “Is Criticism About Inherent Biases in Rigorous Orthopaedic Trials Prone to Biases?”

      We thank Drs. Reito and Karjalainen for their letter to the editor with regards to our level V evidence clinical guideline.
      • Reito A.
      • Karjalainen T.
      Letter to editor: Is criticism about inherent biases in rigorous orthopaedic trials prone to biases?.
      ,
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Indications for arthroscopic subacromial decompression. A level V evidence clinical guideline.
      Subacromial arthroscopic decompression (SAD) in the presence of an intact rotator cuff is a controversial topic, and we welcome debate. In their letter to the editor, Reito and Karjalainen expressed concerns about biases and fierce resistance when “accepted treatments that stood the test in time” are challenged.
      • Reito A.
      • Karjalainen T.
      Letter to editor: Is criticism about inherent biases in rigorous orthopaedic trials prone to biases?.
      The most updated definition of evidence-based medicine was revised in 2000 and defined evidence-based medicine as the “integration of best research evidence with clinical expertise and patient value.”
      • Sackett D.L.
      Evidence-based medicine: how to practice and teach EBM.
      This means that the current best evidence also may be obtained from clinical studies such as case series, retrospective comparative studies, and basic science research.
      • Sackett D.L.
      Evidence-based medicine: how to practice and teach EBM.
      The term “clinical expertise” implies that skills from clinical experience should not be ignored.
      • Hohmann E.
      • Brand J.C.
      • Rossi M.J.
      • Lubowitz J.H.
      Expert opinion is necessary: Delphi panel methodology facilitates a scientific approach to consensus.
      ,
      • Hohmann E.
      • Cote M.P.
      • Brand J.C.
      Research pearls: Expert consensus-based evidence using the Delphi method.
      This is where interpretation of the published evidence comes into play. It will always include an element of subjectivity and inevitably result in myside or confirmation bias. This basically means that we evaluate but also generate evidence in a manner biased toward our own personal opinions and attitudes.
      • Stanovich K.E.
      • West R.F.
      • Toplak M.E.
      Myside bias, rational thinking, and intelligence.
      Are we not all suffering from this problem? Different experiences, training, cultural and health environments, and practice patterns imprint our beliefs and behavior.
      • Phillips R.L.
      • Petterson S.M.
      • Bazemore A.W.
      • Wingrove P.
      • Puffer J.C.
      The effects of training institution, practice costs quality and other characteristics on future practice.
      We admit that we interpret the current available evidence in favor of SAD as a proven surgical intervention if the indications for surgery are correct.
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Indications for arthroscopic subacromial decompression. A level V evidence clinical guideline.
      ,
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Author reply: arthroscopic subacromial decompression. A level V evidence clinical guideline.
      As we have outlined in our response to van den Bekerom and Poolman
      • van den Bekerom M.P.J.
      • Poolman R.W.
      No indications for subacromial decompression in cuff tendinopathy. A level 1 evidence clinical guideline.
      and in the clinical guideline, strong evidence to either support SAD or nonoperative treatment is clearly missing.
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Indications for arthroscopic subacromial decompression. A level V evidence clinical guideline.
      ,
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Author reply: arthroscopic subacromial decompression. A level V evidence clinical guideline.
      In our level V guideline,
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Indications for arthroscopic subacromial decompression. A level V evidence clinical guideline.
      we have very carefully considered the available evidence, summarized the supporting evidence in favor of and against SAD, and merely suggested to consider SAD if 5 specific points are met. Reito and Karjalainen
      • Reito A.
      • Karjalainen T.
      Letter to editor: Is criticism about inherent biases in rigorous orthopaedic trials prone to biases?.
      argue that the placebo-controlled trials by Beard et al.
      • Beard D.J.
      • Rees J.L.
      • Cook J.A.
      • et al.
      Arthroscopic subacromial decompression for subacromial shoulder pain: A multicentre, pragmatic, parallel group, placebo-controlled, three group, randomised surgical trial.
      and Paavola et al.
      • Paavola M.
      • Malmivaara A.
      • Taimela S.
      • et al.
      Subacromial decompression versus diagnostic arthroscopy for shoulder impingement: Randomised, placebo controlled clinical trial.
      were rigorously conducted and of high evidence. Unfortunately, this is fundamentally wrong. As we have outlined already previously,
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Author reply: arthroscopic subacromial decompression. A level V evidence clinical guideline.
      the study by Beard et al.
      • Beard D.J.
      • Rees J.L.
      • Cook J.A.
      • et al.
      Arthroscopic subacromial decompression for subacromial shoulder pain: A multicentre, pragmatic, parallel group, placebo-controlled, three group, randomised surgical trial.
      has been criticized by multiple German Speaking Associations because of its multiple biases, as we have also highlighted in our guideline.
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Indications for arthroscopic subacromial decompression. A level V evidence clinical guideline.
      Similar, the study by Paavola et al. could hardly be defined as being conducted rigorously.
      • Paavola M.
      • Malmivaara A.
      • Taimela S.
      • et al.
      Subacromial decompression versus diagnostic arthroscopy for shoulder impingement: Randomised, placebo controlled clinical trial.
      Twelve percent of patients in the placebo group and 21% of patients in the exercise group eventually crossed over and had surgery.
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Author reply: arthroscopic subacromial decompression. A level V evidence clinical guideline.
      ,
      • Paavola M.
      • Malmivaara A.
      • Taimela S.
      • et al.
      Subacromial decompression versus diagnostic arthroscopy for shoulder impingement: Randomised, placebo controlled clinical trial.
      An intention-to-treat protocol was used for analysis, and these protocols have been widely criticized for violating the principle of randomization, introduction of bias, and effective reduction in sample size and study power.
      • Ranganathan P.
      • Pramesh C.S.
      • Aggarwal R.
      Common pitfalls in statistical analysis: Intention to treat versus per-protocol analysis.
      ,
      • Shrier I.
      • Verhagen E.
      • Stovitz S.D.
      The intention-to-treat analysis is not always the conservative approach.
      Furthermore, sham-controlled studies have methodologic deficiencies, possibly invalidating their conclusions.
      • Sochacki K.R.
      • Mather R.C.
      • Nwachukwu B.
      • et al.
      Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials.
      Of course this does not mean that the studies supporting SAD are better designed or have less limitations.
      With regards to the critical shoulder angle (CSA), the guideline has summarized the available evidence and clearly stated that is a hotly debated topic. In our recommendations we have been very careful in suggesting that a lateral acromioplasty should be considered. We agree with Reito and Karjalainen
      • Reito A.
      • Karjalainen T.
      Letter to editor: Is criticism about inherent biases in rigorous orthopaedic trials prone to biases?.
      that the concept of CSA is new and evolving. For a more balanced perspective on the quoted meta-analysis that was used by Reito and Karjalainen to argue against lateral acromioplasty, it has to be mentioned that the conclusions also state that the pooled data showed a relationship between greater CSA and full-thickness rotator cuff tears with a possible greater re-tear rate following rotator cuff repair.
      • Docter S.
      • Khan M.
      • Ekhtiari S.
      • et al.
      The relationship between the critical shoulder angle and the I Incidence of chronic, full-thickness rotator cuff tears and outcomes after rotator cuff repair: A systematic review.
      Reito and Karjalainen
      • Reito A.
      • Karjalainen T.
      Letter to editor: Is criticism about inherent biases in rigorous orthopaedic trials prone to biases?.
      argue that a Cochrane review from 2019 came to the inevitable conclusion that SAD surgery does not improve pain when compared with exercises.
      • Higgins J.P.T.
      • Green S.
      Cochrane handbook for systematic reviews of interventions. Version 5.1.9 updated March.
      The mean difference in visual analog scale score was 1 point, with a 95% confidence interval ranging from –0.25 to 2.25.
      • Karjalainen T.V.
      • Jain N.B.
      • Page C.M.
      • et al.
      Subacromial decompression surgery for rotator cuff disease.
      In addition, there was low certainty that surgery may improve function.
      • Karjalainen T.V.
      • Jain N.B.
      • Page C.M.
      • et al.
      Subacromial decompression surgery for rotator cuff disease.
      As already outlined in our response to the letter
      • Hohmann E.
      • Shea K.
      • Scheiderer B.
      • Millett P.
      • Imhoff A.
      Author reply: arthroscopic subacromial decompression. A level V evidence clinical guideline.
      by van den Bekerom and Poolman,
      • van den Bekerom M.P.J.
      • Poolman R.W.
      No indications for subacromial decompression in cuff tendinopathy. A level 1 evidence clinical guideline.
      75% of the included studies had a high risk of bias, and because of imprecision of the results, the certainty of evidence was downgraded by the authors of the Cochrane Review.
      • Karjalainen T.V.
      • Jain N.B.
      • Page C.M.
      • et al.
      Subacromial decompression surgery for rotator cuff disease.
      The Cochrane Handbook states that high risk of bias is concerning and therefore lowers the confidence in the results of the analysis.
      • Higgins J.P.T.
      • Green S.
      Cochrane handbook for systematic reviews of interventions. Version 5.1.9 updated March.
      One way to mitigate is to restrict the primary analysis to low-risk studies or to perform sensitivity analysis to show how conclusions might be affected if studies with a high risk of bias were included.
      • Higgins J.P.T.
      • Green S.
      Cochrane handbook for systematic reviews of interventions. Version 5.1.9 updated March.
      The cited Cochrane review has included a sensitivity analysis. Pain at 6 months favored surgery, but the 95% confidence interval was wide (–0.31 [–0.75, 0.12]); pain at 1 year clearly favored surgery (–0.58 [–1.05, –0.12]); function at 6 months favored conservative treatment but the confidence intervals were wide (1.05 [–3.77, 5.87]); function at 1-3 years favored conservative treatment (3.21 [–0.81, 7.23]); and pain at 5 years favored surgery (–0.78 [–1.17, –0.39]).
      • Karjalainen T.V.
      • Jain N.B.
      • Page C.M.
      • et al.
      Subacromial decompression surgery for rotator cuff disease.
      We note that the co-author of this letter to the editor was also the main author of the Cochrane review.
      • Karjalainen T.V.
      • Jain N.B.
      • Page C.M.
      • et al.
      Subacromial decompression surgery for rotator cuff disease.
      Could the authors suffer from confirmation bias themselves? We leave the answer to the readers.
      SAD for impingent remains a controversial topic, and it appears that many of us are strongly opinionated. This certainly reflects the lack of agreement but also the lack of strong and reliable evidence. It may not be time for a paradigm shift but time to create this evidence. Given the difficulties of designing a watertight randomized trial and including most of the potential confounders, this may not be possible in the near future. Until then, we have to rely on interpreting research evidence to the best of our knowledge taking our clinical experience and patient expectations into consideration.
      • Sackett D.L.
      Evidence-based medicine: how to practice and teach EBM.

      Supplementary Data

      References

        • Reito A.
        • Karjalainen T.
        Letter to editor: Is criticism about inherent biases in rigorous orthopaedic trials prone to biases?.
        Arthroscopy. 2020; 37: 8-9
        • Hohmann E.
        • Shea K.
        • Scheiderer B.
        • Millett P.
        • Imhoff A.
        Indications for arthroscopic subacromial decompression. A level V evidence clinical guideline.
        Arthroscopy. 2020; 36: 913-920
        • Sackett D.L.
        Evidence-based medicine: how to practice and teach EBM.
        Churchill-Livingstone, London2000: 1-20
        • Hohmann E.
        • Brand J.C.
        • Rossi M.J.
        • Lubowitz J.H.
        Expert opinion is necessary: Delphi panel methodology facilitates a scientific approach to consensus.
        Arthroscopy. 2018; 34: 349-351
        • Hohmann E.
        • Cote M.P.
        • Brand J.C.
        Research pearls: Expert consensus-based evidence using the Delphi method.
        Arthroscopy. 2018; 24: 3278-3282
        • Stanovich K.E.
        • West R.F.
        • Toplak M.E.
        Myside bias, rational thinking, and intelligence.
        Curr Dir Psych Sci. 2013; 22: 259-264
        • Phillips R.L.
        • Petterson S.M.
        • Bazemore A.W.
        • Wingrove P.
        • Puffer J.C.
        The effects of training institution, practice costs quality and other characteristics on future practice.
        Ann Fam Med. 2017; 15: 140-148
        • Hohmann E.
        • Shea K.
        • Scheiderer B.
        • Millett P.
        • Imhoff A.
        Author reply: arthroscopic subacromial decompression. A level V evidence clinical guideline.
        Arthroscopy. 2020; 36: 1493-1495
        • van den Bekerom M.P.J.
        • Poolman R.W.
        No indications for subacromial decompression in cuff tendinopathy. A level 1 evidence clinical guideline.
        Arthroscopy. 2020; 36: 1492-1493
        • Beard D.J.
        • Rees J.L.
        • Cook J.A.
        • et al.
        Arthroscopic subacromial decompression for subacromial shoulder pain: A multicentre, pragmatic, parallel group, placebo-controlled, three group, randomised surgical trial.
        Lancet. 2018; 391: 329-338
        • Paavola M.
        • Malmivaara A.
        • Taimela S.
        • et al.
        Subacromial decompression versus diagnostic arthroscopy for shoulder impingement: Randomised, placebo controlled clinical trial.
        BMJ. 2018; 362k2860
        • Ranganathan P.
        • Pramesh C.S.
        • Aggarwal R.
        Common pitfalls in statistical analysis: Intention to treat versus per-protocol analysis.
        Perspect Clin Res. 2016; 7: 144-146
        • Shrier I.
        • Verhagen E.
        • Stovitz S.D.
        The intention-to-treat analysis is not always the conservative approach.
        Am J Med. 2017; 130: 867-871
        • Sochacki K.R.
        • Mather R.C.
        • Nwachukwu B.
        • et al.
        Sham surgery studies in orthopaedic surgery may just be a sham: A systematic review of randomized placebo-controlled trials.
        Arthroscopy. 2020; 36: 2750-2762
        • Docter S.
        • Khan M.
        • Ekhtiari S.
        • et al.
        The relationship between the critical shoulder angle and the I Incidence of chronic, full-thickness rotator cuff tears and outcomes after rotator cuff repair: A systematic review.
        Arthroscopy. 2019; 35: 3135-3143
        • Karjalainen T.V.
        • Jain N.B.
        • Page C.M.
        • et al.
        Subacromial decompression surgery for rotator cuff disease.
        Cochrane Database Syst Rev. 2019; 1: CD005619
        • Higgins J.P.T.
        • Green S.
        Cochrane handbook for systematic reviews of interventions. Version 5.1.9 updated March.
        2011 (The Cochrane Collaboration. Chichester: Wiley-Blackwell, 2011.)