Advertisement

“Publish or Perish” Promotes Medical Literature Quantity Over Quality

      Abstract

      When it comes to medical research, incentives align to promote “publish or perish.” This results in quantity over quality. A solution is to change the goal of medical scientist and clinician training from bolstering a curriculum vitae to mastering scientific research methods. In addition, the metric for scholarly authorship should be quality, for which validated measurement tools exist, rather than number of publications.
      It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair, we had everything before us, we had nothing before us, we were all going direct to Heaven, we were all going direct the other way—in short, the period was so far like the present period, that some of its noisiest authorities insisted on its being received, for good or for evil, in the superlative degree of comparison only.—Charles Dickens, A Tale of Two Cities (1859)

      Explosive Growth

      In terms of the explosive growth of medical research, our present could be superlatively described as glorious and also described as ghastly. Specifically, scientific research and development are at risk of being undermined by submission (and sometimes) publication of investigations of low quality, significant bias, low reproducibility, and incorrect conclusions.
      • Frieden T.R.
      Evidence for health decision making—beyond randomized, controlled trials.
      • Chess L.E.
      • Gagnier J.
      Risk of bias of randomized controlled trials published in orthopaedic journals.
      • Buckwalter J.A.
      • Tolo V.T.
      • O’Keefe R.J.
      How do you know it is true? Integrity in research and publications. AOA critical issues.
      • Somford M.P.
      • van Deurzen D.F.
      • Ostedorf M.
      • Eygendaal D.
      • van den Bekerom M.P.
      Quality of research and quality of reporting in elbow surgery trials.

      The Match

      Back in the day, or so we've been told and although it sounds like “a tale” by a great novelist, orthopaedic residents and fellows obtained positions without benefit of the National Resident Matching Program, “the Match.”
      National Resident Matching Program.
      According to legend, a potential orthopaedist might visit a training program and chat with the chairperson. A phone call from a dean or mentor might follow, and a prestigious position might be arranged. With a wink and a nod and a bit of negotiation, a deal could be done. The inequity of the process was disturbing, and while programs benefited from early acceptance, applicants were challenged by exploding offers (job offers that were retracted if not accepted in a short period of time) based more on friendships among directors than qualifications of the candidates.
      Today, the Match purports to provide an algorithm that is “100% objective, 100% accurate, and 100% committed to a fair and transparent process. …The annual Main Residency Match … encompasses more than 42,000 applicants and 30,000 positions,” and more than 60 fellowship subspecialties are also included.
      National Resident Matching Program.
      The system clearly levels the playing field.
      However, quantification of clinical acumen, professionalism, and surgical skill is difficult to measure, and an objective parameter that has been increasingly weighed in the ranking of applications is “demonstrating involvement and interest in research.” Research interest has a significant importance for selecting orthopaedic surgery residency applicants to interview,
      National Resident Matching Program. Results of the 2016 NRMP program director survey.
      and the push to publish is further motivated by the fact that unmatched applicants who publish have a substantially improved chance for eventually finding a position versus similarly unmatched applicants who do not publish.
      • Rivero S.
      • Ippolito J.
      • Martinez M.
      • Beebe K.
      • Benevenia J.
      • Berberian W.
      Analysis of unmatched orthopaedic residency applicants: Options after the match.

      Publish or Perish

      “‘Publish or perish’ is a phrase coined to describe the pressure in academia to rapidly and continually publish academic work to sustain or further one's career. … Frequent publication is one of the few methods at scholars' disposal to demonstrate academic talent. Successful publications bring attention to scholars and their sponsoring institutions, which can facilitate continued funding and an individual's progress through a chosen field. In popular academic perception, scholars who publish infrequently, or who focus on activities that do not result in publications, such as instructing … may lose ground.” The Accreditation Council for Graduate Medical Education (ACGME) also emphasizes publication by both trainees and faculty as a requirement for program accreditation.
      Accreditation Council for Graduate Medical Education. ACGME common program requirements.
      Thus, the incentives for prolific publication include program admission, professorial advancement, ACGME accreditation, fortune, and fame. Taken in sum, these factors sometimes result in quality research, but they promote quantity as the ultimate goal. In addition, bearing in mind that ours is a subspecialty journal and that orthopaedic subspecialty training programs are typically only 12 months in duration, this results in a mandate to publish within a year. One year is too short a time to conduct a prospective trial, necessarily resulting in clinical therapeutic studies of lower levels of evidence and systematic reviews and meta-analyses and basic science investigations necessarily limited in scope. Again, the incentives favor quantity over quality.

      Overwhelming Quantity

      Evidence in support of overwhelming quantity is obvious to your editors. Since January 2017, the rejection rate of Arthroscopy scientific article submissions, based on our peer-review process, has approached 75%. This seems an extreme of quantity at the expense of quality.
      The number of pages we publish annually has exploded, and the journal requires heavy lifting.
      • Lubowitz J.H.
      • Brand J.C.
      • Rossi M.J.
      Quality submissions and editorial commentaries require heavy lifting: Journal growth risks information overload.
      Information overload is a recurring concern.
      • Lubowitz J.H.
      • Brand J.C.
      • Rossi M.J.
      Quality submissions and editorial commentaries require heavy lifting: Journal growth risks information overload.
      • Lubowitz J.H.
      • Brand J.C.
      • Provencher M.T.
      • Rossi M.J.
      Systematic reviews keep arthroscopy up to date.
      • Provencher M.T.
      • Brand J.C.
      • Rossi M.J.
      • Lubowitz J.H.
      Are orthopaedic systematic reviews overly prevalent?.
      There is no question about quantity.

      What Could Be the Solution?

      On identification of a problem, a proactive approach is to develop solutions. When the solution may require alteration of established guidelines or norms, resistance to change is a challenge.
      Nevertheless, we propose that quality research results from strong methods. As we teach in our Arthroscopy journal course for writers and reviewers, Essentials of the Manuscript,
      • Harris J.D.
      • Brand J.C.
      • Cote M.P.
      • Dhawan A.
      Research pearls: The significance of statistics and perils of pooling. Part 3: Pearls and pitfalls of meta-analyses and systematic reviews.
      methods are the most important determinant of quality in medical research, and unlike other aspects of research submissions, fatal flaws in research methods cannot be fixed. We believe that the first step in solving the problem of quantity at the expense of quality is a paradigm shift requiring that the goal of medical scientist and clinician training be to impart expert understanding of research methods rather than requiring that trainees get their names on a robust curriculum vitae of publications. An accompanying paradigm shift could be to weight the metric of scholarly authorship based on quality of publications, for which validated tools exist (see our “Instructions for Authors”), rather than on the number of publications.
      We abhor quantities of low-quality research. The paradigm of publish or perish must pass.

      References

        • Frieden T.R.
        Evidence for health decision making—beyond randomized, controlled trials.
        N Engl J Med. 2017; 377: 465-475
        • Chess L.E.
        • Gagnier J.
        Risk of bias of randomized controlled trials published in orthopaedic journals.
        BMC Med Res Methodol. 2013; 13: 76
        • Buckwalter J.A.
        • Tolo V.T.
        • O’Keefe R.J.
        How do you know it is true? Integrity in research and publications. AOA critical issues.
        J Bone Joint Surg Am. 2015; 97: e2
        • Somford M.P.
        • van Deurzen D.F.
        • Ostedorf M.
        • Eygendaal D.
        • van den Bekerom M.P.
        Quality of research and quality of reporting in elbow surgery trials.
        J Shoulder Elbow Surg. 2015; 24: 1619-1626
      1. National Resident Matching Program.
        (Accessed August 29, 2018)
      2. National Resident Matching Program. Results of the 2016 NRMP program director survey.
        (Accessed August 29, 2018)
        • Rivero S.
        • Ippolito J.
        • Martinez M.
        • Beebe K.
        • Benevenia J.
        • Berberian W.
        Analysis of unmatched orthopaedic residency applicants: Options after the match.
        J Grad Med Educ. 2016; 8: 91-95
      3. Publish or perish.
        (Accessed August 29, 2018)
      4. Accreditation Council for Graduate Medical Education. ACGME common program requirements.
        (Accessed August 29, 2018)
        • Lubowitz J.H.
        • Brand J.C.
        • Rossi M.J.
        Quality submissions and editorial commentaries require heavy lifting: Journal growth risks information overload.
        Arthroscopy. 2018; 34: 1379
        • Lubowitz J.H.
        • Brand J.C.
        • Provencher M.T.
        • Rossi M.J.
        Systematic reviews keep arthroscopy up to date.
        Arthroscopy. 2016; 32: 237
        • Provencher M.T.
        • Brand J.C.
        • Rossi M.J.
        • Lubowitz J.H.
        Are orthopaedic systematic reviews overly prevalent?.
        Arthroscopy. 2016; 32: 955-956
        • Harris J.D.
        • Brand J.C.
        • Cote M.P.
        • Dhawan A.
        Research pearls: The significance of statistics and perils of pooling. Part 3: Pearls and pitfalls of meta-analyses and systematic reviews.
        Arthroscopy. 2017; 33: 1594-1602
      5. (13.)
        Essentials of the manuscript.
        (Accessed August 29, 2018)
      6. Instructions for Authors.
        (Accessed August 29, 2018)