Advertisement

Evaluation of Journal Policies to Increase Promotion of Transparency and Openness in Sport Science Research

      Increased transparency and openness of sport science research will improve the ability to appraise, replicate, and implement research findings. However, sport science journals need to support and prioritize practices that bolster transparency and openness to facilitate this.
      • Ioannidis J.P.
      How to make more published research true.
      ,
      • Munafò M.R.
      • Nosek B.A.
      • Bishop D.V.M.
      • et al.
      A manifesto for reproducible science.
      Journal policies that actively support transparency and openness have been shown to improve the quality, reproducibility, and replicability of research.

      Cashin AG, Bagg MK, Richards GC, Toomey E, McAuley JH, Lee H. Limited engagement with transparent and open science standards in the policies of pain journals: A cross-sectional evaluation [published online January 24, 2020]. BMJ Evid Based Med. https://doi.org/10.1136/bmjebm-2019-111296.

      To date, over 1100 journals and funding agencies have ascribed to implementing 1 or more of the 8 standards for transparent and open research described in the Transparency and Openness Promotion (TOP) guidelines.
      • Nosek B.A.
      • Alter G.
      • Banks G.C.
      • et al.
      Promoting an open research culture.
      We report an appraisal of how well the policies of the leading 38 sport science journals support transparent and open research practices. Arthroscopy was evaluated using the TOP Factor (https://osf.io/t2yu5/)—an alternative metric for evaluating the degree to which journal policies promote transparency and openness
      • Mayo-Wilson E.
      • Grant S.
      • Supplee L.
      • et al.
      Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: The TRUST process for rating journal policies, procedures, and practices.
      —and the International Committee of Medical Journal Editors (ICMJE) disclosure-of-interest requirements.
      International Commitee of Medical Journal Editors
      Conflicts of interest.
      The TOP Factor is calculated as the sum of journal implementation of the 8 modular standards within the TOP guidelines and an additional standard related to publication of replication studies. The requirements for disclosure of conflicts of interest were evaluated using the 4 standards stated on the ICMJE disclosure form.
      International Commitee of Medical Journal Editors
      Conflicts of interest.
      Arthroscopy’s TOP Factor and conflict-of-interest scores are provided in Figures 1 and 2, respectively, alongside summary data from the top 38 sport science journals.
      Figure thumbnail gr1
      Fig 1Summary of scores for each item of Transparency and Openness Promotion (TOP) Factor. The dotted line indicates Arthroscopy’s score for each item, with the color indicating the position relative to the median score of the top 38 sport science journals. Black indicates that the score is equal to the median; red, the score is below the median (worse); and green, the score is above the median (better). For TOP items 1 through 8, a score of 0 indicates not mentioned or “encouraged” by the journal policy; 1, recommended by the journal policy; 2, required by the journal policy; and 3, required and enforced by the journal policy. For item 9, a score of 0 indicates not mentioned by the journal policy; 1, the journal states that significance or novelty is not a criterion for publication; 2, the journal reviews replication studies blinded to results; and 3, the journal accepts registered reports.
      Figure thumbnail gr2
      Fig 2Summary of score for adherence to International Committee of Medical Journal Editors (ICMJE) form for disclosure of potential conflicts of interest. The dotted line indicates the location of Arthroscopy’s score, with the color indicating the position relative to the mean score of the top 38 sport science journals. Black indicates the score is equal to the mean; red, the score is below the mean (worse); and green, the score is above the mean (better).
      In brief, Arthroscopy scored well for TOP Factor item “study preregistration”; scored poorly for items “data citation,” “code citation,” “code transparency,” “data transparency,” “research materials transparency,” “analysis preregistration,” and “replication”; and had a total TOP Factor score of 3 of 27 (a higher score represents increased an requirement for transparency and openness by the journal). Arthroscopy was above the mean TOP Factor score of 2.05 ± 1.99 of 27 and was above the mean ICMJE disclosure-of-conflict score (2.84 ± 1.62 of 4) compared with the top 38 sport science journals.
      We strongly encourage Arthroscopy to review and, where necessary, update the journal’s policies (https://cos.io/top/ provides advice and resources to implement this). Greater transparency and openness will improve the conduct and translation of sport science research to ultimately move the field forward. Moreover, complete and transparent disclosure of conflicts of interest will increase confidence in research findings. By publishing this letter, Arthroscopy is taking action toward improving the transparency and openness of sport science research.

      Supplementary Data

      References

        • Ioannidis J.P.
        How to make more published research true.
        Plos Med. 2014; 2e1001747
        • Munafò M.R.
        • Nosek B.A.
        • Bishop D.V.M.
        • et al.
        A manifesto for reproducible science.
        Nat Hum Behav. 2017; 10021
      1. Cashin AG, Bagg MK, Richards GC, Toomey E, McAuley JH, Lee H. Limited engagement with transparent and open science standards in the policies of pain journals: A cross-sectional evaluation [published online January 24, 2020]. BMJ Evid Based Med. https://doi.org/10.1136/bmjebm-2019-111296.

        • Nosek B.A.
        • Alter G.
        • Banks G.C.
        • et al.
        Promoting an open research culture.
        Science. 2015; 348: 1422-1425
        • Mayo-Wilson E.
        • Grant S.
        • Supplee L.
        • et al.
        Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: The TRUST process for rating journal policies, procedures, and practices.
        Res Integr Peer Rev. 2021; 6: 9
        • International Commitee of Medical Journal Editors
        Conflicts of interest.
        (Accessed June 24, 2021)