Advertisement

Author's Reply

      We thank Roos, Juhl, and Lohmander for their interest in our article and their remarks. They raise concern about the use of an expert panel for determining a priori hypotheses for the assessment of construct validity and responsiveness. They argue that the expert panel should have considered existing literature when formulating a priori hypotheses. In contrast, we do not think that hypotheses should be based on the results of previous validation studies. Classic psychometric works state that hypotheses should be based on theoretical considerations about expected relations among constructs. Cronbach and Meehl
      • Cronbach L.J.
      • Meehl P.E.
      Construct validity in psychological tests.
      proposed to start from theories about the construct and then formulate hypotheses to test whether the scores of the instrument are consistent with a theoretical model of the construct. Our hypotheses were based on theoretical models such as the International Classification of Functioning and the model of Wilson and Cleary.
      • Wilson I.B.
      • Cleary P.D.
      Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes.
      Based on these theories, we hypothesized that scales that intend to measure similar constructs (e.g., visual analogue scale Pain and KOOS Pain) should correlate at least 0.60, scales that intend to measure closely related constructs (e.g., Short Form 36 physical functioning and KOOS sport/recreation) should correlate between 0.40 and 0.60, and scales that intend to measure different constructs (e.g., Short Form 36 bodily pain and KOOS activities of daily living) should correlate ≤0.40.
      • Cronbach L.J.
      • Meehl P.E.
      Construct validity in psychological tests.
      The expected correlations may not always be similar to those found previously in the literature. That is because when setting up a validation study we are not sure a priori that the instrument is valid (otherwise, the study is unnecessary).
      Regarding content validity, we agree with Roos et al. that the current standard considers the patient to be the expert. However, according to the COSMIN guidelines for good content validity,
      • Mokkink L.B.
      • Terwee C.B.
      • Patrick D.L.
      • et al.
      The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study.
      the items of a questionnaire should be relevant for the construct to be measured, for the population being assessed, and for the purpose for which the instrument is being used. Although we consider the patient to be the expert with regard to relevance for the population, we consider the opinion of clinicians and methodologists of additional value for assessing relevance for the construct and purpose.
      • De Vet H.C.W.
      • Terwee C.B.
      • Mokkink L.B.
      • Knol D.L.
      Measurement in medicine. A practical guide.
      Roos et al. raised their concerns about the conclusion of responsiveness of the KOOS and IKDC. We agree with Roos et al. that the effect sizes for the KOOS reflect the difference in improvement for the 5 domains at follow-up, whereas for the IKDC it is impossible to see in which domain the patients have improved. This is an important advantage of the KOOS. However, our conclusions on responsiveness were based on all hypotheses tested, not only on the effect size.
      Finally, we would like to thank Roos et al. for their attention regarding Table 2. Unfortunately, the correlation between the IKDC subjective and the Lysholm score (0.47) was interpreted as in agreement with the predefined hypothesis of ≥0.6. This incorrect interpretation means that the confirmed hypotheses A should be 67% instead of 83%. However, the confirmed hypotheses A + B are still higher than 75% (17 of 21 = 81% instead of 86%).

      References

        • Cronbach L.J.
        • Meehl P.E.
        Construct validity in psychological tests.
        Psychol Bull. 1955; 52: 281-302
        • Wilson I.B.
        • Cleary P.D.
        Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes.
        JAMA. 1995; 273: 59-65
        • Mokkink L.B.
        • Terwee C.B.
        • Patrick D.L.
        • et al.
        The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study.
        Qual Life Res. 2010; 19: 539-549
        • De Vet H.C.W.
        • Terwee C.B.
        • Mokkink L.B.
        • Knol D.L.
        Measurement in medicine. A practical guide.
        Cambridge University Press, Cambridge2011

      Linked Article

      • Expert Panels: Can They Be Trusted?
        ArthroscopyVol. 29Issue 7
        • Preview
          Van Meer et al.1 recently compared the Knee Injury and Osteoarthritis Outcome Score (KOOS)2 with the International Knee Documentation Committee (IKDC) Subjective Knee Form3 to investigate which of the instruments is most useful after anterior cruciate ligament reconstruction (ACLR). Following the COSMIN (COnsensus-based Standards for the selection of health status Measurement INstruments) checklist, an expert panel reached consensus on a priori hypotheses regarding responsiveness and construct validity.
        • Full-Text
        • PDF