Research Pearls: Expert Consensus Based Evidence Using the Delphi Method


      The evolution of a systematic approach to assessing pertinent investigations is known as evidence-based medicine (EBM). EBM is defined as the conscientious and judicious use of current best evidence from clinical care research and integration of clinical expertise in the management of individual patients. There is no doubt that EBM is important but may not give clinically meaningful guidance on topics with clinical equipoise for individual patient care. When EBM has been insufficiently developed for a specific topic, a consensus opinion of experts can be valuable. In principle, there are 2 consensus methods for expert opinion available: the nominal group technique and the Delphi method. The nominal group technique is a structured face-to-face meeting facilitating discussion and allows participants to voice their opinions. The key characteristics of the Delphi method are the use of panel experts to obtain data, no face-to-face discussions, the use of sequential questionnaires, the systematic emergence of a concurrent opinion, use of frequency distributions to identify patterns, and the use of at least 2 rounds with feedback between rounds. We should not dismiss the collective experience of our leading experts, and expert consensus-based evidence should be explored as another tool to improve the quality of treatment for our patients.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Arthroscopy
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Spindler K.P.
        • Kuhn J.E.
        • Dunn W.
        • Matthews C.E.
        • Harrell F.E.
        • Dittus R.S.
        Reading and reviewing the orthopaedic literature: A systematic, evidence-based medicine approach.
        J Am Acad Orthop Surg. 2005; 13: 220-229
        • Barbier O.
        • Hoogmartens M.
        Evidence-based medicine in orthopaedics.
        Acta Orthop Belg. 2004; 70: 91-97
        • Strauss S.E.
        • Sackett D.L.
        Applying evidence to the individual patient.
        Ann Oncol. 1999; 10: 29-32
        • Hohmann E.
        • Feldman M.
        • Hunt T.J.
        • Cote M.
        • Brand J.C.
        Research Pearls: How do we establish the level of evidence?.
        Arthroscopy. 2018; 34: 3271-3277
        • Sackett D.L.
        • Rosenberg W.M.
        • Gray J.A.
        • Haynes R.B.
        • Richardson W.S.
        Evidence-based medicine: What is it and what it isn’t.
        BMJ. 1996; 312: 71-72
        • O’Rourke K.
        An historical perspective on meta-analysis: Dealing quantitatively with varying study results.
        J R Soc Med. 2007; 100: 579-582
        • Rosenberg W.
        • Donald A.
        Evidence-based medicine: An approach to clinical problem-solving.
        BMJ. 1995; 310: 1122-1126
        • Marx R.G.
        • Wilson S.M.
        • Swiontowski M.F.
        Updating the assignment of levels of evidence.
        J Bone Joint Surg. 2015; 97: 1-2
        • Hohmann E.
        • Brand J.C.
        • Rossi M.J.
        • Lubowitz J.H.
        Expert opinion is necessary: Delphi panel methodology facilitates a scientific approach to consensus.
        Arthroscopy. 2018; 34: 349-351
        • Haidich A.B.
        Meta-analysis in medical research.
        Hippokratia. 2010; 14: 29-37
        • Murad M.H.
        • Asi N.
        • Alsawas M.
        • Alahdab F.
        New evidence pyramid.
        Evid Based Med. 2016; 21: 125-127
        • Harris J.D.
        • Brand J.C.
        • Cote M.P.
        • Dhawan A.
        Research Pearls: The significance of statistics and perils of pooling. Part 3: Pearls and pitfalls of meta-analysis and systematic reviews.
        Arthroscopy. 2017; 33: 1594-1602
        • Eysenck H.J.
        Meta-analysis and its problems.
        BMJ. 1994; 309: 789-792
        • Eysenck H.J.
        An exercise in mega silliness.
        Am Psychol. 1978; 33: 517
        • Sharpe D.
        Of apples and oranges, file drawers and garbage: Why validity issues in meta-analysis will not go away.
        Clin Psychol Rev. 1997; 17: 881-901
        • Thompson S.G.
        • Pocock S.J.
        Can meta-analysis be trusted?.
        Lancet. 1991; 338: 1127-1130
        • Ioannidis J.P.A.
        The mass production of redundant, misleading and conflicting systematic reviews and meta-analysis.
        Milbank Q. 2016; 94: 485-514
        • Tonnelli M.R.
        In defense of expert opinion.
        Acad Med. 1999; 74: 1187-1192
        • Tonnelli M.R.
        Integrating evidence into clinical practice: An alternative to evidence-based approaches.
        J Eval Clin Pract. 2006; 12: 248-256
        • Hampton J.R.
        Evidence-based medicine, opinion-based medicine, and real-world medicine.
        Perspect Biol Med. 2002; 54: 549-568
        • Jones J.
        • Hunter D.
        Consensus methods for medical and health services research.
        BMJ. 1995; 311: 376-380
        • McMillan S.S.
        • Kelly F.
        • Sav A.
        • et al.
        Using the nominal group technique: How to analyse across multiple groups.
        Health Serv Outcomes Res Method. 2014; 14: 92-108
        • McMillan S.S.
        • King M.
        • Tully M.P.
        How to use the nominal group and Delphi techniques.
        Int J Clin Pharm. 2016; 38: 655-662
        • Habibi A.
        • Sarafrazi A.
        • Izadyar S.
        Delphi technique theoretical framework in qualitative research.
        Int J Eng Science. 2014; 3: 8-13
        • Sandrey M.A.
        • Bulger S.M.
        The Delphi method. An approach for facilitatingevidecne based practice in athletic training.
        Athl Train Edu J. 2008; 3: 135-142
        • Powell C.
        The Delphi techniques: Myth and realities.
        J Adv Nursing. 2003; 41: 376-382
        • Keeney S.
        • Hasson F.
        • McKenna H.
        Consulting the oracle: Ten lessons learned from using the Delphi technique in nursing research.
        J Adv Nurs. 2006; 53: 205-212
        • Eubank B.H.
        • Mohtadi N.G.
        • Lafave M.R.
        • et al.
        Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology.
        BMC Med Res Methodol. 2016; 16: 56
        • Dalkey N.
        • Helmer O.
        An experimental application of the Delphi method to the use of experts.
        Manage Sci. 1963; 9: 458-467
        • Dalkey N.C.
        The Delphi method: An experimental study of group opinion. The Rand Corporation. RM-5888-PR 1969.
        (Accessed October 23, 2018)
        • McKenna H.P.
        The Delphi technique: A worthwhile research approach for nursing?.
        J Adv Nurs. 1994; 19: 1221-1225
        • Loughlin K.G.
        • Moore L.F.
        Using Delphi to achieve congruent objectives and activities in a paediatrics department.
        J Med Edu. 1979; 54: 101-106
        • Whitman N.I.
        The committee alternative: Using the Delphi technique.
        J Nurse Admin. 1990; 20: 30-36
        • Hasson F.
        • Keeney S.
        • McKenna H.
        Research guidelines for the Delphi survey.
        J Adv Nurs. 2000; 32: 1008-1015
        • Turoff M.
        The design of a policy Delphi.
        Techn Forecast Social Change. 1970; 2: 149-171
        • De Villiers R.
        • De Vielliers P.J.T.
        • Kent A.P.
        The Delphi technique in health science education research.
        Med Teacher. 2005; 27: 639-643
        • Hsu C.C.
        The Delphi technique: Making sense of consensus.
        Pract Assess Res Eval. 2007; 12: 1-8
        • Okoli C.
        • Pawlowski S.D.
        The Delphi method as a research tool: An example, design considerations and applications.
        Inform Manage. 2004; 42: 15-29
        • Norman G.
        Likert scales, levels of measurements and the “laws” of statistics.
        Adv Health Sci Educ Theory Pract. 2010; 15: 625-632
        • Delbecq A.L.
        • Van de Ven A.H.
        • Gustafson D.H.
        Group techniques for program planning: A guide to nominal group and Delphi processes.
        Scott, Foresman and Company, Glenview, IL1975
        • Linstone H.A.
        • Turoff M.
        The Delphi method techniques and applications.
        Addison-Wesley, Boston, MA1975
        • Goodman C.M.
        The Delphi technique a critique.
        J Adv Nurs. 1987; 12: 729-734
        • Rattray J.
        • Jones M.C.
        Essential elements of questionnaires design and development.
        J Clin Nurs. 2007; 16: 234-243
        • Sumsion T.
        The Delphi technique: An adaptive research tool.
        Br J Occup Ther. 1998; 61: 153-156
        • Green B.
        • Jones M.
        • Hughes D.
        • Williams A.
        Applying the Delphi technique in a study of GP’s information requirements.
        Health Social Care Comm. 1999; 7: 198-205
        • Rennie D.
        Consensus statements.
        New Eng J Med. 1981; 304: 665-666
        • Boulkedid R.
        • Abdoul H.
        • Lousta M.
        • Sibony O.
        • Alberti C.
        Using and reporting the Delphi method for selecting healthcare quality indicators: A systematic review.
        PLoS One. 2011; 6: e20476
        • Lee A.C.C.
        • Cousens S.
        • Wall S.N.
        • et al.
        Neonatal resuscitation and immediate newborn assessment and stimulation for the prevention of neonatal deaths: A systematic review, meta-analysis and Delphi estimation of mortality effect.
        BMC Public Health. 2011; 11: S12
        • Stiller K.
        It’s not the evidence, it’s the way you use it: Is clinical practice being tyrannised by evidence?.
        Austral Health Rev. 2008; 32: 204-207