- Email: [email protected]
The Evidence-Based Hierarchy: Is It Time For Change? A Suggested Alternative
COMMENTARIES THE EVIDENCE-BASED HIERARCHY: IS IT TIME CHANGE? A SUGGESTED ALTERNATIVE
FOR
P. J. Miller, BSc, DC, MSc, FCC, and A. R. Jones-Harris, BSc, DC, MSc, FCC
E
vidence-based medicine (EBM) can be defined as bThe integration of best research evidence with clinical expertise and patient values.Q 1 When EBM was introduced, it was considered a paradigm shift in the medical approach to patient management.2 It has been widely adopted and has the potential to help facilitate patient choice in medical care by ensuring that the information on which doctors base their explanations, and which patients use to guide their choices, is of the highest possible standard.3 The EBM process has 5 steps for the clinician to follow; these are shown in Fig 1. To determine the best research evidence, studies are commonly graded according to a hierarchy of evidence. The hierarchy is often shown as a pyramid, with systematic reviews and randomized controlled trials (RCTs) at the top and clinician opinion at the base.1 This particular hierarchy has been challenged by a number of authors as being too restrictive in its descriptions, thereby paradoxically potentially hindering best decision making by limiting the type of research evidence considered in the decision making process.4 - 6 Furthermore, there is no provision made for qualitative research evidence.4,6 - 8 This article offers an alternative to the existing hierarchies that is based upon the nature of the original clinical question. This alternative is presented as a set of short evidence pathways, which is designed to be simple to use and would hopefully facilitate use in practice. It also allows for the inclusion of critically appraised qualitative data in the evidence-informed choices that clinicians make, and may therefore be better suited to the diversities of clinical practice. With the exception of the qualitative data evidence pathways, the ideas presented here are not new but adaptations of existing models that have been simplified to encourage their use by clinicians. Submit requests for reprints to: P. J. Miller, BSc, DC, MSc, FCC, AECC, 13-15 Parkwood Road, Bournemouth, BH5 2DF, UK (e-mail: [email protected]). J Manipulative Physiol Ther 2005;28:454-458. 0161- 4754/$30.00 Copyright D 2005 by National University of Health Sciences. doi:10.1016/j.jmpt.2005.06.010
WHAT IS IN A QUESTION? The first step in EBM is formulating an answerable clinical question based on the perceived knowledge need. These questions fall into 1 of 8 categories as shown in Fig 2. Because these questions are all different in nature, they require different types of research evidence to best answer them. Thus, the idea of one evidence-based hierarchy to encompass them all has been questioned.4,5,9 - 12 One of the problems with the current hierarchy model is the exclusion of qualitative data in the evidence-based decision making process. This problem has been recognized before13 - 19 and has become more pressing with the recent increase in popularity of qualitative research methods.20 The profile of qualitative data has also been increased by the development of the evidence-based patient choice model of patient care in which the external evidence is assessed by the patient and the practitioner together and the decision making process regarding therapy decisions is shared.21 The evidence-based patient choice model requires that the practitioner understands the patients perspective and experience of their condition; the best type of research evidence for this type of information is gained from qualitative studies.22,23 The authors believe that the evidence-based hierarchy needs to be changed. This change must reflect that the nature of posed clinical question determines the external evidence most appropriate to answer that clinical question. Any alternative offered needs to be simple to follow so that clinicians could use it as a bready reckonerQ on their desktop. This would remove at least one of the perceived barriers to the implementation of EBM through helping the practitioner find the best research evidence with which to answer their particular clinical question.24,25 The devised evidence pathways were designed based on the Centre for Evidence-Based Medicine detailed tables of evidence 26 and on the qualitative research literature.13,15,16,18,22,27 - 31 The usefulness of any evidence pathway is limited if used alone because it is vitally important to follow with appraisal the external research literature (Fig 1, step 3) before its use in the decision making process, rather than ranking it solely by any hierarchy. As Greenhalgh 32 says, b. . .not even the most hard line protagonist of EBM would 453
454
Miller and Jones-Harris Commentary
Journal of Manipulative and Physiological Therapeutics July/August 2005
Fig 1. Five steps of EBM based on Sackett et al.1
Fig 2. Clinical question categories based upon works by various authors.19,40-42 place a sloppy meta-analysis or a RCT that was seriously methodologically flawed above a large, well designed cohort study.Q
SYSTEMATIC REVIEWS The role of the systematic review is well established within the quantitative paradigm as a means of improving credibility and generalizability of research data and distilling these data into a format that is easy to access and understand by the clinician.33-37 They are considered the highest level of evidence and are often used as the basis for health-care decisions from government level down.35 To date, these reviews have focused predominantly on effectiveness of treatment interventions and so have been limited to RCTs.35 A systematic review of qualitative data has been attempted,38 and a framework for such systematic reviews is under development.13-16,35,39 However, even the development of a qualitative systematic review framework is controversial because of the context sensitive and highly interpretive nature of the data. The evidence pathways presented in this article use the idea of systematic reviews for both the quantitative and qualitative paradigms in the expectation that acceptable frameworks for systematic reviews of the qualitative literature will be formulated and published.
EVIDENCE PATHWAYS The evidence pathways presented here stem from the 8 types of original clinical questions listed in Fig 2. For convenience, clinical questions that share research evidence pathways are grouped together; these pathways are shown in Fig 3. The clinical question is the focus of the pathways, and the research evidence appropriate to that type of clinical question is ranked according to the strength of evidence they provide. The best evidence available is ranked as gold; the next best evidence is ranked as silver, and the bronze ranking covers the weakest evidence. The ranking of the type of evidence assumes that each is of good quality and that each individual piece of research should be assessed for methodological quality before deciding upon its importance in determining patient care (Fig 1, step 3). The qualitative section of the pathways (outlined with a box around that section in Fig 3) should be considered in the light of considerable overlap between the research types. For instance, if the clinical question was bWhat it is the experience of suffering from a migraine?Q it is likely that phenomenological grounded theory and ethnographic methodologies could be used to answer the question. This is because, if a researcher asks a group about their migraines, this group of bmigraneursQ could be considered
Journal of Manipulative and Physiological Therapeutics Volume 28, Number 6
Miller and Jones-Harris Commentary
Fig 3. Evidence pathways. Gold is considered the highest level of evidence. Silver is considered good level of evidence. Bronze is considered the weakest level of evidence. The area outlined with box represents the qualitative methodologies. a cultural subgroup, and therefore, the results could be published as an ethnography article. Likewise, if a researcher sets out to develop a theory about persons with
migraine, the process may well uncover important information regarding the lived experience of a migraine but be published as a grounded theory article. These
455
456
Miller and Jones-Harris Commentary
papers would be missed if the clinician limited their literature search (Fig 1, step 2) to phenomenology papers only. However, because phenomenology papers on this subject set out to discover lived experience, it is more likely that the information gained from phenomenology papers would be more thorough and relevant. Therefore, it is still worth looking for phenomenology papers first when trying to answer a clinical question regarding lived experience.
CONCLUSION These pathways may help to guide the user to the most appropriate research for the type of clinical question being asked. They are not all-inclusive and are kept simple in an effort to make them easy to use. It is hoped that the presentation of these pathways will stimulate further dialog in the health-care professions as to the role of qualitative research methodologies in EBM.
REFERENCES 1. Sackett DL, Straus S, Richardson WS, Rosenberg WMC, Haynes B. Evidence-based medicine: how to practice and teach EBM. Edinburgh7 Churchill Livingstone; 2000. 2. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420 - 5. 3. Coulter A, Entwistle VA, Gilbert D. Sharing decisions with patients: is the information good enough. BMJ 1999; 318:318 - 22. 4. Buetow S, Kenealy T. Evidence-based medicine: the need for a new definition. J Eval Clin Pract 2000;6:85 - 92. 5. Buetow S. Beyond evidence-based medicine: bridge-building a medicine of meaning. J Eval Clin Pract 2002;8:103 - 8. 6. Tickle-Degnen L, Bedell G. Heterarchy and hierarchy: a critical appraisal of the blevels of evidenceQ as a tool for clinical decision making. Am J Occup Ther 2003;57:234 - 7. 7. Tsafrir J, Grinberg M. Who needs evidence-based health care? Bull Med Libr Assoc 1998;86:40 - 5. 8. Bolton J. The evidence in evidence-based practice: what counts and what doesn’t count. J Manipulative Physiol Ther 2001; 24:362 - 6. 9. Petticrew M, Roberts H. Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health 2003;57: 527 - 9. 10. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ 2003;328:39 - 41. 11. Little M. bBetter than numbers. . .Q A gentle critique of evidence-based medicine. Aust NZ J Surg 2003;73:177 - 82. 12. Clarke JB. Evidence-based practice: a retrograde step? The importance of pluralism in evidence generation for the practice of health care. J Clin Nurs 1999;8:89 - 94. 13. Barbour RS, Barbour M. Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract 2003;9:179 - 86. 14. Evans D. Systematic reviews of interpretive research: interpretive data synthesis of processed data. Aust J Adv Nurs 2002;20:22 - 6.
Journal of Manipulative and Physiological Therapeutics July/August 2005
15. Dixon-Woods M, Fitzpatrick R. Qualitative research in systematic reviews: has established a place for itself. BMJ 2001;323:765 - 6. 16. Dixon-Woods M, Fitzpatrick R, Roberts K. Including qualitative research in systematic reviews: opportunities and problems. J Eval Clin Pract 2001;7:125 - 33. 17. Rosswurm MA, Larrabee JH. A model for change to evidencebased practice. Image J Nurs Sch 1999;31:317 - 22. 18. Sandelowski M, Docherty S, Emden C. Focus on qualitative methods. Qualitative metasynthesis: issues and techniques. Res Nurs Health 1997;20:365 - 71. 19. Leys M. Health care policy: qualitative evidence and health technology assessment. Health Policy 2003;65:217 - 26. 20. Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ 2000;320:50 - 2. 21. Hope T. Evidence-based patient choice. London7 Kings Fund; 1996. 22. Greenhalgh T, Taylor R. How to read a paper: papers that go beyond numbers (qualitative research). BMJ 1997;315:740 - 3. 23. Bolton J. Research design in chiropractic. J Manipulative Physiol Ther 1999;22:483 - 7. 24. Haynes B, Haines A. Barriers and bridges to evidence based clinical practice. BMJ 1998;317:273 - 6. 25. Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Vinson DC, Stevermer JJ, et al. Obstacles to answering doctors’ questions about patient care with evidence: qualitative study. BMJ 2002; 324:710. 26. Philips B, Ball C, Sackett DL, Badenoch D, Straus S, Haynes B, et al. Levels of evidence and grades of recommendation [monograph on the Internet]. Oxford7 Centre for EvidenceBased Medicine; 2001 Available from: http://www.cebm.net/ levels_of_evidence.asp. 27. Bigby M. Challenges to the hierarchy of evidence: does the emperor have no clothes? Arch Dermatol 2001;137:345 - 6. 28. Eakin JM, Mykhalovskiy E. Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract 2003; 9:187 - 94. 29. Greenhalgh T. Narrative based medicine: narrative based medicine in an evidence based world. BMJ 1999;318:323 - 5. 30. Greenhalgh T, Hurwitz B. Narrative based medicine: why study narrative? BMJ 1999;318:48 - 50. 31. Hoddinott P, Pill R. A review of recently published qualitative research in general practice. More methodological questions than answers? Fam Pract 1997;14:313 - 9. 32. Greenhalgh T. How to read a paper: the basics of evidencebased medicine. London7 BMJ Books, 2001 p. 39 - 55. 33. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C. Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999;28: 1 - 9. 34. Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs 2003; 12:77 - 84. 35. Evans D, Pearson A. Systematic reviews: gatekeepers of nursing knowledge. J Clin Nurs 2001;10:593 - 9. 36. Finkel ML, Brown H-A, Gerber LM, Supino PG. Teaching evidence-based medicine to medical students. Med Teach 2003;25:202 - 4. 37. Schunemann HJ, Best D, Vist G, Oxman AD. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. Can Med Assoc J 2003;169: 677 - 80 38. Garcia J, Bricker L, Henderson J, Martin MA, Mugford M, Nielson J, et al. Women’s views of pregnancy ultrasound: a systematic review. Birth 2002;29:225 - 50.
Journal of Manipulative and Physiological Therapeutics Volume 28, Number 6
39. Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res 1998;8:341 - 51. 40. Djulbegovic B, Morris L, Lyman GH. Evidentiary challenges to evidence-based medicine. J Eval Clin Pract 2000;6: 99 - 109.
Miller and Jones-Harris Commentary
41. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and teach EBM. 2nd ed. Edinburgh7 Churchill Livingstone; 2000. p. 29 - 65. 42. Miles A, Charlton B, Ploychronis A, Grey J, Price N. New perspectives in the evidence-based healthcare debate. J Eval Clin Pract 2000;6:77 - 84.
457
FOR
P. J. Miller, BSc, DC, MSc, FCC, and A. R. Jones-Harris, BSc, DC, MSc, FCC
E
vidence-based medicine (EBM) can be defined as bThe integration of best research evidence with clinical expertise and patient values.Q 1 When EBM was introduced, it was considered a paradigm shift in the medical approach to patient management.2 It has been widely adopted and has the potential to help facilitate patient choice in medical care by ensuring that the information on which doctors base their explanations, and which patients use to guide their choices, is of the highest possible standard.3 The EBM process has 5 steps for the clinician to follow; these are shown in Fig 1. To determine the best research evidence, studies are commonly graded according to a hierarchy of evidence. The hierarchy is often shown as a pyramid, with systematic reviews and randomized controlled trials (RCTs) at the top and clinician opinion at the base.1 This particular hierarchy has been challenged by a number of authors as being too restrictive in its descriptions, thereby paradoxically potentially hindering best decision making by limiting the type of research evidence considered in the decision making process.4 - 6 Furthermore, there is no provision made for qualitative research evidence.4,6 - 8 This article offers an alternative to the existing hierarchies that is based upon the nature of the original clinical question. This alternative is presented as a set of short evidence pathways, which is designed to be simple to use and would hopefully facilitate use in practice. It also allows for the inclusion of critically appraised qualitative data in the evidence-informed choices that clinicians make, and may therefore be better suited to the diversities of clinical practice. With the exception of the qualitative data evidence pathways, the ideas presented here are not new but adaptations of existing models that have been simplified to encourage their use by clinicians. Submit requests for reprints to: P. J. Miller, BSc, DC, MSc, FCC, AECC, 13-15 Parkwood Road, Bournemouth, BH5 2DF, UK (e-mail: [email protected]). J Manipulative Physiol Ther 2005;28:454-458. 0161- 4754/$30.00 Copyright D 2005 by National University of Health Sciences. doi:10.1016/j.jmpt.2005.06.010
WHAT IS IN A QUESTION? The first step in EBM is formulating an answerable clinical question based on the perceived knowledge need. These questions fall into 1 of 8 categories as shown in Fig 2. Because these questions are all different in nature, they require different types of research evidence to best answer them. Thus, the idea of one evidence-based hierarchy to encompass them all has been questioned.4,5,9 - 12 One of the problems with the current hierarchy model is the exclusion of qualitative data in the evidence-based decision making process. This problem has been recognized before13 - 19 and has become more pressing with the recent increase in popularity of qualitative research methods.20 The profile of qualitative data has also been increased by the development of the evidence-based patient choice model of patient care in which the external evidence is assessed by the patient and the practitioner together and the decision making process regarding therapy decisions is shared.21 The evidence-based patient choice model requires that the practitioner understands the patients perspective and experience of their condition; the best type of research evidence for this type of information is gained from qualitative studies.22,23 The authors believe that the evidence-based hierarchy needs to be changed. This change must reflect that the nature of posed clinical question determines the external evidence most appropriate to answer that clinical question. Any alternative offered needs to be simple to follow so that clinicians could use it as a bready reckonerQ on their desktop. This would remove at least one of the perceived barriers to the implementation of EBM through helping the practitioner find the best research evidence with which to answer their particular clinical question.24,25 The devised evidence pathways were designed based on the Centre for Evidence-Based Medicine detailed tables of evidence 26 and on the qualitative research literature.13,15,16,18,22,27 - 31 The usefulness of any evidence pathway is limited if used alone because it is vitally important to follow with appraisal the external research literature (Fig 1, step 3) before its use in the decision making process, rather than ranking it solely by any hierarchy. As Greenhalgh 32 says, b. . .not even the most hard line protagonist of EBM would 453
454
Miller and Jones-Harris Commentary
Journal of Manipulative and Physiological Therapeutics July/August 2005
Fig 1. Five steps of EBM based on Sackett et al.1
Fig 2. Clinical question categories based upon works by various authors.19,40-42 place a sloppy meta-analysis or a RCT that was seriously methodologically flawed above a large, well designed cohort study.Q
SYSTEMATIC REVIEWS The role of the systematic review is well established within the quantitative paradigm as a means of improving credibility and generalizability of research data and distilling these data into a format that is easy to access and understand by the clinician.33-37 They are considered the highest level of evidence and are often used as the basis for health-care decisions from government level down.35 To date, these reviews have focused predominantly on effectiveness of treatment interventions and so have been limited to RCTs.35 A systematic review of qualitative data has been attempted,38 and a framework for such systematic reviews is under development.13-16,35,39 However, even the development of a qualitative systematic review framework is controversial because of the context sensitive and highly interpretive nature of the data. The evidence pathways presented in this article use the idea of systematic reviews for both the quantitative and qualitative paradigms in the expectation that acceptable frameworks for systematic reviews of the qualitative literature will be formulated and published.
EVIDENCE PATHWAYS The evidence pathways presented here stem from the 8 types of original clinical questions listed in Fig 2. For convenience, clinical questions that share research evidence pathways are grouped together; these pathways are shown in Fig 3. The clinical question is the focus of the pathways, and the research evidence appropriate to that type of clinical question is ranked according to the strength of evidence they provide. The best evidence available is ranked as gold; the next best evidence is ranked as silver, and the bronze ranking covers the weakest evidence. The ranking of the type of evidence assumes that each is of good quality and that each individual piece of research should be assessed for methodological quality before deciding upon its importance in determining patient care (Fig 1, step 3). The qualitative section of the pathways (outlined with a box around that section in Fig 3) should be considered in the light of considerable overlap between the research types. For instance, if the clinical question was bWhat it is the experience of suffering from a migraine?Q it is likely that phenomenological grounded theory and ethnographic methodologies could be used to answer the question. This is because, if a researcher asks a group about their migraines, this group of bmigraneursQ could be considered
Journal of Manipulative and Physiological Therapeutics Volume 28, Number 6
Miller and Jones-Harris Commentary
Fig 3. Evidence pathways. Gold is considered the highest level of evidence. Silver is considered good level of evidence. Bronze is considered the weakest level of evidence. The area outlined with box represents the qualitative methodologies. a cultural subgroup, and therefore, the results could be published as an ethnography article. Likewise, if a researcher sets out to develop a theory about persons with
migraine, the process may well uncover important information regarding the lived experience of a migraine but be published as a grounded theory article. These
455
456
Miller and Jones-Harris Commentary
papers would be missed if the clinician limited their literature search (Fig 1, step 2) to phenomenology papers only. However, because phenomenology papers on this subject set out to discover lived experience, it is more likely that the information gained from phenomenology papers would be more thorough and relevant. Therefore, it is still worth looking for phenomenology papers first when trying to answer a clinical question regarding lived experience.
CONCLUSION These pathways may help to guide the user to the most appropriate research for the type of clinical question being asked. They are not all-inclusive and are kept simple in an effort to make them easy to use. It is hoped that the presentation of these pathways will stimulate further dialog in the health-care professions as to the role of qualitative research methodologies in EBM.
REFERENCES 1. Sackett DL, Straus S, Richardson WS, Rosenberg WMC, Haynes B. Evidence-based medicine: how to practice and teach EBM. Edinburgh7 Churchill Livingstone; 2000. 2. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992;268:2420 - 5. 3. Coulter A, Entwistle VA, Gilbert D. Sharing decisions with patients: is the information good enough. BMJ 1999; 318:318 - 22. 4. Buetow S, Kenealy T. Evidence-based medicine: the need for a new definition. J Eval Clin Pract 2000;6:85 - 92. 5. Buetow S. Beyond evidence-based medicine: bridge-building a medicine of meaning. J Eval Clin Pract 2002;8:103 - 8. 6. Tickle-Degnen L, Bedell G. Heterarchy and hierarchy: a critical appraisal of the blevels of evidenceQ as a tool for clinical decision making. Am J Occup Ther 2003;57:234 - 7. 7. Tsafrir J, Grinberg M. Who needs evidence-based health care? Bull Med Libr Assoc 1998;86:40 - 5. 8. Bolton J. The evidence in evidence-based practice: what counts and what doesn’t count. J Manipulative Physiol Ther 2001; 24:362 - 6. 9. Petticrew M, Roberts H. Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health 2003;57: 527 - 9. 10. Glasziou P, Vandenbroucke J, Chalmers I. Assessing the quality of research. BMJ 2003;328:39 - 41. 11. Little M. bBetter than numbers. . .Q A gentle critique of evidence-based medicine. Aust NZ J Surg 2003;73:177 - 82. 12. Clarke JB. Evidence-based practice: a retrograde step? The importance of pluralism in evidence generation for the practice of health care. J Clin Nurs 1999;8:89 - 94. 13. Barbour RS, Barbour M. Evaluating and synthesizing qualitative research: the need to develop a distinctive approach. J Eval Clin Pract 2003;9:179 - 86. 14. Evans D. Systematic reviews of interpretive research: interpretive data synthesis of processed data. Aust J Adv Nurs 2002;20:22 - 6.
Journal of Manipulative and Physiological Therapeutics July/August 2005
15. Dixon-Woods M, Fitzpatrick R. Qualitative research in systematic reviews: has established a place for itself. BMJ 2001;323:765 - 6. 16. Dixon-Woods M, Fitzpatrick R, Roberts K. Including qualitative research in systematic reviews: opportunities and problems. J Eval Clin Pract 2001;7:125 - 33. 17. Rosswurm MA, Larrabee JH. A model for change to evidencebased practice. Image J Nurs Sch 1999;31:317 - 22. 18. Sandelowski M, Docherty S, Emden C. Focus on qualitative methods. Qualitative metasynthesis: issues and techniques. Res Nurs Health 1997;20:365 - 71. 19. Leys M. Health care policy: qualitative evidence and health technology assessment. Health Policy 2003;65:217 - 26. 20. Mays N, Pope C. Qualitative research in health care. Assessing quality in qualitative research. BMJ 2000;320:50 - 2. 21. Hope T. Evidence-based patient choice. London7 Kings Fund; 1996. 22. Greenhalgh T, Taylor R. How to read a paper: papers that go beyond numbers (qualitative research). BMJ 1997;315:740 - 3. 23. Bolton J. Research design in chiropractic. J Manipulative Physiol Ther 1999;22:483 - 7. 24. Haynes B, Haines A. Barriers and bridges to evidence based clinical practice. BMJ 1998;317:273 - 6. 25. Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Vinson DC, Stevermer JJ, et al. Obstacles to answering doctors’ questions about patient care with evidence: qualitative study. BMJ 2002; 324:710. 26. Philips B, Ball C, Sackett DL, Badenoch D, Straus S, Haynes B, et al. Levels of evidence and grades of recommendation [monograph on the Internet]. Oxford7 Centre for EvidenceBased Medicine; 2001 Available from: http://www.cebm.net/ levels_of_evidence.asp. 27. Bigby M. Challenges to the hierarchy of evidence: does the emperor have no clothes? Arch Dermatol 2001;137:345 - 6. 28. Eakin JM, Mykhalovskiy E. Reframing the evaluation of qualitative health research: reflections on a review of appraisal guidelines in the health sciences. J Eval Clin Pract 2003; 9:187 - 94. 29. Greenhalgh T. Narrative based medicine: narrative based medicine in an evidence based world. BMJ 1999;318:323 - 5. 30. Greenhalgh T, Hurwitz B. Narrative based medicine: why study narrative? BMJ 1999;318:48 - 50. 31. Hoddinott P, Pill R. A review of recently published qualitative research in general practice. More methodological questions than answers? Fam Pract 1997;14:313 - 9. 32. Greenhalgh T. How to read a paper: the basics of evidencebased medicine. London7 BMJ Books, 2001 p. 39 - 55. 33. Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C. Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999;28: 1 - 9. 34. Evans D. Hierarchy of evidence: a framework for ranking evidence evaluating healthcare interventions. J Clin Nurs 2003; 12:77 - 84. 35. Evans D, Pearson A. Systematic reviews: gatekeepers of nursing knowledge. J Clin Nurs 2001;10:593 - 9. 36. Finkel ML, Brown H-A, Gerber LM, Supino PG. Teaching evidence-based medicine to medical students. Med Teach 2003;25:202 - 4. 37. Schunemann HJ, Best D, Vist G, Oxman AD. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. Can Med Assoc J 2003;169: 677 - 80 38. Garcia J, Bricker L, Henderson J, Martin MA, Mugford M, Nielson J, et al. Women’s views of pregnancy ultrasound: a systematic review. Birth 2002;29:225 - 50.
Journal of Manipulative and Physiological Therapeutics Volume 28, Number 6
39. Popay J, Rogers A, Williams G. Rationale and standards for the systematic review of qualitative literature in health services research. Qual Health Res 1998;8:341 - 51. 40. Djulbegovic B, Morris L, Lyman GH. Evidentiary challenges to evidence-based medicine. J Eval Clin Pract 2000;6: 99 - 109.
Miller and Jones-Harris Commentary
41. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based medicine: how to practice and teach EBM. 2nd ed. Edinburgh7 Churchill Livingstone; 2000. p. 29 - 65. 42. Miles A, Charlton B, Ploychronis A, Grey J, Price N. New perspectives in the evidence-based healthcare debate. J Eval Clin Pract 2000;6:77 - 84.
457