- Email: [email protected]
A practitioner's guide to developing critical appraisal skills
RESEARCH
A practitioner’s guide to developing critical appraisal skills The fundamentals of research Elliot Abt, DDS, MS, MSc; Bruce L. Pihlstrom, DDS, MS
Editor’s note: This is the first in a series of articles summarizing research methods and design, research translation, statistical analysis and critical appraisal of the scientific literature. The purpose of this series is to help practitioners enhance their understanding of research and develop skills in interpreting research in the medical and dental literature.
lthough there are many different definitions of research, the definition applied in this series appears in the U.S. Code of Federal Regulations: “research means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.”1 This definition emphasizes the idea that research involves systematic investigation. As such, it involves specific protocols and methods of analysis. Research commonly is divided into primary and secondary research. Primary research consists
A
AB STRACT Background and Overview. This is the first of a series of articles summarizing the basic principles and methods of research with an emphasis on clinical investigation. In today’s environment of evidence-based practice, the prevention of, diagnosis of, prognosis for and treatment of oral diseases must be based on a critical evaluation of available research. The purpose of this series is to help practitioners understand research and develop skills in interpreting research published in the medical and dental literature. Conclusions and Practice Implications. Research involves systematic investigation and can be categorized in different ways. Primary research may take several forms and includes basic and applied research. Basic research seeks fundamental knowledge without having a specific application; applied research is conducted for the purpose of a specific application and often is predicated on basic research findings. Applied research may take several forms and includes clinical research, which seeks to inform patients, clinicians, public health workers and policymakers about methods of preventing and treating diseases. Secondary research involves the use of existing data and results of published scientific studies. Systematic literature reviews are a form of secondary research that attempt to remove bias often associated with narrative reviews. Key Words. Evidence-based dentistry; research; primary research; clinical trials; secondary research; narrative reviews; systematic reviews. JADA 2012;143(1):54-56.
Dr. Abt is an attending staff member, Department of Dentistry, Advocate Illinois Masonic Medical Center, Chicago. He also maintains a private practice in general dentistry in Skokie, Ill. Address reprint requests to Dr. Abt at Department of Dentistry, Advocate Illinois Masonic Medical Center, 811 W. Wellington Ave., Chicago, Ill. 60657, e-mail “[email protected]”. Dr. Pihlstrom is a professor emeritus, Department of Surgical and Developmental Sciences, School of Dentistry, University of Minnesota, Minneapolis. He also is the associate editor, Research, and the JournalScan section editor for The Journal of the American Dental Association, as well as an independent oral health research consultant.
54 JADA 143(1)
http://jada.ada.org
January 2012
Copyright © 2012 American Dental Association. All rights reserved.
RESEARCH
of basic and applied research, whereas secondary research refers to compilations of existing data. This article briefly describes common types of research involved in dentistry and medicine, as well as reviews that are conducted to summarize existing research (Box). Additional articles in this series are planned for publication in The Journal of the American Dental Association. PRIMARY RESEARCH
Basic and applied research. Basic research. The National Science Foundation2 defines basic research as “systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind.” For example, original research that defines the structure of atoms, ions and molecules would be considered basic research, as it expands our fundamental knowledge and understanding of matter. Investigation of salivary protein structure is another example of basic research. Applied research. Applied research is conducted for practical application and often is founded on basic research findings. It may be defined as “systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need may be met.”2 Examples of applied research include research regarding the effectiveness of fluoride in preventing dental caries and clinical trials to determine the effectiveness of various types of periodontal therapy. Clinical research. Clinical research is applied research involving humans that may involve patient-oriented, epidemiologic, behavioral, outcomes and health services research.3 dPatient-oriented research includes “research conducted with human subjects (or on material of human origin such as tissues, specimens, and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects.” 3 It includes “mechanisms of human disease, therapeutic interventions, clinical trials [and] development of new technologies.”3 However, “in vitro studies that utilize human tissues that cannot be linked to a living individual” are excluded from this definition of clinical research.3 Clinical trials are a specific type of patientoriented research defined as “prospective biomedical or behavioral research [involving] human subjects that is designed to answer specific questions about biomedical or behavioral interventions (drugs, treatments, devices, or new ways of using known drugs, treatments, or devices).”3
BOX
Types of research. PRIMARY RESEARCH Basic Research Applied (Clinical) Research
dPatient-oriented research ■
Clinical trials
dEpidemiologic (observational) research dBehavioral research dOutcomes research dHealth services research SECONDARY RESEARCH Literature Review
dNarrative review dSystematic review ■
Meta-analysis
They are used to determine “whether new biomedical or behavioral interventions are safe, efficacious and effective.”3 In efficacy trials, researchers seek to determine the value of an intervention under carefully controlled conditions, and in effectiveness or pragmatic trials, researchers seek to determine the value of an intervention in the real world of clinical practice.4 dEpidemiologic research often involves observational studies in which investigators record observations of human participants but do not use random assignment of interventions to treat or prevent disease. It frequently involves investigations aimed at determining the prevalence of, incidence of and risk factors for disease. (A subsequent article in this series will present a review of specific types of observational research, such as epidemiologic surveys, casecontrol studies and cohort studies.) dBehavioral research may involve the study of various factors such as cognition, memory, language, perception, personality, emotion and motivation.5 dOutcomes research, as defined by the U.S. Agency for Healthcare Research and Quality6 (AHRQ), is that in which investigators seek “to understand the end results of particular health care practices and interventions” and the “end results include effects that people experience and care about, such as change in the ability to function.”6 dHealth services research, also as defined by the AHRQ,7 is research that “examines how people get access to health care, how much care ABBREVIATION KEY. AHRQ: Agency for Healthcare Research and Quality. PICO: Population of interest, the Intervention, a Comparator group and an Outcome. JADA 143(1)
http://jada.ada.org
Copyright © 2012 American Dental Association. All rights reserved.
January 2012
55
RESEARCH
costs, and what happens to patients as a result of this care,” and it “aims to identify the most effective ways to organize, manage, finance, and deliver high-quality care; reduce medical errors; and improve patient safety.” Although behavioral, outcomes and health services research are not included in this series, it is important to recognize that they are among the types of clinical research. Primary research also can serve as the basis for literature reviews. Reviews provide additional information to practitioners by virtue of the fact that they involve the summary of, and occasionally the pooling of, data found in primary research studies. SECONDARY RESEARCH
Literature reviews. Because they involve the identification and summary of research findings from many sources, reviews have an important place in the health care literature. This is particularly important given the explosion of health care information in scientific journals and electronic media. The two basic types of literature reviews are the narrative review and the systematic review; they differ in a number of important ways. Narrative reviews. Narrative reviews generally cover a broad topic, present summaries of information and often serve to provide background knowledge. For example, if one wanted to understand the role of antibiotics in dentistry, a narrative review would be a good place to start. As an article or a book chapter, a narrative review on this topic likely would provide information about dental anatomy, pathology of infections and mechanisms of action for various antibiotics. Narrative reviews may be biased because they usually do not follow specific rules for study identification, inclusion or quality. Systematic reviews. In systematic reviews, investigators attempt to answer focused, clinically relevant questions. These questions address the Population of interest, the Intervention, a Comparator group and an Outcome (referred to as a PICO format). Researchers use the PICO format to generate clinical questions, aid in evidence searches on databases and address the relevance (external validity) of a study to a clinician’s practice or a patient’s circumstances. For example, clinicians may wish to know if deep scaling and local antibiotic
56
JADA 143(1)
http://jada.ada.org
therapy are more effective than deep scaling alone in reducing pocket depth in patients who have mild to moderate periodontitis. Systematic reviews differ from their narrative counterparts by involving the use of systematic predefined methods of reviewing the literature. This includes an exhaustive literature search to identify all relevant articles; strict criteria for determining which types of studies are to be included in the review; an assessment of the quality (risk of bias) of the included studies; and methods of data extraction, synthesis and summary. Systematic reviews may contain a meta-analysis, which is a statistical pooling of data from different studies. Meta-analyses have yielded statistically significant and clinically relevant summary estimates that investigators in individual studies often are unable to detect. SUMMARY
To practice evidence-based dentistry, practitioners should have a basic understanding of research design, methods, statistical analysis and critical appraisal of the scientific literature. In this article, we have described the fundamentals of basic and applied research, as well as of literature reviews. The next installment in this series (which, as of press time, is scheduled for publication in April) will address the translation of research findings into clinical practice. ■ Disclosure. Drs. Abt and Pihlstrom did not report any disclosures. 1. Protection of Human Subjects; Definitions, 45 USC § 46.102 (2005). “http://ohsr.od.nih.gov/guidelines/45cfr46.html#46.102”. Accessed Nov. 16, 2011. 2. National Science Foundation. Globalization of science and engineering research: a companion to science and engineering indicators 2010—definitions. “www.nsf.gov/statistics/nsb1003/definitions.htm”. Accessed Nov. 16, 2011. 3. U.S. Department of Health and Human Services, National Institutes of Health, Office of Extramural Research. Glossary and acronym list. “http://grants.nih.gov/grants/glossary.htm#C”. Accessed Nov. 16, 2011. 4. Ware JH, Hamel MB. Pragmatic trials: guides to better patient care? N Engl J Med 2011;364(18):1685-1687. 5. U.S. Department of Health and Human Services, National Institutes of Health, Office of Behavioral and Social Sciences Research. Behavioral and social sciences (BSSR) definition. “http://obssr.od.nih. gov/about_obssr/BSSR_CC/BSSR_definition/definition.aspx”. Accessed Nov. 16, 2011. 6. U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. Outcomes research: fact sheet. “www.ahrq.gov/clinic/outfact.htm”. Accessed Nov. 15, 2011. 7. U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. Helping the nation with health services research: fact sheet. “www.ahrq.gov/news/focus/scenarios. htm”. Accessed Nov. 15, 2011.
January 2012
Copyright © 2012 American Dental Association. All rights reserved.
A practitioner’s guide to developing critical appraisal skills The fundamentals of research Elliot Abt, DDS, MS, MSc; Bruce L. Pihlstrom, DDS, MS
Editor’s note: This is the first in a series of articles summarizing research methods and design, research translation, statistical analysis and critical appraisal of the scientific literature. The purpose of this series is to help practitioners enhance their understanding of research and develop skills in interpreting research in the medical and dental literature.
lthough there are many different definitions of research, the definition applied in this series appears in the U.S. Code of Federal Regulations: “research means a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.”1 This definition emphasizes the idea that research involves systematic investigation. As such, it involves specific protocols and methods of analysis. Research commonly is divided into primary and secondary research. Primary research consists
A
AB STRACT Background and Overview. This is the first of a series of articles summarizing the basic principles and methods of research with an emphasis on clinical investigation. In today’s environment of evidence-based practice, the prevention of, diagnosis of, prognosis for and treatment of oral diseases must be based on a critical evaluation of available research. The purpose of this series is to help practitioners understand research and develop skills in interpreting research published in the medical and dental literature. Conclusions and Practice Implications. Research involves systematic investigation and can be categorized in different ways. Primary research may take several forms and includes basic and applied research. Basic research seeks fundamental knowledge without having a specific application; applied research is conducted for the purpose of a specific application and often is predicated on basic research findings. Applied research may take several forms and includes clinical research, which seeks to inform patients, clinicians, public health workers and policymakers about methods of preventing and treating diseases. Secondary research involves the use of existing data and results of published scientific studies. Systematic literature reviews are a form of secondary research that attempt to remove bias often associated with narrative reviews. Key Words. Evidence-based dentistry; research; primary research; clinical trials; secondary research; narrative reviews; systematic reviews. JADA 2012;143(1):54-56.
Dr. Abt is an attending staff member, Department of Dentistry, Advocate Illinois Masonic Medical Center, Chicago. He also maintains a private practice in general dentistry in Skokie, Ill. Address reprint requests to Dr. Abt at Department of Dentistry, Advocate Illinois Masonic Medical Center, 811 W. Wellington Ave., Chicago, Ill. 60657, e-mail “[email protected]”. Dr. Pihlstrom is a professor emeritus, Department of Surgical and Developmental Sciences, School of Dentistry, University of Minnesota, Minneapolis. He also is the associate editor, Research, and the JournalScan section editor for The Journal of the American Dental Association, as well as an independent oral health research consultant.
54 JADA 143(1)
http://jada.ada.org
January 2012
Copyright © 2012 American Dental Association. All rights reserved.
RESEARCH
of basic and applied research, whereas secondary research refers to compilations of existing data. This article briefly describes common types of research involved in dentistry and medicine, as well as reviews that are conducted to summarize existing research (Box). Additional articles in this series are planned for publication in The Journal of the American Dental Association. PRIMARY RESEARCH
Basic and applied research. Basic research. The National Science Foundation2 defines basic research as “systematic study directed toward fuller knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications towards processes or products in mind.” For example, original research that defines the structure of atoms, ions and molecules would be considered basic research, as it expands our fundamental knowledge and understanding of matter. Investigation of salivary protein structure is another example of basic research. Applied research. Applied research is conducted for practical application and often is founded on basic research findings. It may be defined as “systematic study to gain knowledge or understanding necessary to determine the means by which a recognized and specific need may be met.”2 Examples of applied research include research regarding the effectiveness of fluoride in preventing dental caries and clinical trials to determine the effectiveness of various types of periodontal therapy. Clinical research. Clinical research is applied research involving humans that may involve patient-oriented, epidemiologic, behavioral, outcomes and health services research.3 dPatient-oriented research includes “research conducted with human subjects (or on material of human origin such as tissues, specimens, and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects.” 3 It includes “mechanisms of human disease, therapeutic interventions, clinical trials [and] development of new technologies.”3 However, “in vitro studies that utilize human tissues that cannot be linked to a living individual” are excluded from this definition of clinical research.3 Clinical trials are a specific type of patientoriented research defined as “prospective biomedical or behavioral research [involving] human subjects that is designed to answer specific questions about biomedical or behavioral interventions (drugs, treatments, devices, or new ways of using known drugs, treatments, or devices).”3
BOX
Types of research. PRIMARY RESEARCH Basic Research Applied (Clinical) Research
dPatient-oriented research ■
Clinical trials
dEpidemiologic (observational) research dBehavioral research dOutcomes research dHealth services research SECONDARY RESEARCH Literature Review
dNarrative review dSystematic review ■
Meta-analysis
They are used to determine “whether new biomedical or behavioral interventions are safe, efficacious and effective.”3 In efficacy trials, researchers seek to determine the value of an intervention under carefully controlled conditions, and in effectiveness or pragmatic trials, researchers seek to determine the value of an intervention in the real world of clinical practice.4 dEpidemiologic research often involves observational studies in which investigators record observations of human participants but do not use random assignment of interventions to treat or prevent disease. It frequently involves investigations aimed at determining the prevalence of, incidence of and risk factors for disease. (A subsequent article in this series will present a review of specific types of observational research, such as epidemiologic surveys, casecontrol studies and cohort studies.) dBehavioral research may involve the study of various factors such as cognition, memory, language, perception, personality, emotion and motivation.5 dOutcomes research, as defined by the U.S. Agency for Healthcare Research and Quality6 (AHRQ), is that in which investigators seek “to understand the end results of particular health care practices and interventions” and the “end results include effects that people experience and care about, such as change in the ability to function.”6 dHealth services research, also as defined by the AHRQ,7 is research that “examines how people get access to health care, how much care ABBREVIATION KEY. AHRQ: Agency for Healthcare Research and Quality. PICO: Population of interest, the Intervention, a Comparator group and an Outcome. JADA 143(1)
http://jada.ada.org
Copyright © 2012 American Dental Association. All rights reserved.
January 2012
55
RESEARCH
costs, and what happens to patients as a result of this care,” and it “aims to identify the most effective ways to organize, manage, finance, and deliver high-quality care; reduce medical errors; and improve patient safety.” Although behavioral, outcomes and health services research are not included in this series, it is important to recognize that they are among the types of clinical research. Primary research also can serve as the basis for literature reviews. Reviews provide additional information to practitioners by virtue of the fact that they involve the summary of, and occasionally the pooling of, data found in primary research studies. SECONDARY RESEARCH
Literature reviews. Because they involve the identification and summary of research findings from many sources, reviews have an important place in the health care literature. This is particularly important given the explosion of health care information in scientific journals and electronic media. The two basic types of literature reviews are the narrative review and the systematic review; they differ in a number of important ways. Narrative reviews. Narrative reviews generally cover a broad topic, present summaries of information and often serve to provide background knowledge. For example, if one wanted to understand the role of antibiotics in dentistry, a narrative review would be a good place to start. As an article or a book chapter, a narrative review on this topic likely would provide information about dental anatomy, pathology of infections and mechanisms of action for various antibiotics. Narrative reviews may be biased because they usually do not follow specific rules for study identification, inclusion or quality. Systematic reviews. In systematic reviews, investigators attempt to answer focused, clinically relevant questions. These questions address the Population of interest, the Intervention, a Comparator group and an Outcome (referred to as a PICO format). Researchers use the PICO format to generate clinical questions, aid in evidence searches on databases and address the relevance (external validity) of a study to a clinician’s practice or a patient’s circumstances. For example, clinicians may wish to know if deep scaling and local antibiotic
56
JADA 143(1)
http://jada.ada.org
therapy are more effective than deep scaling alone in reducing pocket depth in patients who have mild to moderate periodontitis. Systematic reviews differ from their narrative counterparts by involving the use of systematic predefined methods of reviewing the literature. This includes an exhaustive literature search to identify all relevant articles; strict criteria for determining which types of studies are to be included in the review; an assessment of the quality (risk of bias) of the included studies; and methods of data extraction, synthesis and summary. Systematic reviews may contain a meta-analysis, which is a statistical pooling of data from different studies. Meta-analyses have yielded statistically significant and clinically relevant summary estimates that investigators in individual studies often are unable to detect. SUMMARY
To practice evidence-based dentistry, practitioners should have a basic understanding of research design, methods, statistical analysis and critical appraisal of the scientific literature. In this article, we have described the fundamentals of basic and applied research, as well as of literature reviews. The next installment in this series (which, as of press time, is scheduled for publication in April) will address the translation of research findings into clinical practice. ■ Disclosure. Drs. Abt and Pihlstrom did not report any disclosures. 1. Protection of Human Subjects; Definitions, 45 USC § 46.102 (2005). “http://ohsr.od.nih.gov/guidelines/45cfr46.html#46.102”. Accessed Nov. 16, 2011. 2. National Science Foundation. Globalization of science and engineering research: a companion to science and engineering indicators 2010—definitions. “www.nsf.gov/statistics/nsb1003/definitions.htm”. Accessed Nov. 16, 2011. 3. U.S. Department of Health and Human Services, National Institutes of Health, Office of Extramural Research. Glossary and acronym list. “http://grants.nih.gov/grants/glossary.htm#C”. Accessed Nov. 16, 2011. 4. Ware JH, Hamel MB. Pragmatic trials: guides to better patient care? N Engl J Med 2011;364(18):1685-1687. 5. U.S. Department of Health and Human Services, National Institutes of Health, Office of Behavioral and Social Sciences Research. Behavioral and social sciences (BSSR) definition. “http://obssr.od.nih. gov/about_obssr/BSSR_CC/BSSR_definition/definition.aspx”. Accessed Nov. 16, 2011. 6. U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. Outcomes research: fact sheet. “www.ahrq.gov/clinic/outfact.htm”. Accessed Nov. 15, 2011. 7. U.S. Department of Health and Human Services, Agency for Healthcare Research and Quality. Helping the nation with health services research: fact sheet. “www.ahrq.gov/news/focus/scenarios. htm”. Accessed Nov. 15, 2011.
January 2012
Copyright © 2012 American Dental Association. All rights reserved.