- Email: [email protected]
Case-Control Confusion
Case-Control Confusion Martha A. Hellems, MD, MS; Michael S. Kramer, MD; Gregory F. Hayden, MD Objective.—Critical analysis of journal articles by using principles of evidence-based medicine is important for clinicians applying research results in their practice and is a valuable component of pediatric residency training. Appraisal of an article’s methodological rigor is often tailored to a particular type of study design, so that misclassification of study design can confuse the appraisal. The goal of this study was to determine how often pediatric research articles that are self-declared as casecontrol studies conform to a standard definition for this study design. Methods.—A Medline search identified articles published in two pediatric journals from January 1996 through August 2004 with the phrase “case-control study” in the title or abstract. Articles
that were self-declared as case-control studies were analyzed to determine whether they satisfied a standard definition of a casecontrol study. Results.—Of the 91 purported case-control studies, only 68 (75%) met the standard definition for at least the most important analysis. The remaining 23 articles could be classified as crosssectional studies (N ⫽ 16) or prospective cohort studies (N ⫽ 7). Conclusions.—Ambiguity in the definition of a case-control study can cause confusion in the critical appraisal of published clinical research.
L
cedent exposure among the cases to the rate of similar exposure among a group of persons without the disease (the controls). The comparison of exposure rates is usually based on the odds ratio; as long as the outcome is rare, this serves as a good estimate of the prospective relative risk of the outcome in exposed versus nonexposed subjects. This type of design is more prone to bias than the RCT, but case-control studies can be particularly useful when the outcome in question is rare or delayed, and when the exposure is not easily randomized (eg, smoking or breastfeeding). Case-control studies can produce valid results when careful attention is paid to possible sources of bias in the design and analysis. Unfortunately, confusion can arise about the appropriate classification of an article’s study design. One recent article in a pediatric journal, for example, was described by the authors as a prospective cohort study,4 while an editorial in the same issue referred to the research as a case-control study.5 Another recent article in a different pediatric journal was described by the authors as a casecontrol study, but it involved clinical and laboratory assessment of a cohort of children followed prospectively since birth, some of whom had developed a particular antibody (the cases) and some of whom had not (the controls).6 Another recent article in a family medicine journal was described by the authors as a case-control study, but it used an observational cohort design whereby infants with and without ankyloglossia were identified in the nursery and then followed up at 1 week and 1 month to determine the likelihood of breast-feeding success.7 Studies that include cases and controls may not follow the design of a typical case-control study. The lack of uniformity in nomenclature of study design can have import beyond mere semantics, because many readers attempt to conduct critical appraisals based on study methodology.
KEY WORDS: case-control study; study design Ambulatory Pediatrics 2006;6:96 –99
ifelong learning as a physician demands facility in the assessment and application of clinical evidence from the medical literature. Many pediatric training programs teach residents critical appraisal skills by means of journal clubs that stimulate discussion of relevant studies.1 The appropriateness and rigor of a particular study’s methodology can be analyzed using a systematic critique based on study design. So, for example, a randomized controlled trial (RCT) can be reviewed using guidelines suggested in the literature for “dissecting” and evaluating an RCT. Such guidelines can be helpful in reminding the reader about shortcomings and sources of bias sometimes encountered in that particular type of study and in identifying specific design features requiring careful attention. A systematic critique based on method makes clear assumptions about the type of study design, so that if the design differs from this assumption, the guidelines for analyzing the study’s internal validity may not be appropriate. One study design commonly encountered in the medical literature is the case-control study. In this type of study, the authors begin by selecting a group of persons (the cases) with a particular disease or other outcome of interest (eg, Reye’s syndrome). They then determine whether, at some time in the past, these persons experienced an exposure (eg, aspirin) that might have led to the outcome.2,3 (p 93–112) They compare the rate of such ante-
From the Department of Pediatrics, University of Virginia Children’s Hospital, Charlottesville, Va (Drs Hellems and Hayden); and the Departments of Pediatrics and Epidemiology and Biostatistics, McGill University Faculty of Medicine, Montreal, Canada (Dr Kramer). Address correspondence to Martha A. Hellems, Department of Pediatrics, University of Virginia, P.O. Box 800386, Charlottesville, VA 22908-0386. (e-mail: [email protected].) Received for publication May 31, 2005; accepted November 27, 2005.
AMBULATORY PEDIATRICS Copyright © 2006 by Ambulatory Pediatric Association
96
Volume 6, Number 2 March–April 2006
AMBULATORY PEDIATRICS
For purposes of this study, a case-control study was defined as one that starts by selecting subjects with (cases) or without (controls) a particular outcome, and then ascertaining prior exposure to a suspected risk factor.3(p 40) Our objective was to determine how often studies published in the pediatric literature—self-declared as case-control studies— conformed to this definition. METHODS We conducted a Medline search of articles published in Pediatrics or Archives of Pediatrics & Adolescent Medicine from January 1996 through August 2004 with the phrase “case-control study” in the title or abstract. Pediatrics is the official journal of the American Academy of Pediatrics and has a large circulation and high impact factor; Archives of Pediatrics & Adolescent Medicine is the oldest continuously published pediatric journal in the United States and includes a journal club feature with emphasis on study design and critical appraisal skills. We reviewed the articles to determine whether the study satisfied the following definition of a case-control study: the authors started with the measurement of an outcome, often a classification of subjects into those with and those without a disease or condition of interest. They then ascertain prior exposure to some hypothesized causal or protective factor. The reasoning is “backwards” from effect to 3 cause. (p 40) Key components of this definition are sampling by outcome (case vs control) and directionality (the exposure occurred at some etiologically relevant period in the past). Categorizations of articles as case-control studies were mutually agreed upon by two reviewers (M.A.H. and G.F.H.). Disagreements between the reviewers were resolved through discussion. In one case where agreement was not reached, the more conservative decision was made, that is, the article was classified as a case-control study. Those studies not satisfying the definition of a case-control study were examined to determine whether their designs could be further classified and whether any common design elements could be found. RESULTS The Medline search returned 99 articles: 69 in Pediatrics and 30 in the Archives of Pediatrics & Adolescent Medicine. In eight of these articles, the authors did not claim that their study was a case-control study, although the phrase “case-control” was used in another context in the title or abstract. These studies were excluded from further consideration. Of the remaining 91 articles, 68 studies (46 in Pediatrics and 22 in the Archives of Pediatrics & Adolescent Medicine) met the definition of casecontrol study for at least the most important analysis, as many articles in fact utilized multiple study designs. In these 68 studies, cases and controls were sampled based on outcome and assessed for prior exposure to a risk factor, where exposure clearly preceded development of the outcome. In one article, laboratory data were collected at the time that a child’s status as a case or control was determined, but the laboratory studies tested for exposures or predispositions to disease that presumably preceded devel-
Case-Control Confusion
97
opment of the condition. In the sense that the authors were testing for exposure that occurred prior to the disease in question, this study was classified as a case-control study. The 23 remaining articles that did not meet the casecontrol definition could be classified as cross-sectional studies or cohort studies. A list of these articles is available upon request from the lead author. Sixteen studies met criteria for cross-sectional studies in which temporal relationships between the dependent and independent variables were not established, and therefore causality could not be evaluated.8 In four of these crosssectional studies, the authors defined cases and controls based on outcome, but exposures to the risk factors did not clearly precede development of the outcomes. In another three cross-sectional studies, the authors used the terms “cases” and “controls” to refer to “exposed” and “nonexposed” persons. In another three cross-sectional studies, the authors sampled by presence or absence of a disorder and then measured concurrent behavioral characteristics, psychiatric characteristics, attitudes, and beliefs of subjects (or parents) with or without the disorder. While such factors likely remain stable over a long time, the authors recognized that these attitudes and behaviors might have resulted from the disorder rather than having contributed to its development. These studies did not attempt to infer causation and did not define variables as exposures and outcomes. In another four cross-sectional studies, the authors reported the prevalence of clinical findings in patients with and without a disorder, without attempting to explore etiology. In the final two cross-sectional studies, the authors defined a group of cases with a particular disorder and a group of controls without the disorder, and then assessed the sensitivity and specificity of diagnostic tests for the disorder. The diagnostic marker was not investigated as a “cause” of the outcome, but merely as a concurrent predictor of it. Seven studies best met the definition of a cohort study.8 All of these papers used the terms “cases” and “controls” to denote subjects who were exposed versus nonexposed, respectively, to a suspected risk factor. These exposures may have occurred in an experimental fashion (an interventional cohort such as a RCT) or in a nonexperimental fashion (an observational cohort). The subjects were then followed forward in time to assess development of subsequent outcome(s). DISCUSSION In this analysis of 91 self-defined case-control studies published in two pediatric journals over an 8-year period, 68 of the identified articles met our standard definition of a case-control study. The other 23 articles could be classified as cross-sectional studies or cohort studies, although each was literally a case-control study because it contained both cases and controls. Confusion about the term case-control study is not unique to the pediatric literature. One recent study, for example, found that only 35% of self-identified casecontrol studies published in six general surgical journals actually met the standard definition for this study design.9
98
Hellems et al
Furthermore, case-control confusion is not new. As Feinstein observed some 20 years ago, “The ambiguity of the ‘case-control’ nomenclature can create major scientific and intellectual obstacles for people trying to understand the results. The obstacles are both demonstrated and further increased when the term ‘case-control’ is misunderstood and is misapplied as a label for yet another type of research study, having quite a different type of architectural structure.”10(p 380) Epidemiology textbooks are in general agreement about the design of case-control studies, but the precise definitions are not identical. One text considers a case-control study to be a type of cross-sectional study in which information about exposure and outcome may be obtained at a single point in time, but the temporal direction is backward from outcome to antecedent exposure, and therefore, cause and effect may be inferred.10(p 363, 379 –380) A second text states that a case-control study examines “persons with the disease (or other outcome variable) . . . and a suitable control” and “looks back from the outcome to its causes” as “the past history of exposure to a suspected risk factor is compared.”11 A third text defines a case-control study as one in which subjects are selected for presence or absence of disease, and “cases and controls are compared with respect to existing or past attributes or exposures thought to be relevant to the development of the condition or disease under study.”12 While not all texts specify that exposure must clearly precede outcome, they generally agree that a case-control study involves sampling by outcome and assessing for exposure to etiologic risk factors such that inferences can be drawn regarding cause and effect. Particular types of research questions may lend themselves most aptly to particular study designs. Case-control studies, for example, can be efficient for assessing disorders that are relatively rare or that develop years after exposure to the putative risk factor. Since case-control studies do not require the assembly and prolonged follow-up of a large cohort of patients, they are typically much less expensive than cohort studies. The case-control design also allows one to study the effect of exposures that might be ethically untenable in an interventional trial, for example, the risk of exposure to tobacco smoke. Casecontrol studies are also well suited to exploring multiple etiologic hypotheses (ie, multiple antecedent risk factors), since once the cases and controls are selected, it is efficient to inquire (or consult records) about multiple past exposures. This design might be used, for example, to investigate the cause of an outbreak of infectious disease or other unusual illness found in a setting such as a factory. The determination of temporal direction, and thus the distinction between a case-control study and a cross-sectional study, may not always be clear.3(p 39 – 45) The exposure variable can be a permanent characteristic such as genetic makeup that must have been present before the outcome occurred even though it may not have been ascertained until the study was conducted. In such an instance, simultaneous and prior exposures are equivalent. In other cases, laboratory results obtained at the time of the
AMBULATORY PEDIATRICS
study measure a prior exposure. The cross-sectional and case-control designs can therefore overlap. It may be more useful to think about study designs “not as discrete entities but as lying on a continuum without sharp boundaries.”3 (p 43) The important question is that of directionality and time sequence. This impacts the assessment of mere association (a weaker conclusion) versus the stronger inference of cause and effect. Case-control studies are prone to particular weaknesses and sources of bias. Exposure to risk factors is by definition assessed retrospectively. This can affect the availability and quality of the data and is especially prone to differential recall bias between subjects who do and do not have a disease. Also, the selection of cases and appropriate controls must be made carefully. Because case subjects are selected based on the presence of disease, cases may be biased towards sicker or hospitalized patients. Hospitalized controls may differ from the general population, whereas nonhospitalized controls may not provide valid comparisons with cases. For this reason, case-control studies often include two or more groups of controls. Because outcome prevalence in a case-control study is determined by the selected proportion of cases versus controls, the data analysis typically employs odds ratios to reflect the magnitude of the effect of the exposure on the outcome. Although the type of statistical analysis was not included in our definition of a case-control study, many of the articles used other measures of comparison between the case and control groups, such as t tests or chi-square tests. While these statistical tests are not technically incorrect, they are not helpful in a case-control study in assessing the increased risk of outcome associated with exposure. Our analysis has several important limitations. First, we surveyed only two pediatric journals—the frequency of confusing use of the term case-control study in other pediatric or general medical journals was not assessed. Second, the judgment that a particular study did not meet our definition of a case-control study was based solely on the information contained in the published article and may be open to interpretation. Third, many of the studies incorporated multiple study designs, so our assessment of the “main” design was admittedly somewhat subjective. Finally, we in no way mean to imply that the studies discussed are of poor quality. For purposes of this analysis, we made no attempt to assess the methodological quality of each study. Our intent was simply to determine if the studies met very basic criteria to justify classification of the design as a case-control study and to highlight the importance of careful consideration of methodology when reading a journal article. REFERENCES 1. Edwards KS, Woolf PK, Hetzler T. Pediatric residents as learners and teachers of evidence-based medicine. Acad Med. 2002;77:748. 2. Hayden GF, Kramer MS, Horwitz RI. The case-control study: a practical review for the clinician. JAMA. 1982;247:326 –331. 3. Kramer MS. Clinical Epidemiology and Biostatistics. New York, NY: Springer-Verlag;1988. 4. Perrin EM, Murphy ML, Casey JR, et al. Does group A – hemo-
AMBULATORY PEDIATRICS lytic streptococcal infection increase risk for behavioral and neuropsychiatric symptoms in children? Arch Pediatr Adolesc Med. 2004;158:848 – 856. 5. March JS. Pediatric autoimmune neuropsychiatric disorder associated with streptococcal infections: implications for clinical practice. Arch Pediatr Adolesc Med. 2004;158:927–929. 6. Hoffenberg EJ, Emery LM, Barriga KJ, DeFor TA. Clinical features of children with screening-identified evidence of celiac disease. Pediatrics. 2004;113:1254 –1259. 7. Ricke LA, Baker NJ, Madlon-Kay DJ, et al. Newborn tongue-tie: prevalence and effect on breast feeding. J Am Board Fam Pract. 2005;18:1–7.
Case-Control Confusion
99
8. Kramer MS, Boivin J. Toward an “unconfounded” classification of epidemiologic research design. J Chron Dis. 1987;40:683– 688. 9. Mihailovic A, Bell AM, Urbach DR. Users’ guide to the surgical literature: case-control studies in surgical journals. Can J Surg. 2005;48:148 –151. 10. Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research. Philadelphia, Pa: W. B. Saunders Co; 1985. 11. Last, John, ed. A Dictionary of Epidemiology. NewYork, NY: Oxford University Press; 2001:22–24. 12. Schlesselman, JJ, Stolley PD. Research Strategies. In: Schlesselman JJ. Case-Control Studies. New York, NY: Oxford University Presss; 1982:14.
that were self-declared as case-control studies were analyzed to determine whether they satisfied a standard definition of a casecontrol study. Results.—Of the 91 purported case-control studies, only 68 (75%) met the standard definition for at least the most important analysis. The remaining 23 articles could be classified as crosssectional studies (N ⫽ 16) or prospective cohort studies (N ⫽ 7). Conclusions.—Ambiguity in the definition of a case-control study can cause confusion in the critical appraisal of published clinical research.
L
cedent exposure among the cases to the rate of similar exposure among a group of persons without the disease (the controls). The comparison of exposure rates is usually based on the odds ratio; as long as the outcome is rare, this serves as a good estimate of the prospective relative risk of the outcome in exposed versus nonexposed subjects. This type of design is more prone to bias than the RCT, but case-control studies can be particularly useful when the outcome in question is rare or delayed, and when the exposure is not easily randomized (eg, smoking or breastfeeding). Case-control studies can produce valid results when careful attention is paid to possible sources of bias in the design and analysis. Unfortunately, confusion can arise about the appropriate classification of an article’s study design. One recent article in a pediatric journal, for example, was described by the authors as a prospective cohort study,4 while an editorial in the same issue referred to the research as a case-control study.5 Another recent article in a different pediatric journal was described by the authors as a casecontrol study, but it involved clinical and laboratory assessment of a cohort of children followed prospectively since birth, some of whom had developed a particular antibody (the cases) and some of whom had not (the controls).6 Another recent article in a family medicine journal was described by the authors as a case-control study, but it used an observational cohort design whereby infants with and without ankyloglossia were identified in the nursery and then followed up at 1 week and 1 month to determine the likelihood of breast-feeding success.7 Studies that include cases and controls may not follow the design of a typical case-control study. The lack of uniformity in nomenclature of study design can have import beyond mere semantics, because many readers attempt to conduct critical appraisals based on study methodology.
KEY WORDS: case-control study; study design Ambulatory Pediatrics 2006;6:96 –99
ifelong learning as a physician demands facility in the assessment and application of clinical evidence from the medical literature. Many pediatric training programs teach residents critical appraisal skills by means of journal clubs that stimulate discussion of relevant studies.1 The appropriateness and rigor of a particular study’s methodology can be analyzed using a systematic critique based on study design. So, for example, a randomized controlled trial (RCT) can be reviewed using guidelines suggested in the literature for “dissecting” and evaluating an RCT. Such guidelines can be helpful in reminding the reader about shortcomings and sources of bias sometimes encountered in that particular type of study and in identifying specific design features requiring careful attention. A systematic critique based on method makes clear assumptions about the type of study design, so that if the design differs from this assumption, the guidelines for analyzing the study’s internal validity may not be appropriate. One study design commonly encountered in the medical literature is the case-control study. In this type of study, the authors begin by selecting a group of persons (the cases) with a particular disease or other outcome of interest (eg, Reye’s syndrome). They then determine whether, at some time in the past, these persons experienced an exposure (eg, aspirin) that might have led to the outcome.2,3 (p 93–112) They compare the rate of such ante-
From the Department of Pediatrics, University of Virginia Children’s Hospital, Charlottesville, Va (Drs Hellems and Hayden); and the Departments of Pediatrics and Epidemiology and Biostatistics, McGill University Faculty of Medicine, Montreal, Canada (Dr Kramer). Address correspondence to Martha A. Hellems, Department of Pediatrics, University of Virginia, P.O. Box 800386, Charlottesville, VA 22908-0386. (e-mail: [email protected].) Received for publication May 31, 2005; accepted November 27, 2005.
AMBULATORY PEDIATRICS Copyright © 2006 by Ambulatory Pediatric Association
96
Volume 6, Number 2 March–April 2006
AMBULATORY PEDIATRICS
For purposes of this study, a case-control study was defined as one that starts by selecting subjects with (cases) or without (controls) a particular outcome, and then ascertaining prior exposure to a suspected risk factor.3(p 40) Our objective was to determine how often studies published in the pediatric literature—self-declared as case-control studies— conformed to this definition. METHODS We conducted a Medline search of articles published in Pediatrics or Archives of Pediatrics & Adolescent Medicine from January 1996 through August 2004 with the phrase “case-control study” in the title or abstract. Pediatrics is the official journal of the American Academy of Pediatrics and has a large circulation and high impact factor; Archives of Pediatrics & Adolescent Medicine is the oldest continuously published pediatric journal in the United States and includes a journal club feature with emphasis on study design and critical appraisal skills. We reviewed the articles to determine whether the study satisfied the following definition of a case-control study: the authors started with the measurement of an outcome, often a classification of subjects into those with and those without a disease or condition of interest. They then ascertain prior exposure to some hypothesized causal or protective factor. The reasoning is “backwards” from effect to 3 cause. (p 40) Key components of this definition are sampling by outcome (case vs control) and directionality (the exposure occurred at some etiologically relevant period in the past). Categorizations of articles as case-control studies were mutually agreed upon by two reviewers (M.A.H. and G.F.H.). Disagreements between the reviewers were resolved through discussion. In one case where agreement was not reached, the more conservative decision was made, that is, the article was classified as a case-control study. Those studies not satisfying the definition of a case-control study were examined to determine whether their designs could be further classified and whether any common design elements could be found. RESULTS The Medline search returned 99 articles: 69 in Pediatrics and 30 in the Archives of Pediatrics & Adolescent Medicine. In eight of these articles, the authors did not claim that their study was a case-control study, although the phrase “case-control” was used in another context in the title or abstract. These studies were excluded from further consideration. Of the remaining 91 articles, 68 studies (46 in Pediatrics and 22 in the Archives of Pediatrics & Adolescent Medicine) met the definition of casecontrol study for at least the most important analysis, as many articles in fact utilized multiple study designs. In these 68 studies, cases and controls were sampled based on outcome and assessed for prior exposure to a risk factor, where exposure clearly preceded development of the outcome. In one article, laboratory data were collected at the time that a child’s status as a case or control was determined, but the laboratory studies tested for exposures or predispositions to disease that presumably preceded devel-
Case-Control Confusion
97
opment of the condition. In the sense that the authors were testing for exposure that occurred prior to the disease in question, this study was classified as a case-control study. The 23 remaining articles that did not meet the casecontrol definition could be classified as cross-sectional studies or cohort studies. A list of these articles is available upon request from the lead author. Sixteen studies met criteria for cross-sectional studies in which temporal relationships between the dependent and independent variables were not established, and therefore causality could not be evaluated.8 In four of these crosssectional studies, the authors defined cases and controls based on outcome, but exposures to the risk factors did not clearly precede development of the outcomes. In another three cross-sectional studies, the authors used the terms “cases” and “controls” to refer to “exposed” and “nonexposed” persons. In another three cross-sectional studies, the authors sampled by presence or absence of a disorder and then measured concurrent behavioral characteristics, psychiatric characteristics, attitudes, and beliefs of subjects (or parents) with or without the disorder. While such factors likely remain stable over a long time, the authors recognized that these attitudes and behaviors might have resulted from the disorder rather than having contributed to its development. These studies did not attempt to infer causation and did not define variables as exposures and outcomes. In another four cross-sectional studies, the authors reported the prevalence of clinical findings in patients with and without a disorder, without attempting to explore etiology. In the final two cross-sectional studies, the authors defined a group of cases with a particular disorder and a group of controls without the disorder, and then assessed the sensitivity and specificity of diagnostic tests for the disorder. The diagnostic marker was not investigated as a “cause” of the outcome, but merely as a concurrent predictor of it. Seven studies best met the definition of a cohort study.8 All of these papers used the terms “cases” and “controls” to denote subjects who were exposed versus nonexposed, respectively, to a suspected risk factor. These exposures may have occurred in an experimental fashion (an interventional cohort such as a RCT) or in a nonexperimental fashion (an observational cohort). The subjects were then followed forward in time to assess development of subsequent outcome(s). DISCUSSION In this analysis of 91 self-defined case-control studies published in two pediatric journals over an 8-year period, 68 of the identified articles met our standard definition of a case-control study. The other 23 articles could be classified as cross-sectional studies or cohort studies, although each was literally a case-control study because it contained both cases and controls. Confusion about the term case-control study is not unique to the pediatric literature. One recent study, for example, found that only 35% of self-identified casecontrol studies published in six general surgical journals actually met the standard definition for this study design.9
98
Hellems et al
Furthermore, case-control confusion is not new. As Feinstein observed some 20 years ago, “The ambiguity of the ‘case-control’ nomenclature can create major scientific and intellectual obstacles for people trying to understand the results. The obstacles are both demonstrated and further increased when the term ‘case-control’ is misunderstood and is misapplied as a label for yet another type of research study, having quite a different type of architectural structure.”10(p 380) Epidemiology textbooks are in general agreement about the design of case-control studies, but the precise definitions are not identical. One text considers a case-control study to be a type of cross-sectional study in which information about exposure and outcome may be obtained at a single point in time, but the temporal direction is backward from outcome to antecedent exposure, and therefore, cause and effect may be inferred.10(p 363, 379 –380) A second text states that a case-control study examines “persons with the disease (or other outcome variable) . . . and a suitable control” and “looks back from the outcome to its causes” as “the past history of exposure to a suspected risk factor is compared.”11 A third text defines a case-control study as one in which subjects are selected for presence or absence of disease, and “cases and controls are compared with respect to existing or past attributes or exposures thought to be relevant to the development of the condition or disease under study.”12 While not all texts specify that exposure must clearly precede outcome, they generally agree that a case-control study involves sampling by outcome and assessing for exposure to etiologic risk factors such that inferences can be drawn regarding cause and effect. Particular types of research questions may lend themselves most aptly to particular study designs. Case-control studies, for example, can be efficient for assessing disorders that are relatively rare or that develop years after exposure to the putative risk factor. Since case-control studies do not require the assembly and prolonged follow-up of a large cohort of patients, they are typically much less expensive than cohort studies. The case-control design also allows one to study the effect of exposures that might be ethically untenable in an interventional trial, for example, the risk of exposure to tobacco smoke. Casecontrol studies are also well suited to exploring multiple etiologic hypotheses (ie, multiple antecedent risk factors), since once the cases and controls are selected, it is efficient to inquire (or consult records) about multiple past exposures. This design might be used, for example, to investigate the cause of an outbreak of infectious disease or other unusual illness found in a setting such as a factory. The determination of temporal direction, and thus the distinction between a case-control study and a cross-sectional study, may not always be clear.3(p 39 – 45) The exposure variable can be a permanent characteristic such as genetic makeup that must have been present before the outcome occurred even though it may not have been ascertained until the study was conducted. In such an instance, simultaneous and prior exposures are equivalent. In other cases, laboratory results obtained at the time of the
AMBULATORY PEDIATRICS
study measure a prior exposure. The cross-sectional and case-control designs can therefore overlap. It may be more useful to think about study designs “not as discrete entities but as lying on a continuum without sharp boundaries.”3 (p 43) The important question is that of directionality and time sequence. This impacts the assessment of mere association (a weaker conclusion) versus the stronger inference of cause and effect. Case-control studies are prone to particular weaknesses and sources of bias. Exposure to risk factors is by definition assessed retrospectively. This can affect the availability and quality of the data and is especially prone to differential recall bias between subjects who do and do not have a disease. Also, the selection of cases and appropriate controls must be made carefully. Because case subjects are selected based on the presence of disease, cases may be biased towards sicker or hospitalized patients. Hospitalized controls may differ from the general population, whereas nonhospitalized controls may not provide valid comparisons with cases. For this reason, case-control studies often include two or more groups of controls. Because outcome prevalence in a case-control study is determined by the selected proportion of cases versus controls, the data analysis typically employs odds ratios to reflect the magnitude of the effect of the exposure on the outcome. Although the type of statistical analysis was not included in our definition of a case-control study, many of the articles used other measures of comparison between the case and control groups, such as t tests or chi-square tests. While these statistical tests are not technically incorrect, they are not helpful in a case-control study in assessing the increased risk of outcome associated with exposure. Our analysis has several important limitations. First, we surveyed only two pediatric journals—the frequency of confusing use of the term case-control study in other pediatric or general medical journals was not assessed. Second, the judgment that a particular study did not meet our definition of a case-control study was based solely on the information contained in the published article and may be open to interpretation. Third, many of the studies incorporated multiple study designs, so our assessment of the “main” design was admittedly somewhat subjective. Finally, we in no way mean to imply that the studies discussed are of poor quality. For purposes of this analysis, we made no attempt to assess the methodological quality of each study. Our intent was simply to determine if the studies met very basic criteria to justify classification of the design as a case-control study and to highlight the importance of careful consideration of methodology when reading a journal article. REFERENCES 1. Edwards KS, Woolf PK, Hetzler T. Pediatric residents as learners and teachers of evidence-based medicine. Acad Med. 2002;77:748. 2. Hayden GF, Kramer MS, Horwitz RI. The case-control study: a practical review for the clinician. JAMA. 1982;247:326 –331. 3. Kramer MS. Clinical Epidemiology and Biostatistics. New York, NY: Springer-Verlag;1988. 4. Perrin EM, Murphy ML, Casey JR, et al. Does group A – hemo-
AMBULATORY PEDIATRICS lytic streptococcal infection increase risk for behavioral and neuropsychiatric symptoms in children? Arch Pediatr Adolesc Med. 2004;158:848 – 856. 5. March JS. Pediatric autoimmune neuropsychiatric disorder associated with streptococcal infections: implications for clinical practice. Arch Pediatr Adolesc Med. 2004;158:927–929. 6. Hoffenberg EJ, Emery LM, Barriga KJ, DeFor TA. Clinical features of children with screening-identified evidence of celiac disease. Pediatrics. 2004;113:1254 –1259. 7. Ricke LA, Baker NJ, Madlon-Kay DJ, et al. Newborn tongue-tie: prevalence and effect on breast feeding. J Am Board Fam Pract. 2005;18:1–7.
Case-Control Confusion
99
8. Kramer MS, Boivin J. Toward an “unconfounded” classification of epidemiologic research design. J Chron Dis. 1987;40:683– 688. 9. Mihailovic A, Bell AM, Urbach DR. Users’ guide to the surgical literature: case-control studies in surgical journals. Can J Surg. 2005;48:148 –151. 10. Feinstein AR. Clinical Epidemiology: The Architecture of Clinical Research. Philadelphia, Pa: W. B. Saunders Co; 1985. 11. Last, John, ed. A Dictionary of Epidemiology. NewYork, NY: Oxford University Press; 2001:22–24. 12. Schlesselman, JJ, Stolley PD. Research Strategies. In: Schlesselman JJ. Case-Control Studies. New York, NY: Oxford University Presss; 1982:14.