- Email: [email protected]
Reading research critically: The discussion section
Reading Research Critically: The Discussion Section MAUREEN
GIUFFRE,
PhD,
RN
The discussion section of a research manuscript is the last thing we read and often the section that leaves the most lasting impression. It is important to know what to reasonably expect to be in the discussion section. This article presents the areas that are usually covered. Vver-intexpretation of the results of a research project occurs all too often. This article will help you avoid being taken in by this problem. 0 1996 by American Society of PeriAnesthesia Nurses.
M
ANY OF US who regularly conduct research and publish the findings almost intuitively know what the discussion section of a manuscript is to contain. Nonetheless, it is still difficult to define. Briefly, the discussion section is where the author has the opportunity to take the raw facts and bring them to life. The major goal of this section is the presentation of the researchers’ conclusions based on the facts, and an elaboration of any theoretical or clinical implications. Although authors have some discretion in how they structure this section, it generally contains one or more of the following elements: a brief summary of the findings, comparison with past research, evaluation of the theoretical framework, critique of this research, clinical implications, and/or suggestions for future research. SUMMARY
OF FINDINGS
The discussion section is often begun with a repetition of the findings that have already been presented in the results section, leaving off the statistics. This summation should be very brief. As a result, the discussion section generally makes better and easier reading than the rest of the manuscript. Novice researchers and busy nurses are often tempted to read the review of
Jownd
of f.w~AmsThes;a
Mushg,
Vol 11, No 6 (December),
literature and skip right to the discussion section. After all, the author generally tells the reader what she found but leaves out all the technical jargon, right? Wrong! Or at least not always. Take this (disguised) example from a discussion section of a published JoPAN manuscript. “Based on the findings of these researchers, use of. . . is strongly recommended in the postoperative period.” Compare that statement from the discussion section with these statements from the results section. “ANOVA with repeated measure showed no significant difference in (primary outcome measure) among the groups throughout the study,” and, “ANOVA with repeated measure did not show any significant difference between the groups (in secondary outcome measure) . . . Results of t-tests ~ . . showed no significant differences.” My reading of the results section of this manuscript is that there is no effect from the intervention, but the authors go on to “strongly
MaureenGiuffres PhD, RN, is a Clinical Research Consultant in private practice. Address correspondence to Maureen Gir&re, PhD, RN, 26361 High Banks Dr, ,%&bury, MD 21801. 0 1996 by American Socie@ of PeriAnesthesia Nurses. 1089-9472/96/1106-0009$03,00/0
1996: pp 417-420
417
MAUREEN
418
recommend” its use in the discussion section. It is difficult to believe that these statements came from the same manuscript, but they did. Consequently, it is important for you to carefully read the results section before reading the discussion section to make sure the authors are discussing what they found, not what they wanted to find or what they believe. The discussion section does not substitute for the results section. COMPARISONS
WITH
PAST
RESEARCH
It is appropriate for the discussion section to be used as an opportunity to point out and make sense of discrepancies between the findings of the current research and those of past research. Frequently, research appearing in the medical literature forgoes the literature review section almost entirely, saving the presentation of past research for the discussion section. For example the authors might say “Researcher Jones found A, this is in contrast to our findings of B . . . the reason for this discrepancy might be. . . .” It is at this point that the authors may see trends that were not clear before. For example, before doing their own research the authors may interpret the existing body of literature as a complete jumble. It may seem that half of the research has found support for a relationship and the other half has not. With the learning experience of their own research, what may become obvious to the researchers is if they define (and consequently measure) the major variable in one way the relationship works. If they define the variable in another way, the relationship does not work. The authors then realize that where they thought they were dealing with a single concept they may be dealing with two and the discrepancies in the past research become clear. This sort of presentation can be very helpful. EVALUATION OF THEORETICAL FRAMEWORK
The discussion is where the author has the opportunity to evaluate the findings of the research in terms of the theoretical framework. If the research was designed to test a theory the author might say, “If the relationships proposed in the xy theory were sound we would have found z, but we did not. This may be because .” The research done by nurses is seldom de-
GIUFFRE
signed to test theory. More often a theoretical framework is “used” to explain how the variables that are being investigated relate to each other. Unfortunately, more often than not the theoretical framework discussed early in the manuscript is never mentioned again. Look for a reiteration of the theoretical framework in the discussion section. The author should be able to tell you why the findings of this study give support the theoretical framework. If the findings were not as expected, the author should be able to clarify why this is so. The author will either have to find fault with the theory or with the methods of the current study. Usually the author will tell the reader that there were weaknesses in the current study’s design so that the study can no longer be viewed as a test of the theory. Occasionally, the authors are convinced that their study was a trne test of a theory. In this case, not finding support for the theory is given as refutation of the theory. If the author does not mention the theoretical framework, you should remind yourself of it. Ask yourself what are the theoretical consequences of the results. Ask yourself why the author has failed to clarify this for you. CRITIQUE
OF METHODS
When the research does not turn out the way the authors have hypothesized the authors will use the discussion section to explain this. As mentioned earlier they can either find fault with theoretical relationships proposed or they can find fault with their own research methods. Usually they do the latter. It is not uncommon, especially among novices, for the researcher to be so wedded to the intervention that failure to find support for the intervention causes the researcher to contradict everything said earlier in the manuscript. A number of years ago I attended a research presentation where the researcher was looking for a relationship between childbirth pain and serum endorphin levels. I cannot even remember if she expected the sernm levels to go up or down with the progressing labor and increasing pain, but she expected some type of relationship. She described her methods in detail, including the support for using the visual analogue scale (VAS) for the measurement of pain. The disappointed researcher found no relation-
THE DISCUSSION
SECTION
ship between VAS score and serum endorphin level. Because the researcher was convinced that a relationship had to exist, she proceeded to suggest that the problem lay with the VAS. Although it is perfectly legitimate and appropriate to critique the instruments used in the study, it is important not to get caught in the trap of believing that finding fault with the methods is the same as finding support for the original concept. That is, if the researcher had measured pain in a different way she would have been able to show the relationship she was looking for. A very clear (if older) example of this is by Reichert and Fuller.’ These authors were seeking to show that intravenous sodium bicarbonate (NaHC03) given to premature babies caused intraventricular hemorrhage (IVH). They found no support for their thesis. In the very lengthy discussion section they state “The results of the present study indicate that the danger of iatrogenically induced IVH is being prevented in most cases by conservative use of NaHC03 .” In other words, even if we did not find a relationship we still think that NaHC03 causes IVH, so the reason we did not find it is that the nurses inject it slowly. They have gone from trying to show the relationship to assuming the relationship exists. CLINICAL
IMPLICATIONS
Clinical journals frequently have a section separate from the discussion section called Clinical Implications. Technically, clinical implications are part of the discussion. I suspect this separation is a throw-back to the time when clinicians were not expected to understand or read the presentation of the methods or the findings and it was assumed that they just wanted to be told what to do with the findings. The clinical implications of a research study will depend on the clinicians’ practice area. Let us say the finding of a research study is that a increasing portion of young women are taking up smoking. The clinical implications for this are different for a school nurse than they are for a nurse who works in oncology. When reading the discussion the point is not to limit yourself to thinking of the clinical implications only in terms of those laid out by the authors. Come up with your own. The other important thing to keep in mind is
419
when reading the clinical implications (or any other portion) is not to be taken in by over-mterpretation of the findings. Sometimes authors can get a bit carried away, For example, take the findings by Albrecht and London.’ These authors claim to have found an atypical season of birth pattern amongst women with breast cancer. It seems as though there might be a greater incidence of breast cancer among American women born during the 6 months of November to April. That this relationship does not reach statistical significance suggests that on a statistical basis as well as a common sense basis that this finding may be spurious. A subgroup of women less than 45 years old without a family history does reach statistical significance. These authors combine their findings with another study that showed an excess of breast cancer among Japanese women born May through October. (Because Japan and the United States are in the same hemisphere and have the same seasons it would seem that the opposite findings warrant explanation-but none are forthcoming.) Although I believe that if you divide your data in enough ways you will surely find something mat reaches statistical significance, the point I want to make is about their clinical implications. Although at no point in this manuscript do these authors present the relative risks of breast cancer faced by the children of women who develop the disease, the authors concluded with the statement “If some women develop breast cancer because of a prenatal seasonal factor, which is environmental, their offspring need not be considered at increased risk for breast cancer.” Having seen what little evidence is required by some clinicians to change practice, this latter statement frightens me. Although authors do have some right to speculate, this speculation needs to be closely and logically related to the data or theory.3 This is clearly not the case here. DIRECTIONS
FOR FUTURE
RESEARCH
Finally the authors can use this section to briefly comment on research questions that logically spring from the findings of this study. For example, an early stage of a human drug study is usually an open study. Everyone knows who is receiving the experimental drug. Once the drug appears to have passed certain requirements for
420
safety and efficacy, the drug is then subjected to a blinded study. This is the next logical step. Unfortunately it is not uncommon for the researcher who has now conducted a preliminary study of 20 people to propose future research that neither logically follows nor is reasonably possible. The authors who propose the logical and doable next step generally has the intention of doing it herself. The author who suggests that the next step is collecting international data on the World Wide Web from 10,000 subjects on 200 variables including the subjects’ philosophy of life and current state of karma, is probably suggesting that this might make an interesting Master’s Thesis for you to consider. After all, she is busy reevaluating the data from the first 20 subjects.
MAUREEN
GIUFFRE
In summary, the discussion section should be a thoughtful section. It should be used to help you clarify theoretical and possible clinical issues. It is not the meat of the research. Do not confuse it with the results.
REFERENCES 1. Reichert EM, Fuller PW: Relationship of sodium bicxbonate to intraventricular hemorrhage in premature infants with respiratory distress syndrome. Nurs Res 29:357-361, 1980 2. Albrecht SA, London WP: Season of birth and laterality of breast cancer. Nurs Res 39:118-120, 1990 3. American Psychological Association: Publication Manual of the American Psychological Association (ed 4). Washington, DC, American Psychological Association, 1994
GIUFFRE,
PhD,
RN
The discussion section of a research manuscript is the last thing we read and often the section that leaves the most lasting impression. It is important to know what to reasonably expect to be in the discussion section. This article presents the areas that are usually covered. Vver-intexpretation of the results of a research project occurs all too often. This article will help you avoid being taken in by this problem. 0 1996 by American Society of PeriAnesthesia Nurses.
M
ANY OF US who regularly conduct research and publish the findings almost intuitively know what the discussion section of a manuscript is to contain. Nonetheless, it is still difficult to define. Briefly, the discussion section is where the author has the opportunity to take the raw facts and bring them to life. The major goal of this section is the presentation of the researchers’ conclusions based on the facts, and an elaboration of any theoretical or clinical implications. Although authors have some discretion in how they structure this section, it generally contains one or more of the following elements: a brief summary of the findings, comparison with past research, evaluation of the theoretical framework, critique of this research, clinical implications, and/or suggestions for future research. SUMMARY
OF FINDINGS
The discussion section is often begun with a repetition of the findings that have already been presented in the results section, leaving off the statistics. This summation should be very brief. As a result, the discussion section generally makes better and easier reading than the rest of the manuscript. Novice researchers and busy nurses are often tempted to read the review of
Jownd
of f.w~AmsThes;a
Mushg,
Vol 11, No 6 (December),
literature and skip right to the discussion section. After all, the author generally tells the reader what she found but leaves out all the technical jargon, right? Wrong! Or at least not always. Take this (disguised) example from a discussion section of a published JoPAN manuscript. “Based on the findings of these researchers, use of. . . is strongly recommended in the postoperative period.” Compare that statement from the discussion section with these statements from the results section. “ANOVA with repeated measure showed no significant difference in (primary outcome measure) among the groups throughout the study,” and, “ANOVA with repeated measure did not show any significant difference between the groups (in secondary outcome measure) . . . Results of t-tests ~ . . showed no significant differences.” My reading of the results section of this manuscript is that there is no effect from the intervention, but the authors go on to “strongly
MaureenGiuffres PhD, RN, is a Clinical Research Consultant in private practice. Address correspondence to Maureen Gir&re, PhD, RN, 26361 High Banks Dr, ,%&bury, MD 21801. 0 1996 by American Socie@ of PeriAnesthesia Nurses. 1089-9472/96/1106-0009$03,00/0
1996: pp 417-420
417
MAUREEN
418
recommend” its use in the discussion section. It is difficult to believe that these statements came from the same manuscript, but they did. Consequently, it is important for you to carefully read the results section before reading the discussion section to make sure the authors are discussing what they found, not what they wanted to find or what they believe. The discussion section does not substitute for the results section. COMPARISONS
WITH
PAST
RESEARCH
It is appropriate for the discussion section to be used as an opportunity to point out and make sense of discrepancies between the findings of the current research and those of past research. Frequently, research appearing in the medical literature forgoes the literature review section almost entirely, saving the presentation of past research for the discussion section. For example the authors might say “Researcher Jones found A, this is in contrast to our findings of B . . . the reason for this discrepancy might be. . . .” It is at this point that the authors may see trends that were not clear before. For example, before doing their own research the authors may interpret the existing body of literature as a complete jumble. It may seem that half of the research has found support for a relationship and the other half has not. With the learning experience of their own research, what may become obvious to the researchers is if they define (and consequently measure) the major variable in one way the relationship works. If they define the variable in another way, the relationship does not work. The authors then realize that where they thought they were dealing with a single concept they may be dealing with two and the discrepancies in the past research become clear. This sort of presentation can be very helpful. EVALUATION OF THEORETICAL FRAMEWORK
The discussion is where the author has the opportunity to evaluate the findings of the research in terms of the theoretical framework. If the research was designed to test a theory the author might say, “If the relationships proposed in the xy theory were sound we would have found z, but we did not. This may be because .” The research done by nurses is seldom de-
GIUFFRE
signed to test theory. More often a theoretical framework is “used” to explain how the variables that are being investigated relate to each other. Unfortunately, more often than not the theoretical framework discussed early in the manuscript is never mentioned again. Look for a reiteration of the theoretical framework in the discussion section. The author should be able to tell you why the findings of this study give support the theoretical framework. If the findings were not as expected, the author should be able to clarify why this is so. The author will either have to find fault with the theory or with the methods of the current study. Usually the author will tell the reader that there were weaknesses in the current study’s design so that the study can no longer be viewed as a test of the theory. Occasionally, the authors are convinced that their study was a trne test of a theory. In this case, not finding support for the theory is given as refutation of the theory. If the author does not mention the theoretical framework, you should remind yourself of it. Ask yourself what are the theoretical consequences of the results. Ask yourself why the author has failed to clarify this for you. CRITIQUE
OF METHODS
When the research does not turn out the way the authors have hypothesized the authors will use the discussion section to explain this. As mentioned earlier they can either find fault with theoretical relationships proposed or they can find fault with their own research methods. Usually they do the latter. It is not uncommon, especially among novices, for the researcher to be so wedded to the intervention that failure to find support for the intervention causes the researcher to contradict everything said earlier in the manuscript. A number of years ago I attended a research presentation where the researcher was looking for a relationship between childbirth pain and serum endorphin levels. I cannot even remember if she expected the sernm levels to go up or down with the progressing labor and increasing pain, but she expected some type of relationship. She described her methods in detail, including the support for using the visual analogue scale (VAS) for the measurement of pain. The disappointed researcher found no relation-
THE DISCUSSION
SECTION
ship between VAS score and serum endorphin level. Because the researcher was convinced that a relationship had to exist, she proceeded to suggest that the problem lay with the VAS. Although it is perfectly legitimate and appropriate to critique the instruments used in the study, it is important not to get caught in the trap of believing that finding fault with the methods is the same as finding support for the original concept. That is, if the researcher had measured pain in a different way she would have been able to show the relationship she was looking for. A very clear (if older) example of this is by Reichert and Fuller.’ These authors were seeking to show that intravenous sodium bicarbonate (NaHC03) given to premature babies caused intraventricular hemorrhage (IVH). They found no support for their thesis. In the very lengthy discussion section they state “The results of the present study indicate that the danger of iatrogenically induced IVH is being prevented in most cases by conservative use of NaHC03 .” In other words, even if we did not find a relationship we still think that NaHC03 causes IVH, so the reason we did not find it is that the nurses inject it slowly. They have gone from trying to show the relationship to assuming the relationship exists. CLINICAL
IMPLICATIONS
Clinical journals frequently have a section separate from the discussion section called Clinical Implications. Technically, clinical implications are part of the discussion. I suspect this separation is a throw-back to the time when clinicians were not expected to understand or read the presentation of the methods or the findings and it was assumed that they just wanted to be told what to do with the findings. The clinical implications of a research study will depend on the clinicians’ practice area. Let us say the finding of a research study is that a increasing portion of young women are taking up smoking. The clinical implications for this are different for a school nurse than they are for a nurse who works in oncology. When reading the discussion the point is not to limit yourself to thinking of the clinical implications only in terms of those laid out by the authors. Come up with your own. The other important thing to keep in mind is
419
when reading the clinical implications (or any other portion) is not to be taken in by over-mterpretation of the findings. Sometimes authors can get a bit carried away, For example, take the findings by Albrecht and London.’ These authors claim to have found an atypical season of birth pattern amongst women with breast cancer. It seems as though there might be a greater incidence of breast cancer among American women born during the 6 months of November to April. That this relationship does not reach statistical significance suggests that on a statistical basis as well as a common sense basis that this finding may be spurious. A subgroup of women less than 45 years old without a family history does reach statistical significance. These authors combine their findings with another study that showed an excess of breast cancer among Japanese women born May through October. (Because Japan and the United States are in the same hemisphere and have the same seasons it would seem that the opposite findings warrant explanation-but none are forthcoming.) Although I believe that if you divide your data in enough ways you will surely find something mat reaches statistical significance, the point I want to make is about their clinical implications. Although at no point in this manuscript do these authors present the relative risks of breast cancer faced by the children of women who develop the disease, the authors concluded with the statement “If some women develop breast cancer because of a prenatal seasonal factor, which is environmental, their offspring need not be considered at increased risk for breast cancer.” Having seen what little evidence is required by some clinicians to change practice, this latter statement frightens me. Although authors do have some right to speculate, this speculation needs to be closely and logically related to the data or theory.3 This is clearly not the case here. DIRECTIONS
FOR FUTURE
RESEARCH
Finally the authors can use this section to briefly comment on research questions that logically spring from the findings of this study. For example, an early stage of a human drug study is usually an open study. Everyone knows who is receiving the experimental drug. Once the drug appears to have passed certain requirements for
420
safety and efficacy, the drug is then subjected to a blinded study. This is the next logical step. Unfortunately it is not uncommon for the researcher who has now conducted a preliminary study of 20 people to propose future research that neither logically follows nor is reasonably possible. The authors who propose the logical and doable next step generally has the intention of doing it herself. The author who suggests that the next step is collecting international data on the World Wide Web from 10,000 subjects on 200 variables including the subjects’ philosophy of life and current state of karma, is probably suggesting that this might make an interesting Master’s Thesis for you to consider. After all, she is busy reevaluating the data from the first 20 subjects.
MAUREEN
GIUFFRE
In summary, the discussion section should be a thoughtful section. It should be used to help you clarify theoretical and possible clinical issues. It is not the meat of the research. Do not confuse it with the results.
REFERENCES 1. Reichert EM, Fuller PW: Relationship of sodium bicxbonate to intraventricular hemorrhage in premature infants with respiratory distress syndrome. Nurs Res 29:357-361, 1980 2. Albrecht SA, London WP: Season of birth and laterality of breast cancer. Nurs Res 39:118-120, 1990 3. American Psychological Association: Publication Manual of the American Psychological Association (ed 4). Washington, DC, American Psychological Association, 1994