International Journal of Orthopaedic and Trauma Nursing (2012) 16, 160–161
www.elsevier.com/locate/ijotn
LETTER TO THE EDITOR
Online and face-to-face orthopaedic surgery education methods Erin Austin, L. Lee Glenn
*
College of Nursing and Institute for Quantitative Biology, East Tennessee State University, P.O. Box 70658, Johnson City, TN 37604, United States The recent study by Heikkinen et al. (2012) concluded that symptoms related to surgery were not impacted by whether internet-based or face-toface patient education was used. However, this conclusion is not actually supported by the findings in their study because of missing statistical information and very low measurement validity for pain and disability, possibly due to questionnaire miswording. First, the low validity of the measurements is obvious by viewing their Table 1. For example, the percentage of patients reporting a high disability level nearly quadruples from 5.6% to 20% after the internet-based education session compared to before the session. This difference is statistically significant by the chi squared test at p < 0.001. If the disability measurement is taken as valid, then the conclusion would be an internet education session quadruples a patient’s disability level. Even in the face-to-face patient education session the percentage of patients reporting high levels of disability doubled from 1.4% to 2.9%. There are dozens of other examples which indicate that the pain and disabilities measurements were not consistent or accurate and consequently had little or no validity. A possible explanation for the validity problems is found in the VAS questionnaire wording and procedures. One of the most important determinants
* Corresponding author. Tel.: +1 (423) 439 4871; fax: +1 (423) 283 0214. E-mail address:
[email protected] (L. Lee Glenn).
of the measurement quality of any scale is the wording of the question that is posed to the participant, but the authors never state what question or instruction was given to the patients before the questionnaire was answered. The only information provided is an unclear statement that the instructions were based on theoretical literature. This is not of much help in understanding what exactly the patients were asked, and in particular, whether the same instruction was given consistently to all patients at all measurement times. With regard to the response choices, the authors state that 100 indicates intolerable intensity and that a 0 indicates low intensity. If this is correct then there is no way for a patient can indicate when they have no pain or disability. This may explain the sum of the measurement validity weakness. Also, instead of conducting statistics using the 0–100 mm data, the authors degrade the data by nominalizing into three arbitrary categories (low, moderate, and high), the justification for which was not based on research evidence. These shortcomings would be fine if alphas, r’s or kappas showing measurement reliability or validity were provided, but despite the ease of calculating such values for a questionnaire of this type, none were reported. The content validity was indicated to have been tested, but there is no detailed information of the opinions of the experts cited, nor a description of the experts themselves or why they were considered to be experts. All of the above are plausible sources of the low validity of the questionnaire.
1878-1241/$ - see front matter c 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.ijotn.2012.04.004
Online and face-to-face orthopaedic surgery education methods Third, with regard to missing statistics, there are no statistics missing when the point is made that surgery causes pain. The authors are very specific with their statistics and present 11 odds ratios and 18 individual p values in a single paragraph, and even more in Table 1, all of which show that pain at first increases after surgery and then decreases with time. Of course, the demonstration that surgery causes pain and disability is a trivial, obvious finding that did not address the stated hypothesis for the study. For the findings that do address the hypothesis, very little information is given. The p values for the differences are provided in ranges in a few places, with no odds ratios provided. Therefore, the conclusion in the study, that the two educational programs have the same effect, cannot be supported for two reasons: (1) the absence of sufficient detail in the crucial statistics the authors calculated and (2) because the p values that were provided, such as 0.033 and 0.084, indicate that education may have had a small, but non-significant, effect. Given the lack of statistical information on the study hypothesis, we recomputed the statistics by returning the data to the interval level by assigning 1 point for low, 10 for moderate and 100 for high levels of pain or disability. The average scores were calculated across time and all five categories of pain and disability for both the internet and faceto-face group. A simple chi squared test showed that patients with face-to-face education had a 32% reduction in pain and disability across all time periods (p < 0.001), and also for four of the five measures for the majority of the time periods. Consequently, no matter how you view it, the author’s conclusions are not supported by the data.
161
The study has many strengths, including a large sample size, low dropout rate, clear explanation of why the study is important, the manner in which the hypothesis is constructed on the basis of previous studies, the frank specification of many limitations, the quality of the writing, the advanced statistical methods used and other strengths. Despite these strengths, the conclusion that internet-based patient education and face-to-face education have the same effectiveness in ambulatory orthopaedic surgery patients is not supported by the findings. If anything, face-to-face education is more effective in reducing pain and disability. Health care providers are advised against replacing face-to-face patient education with internet-based education on the basis of the study findings at this time.
Conflict of Interest Statement The authors have no financial or other form of conflict of interest with regard to this study.
Funding Source No funding was obtained for this study.
Reference Heikkinen, K. et al., 2012. Ambulatory orthopaedic surgery patients’ symptoms with two different patient education methods. International Journal of Orthopaedic and Trauma Nursing 16 (1), 13–20.