- Email: [email protected]
Impact of medical audit on electrodiagnostic medicine in polyneuropathy
Clinical Neurophysiology 122 (2011) 2523–2529
Contents lists available at ScienceDirect
Clinical Neurophysiology journal homepage: www.elsevier.com/locate/clinph
Impact of medical audit on electrodiagnostic medicine in polyneuropathy Kirsten Pugdahl a, Anders Fuglsang-Frederiksen a,⇑, Hatice Tankisi a, Birger Johnsen a, Mamede de Carvalho b, Peter R.W. Fawcett c, Annick Labarre-Vila d, Rocco Liguori e, Wilfred Nix f, Ian S. Schofield c a
Department of Clinical Neurophysiology, Aarhus University Hospital, Aarhus, Denmark Department of Neurology, Hospital de Santa Maria, Laboratory of Electromyography, Institute of Molecular Medicine, Faculty of Medicine, Lisbon, Portugal c Department of Clinical Neurophysiology, Newcastle General Hospital, Newcastle upon Tyne, UK d Unité ENMG et Pathologie neuromusculaire, Département de Neurologie, Centre Hospitalier Universitaire, Grenoble, France e Department of Neurological Sciences, University of Bologna, Bologna, Italy f Department of Neurology, University Clinics Mainz, Mainz, Germany b
a r t i c l e
i n f o
Article history: Accepted 22 May 2011 Available online 23 June 2011 Keywords: Polyneuropathy Neurophysiology Electromyography Nerve conduction studies Medical audit
h i g h l i g h t s Studies on quality development are important, but sparse, in clinical neurophysiology. Experienced neurophysiologists change their practice after participation in peer review medical audit. Profound and lasting changes in electrodiagnostic practice can be obtained but require several years to develop.
a b s t r a c t Objective: The aim of the study was to investigate whether experienced physicians’ electrodiagnostic practice and criteria can be influenced by international collaboration involving peer review medical audit. Methods: Data was obtained from the ESTEEM project, an ongoing collaboration since 1991 among European neurophysiologists concerned with quality improvement in electrodiagnostic medicine. Three sets of the physicians’ polyneuropathy examinations performed with intervals of 2–4 years were analysed. Results: Changes towards increased homogeneity among the physicians were found in (1) the average number of studies performed per patient and the number of abnormal studies required for accepting the diagnosis of polyneuropathy, with the most pronounced changes seen for abnormal motor nerve segments, abnormal F-wave studies, and electromyographic studies, and (2) the agreement on pathophysiological interpretation of nerve conduction studies and classification of polyneuropathy. Conclusions: Changes towards increased homogeneity contributed to years of participation in peer review medical audit, were seen among a group of experienced physicians. Peer review medical audit as carried out here is however difficult to scale up. Therefore guidelines or minimal criteria should ideally supplement a medical audit process to disseminate the results obtained to a larger audience. Significance: These results support the role of international peer review medical audit in quality improvement of electrodiagnostic medicine. Ó 2011 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved.
1. Introduction Polyneuropathies are common disorders of the peripheral nerve, affecting axons, myelin sheaths, or both. The population prevalence is about 0.04–0.1%, but in people older than 55 years the prevalence rises to about 1.6% (Beghi and Monticelli, 1998; ⇑ Corresponding author. Address: Department of Neurophysiology, Aarhus University Hospital, Nørrebrogade 44, Building 10, 8000 Aarhus C, Denmark. Tel.: +45 8949 3100; fax: +45 8949 3140. E-mail addresses: [email protected], [email protected] (A. FuglsangFrederiksen).
Kurtzke, 1984; MacDonald et al., 2000). The clinical examination is very useful to determine the severity of a polyneuropathy and to test for predominant sensory or motor involvement; however, it can be limited in identifying the type of the neuropathy (axonal vs. demyelinating). As opposed to this limitation of the clinical examination, the neurophysiological investigation approaches objectively the pathophysiological dysfunction of the nerve, as nerve conduction studies measure the function of the peripheral nerves directly without subjective bias and without contamination by the central nervous system. The neurophysiological examination is, however, prone to show a great deal of variation as a series of diagnostic tests is applied to a
1388-2457/$36.00 Ó 2011 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.clinph.2011.05.018
2524
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
clinical problem, with the performance and selection of tests being highly operator dependent (Veloso et al., 1995). In addition, a wide range of different neurophysiological techniques are available and a huge number different nerve segments can be selected for examination (Johnsen et al., 1994a). A physician’s examination strategy often relies on more or less well-defined rules based mainly on tradition and personal experience, and only to a lesser extent on documented results. Similarly, criteria for classification of polyneuropathy are not consistent in the literature. Various criteria for definition of demyelination have been proposed but none are consistently used in practice (Fuglsang-Frederiksen and Pugdahl, 2011). In contrast, the literature on criteria for axonal loss is sparse (Hadden et al., 1998; Ho et al., 1997; Tankisi et al., 2005; van der Meche et al., 2001). Once an optimal medical practice is identified it has to be integrated into practice by appropriately changing physician behaviour (Landry and Sibbald, 2002). However, changing of physicians’ behaviour is a complex and difficult task that is strongly dependent on the willingness to learn from other physicians and compromise on own approaches. Peer review medical audit has the potential of changing physician’s behaviour as a widely accepted method of quality improvement in medical practice, because it encourages professional autonomy and supports critical insight and appraisal of quality of care (Beyer et al., 2003). It appears logical that health care professionals given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines would be prompted to modify their practice (Jamtvedt et al., 2006b). Yet, medical audit and feedback is not consistently effective in changing clinical behaviours and the effects are generally small to moderate and dependent on the intensity of feedback (Bloom, 2005; Jamtvedt et al., 2006a).
and all physicians have reported that they have changed their routines somewhat during the project. The aim of the present study was to examine whether the variation among the physicians in the ESTEEM project had been further reduced in a recently submitted and peer reviewed set of polyneuropathy examinations, i.e. to examine to what extent continuing international peer review medical audit can change experienced physicians’ routine. 2. Methods 2.1. Data sets Three sets of peer reviewed examinations with a consensus diagnosis of definite or probable polyneuropathy were compared. Each set was based on 12 prospectively collected consecutive examinations with a referral diagnosis of polyneuropathy from each physician. The sets are specified in Table 1. The patients were examined with standard up-to-date EMG equipment used routinely in the participating labs and data transferred to a standardised data structure implemented in a PC program consisting, in accordance with the clinical neurophysiological consultation, of three parts (Johnsen et al., 1994b): (a) General data: Patient age, sex, and height, as well as free text clinical information. (b) Examination data: Measured values from each EMG or nerve conduction study with accompanying locally used reference values, percentage deviation from mean of controls, and a symbolic value (normal, increased, decreased, borderline increased/decreased, or inconclusive), and data on examination conditions, such as information on muscle force and atrophy, patient cooperation for EMG studies, electrode type (needle, surface), temperature, and segment length for nerve conduction studies. (c) Inferred data: Each individual EMG or nerve conduction study is given a pathophysiological test conclusion (PTC) based on the electrophysiological data. The available PTC’s for a nerve conduction study are: ’Normal’, ‘‘Demyelinating’, ‘Axonal loss’, ‘Neuropathic’, or ‘Other’, with the term neuropathic referring to a grey zone, where at distinction between demyelination and axonal loss is not possible (Tankisi et al., 2005). Finally, one or more diagnoses combined with the logical operators ‘‘and/or’’ are given.
1.1. The ESTEEM project The European multicenter project ESTEEM (European standardised telematic tool to evaluate electrodiagnostic methods) was initiated in 1992 with the aim of improving the quality of the neurophysiological examination (Vingtoft et al., 1994, 1995; Fuglsang-Frederiksen et al., 1996). The main objectives of the ESTEEM project were (1) to create an environment for storage and exchange of data from electrodiagnostic examinations and (2) to establish a peer review medical audit procedure across borders in order to build a multicentre reference database consisting of electrodiagnostic examinations, in which all contained decisions had been evaluated and a consensus diagnosis obtained (Veloso et al., 1995). Since the beginning of the project 7 neurophysiologists from 6 European countries have prospectively collected and discussed samples of their examinations. All are experienced specialists who, when joining the project, had practised clinical neurophysiology for between 7 and 16 years. Changes towards a more uniform practice among the physicians, attributed to the years of collaboration, have been indicated (Finnerup et al., 1998; Pugdahl et al., 2005; Tankisi et al., 2006),
2.2. Peer review Examinations in the ESTEEM project are peer reviewed as follows: An examination sampled at one of the participating laboratories is sent via the Internet to Aarhus, Denmark, where a moderator validates the data and stores a copy of the examination in the data base. The moderator prepares the examination for further analysis
Table 1 Examinations provided by the individual physicians (P1–P7) in the three sets. Physician P1 P2 P3 P4 P5 P6 P7 Total *
Set 1 No. of patients
Exam. time
Set 2 No. of patients
Exam. time
Set 3 No. of patients
Exam. time
12 8 8 12 10 12 7 69*
1992–93 1992–93 1992–93 1992 1994 1989–93 1992–95 1989–95
10 8 10 11 12 9 10 70*
1994–96 1995–97 1995–97 1996 1996 1996–97 1996–97 1994–97
12 12 11 12 11 12 11 81*
2003 2001–02 2003–04 2003 2002–04 2003 2002 2001–04
No difference in the number of examinations provided by the individual physicians among the three sets (P = 1.0).
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529 Table 2 Number of nerve conduction studies in the three sets.
3. Results
Set
Motor NCS
Sensory NCS Surface recording
Sensory NCS Needle recording
F-wave studies
Total
1
636 (52.2%) 724 (52.5%) 771 (50.1%)
255 (20.9%)
149 (12,2%)
178 (14.6%)
1218
296 (21.5%)
128 (9.3%)
231 (16.8%)
1379
355 (23.1%)
122 (7.9 %)
292 (19.1%)
1540
2 3
2525
by removing laboratory identity, the sampling physician’s test conclusions and diagnoses, as well as all clinical information except the reason for referral, thus leaving only patient information, examination conditions, and measured data with the local reference values and the assigned symbolic value. The modified examination is then sent to all physicians for individual blind interpretation. The resulting diagnoses serve as the starting point at biannual workshop discussions among all participating physicians with the aim of reaching consensus on the diagnosis. First consensus on a diagnosis based solely on electrophysiological data is sought, and afterwards the clinical information is reviewed to arrive at a final consensus diagnosis. For some patients a clinical follow-up is further requested. 2.3. Data analysis The following possible changes between the three sets were assessed for the group of physicians and for each individual physician: A change in the average number of motor and sensory nerve segments, F-wave studies, and muscles examined per patient. A change in the average number of abnormal motor and sensory nerve segments, F-wave studies, and muscles per patient. A change in the agreement on the pathophysiological interpretation of a nerve conduction study. For each nerve conduction study, the PTC (axonal loss, demyelinating, neuropathic or normal) given by each physician at the blind review was pair-wise compared with each of the other physicians’ interpretation. The agreement among the group of physicians was calculated separately for the different types of PTCs in general and for motor and sensory nerve conduction studies and F-wave studies separately. These analyses included only 6 physicians as one (P7) used software for test interpretation. A change in the agreement on the pathophysiological classification of a polyneuropathy as axonal, demyelinating, or unclassifiable. The classification in each individual physician’s interpretation diagnosis was compared with the consensus classification and pair-wise with each of the other physicians’ classifications. Each physician’s (P1–P6) agreement rate was determined as the average of his pair-wise agreements with the other physicians. 2.4. Statistical analysis The mean number of examined and abnormal studies for the group of physicians was compared using Kruskal–Wallis nonparametric ANOVA, while the physicians’ individual changes were estimated using the Mann–Whitney U test. Changes in proportions, i.e. distribution of test types and agreement on PTCs were determined using chi-square tests or Fisher’s exact test. Changes towards improved agreement on PNP classification were estimated using the Wilcoxon signed rank test. For all tests, a P-value < 0.05 was considered significant.
3.1. Proportion of test types The number of motor nerve studies, F-wave studies, and sensory nerve conduction studies performed with surface and near-nerve needle technique respectively in the 3 sets are shown in Table 2. The proportion of motor nerve conduction studies was constant in all 3 sets, comprising around 50% of all tests conducted, however the proportion of F-wave studies increased from set 1 (14.6%) to set 3 (19.1%) (P = 0.012). The proportion of sensory nerve conduction studies performed was constant – around 30% – in all 3 sets, but a lower proportion of the sensory nerve conduction studies were performed using the near-nerve needle technique in set 3 (26%) compared to set 1 (37%) (P = 0.01) (Table 2). 3.2. Motor nerve segments The mean number of examined motor nerve segments per patient increased from set 1 (mean 7.7; range 3.3–13.9) to set 2 (mean 8.8; range 4.9–12.5) (P = 0.039) and from set 1 to set 3 (mean 9.1; range 6.3–13.8) (P = 0.04). Individual changes were seen for two physicians (P1 and P2), who both examined a higher number of motor nerve segments in set 2 and set 3 compared to set 1 (P < 0.05). For P2 an increase was further seen from set 2 to set 3 (P = 0.008). Despite the changes a significant variation among the physicians were still seen in all 3 sets (P = 0.000) (Fig. 1). The mean number of abnormal motor nerve segments per patient increased from set 1 (mean 5.2; range 1.8–11.5) to set 2 (mean 6.5; range 4–8.9) (P = 0.009). Individual changes were seen for three physicians (P1, P2, and P4). P1 and P2 both examined a higher number of abnormal motor nerve segments in set 2 and 3 compared to set 1 (P < 0.01). P4 had a higher number of abnormal motor nerve segments in set 2 compared to set 1 (P = 0.007); however this number dropped in set 3 and became similar to the number in set 1 (P = 0.03). Variation among the physicians was found in the first two sets (P < 0.01), however in the last set there was no significant variation (P = 0.13) (Fig. 1). 3.3. F-wave studies The mean number of F-wave studies performed per patient increased from set 1 (mean 2.1; range 0.3–4.7) to set 2 (mean 2.7; range 1.6–4.6) (P = 0.039) and from set 2 to set 3 (mean 3.5; range 2.4–4.5) (P = 0.014). Individual changes were seen for two physicians (P1 and P2), who both studied more F-waves in set 2 and set 3 compared to set 1 (P < 0.01). For P2 an increase was further seen from set 2 to set 3 (P = 0.006). A variation among the physicians was seen in all 3 sets (P < 0.01) (Fig. 1). The mean number of abnormal F-wave studies per patient increased from set 1 (mean 1.8; range 0.3–4.6) to set 3 (mean 2.5; range 1.6–3.8) (P = 0.008). Individual changes were seen for three physicians (P1, P2, and P5). All had a higher number of abnormal F-wave studies in set 3 compared to set 1 (P < 0.05). Variation among the physicians was found in the first two sets (P < 0.01), however in the last set there was no significant variation (P = 0.054) (Fig. 1). 3.4. Sensory nerve segments The number of sensory nerve segments examined per patient increased from set 1 (mean 4.8; range 3.1–8.1) to set 3 (mean 8.8; range 4.9–12.5) (P = 0.031). Individual changes were seen for one physician (P1), who examined a higher number of sensory nerve segments in set 2 and 3 compared to set 1 (P < 0.01). Variation among the physicians was found in all 3 sets (P = 0.000)(Fig. 1).
2526
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
Motor Nerve Segments Abnormal
16
16
14
14
12
12
10
10
8
8
6
6
4
4
2
2
Examined
Avg. no per physician
Avg. no per physician
Examined
F-wave Studies
0
0
1
2
Abnormal
6
6
4
4
2
2
0
0
1
3
2
3
1
2
Set
Set
1
3
6
6
4
4
2
2
0
0
2
Set
3
Avg. no per physician
Avg. no per physician
8
1
Examined
Abnormal
8
Abnormal
6
6
4
4
2
2
0
1
2
3
Muscles
Sensory Nerve Segments Examined
2
Set
Set
3
Set
0 1
2
3
1
Set
2
3
Set
Fig. 1. Each of the physician’s average number per patient of examined and abnormal motor nerve segments, F-wave studies, sensory nerve segments and muscles in each set is plotted. The physicians are represented by the following symbols: P1 (d), P2 (s), P3 (.), P4 (O), P5 (j), P6 (h), P7 ().
No change in the number of abnormal sensory nerve segments could be documented for the group of physicians; however, one physician (P1) examined a higher number of abnormal nerve segments in set 3 than in set 1 (P = 0.009). Variation among the physicians was found in all three sets (P < 0.05) (Fig. 1).
3.5. Electromyography There were no changes in the mean number of examined or abnormal muscles for the group of physicians in the three sets. However, in contrast to what was found in the first two sets (P < 0.01), no variation among the physicians was found in the last set regarding both number of examined and abnormal muscles (P > 0.6). In addition, several individual changes were present: Three physicians (P5, P6, and P7) changed their average number of examined muscles from set 2 to set 3 (P < 0.05), and P5 also changed from set 1 to set 2 (P = 0.01). For abnormal muscles, changes were seen for 4 physicians. P5 and P6 changed from set 2 to set 3 and P3 and P7 changed from set 1 to set 3 (P < 0.05).
3.6. Interpretation of nerve conduction studies The agreement on pathophysiological interpretation of all types of nerve conduction studies increased in the group of physicians from set 1 to set 2 for interpretation as axonal loss, demyelination or neuropathic (P < 0.01), however the agreement decreased slightly from set 2 to set 3 for all three test conclusions (P < 0.05). It remained, however, significantly better in the third set than in the first (P < 0.01) (Fig. 2). The agreement on interpretation as normal was high in all three set (94.1%–94.7%). Besides from normal, the highest agreement was found on interpretation as neuropathic (81.2%–87.2%) in the three sets. The agreement on axonal loss ranged from 61.3% in set 1 to 78.3% in set 2, while the agreement on demyelination ranged from 46.5% in set 1 to 56.2% in set 2 (Fig. 2). When analysing the agreement on the separate test types, increases on the agreement on interpretation of motor nerve conduction studies as demyelination, axonal loss and neuropathic were seen from set 1 to set 2 (P < 0.01) with further increases from set 2 to set 3 for demyelination and axonal loss (P = 0.000). The
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
Motor NCS
% Agreement
All NCS 100
100
80
80
60
60
40
40
20
20
0
0
1
2
3
Set
1
2
3
Set
Fig. 2. The mean agreement for the group of physicians on pathophysiological interpretation of nerve conduction studies as normal (O), neuropathic (s), demyelination (}) or axonal loss (h) in each of the three sets of polyneuropathy examinations. Results are shown for all nerve conduction studies (left) and motor nerve conduction studies (right).
agreement on axonal loss increased from 69.8% in set to 90.8% in set 3, for demyelination from 50.13% in set 1 to 76.1% in set 3 and for neuropathic from 75.6% in set 1 to 83.2% in set 3 (Fig. 2). For F-wave studies, there was an increase in the agreement on interpretation as demyelination from 27.6% in set 1 to 59.1% in set 2 (P < 0.01) that again decreased in set 3 to 45.3% (P = 0.002) but remained higher than in set 1 (P = 0.000). There was high agreement on interpretation as neuropathic (90.6%–93.7%) and normal (94.0%–96.7%) in all three sets with no changes observed. For sensory nerve conduction studies, increases on the agreement from set 1 to set 2 were seen for interpretation as axonal loss, neuropathic and normal (P < 0.05). The agreement decreased again from set 2 to set 3 for axonal loss and neuropathic findings (P < 0.05), but remained higher for axonal loss in set 3 compared to set 1 (P < 0.05). There was a good agreement on interpretation as neuropathic (81.1%–89.5%) and normal (87.0%–94.6%) in all three sets, while the agreement on axonal loss ranged from 51.3% to 62.9% and on demyelination from 44.2% to 51.4% (results no shown). 3.7. Classification of polyneuropathy An increased agreement on classification of all polyneuropathies was seen in set 2 and set 3 compared to set1 for the physicians as a group; both regarding agreement with the consensus group (P < 0.05) and with the rest of the physicians (P < 0.000) and for all individual physicians. One physician (P6), however, decreased his agreement with the consensus group from set 2 to set 3 (P = 0.043). The agreement between the individual physicians and the consensus group ranged from 49.3%–75.7% in set 1 to 77.5%– 89.6% in set 3 (Fig. 3). A similar picture was seen on the agreement on unclassified polyneuropathies, where an increased agreement among the physicians was seen from set 1 to sets 2 and 3 (P = 0.000). One physician (P1) further increased his agreement with the consensus group from set 2 to set 3 (P = 0.043). The agreement between the individual physicians and the consensus group on which polyneuropathies were unclassified ranged from 32.4% to 69.8% in set 1 to 73.3% to 87.7% in set 3 (Fig. 3). For demyelinating polyneuropathy increases in the agreement were seen for the
2527
agreement between the physicians and the consensus group from set 1 to sets 2 and 3 (P < 0.05), and for the physicians’ agreement with each other from set 1 to sets 2 and 3 (P = 0.000), as well as from set 2 to set 3 (P = 0.014). Individual changes were seen for all physicians: P6 increased his agreement from set 1 to set 2 and P3 from set 2 to set 3, while all physicians except P4 had a higher agreement in the third set than in the first (P = 0.043). The agreement between the individual physicians and the consensus group on classification as demyelinating polyneuropathy ranged from 42.9%–60% in set 1 to 50%–91.7% in set 3 (Fig. 3). For classification as axonal polyneuropathy, an increased agreement between the physicians and the consensus group was seen from set 1 to set 2 (P < 0.05), and between the physicians from set 1 to sets 2 and 3 (P < 0.01). Individual changes were seen for three physicians: P1 and P6 had higher agreements in sets 2 and 3 than in set 1 and physician P2 increased his agreement from set 1 to set 2 (P = 0.043). The agreement between the individual physicians and the consensus group ranged from 12.5%–69.2% in set 1 to 11.1%– 80% in set 3. Very low agreement in the third set was however only seen for one physician (P3), except from this all agreements were P50% (Fig. 3). 4. Discussion Based on three sets of polyneuropathy examinations examined with intervals of 2–4 years, the present study demonstrated that a more uniform practice was introduced in a group of experienced neurophysiologists working closely together in the European multicentre project ESTEEM since 1992. For the number of nerve conduction studies performed, the most pronounced changes were seen for F-wave studies, where a higher number was performed in the last set than in the first two sets. Regarding performed motor and sensory nerve conduction studies, a more uniform practice was adopted as the physicians initially performing the fewest studies increased their number of studies during the project. Some variation, however, persisted in the last set, much of which can probably be explained by the use of different techniques, especially use of surface versus needle electrodes for sensory nerve conduction studies. More pronounced changes were seen for the number of abnormal studies. Most importantly no variation was present among the physicians for abnormal motor nerve conduction studies and F-wave studies in the last set, whereas a significant variation was seen in the first two sets. Variation among the physicians persisted for the number of abnormal sensory nerve conduction studies, however the physicians who initially required the lowest number of studies increased towards the mean in the last set. The results indicate that the physicians have influenced each other in the selection of which nerves to study, and in the number of abnormal studies required for diagnosing. Agreement on the optimal diagnostic strategy, i.e. the composition and amount of abnormal and normal test results required by a physician for making a diagnosis, is important for the quality of the diagnosis. Finally, a more homogenous practice was adopted in the number of performed and abnormal EMG studies in polyneuropathy. The changes were towards a more limited use of EMG, either because the physicians started to relying less on the use of EMG for classification of polyneuropathy (Tankisi et al., 2007) or a tendency to use a less invasive approach. The agreement on pathophysiological interpretation of nerve conduction studies was generally good in the last two sets of examinations. When analysing the agreement separately for the different test types, the best agreement was found on interpretation of motor nerve conduction studies, indicating that the physicians started to apply common criteria in a more consistent fashion during their participation in ESTEEM. However, in sensory nerve conduction studies a considerable disagreement remained.
2528
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
Unclassified
% Agreement
All Polyneuropathies
Axonal
100
100
100
80
80
80
80
60
60
60
60
40
40
40
40
20
20
20
20
0 SET 2
SET 3
0
0
0 SET 1
SET 1
SET 2
SET 1
SET 3
SET 2
SET 3
100
100
100
100
80
80
80
80
60
60
60
60
40
40
40
40
20
20
20
20
SET 1
SET 2
SET 3
SET 1
SET 2
SET 3
g
% Agreement
Demyelinating
100
0
0 SET 1
SET 2
SET 3
0
0 SET 1
SET 2
SET 3
SET 1
SET 2
SET 3
Fig. 3. Part A shows each physician’s agreement with the consensus group on classification of polyneuropathy in general (all) and on unclassified, demyelinating and axonal polyneuropathy in each of the three sets. The physicians are represented by the following symbols: P1 (d), P2 (s), P3 (.), P4 (O), P5 (j), P6 (h). Part B shows all pair-wise agreements between the physicians in each set.
This may be due to a general lack of well-defined international criteria for sensory nerve conduction studies (Tankisi et al., 2005; Fuglsang-Frederiksen and Pugdahl, 2011) but also due to the use of different techniques. It may be argued that choosing the proper pathophysiological test conclusion is of minor significance compared with the significance of making the correct diagnosis. Nevertheless, exact knowledge about which pathophysiological information that can be inferred from individual nerve tests may promote a broader understanding of electrophysiology with advantages for education of clinical neurophysiologists, and for selection of patients for scientific studies (Johnsen et al., 1995). As a pathological test conclusion concerns only the findings of a single nerve test while a diagnosis depends on a constellation of abnormalities in a series of motor and sensory nerves together with EMG findings, a better agreement may be expected on the diagnosis level. Certainly, the agreement among the physicians appeared to be better on the classification of polyneuropathy than on the classification of an individual nerve segment. A good agreement on classification of demyelinating polyneuropathy was present in the last set for most of the physicians. Similarly, an overall good agreement rate on classification of axonal neuropathy was seen for most physicians in the last set, however one physician remained having a very low agreement with the consensus group and with several of the other physicians. The best agreement
was, however, seen on unclassified polyneuropathies. An earlier study showed that the consensus group was more cautious in assigning a specific classification to a polyneuropathy than most of the individual physicians (Tankisi et al., 2003). This is probably because the group of peers was more likely than the individual physicians to follow classification criteria strictly and only classified a polyneuropathy in case of strong evidence of demyelination or axonal loss that everybody could agree on. The increased agreement on unclassified polyneuropathies shown here indicates that the physicians in a similar manner accustomed them to apply criteria for polyneuropathy classification more carefully during the project participation. In conclusion, peer review medical audit seems to be an effective means of changing physicians’ clinical practice as continuous changes towards a more homogenous practice regarding performance and interpretation of the electrodiagnostic examination of polyneuropathy have been achieved in a group of European neurophysiologists participating in peer review medical audit through several years. Generally, the most pronounced changes were seen from the first to the second set of examinations, which is not surprising as the greatest differences among the physicians were present in the beginning of the project (Fuglsang-Frederiksen et al., 1995, 1999; Johnsen et al., 1994a, 1995). It was also in the earliest phase of the project that the participants had the greatest ‘‘aha
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
experiences’’ when seeing how things were done in other electrodiagnostic labs. Very importantly, however, almost all changes were maintained in the third set indicating that a consistent practice had been introduced. Probably the changes in examination strategies presented here has resulted in examinations of higher quality for the ESTEEM physicians. An improved quality can, however, not be documented by the data in this report, but is expected as the changes most likely results from the numerous consensus discussions where each physician has argued strongly for his reason for doing examinations in a certain way. In this way the physicians have probably picked up the best from each others’ practices. Although it seems to be an efficient step in quality improvement of electrodiagnostic medicine and probably most other medical fields, peer review medical audit as carried out here is difficult to scale up. Therefore guidelines or minimal criteria should ideally supplement a medical audit process to disseminate the results obtained to a larger audience. It is, however, of great importance that the guidelines are carefully developed and evidence-based, as the implementation of standards and guidelines may raise some concerns, for example there may be risks of normalising unsound practices, standardising practices around the average, preventing progress, and introducing ‘‘cookbook medicine’’ limiting individual judgement (Veloso et al., 1995). However, the advances of evidence-based guidelines include improvement of the quality of medical care by basing it on scientific evidence, reduction of the variations between physicians promoting a standard management of medical care, reduction of the cost of health care, and reduction of the risk of human error (James and Hammond, 2000). References Beghi E, Monticelli ML. Chronic symmetric symptomatic polyneuropathy in the elderly: a field screening investigation of risk factors for polyneuropathy in two Italian communities. Italian General Practitioner Study Group (IGPST). J Clin Epidemiol 1998;51:697–702. Beyer M, Gerlach FM, Flies U, Grol R, Krol Z, Munck A, et al. The development of quality circles/peer review groups as a method of quality improvement in Europe. Results of a survey in 26 European countries. Fam Pract 2003;20:443–51. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care 2005;21:380–5. Finnerup NB, Johnsen B, Fuglsang-Frederiksen A, de Carvalho M, Fawcett P, Liguori R, et al. Can medical audit change electromyographic practice? Electroencephalogr Clin Neurophysiol 1998;109:496–501. Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, Liguori R, Nix W, et al. Variation in diagnostic strategy of the EMG examination – a multicentre study. Clin Neurophysiol 1999;110:1814–24. Fuglsang-Frederiksen A, Johnsen B, Vingtoft S, Carvalho M, Fawcett P, Liguori R, et al. Variation in performance of the EMG examination at six European laboratories. Electroencephalogr Clin Neurophysiol 1995;97:444–50. Fuglsang-Frederiksen A, Johnsen B, Vingtoft S, the Clinical ESTEEM group. Expert systems and quality development in electromyography. 1996. Elsevier Science B.V. Recent Advances in Clinical Neurophysiology.
2529
Fuglsang-Frederiksen A, Pugdahl K. Current status on electrodiagnostic standards and guidelines in neuromuscular disorders. Clin Neurophysiol 2011;122:440–55. Hadden RD, Cornblath DR, Hughes RA, Zielasek J, Hartung HP, Toyka KV, et al. Electrophysiological classification of Guillain-Barre syndrome: clinical associations and outcome. Plasma Exchange/Sandoglobulin Guillain-Barre Syndrome Trial Group. Ann Neurol 1998;44:780–8. Ho TW, Li CY, Cornblath DR, Gao CY, Asbury AK, Griffin JW, et al. Patterns of recovery in the Guillain-Barre syndromes. Neurology 1997;48:695–700. James BC, Hammond ME. The challenge of variation in medical practice. Arch Pathol Lab Med 2000;124:1001–3. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006a; CD000259. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 2006b;15:433–6. Johnsen B, Fuglsang-Frederiksen A, Vingtoft S, Fawcett P, Liguori R, Nix W, et al. Differences in the handling of the EMG examination at seven European laboratories. Electroencephalogr Clin Neurophysiol 1994a;93:155–8. Johnsen B, Fuglsang-Frederiksen A, Vingtoft S, Fawcett P, Liguori R, Nix W, et al. Inter- and intraobserver variation in the interpretation of electromyographic tests. Electroencephalogr Clin Neurophysiol 1995;97:432–43. Johnsen B, Vingtoft S, Fuglsang-Frederiksen A, Barahona P, Fawcett P. A common structure for the representation of data and diagnostic processes within clinical neurophysiology. Proc. Twelfth Internat Congr Eur Federat Med Informat 1994:150–5. Kurtzke JF. Neuroepidemiology. Ann Neurol 1984;16:265–77. Landry MD, Sibbald WJ. Changing physician behavior: a review of patient safety in critical care medicine. J Crit Care 2002;17:138–45. MacDonald BK, Cockerell OC, Sander JW, Shorvon SD. The incidence and lifetime prevalence of neurological disorders in a prospective community-based study in the UK. Brain 2000;123(Pt 4):665–76. Pugdahl K, Tankisi H, Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, et al. Influence of medical audit on electrodiagnostic evaluation of polyneuropathy. A multicentre study. Clin Neurophysiol 2005;116:49–55. Tankisi H, Johnsen B, Fuglsang-Frederiksen A, de Carvalho M, Fawcett PR, LabarreVila A, et al. Variation in the classification of polyneuropathies among European physicians. Clin Neurophysiol 2003;114:496–503. Tankisi H, Pugdahl K, Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, et al. Influence of peer review medical audit on pathophysiological interpretation of nerve conduction studies in polyneuropathies. Clin Neurophysiol 2006;117:979–83. Tankisi H, Pugdahl K, Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, et al. Pathophysiology inferred from electrodiagnostic nerve tests and classification of polyneuropathies. Suggested guidelines. Clin Neurophysiol 2005;116:1571–80. Tankisi H, Pugdahl K, Johnsen B, Fuglsang-Frederiksen A. Correlations of nerve conduction measures in axonal and demyelinating polyneuropathies. Clin Neurophysiol 2007;118:2383–92. van der Meche FG, van Doorn PA, Meulstee J, Jennekens FG. Diagnostic and classification criteria for the Guillain–Barre syndrome. Eur Neurol 2001;45:133–9. Veloso M, Vingtoft S, Carvalho M, Fawcett P, Fuglsang-Frederiksen A, Johnsen B, et al. Quality assurance in clinical neurophysiology: the ESTEEM project example. Stud Health Technol Inform 1995;16:125–33. Vingtoft S, Johnsen B, Fuglsang-Frederiksen A, Veloso M, Barahona P, Vila A, et al. ESTEEM: a European telematic project for quality assurance within clinical neurophysiology. Medinfo 1995;8(Pt 2):1047–51. Vingtoft S, Veloso M, Fuglsang-Frederiksen A, Johnsen B, Vila A, Barahona P, et al. ESTEEM (European Standardised Telematic Tool to Evaluate EMG KnowledgeBased Systems and Methods): AIM Project A2010. Comput Methods Programs Biomed 1994;45:61–3.
Contents lists available at ScienceDirect
Clinical Neurophysiology journal homepage: www.elsevier.com/locate/clinph
Impact of medical audit on electrodiagnostic medicine in polyneuropathy Kirsten Pugdahl a, Anders Fuglsang-Frederiksen a,⇑, Hatice Tankisi a, Birger Johnsen a, Mamede de Carvalho b, Peter R.W. Fawcett c, Annick Labarre-Vila d, Rocco Liguori e, Wilfred Nix f, Ian S. Schofield c a
Department of Clinical Neurophysiology, Aarhus University Hospital, Aarhus, Denmark Department of Neurology, Hospital de Santa Maria, Laboratory of Electromyography, Institute of Molecular Medicine, Faculty of Medicine, Lisbon, Portugal c Department of Clinical Neurophysiology, Newcastle General Hospital, Newcastle upon Tyne, UK d Unité ENMG et Pathologie neuromusculaire, Département de Neurologie, Centre Hospitalier Universitaire, Grenoble, France e Department of Neurological Sciences, University of Bologna, Bologna, Italy f Department of Neurology, University Clinics Mainz, Mainz, Germany b
a r t i c l e
i n f o
Article history: Accepted 22 May 2011 Available online 23 June 2011 Keywords: Polyneuropathy Neurophysiology Electromyography Nerve conduction studies Medical audit
h i g h l i g h t s Studies on quality development are important, but sparse, in clinical neurophysiology. Experienced neurophysiologists change their practice after participation in peer review medical audit. Profound and lasting changes in electrodiagnostic practice can be obtained but require several years to develop.
a b s t r a c t Objective: The aim of the study was to investigate whether experienced physicians’ electrodiagnostic practice and criteria can be influenced by international collaboration involving peer review medical audit. Methods: Data was obtained from the ESTEEM project, an ongoing collaboration since 1991 among European neurophysiologists concerned with quality improvement in electrodiagnostic medicine. Three sets of the physicians’ polyneuropathy examinations performed with intervals of 2–4 years were analysed. Results: Changes towards increased homogeneity among the physicians were found in (1) the average number of studies performed per patient and the number of abnormal studies required for accepting the diagnosis of polyneuropathy, with the most pronounced changes seen for abnormal motor nerve segments, abnormal F-wave studies, and electromyographic studies, and (2) the agreement on pathophysiological interpretation of nerve conduction studies and classification of polyneuropathy. Conclusions: Changes towards increased homogeneity contributed to years of participation in peer review medical audit, were seen among a group of experienced physicians. Peer review medical audit as carried out here is however difficult to scale up. Therefore guidelines or minimal criteria should ideally supplement a medical audit process to disseminate the results obtained to a larger audience. Significance: These results support the role of international peer review medical audit in quality improvement of electrodiagnostic medicine. Ó 2011 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved.
1. Introduction Polyneuropathies are common disorders of the peripheral nerve, affecting axons, myelin sheaths, or both. The population prevalence is about 0.04–0.1%, but in people older than 55 years the prevalence rises to about 1.6% (Beghi and Monticelli, 1998; ⇑ Corresponding author. Address: Department of Neurophysiology, Aarhus University Hospital, Nørrebrogade 44, Building 10, 8000 Aarhus C, Denmark. Tel.: +45 8949 3100; fax: +45 8949 3140. E-mail addresses: [email protected], [email protected] (A. FuglsangFrederiksen).
Kurtzke, 1984; MacDonald et al., 2000). The clinical examination is very useful to determine the severity of a polyneuropathy and to test for predominant sensory or motor involvement; however, it can be limited in identifying the type of the neuropathy (axonal vs. demyelinating). As opposed to this limitation of the clinical examination, the neurophysiological investigation approaches objectively the pathophysiological dysfunction of the nerve, as nerve conduction studies measure the function of the peripheral nerves directly without subjective bias and without contamination by the central nervous system. The neurophysiological examination is, however, prone to show a great deal of variation as a series of diagnostic tests is applied to a
1388-2457/$36.00 Ó 2011 International Federation of Clinical Neurophysiology. Published by Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.clinph.2011.05.018
2524
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
clinical problem, with the performance and selection of tests being highly operator dependent (Veloso et al., 1995). In addition, a wide range of different neurophysiological techniques are available and a huge number different nerve segments can be selected for examination (Johnsen et al., 1994a). A physician’s examination strategy often relies on more or less well-defined rules based mainly on tradition and personal experience, and only to a lesser extent on documented results. Similarly, criteria for classification of polyneuropathy are not consistent in the literature. Various criteria for definition of demyelination have been proposed but none are consistently used in practice (Fuglsang-Frederiksen and Pugdahl, 2011). In contrast, the literature on criteria for axonal loss is sparse (Hadden et al., 1998; Ho et al., 1997; Tankisi et al., 2005; van der Meche et al., 2001). Once an optimal medical practice is identified it has to be integrated into practice by appropriately changing physician behaviour (Landry and Sibbald, 2002). However, changing of physicians’ behaviour is a complex and difficult task that is strongly dependent on the willingness to learn from other physicians and compromise on own approaches. Peer review medical audit has the potential of changing physician’s behaviour as a widely accepted method of quality improvement in medical practice, because it encourages professional autonomy and supports critical insight and appraisal of quality of care (Beyer et al., 2003). It appears logical that health care professionals given feedback that their clinical practice was inconsistent with that of their peers or accepted guidelines would be prompted to modify their practice (Jamtvedt et al., 2006b). Yet, medical audit and feedback is not consistently effective in changing clinical behaviours and the effects are generally small to moderate and dependent on the intensity of feedback (Bloom, 2005; Jamtvedt et al., 2006a).
and all physicians have reported that they have changed their routines somewhat during the project. The aim of the present study was to examine whether the variation among the physicians in the ESTEEM project had been further reduced in a recently submitted and peer reviewed set of polyneuropathy examinations, i.e. to examine to what extent continuing international peer review medical audit can change experienced physicians’ routine. 2. Methods 2.1. Data sets Three sets of peer reviewed examinations with a consensus diagnosis of definite or probable polyneuropathy were compared. Each set was based on 12 prospectively collected consecutive examinations with a referral diagnosis of polyneuropathy from each physician. The sets are specified in Table 1. The patients were examined with standard up-to-date EMG equipment used routinely in the participating labs and data transferred to a standardised data structure implemented in a PC program consisting, in accordance with the clinical neurophysiological consultation, of three parts (Johnsen et al., 1994b): (a) General data: Patient age, sex, and height, as well as free text clinical information. (b) Examination data: Measured values from each EMG or nerve conduction study with accompanying locally used reference values, percentage deviation from mean of controls, and a symbolic value (normal, increased, decreased, borderline increased/decreased, or inconclusive), and data on examination conditions, such as information on muscle force and atrophy, patient cooperation for EMG studies, electrode type (needle, surface), temperature, and segment length for nerve conduction studies. (c) Inferred data: Each individual EMG or nerve conduction study is given a pathophysiological test conclusion (PTC) based on the electrophysiological data. The available PTC’s for a nerve conduction study are: ’Normal’, ‘‘Demyelinating’, ‘Axonal loss’, ‘Neuropathic’, or ‘Other’, with the term neuropathic referring to a grey zone, where at distinction between demyelination and axonal loss is not possible (Tankisi et al., 2005). Finally, one or more diagnoses combined with the logical operators ‘‘and/or’’ are given.
1.1. The ESTEEM project The European multicenter project ESTEEM (European standardised telematic tool to evaluate electrodiagnostic methods) was initiated in 1992 with the aim of improving the quality of the neurophysiological examination (Vingtoft et al., 1994, 1995; Fuglsang-Frederiksen et al., 1996). The main objectives of the ESTEEM project were (1) to create an environment for storage and exchange of data from electrodiagnostic examinations and (2) to establish a peer review medical audit procedure across borders in order to build a multicentre reference database consisting of electrodiagnostic examinations, in which all contained decisions had been evaluated and a consensus diagnosis obtained (Veloso et al., 1995). Since the beginning of the project 7 neurophysiologists from 6 European countries have prospectively collected and discussed samples of their examinations. All are experienced specialists who, when joining the project, had practised clinical neurophysiology for between 7 and 16 years. Changes towards a more uniform practice among the physicians, attributed to the years of collaboration, have been indicated (Finnerup et al., 1998; Pugdahl et al., 2005; Tankisi et al., 2006),
2.2. Peer review Examinations in the ESTEEM project are peer reviewed as follows: An examination sampled at one of the participating laboratories is sent via the Internet to Aarhus, Denmark, where a moderator validates the data and stores a copy of the examination in the data base. The moderator prepares the examination for further analysis
Table 1 Examinations provided by the individual physicians (P1–P7) in the three sets. Physician P1 P2 P3 P4 P5 P6 P7 Total *
Set 1 No. of patients
Exam. time
Set 2 No. of patients
Exam. time
Set 3 No. of patients
Exam. time
12 8 8 12 10 12 7 69*
1992–93 1992–93 1992–93 1992 1994 1989–93 1992–95 1989–95
10 8 10 11 12 9 10 70*
1994–96 1995–97 1995–97 1996 1996 1996–97 1996–97 1994–97
12 12 11 12 11 12 11 81*
2003 2001–02 2003–04 2003 2002–04 2003 2002 2001–04
No difference in the number of examinations provided by the individual physicians among the three sets (P = 1.0).
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529 Table 2 Number of nerve conduction studies in the three sets.
3. Results
Set
Motor NCS
Sensory NCS Surface recording
Sensory NCS Needle recording
F-wave studies
Total
1
636 (52.2%) 724 (52.5%) 771 (50.1%)
255 (20.9%)
149 (12,2%)
178 (14.6%)
1218
296 (21.5%)
128 (9.3%)
231 (16.8%)
1379
355 (23.1%)
122 (7.9 %)
292 (19.1%)
1540
2 3
2525
by removing laboratory identity, the sampling physician’s test conclusions and diagnoses, as well as all clinical information except the reason for referral, thus leaving only patient information, examination conditions, and measured data with the local reference values and the assigned symbolic value. The modified examination is then sent to all physicians for individual blind interpretation. The resulting diagnoses serve as the starting point at biannual workshop discussions among all participating physicians with the aim of reaching consensus on the diagnosis. First consensus on a diagnosis based solely on electrophysiological data is sought, and afterwards the clinical information is reviewed to arrive at a final consensus diagnosis. For some patients a clinical follow-up is further requested. 2.3. Data analysis The following possible changes between the three sets were assessed for the group of physicians and for each individual physician: A change in the average number of motor and sensory nerve segments, F-wave studies, and muscles examined per patient. A change in the average number of abnormal motor and sensory nerve segments, F-wave studies, and muscles per patient. A change in the agreement on the pathophysiological interpretation of a nerve conduction study. For each nerve conduction study, the PTC (axonal loss, demyelinating, neuropathic or normal) given by each physician at the blind review was pair-wise compared with each of the other physicians’ interpretation. The agreement among the group of physicians was calculated separately for the different types of PTCs in general and for motor and sensory nerve conduction studies and F-wave studies separately. These analyses included only 6 physicians as one (P7) used software for test interpretation. A change in the agreement on the pathophysiological classification of a polyneuropathy as axonal, demyelinating, or unclassifiable. The classification in each individual physician’s interpretation diagnosis was compared with the consensus classification and pair-wise with each of the other physicians’ classifications. Each physician’s (P1–P6) agreement rate was determined as the average of his pair-wise agreements with the other physicians. 2.4. Statistical analysis The mean number of examined and abnormal studies for the group of physicians was compared using Kruskal–Wallis nonparametric ANOVA, while the physicians’ individual changes were estimated using the Mann–Whitney U test. Changes in proportions, i.e. distribution of test types and agreement on PTCs were determined using chi-square tests or Fisher’s exact test. Changes towards improved agreement on PNP classification were estimated using the Wilcoxon signed rank test. For all tests, a P-value < 0.05 was considered significant.
3.1. Proportion of test types The number of motor nerve studies, F-wave studies, and sensory nerve conduction studies performed with surface and near-nerve needle technique respectively in the 3 sets are shown in Table 2. The proportion of motor nerve conduction studies was constant in all 3 sets, comprising around 50% of all tests conducted, however the proportion of F-wave studies increased from set 1 (14.6%) to set 3 (19.1%) (P = 0.012). The proportion of sensory nerve conduction studies performed was constant – around 30% – in all 3 sets, but a lower proportion of the sensory nerve conduction studies were performed using the near-nerve needle technique in set 3 (26%) compared to set 1 (37%) (P = 0.01) (Table 2). 3.2. Motor nerve segments The mean number of examined motor nerve segments per patient increased from set 1 (mean 7.7; range 3.3–13.9) to set 2 (mean 8.8; range 4.9–12.5) (P = 0.039) and from set 1 to set 3 (mean 9.1; range 6.3–13.8) (P = 0.04). Individual changes were seen for two physicians (P1 and P2), who both examined a higher number of motor nerve segments in set 2 and set 3 compared to set 1 (P < 0.05). For P2 an increase was further seen from set 2 to set 3 (P = 0.008). Despite the changes a significant variation among the physicians were still seen in all 3 sets (P = 0.000) (Fig. 1). The mean number of abnormal motor nerve segments per patient increased from set 1 (mean 5.2; range 1.8–11.5) to set 2 (mean 6.5; range 4–8.9) (P = 0.009). Individual changes were seen for three physicians (P1, P2, and P4). P1 and P2 both examined a higher number of abnormal motor nerve segments in set 2 and 3 compared to set 1 (P < 0.01). P4 had a higher number of abnormal motor nerve segments in set 2 compared to set 1 (P = 0.007); however this number dropped in set 3 and became similar to the number in set 1 (P = 0.03). Variation among the physicians was found in the first two sets (P < 0.01), however in the last set there was no significant variation (P = 0.13) (Fig. 1). 3.3. F-wave studies The mean number of F-wave studies performed per patient increased from set 1 (mean 2.1; range 0.3–4.7) to set 2 (mean 2.7; range 1.6–4.6) (P = 0.039) and from set 2 to set 3 (mean 3.5; range 2.4–4.5) (P = 0.014). Individual changes were seen for two physicians (P1 and P2), who both studied more F-waves in set 2 and set 3 compared to set 1 (P < 0.01). For P2 an increase was further seen from set 2 to set 3 (P = 0.006). A variation among the physicians was seen in all 3 sets (P < 0.01) (Fig. 1). The mean number of abnormal F-wave studies per patient increased from set 1 (mean 1.8; range 0.3–4.6) to set 3 (mean 2.5; range 1.6–3.8) (P = 0.008). Individual changes were seen for three physicians (P1, P2, and P5). All had a higher number of abnormal F-wave studies in set 3 compared to set 1 (P < 0.05). Variation among the physicians was found in the first two sets (P < 0.01), however in the last set there was no significant variation (P = 0.054) (Fig. 1). 3.4. Sensory nerve segments The number of sensory nerve segments examined per patient increased from set 1 (mean 4.8; range 3.1–8.1) to set 3 (mean 8.8; range 4.9–12.5) (P = 0.031). Individual changes were seen for one physician (P1), who examined a higher number of sensory nerve segments in set 2 and 3 compared to set 1 (P < 0.01). Variation among the physicians was found in all 3 sets (P = 0.000)(Fig. 1).
2526
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
Motor Nerve Segments Abnormal
16
16
14
14
12
12
10
10
8
8
6
6
4
4
2
2
Examined
Avg. no per physician
Avg. no per physician
Examined
F-wave Studies
0
0
1
2
Abnormal
6
6
4
4
2
2
0
0
1
3
2
3
1
2
Set
Set
1
3
6
6
4
4
2
2
0
0
2
Set
3
Avg. no per physician
Avg. no per physician
8
1
Examined
Abnormal
8
Abnormal
6
6
4
4
2
2
0
1
2
3
Muscles
Sensory Nerve Segments Examined
2
Set
Set
3
Set
0 1
2
3
1
Set
2
3
Set
Fig. 1. Each of the physician’s average number per patient of examined and abnormal motor nerve segments, F-wave studies, sensory nerve segments and muscles in each set is plotted. The physicians are represented by the following symbols: P1 (d), P2 (s), P3 (.), P4 (O), P5 (j), P6 (h), P7 ().
No change in the number of abnormal sensory nerve segments could be documented for the group of physicians; however, one physician (P1) examined a higher number of abnormal nerve segments in set 3 than in set 1 (P = 0.009). Variation among the physicians was found in all three sets (P < 0.05) (Fig. 1).
3.5. Electromyography There were no changes in the mean number of examined or abnormal muscles for the group of physicians in the three sets. However, in contrast to what was found in the first two sets (P < 0.01), no variation among the physicians was found in the last set regarding both number of examined and abnormal muscles (P > 0.6). In addition, several individual changes were present: Three physicians (P5, P6, and P7) changed their average number of examined muscles from set 2 to set 3 (P < 0.05), and P5 also changed from set 1 to set 2 (P = 0.01). For abnormal muscles, changes were seen for 4 physicians. P5 and P6 changed from set 2 to set 3 and P3 and P7 changed from set 1 to set 3 (P < 0.05).
3.6. Interpretation of nerve conduction studies The agreement on pathophysiological interpretation of all types of nerve conduction studies increased in the group of physicians from set 1 to set 2 for interpretation as axonal loss, demyelination or neuropathic (P < 0.01), however the agreement decreased slightly from set 2 to set 3 for all three test conclusions (P < 0.05). It remained, however, significantly better in the third set than in the first (P < 0.01) (Fig. 2). The agreement on interpretation as normal was high in all three set (94.1%–94.7%). Besides from normal, the highest agreement was found on interpretation as neuropathic (81.2%–87.2%) in the three sets. The agreement on axonal loss ranged from 61.3% in set 1 to 78.3% in set 2, while the agreement on demyelination ranged from 46.5% in set 1 to 56.2% in set 2 (Fig. 2). When analysing the agreement on the separate test types, increases on the agreement on interpretation of motor nerve conduction studies as demyelination, axonal loss and neuropathic were seen from set 1 to set 2 (P < 0.01) with further increases from set 2 to set 3 for demyelination and axonal loss (P = 0.000). The
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
Motor NCS
% Agreement
All NCS 100
100
80
80
60
60
40
40
20
20
0
0
1
2
3
Set
1
2
3
Set
Fig. 2. The mean agreement for the group of physicians on pathophysiological interpretation of nerve conduction studies as normal (O), neuropathic (s), demyelination (}) or axonal loss (h) in each of the three sets of polyneuropathy examinations. Results are shown for all nerve conduction studies (left) and motor nerve conduction studies (right).
agreement on axonal loss increased from 69.8% in set to 90.8% in set 3, for demyelination from 50.13% in set 1 to 76.1% in set 3 and for neuropathic from 75.6% in set 1 to 83.2% in set 3 (Fig. 2). For F-wave studies, there was an increase in the agreement on interpretation as demyelination from 27.6% in set 1 to 59.1% in set 2 (P < 0.01) that again decreased in set 3 to 45.3% (P = 0.002) but remained higher than in set 1 (P = 0.000). There was high agreement on interpretation as neuropathic (90.6%–93.7%) and normal (94.0%–96.7%) in all three sets with no changes observed. For sensory nerve conduction studies, increases on the agreement from set 1 to set 2 were seen for interpretation as axonal loss, neuropathic and normal (P < 0.05). The agreement decreased again from set 2 to set 3 for axonal loss and neuropathic findings (P < 0.05), but remained higher for axonal loss in set 3 compared to set 1 (P < 0.05). There was a good agreement on interpretation as neuropathic (81.1%–89.5%) and normal (87.0%–94.6%) in all three sets, while the agreement on axonal loss ranged from 51.3% to 62.9% and on demyelination from 44.2% to 51.4% (results no shown). 3.7. Classification of polyneuropathy An increased agreement on classification of all polyneuropathies was seen in set 2 and set 3 compared to set1 for the physicians as a group; both regarding agreement with the consensus group (P < 0.05) and with the rest of the physicians (P < 0.000) and for all individual physicians. One physician (P6), however, decreased his agreement with the consensus group from set 2 to set 3 (P = 0.043). The agreement between the individual physicians and the consensus group ranged from 49.3%–75.7% in set 1 to 77.5%– 89.6% in set 3 (Fig. 3). A similar picture was seen on the agreement on unclassified polyneuropathies, where an increased agreement among the physicians was seen from set 1 to sets 2 and 3 (P = 0.000). One physician (P1) further increased his agreement with the consensus group from set 2 to set 3 (P = 0.043). The agreement between the individual physicians and the consensus group on which polyneuropathies were unclassified ranged from 32.4% to 69.8% in set 1 to 73.3% to 87.7% in set 3 (Fig. 3). For demyelinating polyneuropathy increases in the agreement were seen for the
2527
agreement between the physicians and the consensus group from set 1 to sets 2 and 3 (P < 0.05), and for the physicians’ agreement with each other from set 1 to sets 2 and 3 (P = 0.000), as well as from set 2 to set 3 (P = 0.014). Individual changes were seen for all physicians: P6 increased his agreement from set 1 to set 2 and P3 from set 2 to set 3, while all physicians except P4 had a higher agreement in the third set than in the first (P = 0.043). The agreement between the individual physicians and the consensus group on classification as demyelinating polyneuropathy ranged from 42.9%–60% in set 1 to 50%–91.7% in set 3 (Fig. 3). For classification as axonal polyneuropathy, an increased agreement between the physicians and the consensus group was seen from set 1 to set 2 (P < 0.05), and between the physicians from set 1 to sets 2 and 3 (P < 0.01). Individual changes were seen for three physicians: P1 and P6 had higher agreements in sets 2 and 3 than in set 1 and physician P2 increased his agreement from set 1 to set 2 (P = 0.043). The agreement between the individual physicians and the consensus group ranged from 12.5%–69.2% in set 1 to 11.1%– 80% in set 3. Very low agreement in the third set was however only seen for one physician (P3), except from this all agreements were P50% (Fig. 3). 4. Discussion Based on three sets of polyneuropathy examinations examined with intervals of 2–4 years, the present study demonstrated that a more uniform practice was introduced in a group of experienced neurophysiologists working closely together in the European multicentre project ESTEEM since 1992. For the number of nerve conduction studies performed, the most pronounced changes were seen for F-wave studies, where a higher number was performed in the last set than in the first two sets. Regarding performed motor and sensory nerve conduction studies, a more uniform practice was adopted as the physicians initially performing the fewest studies increased their number of studies during the project. Some variation, however, persisted in the last set, much of which can probably be explained by the use of different techniques, especially use of surface versus needle electrodes for sensory nerve conduction studies. More pronounced changes were seen for the number of abnormal studies. Most importantly no variation was present among the physicians for abnormal motor nerve conduction studies and F-wave studies in the last set, whereas a significant variation was seen in the first two sets. Variation among the physicians persisted for the number of abnormal sensory nerve conduction studies, however the physicians who initially required the lowest number of studies increased towards the mean in the last set. The results indicate that the physicians have influenced each other in the selection of which nerves to study, and in the number of abnormal studies required for diagnosing. Agreement on the optimal diagnostic strategy, i.e. the composition and amount of abnormal and normal test results required by a physician for making a diagnosis, is important for the quality of the diagnosis. Finally, a more homogenous practice was adopted in the number of performed and abnormal EMG studies in polyneuropathy. The changes were towards a more limited use of EMG, either because the physicians started to relying less on the use of EMG for classification of polyneuropathy (Tankisi et al., 2007) or a tendency to use a less invasive approach. The agreement on pathophysiological interpretation of nerve conduction studies was generally good in the last two sets of examinations. When analysing the agreement separately for the different test types, the best agreement was found on interpretation of motor nerve conduction studies, indicating that the physicians started to apply common criteria in a more consistent fashion during their participation in ESTEEM. However, in sensory nerve conduction studies a considerable disagreement remained.
2528
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
Unclassified
% Agreement
All Polyneuropathies
Axonal
100
100
100
80
80
80
80
60
60
60
60
40
40
40
40
20
20
20
20
0 SET 2
SET 3
0
0
0 SET 1
SET 1
SET 2
SET 1
SET 3
SET 2
SET 3
100
100
100
100
80
80
80
80
60
60
60
60
40
40
40
40
20
20
20
20
SET 1
SET 2
SET 3
SET 1
SET 2
SET 3
g
% Agreement
Demyelinating
100
0
0 SET 1
SET 2
SET 3
0
0 SET 1
SET 2
SET 3
SET 1
SET 2
SET 3
Fig. 3. Part A shows each physician’s agreement with the consensus group on classification of polyneuropathy in general (all) and on unclassified, demyelinating and axonal polyneuropathy in each of the three sets. The physicians are represented by the following symbols: P1 (d), P2 (s), P3 (.), P4 (O), P5 (j), P6 (h). Part B shows all pair-wise agreements between the physicians in each set.
This may be due to a general lack of well-defined international criteria for sensory nerve conduction studies (Tankisi et al., 2005; Fuglsang-Frederiksen and Pugdahl, 2011) but also due to the use of different techniques. It may be argued that choosing the proper pathophysiological test conclusion is of minor significance compared with the significance of making the correct diagnosis. Nevertheless, exact knowledge about which pathophysiological information that can be inferred from individual nerve tests may promote a broader understanding of electrophysiology with advantages for education of clinical neurophysiologists, and for selection of patients for scientific studies (Johnsen et al., 1995). As a pathological test conclusion concerns only the findings of a single nerve test while a diagnosis depends on a constellation of abnormalities in a series of motor and sensory nerves together with EMG findings, a better agreement may be expected on the diagnosis level. Certainly, the agreement among the physicians appeared to be better on the classification of polyneuropathy than on the classification of an individual nerve segment. A good agreement on classification of demyelinating polyneuropathy was present in the last set for most of the physicians. Similarly, an overall good agreement rate on classification of axonal neuropathy was seen for most physicians in the last set, however one physician remained having a very low agreement with the consensus group and with several of the other physicians. The best agreement
was, however, seen on unclassified polyneuropathies. An earlier study showed that the consensus group was more cautious in assigning a specific classification to a polyneuropathy than most of the individual physicians (Tankisi et al., 2003). This is probably because the group of peers was more likely than the individual physicians to follow classification criteria strictly and only classified a polyneuropathy in case of strong evidence of demyelination or axonal loss that everybody could agree on. The increased agreement on unclassified polyneuropathies shown here indicates that the physicians in a similar manner accustomed them to apply criteria for polyneuropathy classification more carefully during the project participation. In conclusion, peer review medical audit seems to be an effective means of changing physicians’ clinical practice as continuous changes towards a more homogenous practice regarding performance and interpretation of the electrodiagnostic examination of polyneuropathy have been achieved in a group of European neurophysiologists participating in peer review medical audit through several years. Generally, the most pronounced changes were seen from the first to the second set of examinations, which is not surprising as the greatest differences among the physicians were present in the beginning of the project (Fuglsang-Frederiksen et al., 1995, 1999; Johnsen et al., 1994a, 1995). It was also in the earliest phase of the project that the participants had the greatest ‘‘aha
K. Pugdahl et al. / Clinical Neurophysiology 122 (2011) 2523–2529
experiences’’ when seeing how things were done in other electrodiagnostic labs. Very importantly, however, almost all changes were maintained in the third set indicating that a consistent practice had been introduced. Probably the changes in examination strategies presented here has resulted in examinations of higher quality for the ESTEEM physicians. An improved quality can, however, not be documented by the data in this report, but is expected as the changes most likely results from the numerous consensus discussions where each physician has argued strongly for his reason for doing examinations in a certain way. In this way the physicians have probably picked up the best from each others’ practices. Although it seems to be an efficient step in quality improvement of electrodiagnostic medicine and probably most other medical fields, peer review medical audit as carried out here is difficult to scale up. Therefore guidelines or minimal criteria should ideally supplement a medical audit process to disseminate the results obtained to a larger audience. It is, however, of great importance that the guidelines are carefully developed and evidence-based, as the implementation of standards and guidelines may raise some concerns, for example there may be risks of normalising unsound practices, standardising practices around the average, preventing progress, and introducing ‘‘cookbook medicine’’ limiting individual judgement (Veloso et al., 1995). However, the advances of evidence-based guidelines include improvement of the quality of medical care by basing it on scientific evidence, reduction of the variations between physicians promoting a standard management of medical care, reduction of the cost of health care, and reduction of the risk of human error (James and Hammond, 2000). References Beghi E, Monticelli ML. Chronic symmetric symptomatic polyneuropathy in the elderly: a field screening investigation of risk factors for polyneuropathy in two Italian communities. Italian General Practitioner Study Group (IGPST). J Clin Epidemiol 1998;51:697–702. Beyer M, Gerlach FM, Flies U, Grol R, Krol Z, Munck A, et al. The development of quality circles/peer review groups as a method of quality improvement in Europe. Results of a survey in 26 European countries. Fam Pract 2003;20:443–51. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care 2005;21:380–5. Finnerup NB, Johnsen B, Fuglsang-Frederiksen A, de Carvalho M, Fawcett P, Liguori R, et al. Can medical audit change electromyographic practice? Electroencephalogr Clin Neurophysiol 1998;109:496–501. Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, Liguori R, Nix W, et al. Variation in diagnostic strategy of the EMG examination – a multicentre study. Clin Neurophysiol 1999;110:1814–24. Fuglsang-Frederiksen A, Johnsen B, Vingtoft S, Carvalho M, Fawcett P, Liguori R, et al. Variation in performance of the EMG examination at six European laboratories. Electroencephalogr Clin Neurophysiol 1995;97:444–50. Fuglsang-Frederiksen A, Johnsen B, Vingtoft S, the Clinical ESTEEM group. Expert systems and quality development in electromyography. 1996. Elsevier Science B.V. Recent Advances in Clinical Neurophysiology.
2529
Fuglsang-Frederiksen A, Pugdahl K. Current status on electrodiagnostic standards and guidelines in neuromuscular disorders. Clin Neurophysiol 2011;122:440–55. Hadden RD, Cornblath DR, Hughes RA, Zielasek J, Hartung HP, Toyka KV, et al. Electrophysiological classification of Guillain-Barre syndrome: clinical associations and outcome. Plasma Exchange/Sandoglobulin Guillain-Barre Syndrome Trial Group. Ann Neurol 1998;44:780–8. Ho TW, Li CY, Cornblath DR, Gao CY, Asbury AK, Griffin JW, et al. Patterns of recovery in the Guillain-Barre syndromes. Neurology 1997;48:695–700. James BC, Hammond ME. The challenge of variation in medical practice. Arch Pathol Lab Med 2000;124:1001–3. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2006a; CD000259. Jamtvedt G, Young JM, Kristoffersen DT, O’Brien MA, Oxman AD. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 2006b;15:433–6. Johnsen B, Fuglsang-Frederiksen A, Vingtoft S, Fawcett P, Liguori R, Nix W, et al. Differences in the handling of the EMG examination at seven European laboratories. Electroencephalogr Clin Neurophysiol 1994a;93:155–8. Johnsen B, Fuglsang-Frederiksen A, Vingtoft S, Fawcett P, Liguori R, Nix W, et al. Inter- and intraobserver variation in the interpretation of electromyographic tests. Electroencephalogr Clin Neurophysiol 1995;97:432–43. Johnsen B, Vingtoft S, Fuglsang-Frederiksen A, Barahona P, Fawcett P. A common structure for the representation of data and diagnostic processes within clinical neurophysiology. Proc. Twelfth Internat Congr Eur Federat Med Informat 1994:150–5. Kurtzke JF. Neuroepidemiology. Ann Neurol 1984;16:265–77. Landry MD, Sibbald WJ. Changing physician behavior: a review of patient safety in critical care medicine. J Crit Care 2002;17:138–45. MacDonald BK, Cockerell OC, Sander JW, Shorvon SD. The incidence and lifetime prevalence of neurological disorders in a prospective community-based study in the UK. Brain 2000;123(Pt 4):665–76. Pugdahl K, Tankisi H, Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, et al. Influence of medical audit on electrodiagnostic evaluation of polyneuropathy. A multicentre study. Clin Neurophysiol 2005;116:49–55. Tankisi H, Johnsen B, Fuglsang-Frederiksen A, de Carvalho M, Fawcett PR, LabarreVila A, et al. Variation in the classification of polyneuropathies among European physicians. Clin Neurophysiol 2003;114:496–503. Tankisi H, Pugdahl K, Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, et al. Influence of peer review medical audit on pathophysiological interpretation of nerve conduction studies in polyneuropathies. Clin Neurophysiol 2006;117:979–83. Tankisi H, Pugdahl K, Fuglsang-Frederiksen A, Johnsen B, de Carvalho M, Fawcett PR, et al. Pathophysiology inferred from electrodiagnostic nerve tests and classification of polyneuropathies. Suggested guidelines. Clin Neurophysiol 2005;116:1571–80. Tankisi H, Pugdahl K, Johnsen B, Fuglsang-Frederiksen A. Correlations of nerve conduction measures in axonal and demyelinating polyneuropathies. Clin Neurophysiol 2007;118:2383–92. van der Meche FG, van Doorn PA, Meulstee J, Jennekens FG. Diagnostic and classification criteria for the Guillain–Barre syndrome. Eur Neurol 2001;45:133–9. Veloso M, Vingtoft S, Carvalho M, Fawcett P, Fuglsang-Frederiksen A, Johnsen B, et al. Quality assurance in clinical neurophysiology: the ESTEEM project example. Stud Health Technol Inform 1995;16:125–33. Vingtoft S, Johnsen B, Fuglsang-Frederiksen A, Veloso M, Barahona P, Vila A, et al. ESTEEM: a European telematic project for quality assurance within clinical neurophysiology. Medinfo 1995;8(Pt 2):1047–51. Vingtoft S, Veloso M, Fuglsang-Frederiksen A, Johnsen B, Vila A, Barahona P, et al. ESTEEM (European Standardised Telematic Tool to Evaluate EMG KnowledgeBased Systems and Methods): AIM Project A2010. Comput Methods Programs Biomed 1994;45:61–3.