- Email: [email protected]
Quality assessment of malaria laboratory diagnosis in South Africa
TRANSACTIONSOF THE ROYALSOCIETYOF TROPICALMEDICINEAND HYGIENE (2003) 97, 675-677
Quality assessment of malaria laboratory diagnosis in South Africa Leigh D i n i and John Frean Parasitology Reference Unit, National Institute for Communicable Diseases, National Health Laboratory Service, Box 1038, Johannesburg 2000, South Africa Abstract T o assess the quality of malaria diagnosis in 115 South African laboratories participating in the National Health Laboratory Service Parasitology External Quality Assessment Programme we reviewed the results from 7 surveys from January 2000 to August 2002. Th e mean percentage incorrect result rate was 13.8% (95% CI 11.3-16.9%), which is alarmingly high, with about 1 in 7 blood films being incorrectly interpreted. Most participants with incorrect blood film interpretations had acceptable Giemsa staining quality, indicating that there is less of a problem with staining technique than with blood film interpretation. Laboratories in provinces in which malaria is endemic did not necessarily perform better than those in non-endemic areas. T h e results clearly suggest that malaria laboratory diagnosis throughout South Africa needs strengthening by improving laboratory standardization and auditing, training, quality assurance and referral resources. Keywords: malaria, diagnosis, quality assessment, South Africa Introduction Malaria is one of the 6 major causes of 90% of deaths from communicable diseases. Malaria deaths worldwide amount to over one million annually (Samba, 2001), and 100-300 deaths are notified in South Africa each year (Anonymous, 2001). Prompt, reliable laboratory diagnosis is recognized as an important component of effective malaria case management and control. Strengthening laboratory diagnosis should, ultimately, decrease malaria morbidity and mortality. Surprisingly, at a South African national malaria conference in Malelane in August 2002, there were no contributions dealing with the laboratory diagnosis of malaria. It seems that there are very few published data on the quality of malaria laboratory diagnosis in South Africa (Durrheim et al., 1997; Hansford & van Vuuren, 1997). Durrheim et al. (1997) noted minimal agreement in microscopical malaria slide examination amongst 4 diagnostic laboratories and pointed to the absence of uniform training and quality assessment programmes. T h e aim of our review was to assess the quality of malaria laboratory diagnosis in 115 South African laboratories participating in the National Health Laboratory Service (NHLS) Parasitology External Quality Assessment (EQA) Programme, from January 2000 to August 2002. T h e Parasitology Reference Unit of the National Institute for Communicable Diseases in Johannesburg coordinates an E Q A programme for over 100 medical diagnostic laboratories in southern Africa. As part of each EQA survey the laboratories are assessed on their ability to identify blood parasites. In addition, their ability to correctly stain a blood film and perform parasite counts are regularly assessed. T h e aims of the Parasitology EQA Programme include the following: to build capacity in the field of human diagnostic parasitology in southern Africa, to obtain an objective measure of the diagnostic ability of participants, and to improve knowledge about human diagnostic parasitology. E QA surveys are conducted 4 times a year and each survey comprises one faecal and one blood sample plus a practical laboratory technique exercise. T o ensure the quality of material sent out for each survey, the expected (model) results are compared against those returned by a subset of 10 historically good participants (reference laboratories). After analysis of results, each participant receives an individual report and comprehensive teaching material relevant to that survey.
Address for correspondence: Leigh Dini, Parasitology Reference Unit, NICD NHLS, Box 1038, Johannesburg 2000, South Africa; phone +27 11 4899342, fax +27 11 4899357, e-mail leigh, [email protected]
Methods Th e majority of participants included in the review were public sector laboratories, comprising mainly former South African Institute for Medical Research laboratories and a few former government laboratories. In total, there were 115 participants and 23 were mine and private laboratories from within South Africa. Participants were grouped according to provinces with the Northern Cape and Free State combined, whilst mine and private laboratories were incorporated into one group. It should be noted that most of the dedicated National Malaria Control Programme laboratories do not participate in this E Q A programme and were not included in this review. We included only the results of parasitology E Q A surveys relevant to the diagnosis of malaria from participants within South Africa from January 2000 to August 2002. The results from 7 surveys were reviewed and Table 1 shows the types of challenges and expected results from each of the 7 surveys. The results were graded as percentage correct, partially correct, incorrect, or no result. A 'correct result' was defined as reporting the expected result; a 'partially correct result' was defined as appropriately reporting a Plasmodium sp. present but unable to identify it further; an 'incorrect result' was defined as a false positive or false negative report; and 'no result' was defined as a non-returned answer for that survey. Th e 'no result' responses were excluded from the calculations for percentage correct, percentage partially correct, and percentage incorrect results. Th e assessment of the quality of the Giemsa staining of blood films was based on microscopical examination by one observer of the stained films returned by participants. Staining quality was rated 'acceptable' if the film was not markedly understained or did not have so much stain precipitate present that blood cells or parasites were obscured. Results and D i s c u s s i o n The review of EQ A results for malaria laboratory diagnosis is presented in the Figure. This shows that the N H L S laboratories in the Western Cape had the highest percentage of incorrect results (20.9%), followed by the Northern Cape and Free State (18.5%), and Eastern Cape (17.6%). Malaria is not endemic to these areas, which treat fewer malaria cases than the rest of South Africa. A lack of experience and a lower frequency of malaria specimens may account for the higher percentage of incorrect results. However, the laboratories in North West, which has no malariaendemic areas, had a lower percentage of incorrect results (12.1%) than KwaZulu-Natal (16.7%), which has malaria-endemic areas. Limpopo and Mpumalanga, which both have malaria-endemic areas, had similar
676
L. DINI AND J. FREAN
Table 1. T y p e s o f challenges for s e v e n surveys of the South African National H e a l t h Laboratory S e r v i c e P a r a s i t o l o g y External Quality A s s e s s m e n t P r o g r a m m e , January 2000 to August 2002 Survey no.
Type of challenge
Expected result
1 2 3 4 5 6 7
Interpret stained thick blood film Stain and interpret thin blood film Interpret stained thin blood film Stain and interpret thick blood film Stain and interpret thin blood film Stain and interpret thin blood film Stain and interpret thin blood film
No parasite seen No parasite seen
Plasmodium malariae No parasite seen
Plasmodium falciparum No parasite seen
Plasmodiumfalciparum
100=
78
n
=
113
n =
33
n
=
66
n
=
54
n
=
74
n =
43
n
=
78
n =
77
9080 70 60 50
i
40 30 ¸
20 ¸
i0,
/o// 0
-"
-"
-"
,/
r
-"
-°
°
/o;,;9J
o-"
r~
J
o,.
J
Regions
~
# % correct % partially correct % incorrect % no result
Figure. South African National Health Laboratory Service Parasitology External Quality Assessment Programme results per region for laboratory diagnosis of malaria, January 2000-August 2002. n = total number of responses, excluding 'no result'. percentages of incorrect results: namely 10.6% and 10.3%, respectively. Gauteng had the lowest percentage incorrect (8.8%), whilst the mine and private laboratories had 14.3% incorrect. T h e mean incorrect result rate was 13.8% (95% CI 11.3-16.9), which is alarmingly high, with about 1 in 7 blood films being incorrectly interpreted. Mpumalanga and Gauteng had the highest percentage of correct results, 87.2% and 84.1%, respectively, followed by Limpopo, North West and mine and private laboratories, which all had 81.8% correct. KwaZulu-Natal had the lowest percentage of correct results with 73%. T h e mean correct result rate was 80% (95% CI 77-83%). T h e mean partially correct result rate was 6% (95% CI 5.8-6.6). One would expect the provinces with malaria-endemic areas to perform better in malaria proficiency testing. However, this is clearly not the case. It should be noted that only certain areas of KwaZulu-Natal, Mpumalanga and Limpopo experience malaria transmission and results from laboratories serving these areas have not been separated. Table 2 shows an analysis of the Giemsa staining
T a b l e 2 . Analysis o f G i e m s a staining quality linked to blood f i l m i n t e r p r e t a t i o n for s e v e n surv e y s o f the South African N a t i o n a l H e a l t h Lab o r a t o r y S e r v i c e P a r a s i t o l o g y E x t e r n a l Quality A s s e s s m e n t P r o g r a m m e , January 2000 to August 2002 Giemsa stain result Blood film interpretation Correct Incorrect
Acceptable
Unacceptable
125 18
7 3
results compared with the interpretation of blood films from 2 surveys. There was no significant association between Giemsa stain quality and blood film interpretation (Fisher's exact test, P = 0.28). In fact, most participants with incorrect blood film interpretations had acceptable Giemsa staining quality, indicating that there is less of a problem with staining technique than with microscopical interpretation of a blood film.
QUALITY ASSESSMENT OF MALARIALABORATORYDIAGNOSIS IN SOUTH AFRICA Conclusions
T h e results from this E Q A review clearly suggest that malaria laboratory diagnosis throughout South Africa needs strengthening. Laboratories in n o n - e n d e m i c malaria areas need to be able to diagnose malaria, as both infected persons and infected mosquitoes travel outside endemic areas (Isaficson & Frean, 2001). F u r t h e r teaching and training in malaria laboratory diagnosis are urgently required. W e put forward the following r e c o m m e n d a t i o n s to strengthen malaria diagnosis in South African public and private sector laboratories: • extensive laboratory reviews and quality audits should be c o n d u c t e d to identify the areas that need strengthening; • national standards for malaria laboratory diagnosis need to be developed as a matter of urgency; • quality assurance systems should be put in place and maintained in laboratories; • laboratory infrastructure and transport need to be improved; • regional malaria laboratory reference centres need to be identified to assist in training, E Q A , and referral of specimens; • a comprehensive training course and manual for malaria laboratory diagnosis needs to be developed for South Africa; • all laboratory staff involved in malaria diagnosis should be well trained and their skills regularly assessed; • assessment of skills needs to be extended to new
677
technology, in particular rapid i m m u n o c h r o m a t o graphic antigen tests. T h e need for participation in a Parasitology E Q A P r o g r a m m e cannot be overemphasized and it is of utmost importance that the quality of laboratory results is monitored. T h e r e is reason for optimism, as we have shown that participation in this E Q A p r o g r a m m e has led to an overall i m p r o v e m e n t in performance of laboratories ( D i n i e t al., 2001). References
Anonymous (2001). National Department of Health: notifiable medical conditions. EpidemiologicalComments, 4, 35. Dini, L., Frean, J. & Stewart, M. (2001). The South African parasitology quality assurance programme. Epidemiological Comments, 4, 17-20. Durrheim, D. N., Becker, P. J., Billinghurst, K. & Brink, A. (1997). Diagnostic disagreement--the lessons learnt from malaria diagnosis in Mpumalanga. South African Medical Journal, 87, 609-611. Hansford, C. F. & van Vuuren, C. M. (1997). Diagnostic disagreement with malaria diagnosis in Mpumalanga. South African Medical Journal, 87, 910-912. Isa~icson, M. & Frean, J. A. (2001). Odyssean and nonmosquito-transmitted forms of malaria. In: Travelers' Malaria. Schlagenhauf, P. (editor). Hamilton, Ontario: B.C. Dekker, pp. 463-473. Samba, E. (2001). Preface. The malaria burden and Africa. American Journal of Tropical Medicine and Hygiene, 64, supplement, ii.
Received 23 April 2003; revised 23 June 2003; accepted for publication 23 June 2003
Quality assessment of malaria laboratory diagnosis in South Africa Leigh D i n i and John Frean Parasitology Reference Unit, National Institute for Communicable Diseases, National Health Laboratory Service, Box 1038, Johannesburg 2000, South Africa Abstract T o assess the quality of malaria diagnosis in 115 South African laboratories participating in the National Health Laboratory Service Parasitology External Quality Assessment Programme we reviewed the results from 7 surveys from January 2000 to August 2002. Th e mean percentage incorrect result rate was 13.8% (95% CI 11.3-16.9%), which is alarmingly high, with about 1 in 7 blood films being incorrectly interpreted. Most participants with incorrect blood film interpretations had acceptable Giemsa staining quality, indicating that there is less of a problem with staining technique than with blood film interpretation. Laboratories in provinces in which malaria is endemic did not necessarily perform better than those in non-endemic areas. T h e results clearly suggest that malaria laboratory diagnosis throughout South Africa needs strengthening by improving laboratory standardization and auditing, training, quality assurance and referral resources. Keywords: malaria, diagnosis, quality assessment, South Africa Introduction Malaria is one of the 6 major causes of 90% of deaths from communicable diseases. Malaria deaths worldwide amount to over one million annually (Samba, 2001), and 100-300 deaths are notified in South Africa each year (Anonymous, 2001). Prompt, reliable laboratory diagnosis is recognized as an important component of effective malaria case management and control. Strengthening laboratory diagnosis should, ultimately, decrease malaria morbidity and mortality. Surprisingly, at a South African national malaria conference in Malelane in August 2002, there were no contributions dealing with the laboratory diagnosis of malaria. It seems that there are very few published data on the quality of malaria laboratory diagnosis in South Africa (Durrheim et al., 1997; Hansford & van Vuuren, 1997). Durrheim et al. (1997) noted minimal agreement in microscopical malaria slide examination amongst 4 diagnostic laboratories and pointed to the absence of uniform training and quality assessment programmes. T h e aim of our review was to assess the quality of malaria laboratory diagnosis in 115 South African laboratories participating in the National Health Laboratory Service (NHLS) Parasitology External Quality Assessment (EQA) Programme, from January 2000 to August 2002. T h e Parasitology Reference Unit of the National Institute for Communicable Diseases in Johannesburg coordinates an E Q A programme for over 100 medical diagnostic laboratories in southern Africa. As part of each EQA survey the laboratories are assessed on their ability to identify blood parasites. In addition, their ability to correctly stain a blood film and perform parasite counts are regularly assessed. T h e aims of the Parasitology EQA Programme include the following: to build capacity in the field of human diagnostic parasitology in southern Africa, to obtain an objective measure of the diagnostic ability of participants, and to improve knowledge about human diagnostic parasitology. E QA surveys are conducted 4 times a year and each survey comprises one faecal and one blood sample plus a practical laboratory technique exercise. T o ensure the quality of material sent out for each survey, the expected (model) results are compared against those returned by a subset of 10 historically good participants (reference laboratories). After analysis of results, each participant receives an individual report and comprehensive teaching material relevant to that survey.
Address for correspondence: Leigh Dini, Parasitology Reference Unit, NICD NHLS, Box 1038, Johannesburg 2000, South Africa; phone +27 11 4899342, fax +27 11 4899357, e-mail leigh, [email protected]
Methods Th e majority of participants included in the review were public sector laboratories, comprising mainly former South African Institute for Medical Research laboratories and a few former government laboratories. In total, there were 115 participants and 23 were mine and private laboratories from within South Africa. Participants were grouped according to provinces with the Northern Cape and Free State combined, whilst mine and private laboratories were incorporated into one group. It should be noted that most of the dedicated National Malaria Control Programme laboratories do not participate in this E Q A programme and were not included in this review. We included only the results of parasitology E Q A surveys relevant to the diagnosis of malaria from participants within South Africa from January 2000 to August 2002. The results from 7 surveys were reviewed and Table 1 shows the types of challenges and expected results from each of the 7 surveys. The results were graded as percentage correct, partially correct, incorrect, or no result. A 'correct result' was defined as reporting the expected result; a 'partially correct result' was defined as appropriately reporting a Plasmodium sp. present but unable to identify it further; an 'incorrect result' was defined as a false positive or false negative report; and 'no result' was defined as a non-returned answer for that survey. Th e 'no result' responses were excluded from the calculations for percentage correct, percentage partially correct, and percentage incorrect results. Th e assessment of the quality of the Giemsa staining of blood films was based on microscopical examination by one observer of the stained films returned by participants. Staining quality was rated 'acceptable' if the film was not markedly understained or did not have so much stain precipitate present that blood cells or parasites were obscured. Results and D i s c u s s i o n The review of EQ A results for malaria laboratory diagnosis is presented in the Figure. This shows that the N H L S laboratories in the Western Cape had the highest percentage of incorrect results (20.9%), followed by the Northern Cape and Free State (18.5%), and Eastern Cape (17.6%). Malaria is not endemic to these areas, which treat fewer malaria cases than the rest of South Africa. A lack of experience and a lower frequency of malaria specimens may account for the higher percentage of incorrect results. However, the laboratories in North West, which has no malariaendemic areas, had a lower percentage of incorrect results (12.1%) than KwaZulu-Natal (16.7%), which has malaria-endemic areas. Limpopo and Mpumalanga, which both have malaria-endemic areas, had similar
676
L. DINI AND J. FREAN
Table 1. T y p e s o f challenges for s e v e n surveys of the South African National H e a l t h Laboratory S e r v i c e P a r a s i t o l o g y External Quality A s s e s s m e n t P r o g r a m m e , January 2000 to August 2002 Survey no.
Type of challenge
Expected result
1 2 3 4 5 6 7
Interpret stained thick blood film Stain and interpret thin blood film Interpret stained thin blood film Stain and interpret thick blood film Stain and interpret thin blood film Stain and interpret thin blood film Stain and interpret thin blood film
No parasite seen No parasite seen
Plasmodium malariae No parasite seen
Plasmodium falciparum No parasite seen
Plasmodiumfalciparum
100=
78
n
=
113
n =
33
n
=
66
n
=
54
n
=
74
n =
43
n
=
78
n =
77
9080 70 60 50
i
40 30 ¸
20 ¸
i0,
/o// 0
-"
-"
-"
,/
r
-"
-°
°
/o;,;9J
o-"
r~
J
o,.
J
Regions
~
# % correct % partially correct % incorrect % no result
Figure. South African National Health Laboratory Service Parasitology External Quality Assessment Programme results per region for laboratory diagnosis of malaria, January 2000-August 2002. n = total number of responses, excluding 'no result'. percentages of incorrect results: namely 10.6% and 10.3%, respectively. Gauteng had the lowest percentage incorrect (8.8%), whilst the mine and private laboratories had 14.3% incorrect. T h e mean incorrect result rate was 13.8% (95% CI 11.3-16.9), which is alarmingly high, with about 1 in 7 blood films being incorrectly interpreted. Mpumalanga and Gauteng had the highest percentage of correct results, 87.2% and 84.1%, respectively, followed by Limpopo, North West and mine and private laboratories, which all had 81.8% correct. KwaZulu-Natal had the lowest percentage of correct results with 73%. T h e mean correct result rate was 80% (95% CI 77-83%). T h e mean partially correct result rate was 6% (95% CI 5.8-6.6). One would expect the provinces with malaria-endemic areas to perform better in malaria proficiency testing. However, this is clearly not the case. It should be noted that only certain areas of KwaZulu-Natal, Mpumalanga and Limpopo experience malaria transmission and results from laboratories serving these areas have not been separated. Table 2 shows an analysis of the Giemsa staining
T a b l e 2 . Analysis o f G i e m s a staining quality linked to blood f i l m i n t e r p r e t a t i o n for s e v e n surv e y s o f the South African N a t i o n a l H e a l t h Lab o r a t o r y S e r v i c e P a r a s i t o l o g y E x t e r n a l Quality A s s e s s m e n t P r o g r a m m e , January 2000 to August 2002 Giemsa stain result Blood film interpretation Correct Incorrect
Acceptable
Unacceptable
125 18
7 3
results compared with the interpretation of blood films from 2 surveys. There was no significant association between Giemsa stain quality and blood film interpretation (Fisher's exact test, P = 0.28). In fact, most participants with incorrect blood film interpretations had acceptable Giemsa staining quality, indicating that there is less of a problem with staining technique than with microscopical interpretation of a blood film.
QUALITY ASSESSMENT OF MALARIALABORATORYDIAGNOSIS IN SOUTH AFRICA Conclusions
T h e results from this E Q A review clearly suggest that malaria laboratory diagnosis throughout South Africa needs strengthening. Laboratories in n o n - e n d e m i c malaria areas need to be able to diagnose malaria, as both infected persons and infected mosquitoes travel outside endemic areas (Isaficson & Frean, 2001). F u r t h e r teaching and training in malaria laboratory diagnosis are urgently required. W e put forward the following r e c o m m e n d a t i o n s to strengthen malaria diagnosis in South African public and private sector laboratories: • extensive laboratory reviews and quality audits should be c o n d u c t e d to identify the areas that need strengthening; • national standards for malaria laboratory diagnosis need to be developed as a matter of urgency; • quality assurance systems should be put in place and maintained in laboratories; • laboratory infrastructure and transport need to be improved; • regional malaria laboratory reference centres need to be identified to assist in training, E Q A , and referral of specimens; • a comprehensive training course and manual for malaria laboratory diagnosis needs to be developed for South Africa; • all laboratory staff involved in malaria diagnosis should be well trained and their skills regularly assessed; • assessment of skills needs to be extended to new
677
technology, in particular rapid i m m u n o c h r o m a t o graphic antigen tests. T h e need for participation in a Parasitology E Q A P r o g r a m m e cannot be overemphasized and it is of utmost importance that the quality of laboratory results is monitored. T h e r e is reason for optimism, as we have shown that participation in this E Q A p r o g r a m m e has led to an overall i m p r o v e m e n t in performance of laboratories ( D i n i e t al., 2001). References
Anonymous (2001). National Department of Health: notifiable medical conditions. EpidemiologicalComments, 4, 35. Dini, L., Frean, J. & Stewart, M. (2001). The South African parasitology quality assurance programme. Epidemiological Comments, 4, 17-20. Durrheim, D. N., Becker, P. J., Billinghurst, K. & Brink, A. (1997). Diagnostic disagreement--the lessons learnt from malaria diagnosis in Mpumalanga. South African Medical Journal, 87, 609-611. Hansford, C. F. & van Vuuren, C. M. (1997). Diagnostic disagreement with malaria diagnosis in Mpumalanga. South African Medical Journal, 87, 910-912. Isa~icson, M. & Frean, J. A. (2001). Odyssean and nonmosquito-transmitted forms of malaria. In: Travelers' Malaria. Schlagenhauf, P. (editor). Hamilton, Ontario: B.C. Dekker, pp. 463-473. Samba, E. (2001). Preface. The malaria burden and Africa. American Journal of Tropical Medicine and Hygiene, 64, supplement, ii.
Received 23 April 2003; revised 23 June 2003; accepted for publication 23 June 2003