- Email: [email protected]
Comparison of the discriminatory accuracy of four risk criteria for preeclampsia
Accepted Manuscript Comparison of the discriminatory accuracy of four risk criteria for preeclampsia Hid Felizardo Cordero-Franco, Ana María Salinas-Martínez, Tania Abigail García-Alvarez, Gloria Estefanía Medina-Franco, Francisco Javier Guzmán-de la Garza, Oscar Díaz-Sánchez, Gerardo Ramírez-Sandoval PII: DOI: Reference:
S2210-7789(18)30040-0 https://doi.org/10.1016/j.preghy.2018.06.007 PREGHY 457
To appear in:
Pregnancy Hypertension: An International Journal of Women's Cardiovascular Health
Received Date: Revised Date: Accepted Date:
12 February 2018 13 April 2018 9 June 2018
Please cite this article as: Felizardo Cordero-Franco, H., María Salinas-Martínez, A., Abigail García-Alvarez, T., Estefanía Medina-Franco, G., Javier Guzmán-de la Garza, F., Díaz-Sánchez, O., Ramírez-Sandoval, G., Comparison of the discriminatory accuracy of four risk criteria for preeclampsia, Pregnancy Hypertension: An International Journal of Women's Cardiovascular Health (2018), doi: https://doi.org/10.1016/j.preghy.2018.06.007
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
COMPARISON OF THE DISCRIMINATORY ACCURACY OF FOUR RISK CRITERIA FOR PREECLAMPSIA Hid Felizardo Cordero-Francoa,b*, Ana María Salinas-Martíneza,c, Tania Abigail GarcíaAlvarezd, Gloria Estefanía Medina-Francoa, Francisco Javier Guzmán-de la Garzaa,b, Oscar Díaz-Sánchezd, Gerardo Ramírez-Sandovald. a
Unidad de Investigación Epidemiológica y en Servicios de Salud/CIBIN, Delegación Nuevo León, Instituto Mexicano del Seguro Social. Monterrey, Mexico. b
Universidad Autónoma de Nuevo León, Facultad de Medicina, Monterrey, Mexico.
c
Universidad Autónoma de Nuevo León, Facultad de Salud Pública y Nutrición, Monterrey, Mexico. d
Unidad de Medicina Familiar No. 31, Instituto Mexicano del Seguro Social, San Nicolás de los Garza, Mexico. *Corresponding author: Hid Felizardo Cordero-Franco, Conjunto Lincoln (Contiguo Servicio de Urgencias Hospital No. 34), María de Jesús Candia and Ave Lincoln s/n, Monterrey, Nuevo León, Mexico. Zip code 64730. Phone: (+5281) 1257-3125. E-mail: [email protected]
Abbreviations: WHO: World Health Organization; NICE: National Institute for Health and Care Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health; TP: true positives; FP: false positives; PPV: positive predictive value; NPV: negative predictive value; LR +: positive likelihood ratio; LR-: negative likelihood ratio; DOR: diagnostic odds ratio; AUROC: area under the receiver operating characteristics curve; 95% CI: 95% confidence interval; HRF: high risk factor; MRF: moderate risk factor; NTA: not taken into account as a risk factor; BMI: body mass index; SBP: systolic blood pressure DBP: diastolic blood pressure; MAP: mean arterial pressure.
1
ABSTRACT Objectives: Several criteria have been proposed to categorize the risk of preeclampsia, with notable differences between these criteria. We compared the discriminatory accuracy of criteria for categorizing preeclampsia risk established by four institutions, namely, the World Health Organization (WHO), National Institute for Health and Care Excellence (NICE), American College of Obstetricians and Gynecologists (ACOG), and National Center for Technological Excellence in Health (CENETEC), and estimated the concordance between these criteria. Study design: We performed a secondary data analysis of 590 Mexican obstetric patients who received prenatal care in primary care between 2016 and 2017; 160 had a diagnosis of preeclampsia. Main outcome measures: We estimated the true (TP) and false positive (FP) fractions, positive (PPV) and negative predictive values (NPV), positive (LR+) and negative (LR-) likelihood ratios, diagnostic odds ratio (DOR), area under the receiver operating characteristic curve (AUROC), and Kappa coefficient with corresponding 95% confidence intervals (CIs). Results: Only the WHO criteria, followed by the NICE criteria, had the greatest number of accuracy indicators with ideal or acceptable results: TP 83.6%, PPV 60.5%, NPV 90.3%, DOR 14.3, and AUROC 0.79 and TP 84.5%, PPV 51.0 %, NPV 90.3%, DOR 9.7, and AUROC 0.74, respectively. The Kappa coefficient between WHO and NICE criteria was 0.78 (95% CI 0.71-0.85).
2
Conclusions: The discriminatory accuracies of the WHO and NICE criteria were superior to those of the ACOG and CENETEC criteria for classifying preeclampsia risk. Their concordance was good; thus, both criteria seem appropriate for screening preeclampsia in primary care. Keywords: preeclampsia; guideline; risk; sensitivity; specificity; predictive value
3
INTRODUCTION Preeclampsia is a syndrome characterized by arterial hypertension and proteinuria that appears for the first time after week 20 of gestation [1]. It is considered a serious public health problem; the World Health Organization (WHO) estimates an overall preeclampsia incidence of 2.2%, and the incidence is higher in the Americas and Mexico (3.9% and 3.8%, respectively) [2]. Preeclampsia leads to a significant increase in maternal and perinatal morbidity and mortality [2,3]. The origin of preeclampsia is multifactorial. The association between preeclampsia and the pre-existing diseases hypertension, nephropathy, diabetes, antiphospholipid antibody syndrome, thrombophilia, systemic lupus erythematosus, and obesity has long been recognized. Moreover, a personal or family history of preeclampsia, age, first pregnancy, pregnancy interval >10 years or < 2 years, multiple pregnancy, and a diastolic blood pressure of 80-89 mmHg at the beginning of gestation have been recognized as risk factors of preeclampsia [4-7]. However, the strength of association between each of these factors and preeclampsia varies [8]; thus, several risk categorization criteria have been proposed. Noticeably, important differences exist between the several proposed criteria. For example, WHO does not consider age, pregnancy interval or multiple pregnancy as risk factors. Additionally, the National Institute for Health and Care Excellence (NICE) and the National Center for Technological Excellence in Health (CENETEC) distinguish high from moderate-risk factors while WHO and the American College of Obstetricians and Gynecologists (ACOG) do not [4-7]. Such discrepancies create confusion and precaution is required when comparing data because variations can be explained by the risk criteria applied. Moreover, the primary care clinician must choose one for screening purposes. 4
Preventive measures are needed for reducing preeclampsia incidence in primary care. The literature regarding the evaluation of discriminatory accuracy criteria for preeclampsia risk classification is scarce and mostly focuses on NICE and ACOG [9-13]. Studies indicate a higher sensitivity and higher specificity for the ACOG and NICE criteria, respectively [1113]. Comparing discriminatory accuracy criteria seems meaningful for identifying the criteria with the highest precision, allowing less unnecessary exposure to acetylsalicylic acid and fewer unjustified secondary care referrals due to misclassification. The main objective of the present study was to compare the discriminatory accuracy of four risk criteria for preeclampsia in primary care: WHO, NICE, ACOG, and CENETEC criteria. We also estimated the concordance between these criteria.
5
MATERIALS AND METHODS Design, patients, and procedures We performed a secondary data analysis of 590 Mexican obstetric patients who received prenatal care in primary care between 2016 and 2017; 160 had a diagnosis of preeclampsia (systolic and/or diastolic blood pressure ≥ 140/90 mmHg after week 20 of gestation, along with proteinuria [1,2]). The study was approved by the Local Committee of Ethics and Institutional Research and the confidentiality of patients’ data was ensured. Study variables We selected variables that represented known preeclampsia risk factors for categorizing preeclampsia risk according to four different criteria: WHO, ≥1 high-risk factor [4]; NICE, ≥1 high-risk factor or ≥ 2 moderate-risk factors [5]; ACOG, ≥1 risk factor [6], and CENETEC, ≥1 high-risk factor or ≥ 2 moderate-risk factors [7] (Table 1). We also included sociodemographic variables such as age, weight, and height; gynecological and obstetrical variables such as menarche, age of onset of sexual life, gestational age at first and last prenatal visit, and number of prenatal visits; and blood pressure at first prenatal care. Data analysis The analysis consisted of descriptive statistics and the Mann–Whitney U-test test was used to compare clinical and obstetrical characteristics between patients with and without preeclampsia; a p value <0.05 was considered significant. We estimated the true and false positive fractions (TP and FP, respectively), positive and negative predictive values (PPV and NPV, respectively), positive and negative likelihood ratios (LR + and LR-,
6
respectively), diagnostic odds ratio (DOR), and the area under the receiver operating characteristics curve (AUROC) [14-19]. Two reference populations were used: (1) women with no risk factors under analysis (as defined by each institution) and (2) women without any known risk factor, that is, women between 18 and 39 years without diabetes, hypertension, or a history of preeclampsia, with a body mass index <25 kg/m2, non-first pregnancy, without multiple pregnancies, with a diastolic blood pressure <80 mmHg at the first prenatal visit, and with a pregnancy interval between 2 and 10 years. The latter was comprised of 93 women: 9 with preeclampsia and 84 without preeclampsia. The discriminatory accuracy indicators were labeled as ideal, acceptable, and unacceptable as follows. For TP, PPV, and NPV: ideal ≥80%, acceptable 60-79%, and unacceptable <60%. For FP: ideal ≤5%, acceptable 5.1-10%, and unacceptable >10%. For LR +: ideal >10, acceptable 5-10, and unacceptable <5. For LR-: ideal <0.1, acceptable 0.1-0.4, and unacceptable >0.4. For DOR: ideal >20, acceptable 9-20, and - <9; and for AUROC: ideal >0.90, acceptable 0.70-0.90, and unacceptable <0.70 [15-17, 19]. The Kappa coefficient was used to determine the concordance between the criteria; a value of 0.00-0.20 was considered poor; 0.21-0.40, weak; 0.40- 0.60, moderate; 0.61-0.80, good; and 0.81-1.00, very good [20,21]. EPIDAT® version 3.1 was used for accuracy calculations (Xunta de Galicia / Pan-American Health Organization, A Coruña, Spain).
7
RESULTS The mean maternal age was 28.1 ± 5.7 years; 36.8% of the patients were in their first pregnancy. Patients with preeclampsia were characterized by a greater weight at the beginning and end of the index pregnancy, and by a higher blood pressure at the first prenatal care appointment (Table 2). Comparison of discriminatory accuracy based on women with no risk factors under analysis as the reference population (as defined by each institution) NPV was the only indicator with an acceptable result for categorizing preeclampsia risk between the four criteria. The WHO criteria had the greatest number of accuracy indicators with ideal or acceptable values (Tables 3 and 4). Comparison of discriminatory accuracy based on women without any known risk factor as the reference population Improvement in discriminatory accuracy was noticed in this reference population. Again, the WHO criteria had the greatest number of indicators with ideal or acceptable values, followed by the NICE, ACOG, and CENETEC criteria. TP and NPV demonstrated optimal results in the four criteria (Tables 3 and 4). Concordance between risk categorization criteria The mean Kappa between risk categorization criteria was 0.45 (95% CI 0.27-0.60). Only the WHO and NICE criteria achieved a good level of agreement for categorizing preeclampsia risk (Table 5).
8
DISCUSSION In this study, the WHO criteria for classifying preeclampsia risk demonstrated the best discriminatory accuracy compared to the NICE, ACOG, and CENETEC criteria. Particularly, the use of a reference population with no known risk factors improved the results of all four criteria, and a moderate concordance between the criteria was observed. The main outcomes of this study and their implications are discussed below. Reports have shown that the NICE criteria are less sensitive than the ACOG criteria but are more specific for categorizing preeclampsia risk [9-13], and our results support this assertion. A possible reason is that the NICE criteria differentiate high from moderate-risk factors [4]. A previous Swedish study showed that the use of "any factor" or "one or more factors” was not enough for discriminating preeclampsia risk, but rather the use of a particular single factor or a particular combination of factors [12]. It is noteworthy that accuracy was low when calculations were made using women without the risk factors under analysis as the reference population. This implied that concurrent potential risk factors could be present, and were ignored. For this reason, we also evaluated accuracy using women without any known risk factors as the reference population in order to properly clarify discriminatory ability. Several accuracy results changed; some improved while others did not. Particularly, the FP rate increased and reached unacceptable values, evidencing poor specificity results, especially for ACOG criteria. We could not identify WHO or CENETEC discriminatory accuracy information in the literature or statistics on the concordance between the classification criteria for preeclampsia risk. In this study, a moderate mean concordance was found. Pairwise, only
9
two institutions’ criteria showed a good agreement: the WHO and NICE criteria. This outcome, together with the previous best discriminatory accuracy results, suggests that the WHO or NICE criteria can be recommended for screening preeclampsia in primary care, with a reasonable level of confidence for deciding in favor of prophylactic and/or therapeutic actions. Limitations of the study One limitation of this study was the lack of patients with less common risk factors such as antiphospholipid antibody syndrome and chronic kidney diseases. Patients with these types of conditions do not remain in primary care but are referred to secondary or tertiary care. In the future, women with such factors should be included in order to explore the effect of these factors on accuracy. The likelihood ratios provide greater certainty about the ability of criteria to classify a disease [14]. Unfortunately, no criteria had LR + >10 or LR- <0.1, which are the ideal values for a screening test [22]. Our study also had its strengths. The use of various accuracy measures such as LR +, LR-, and DOR added a wider-ranging assessment. In this study, the WHO criteria had the highest DOR, with several main practical advantages. Specifically, DOR is a measure easily interpretable by physicians. It is also a good indicator of a test performance [18] indicating higher capacity for a correct risk classification, which gives the opportunity of disease prevention from the first appointments of prenatal care. Therefore, it could contribute to reduction of maternal and perinatal morbidity and mortality.
10
CONCLUSIONS The WHO and NICE criteria were superior to the ACOG and CENETEC criteria for accurately categorizing preeclampsia risk. They performed even better when women without any known risk factors were used as the reference population. Moreover, these two criteria demonstrated a good concordance and may be considered the most appropriate for preeclampsia screening in primary care. There is a need to redefine preeclampsia risk criteria, based on discriminatory accuracy. A higher precision of categorization would contribute to increased favorable outcomes in pregnancy.
Acknowledgments We want to thank the personnel of the Clinical File Department of the General Hospital of Zone No. 6 of the Mexican Institute of Social Security. Funding information The authors received no funding from an external source. List of contributions -
Hid Felizardo Cordero-Franco. I declare that I participated in the conception and design of the research, the analysis and interpretation of the data, the writing and 11
critical revision of the manuscript and the approval of its final version. I have no conflict of interest. -
Ana María Salinas-Martínez. I declare that I participated in the design of the research, the analysis and interpretation of the data, the writing and critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Tania Abigail García-Alvarez. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Gloria Estefanía Medina-Franco. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Francisco Javier Guzmán-de la Garza. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Oscar Díaz-Sánchez. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Gerardo Ramírez-Sandoval. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
12
REFERENCES [1]
B.W.J. Mol, C.T. Roberts, S. Thangaratinam, L.A. Magee, C.J.M. de Groot, G.J.
Hofmeyr, Pre-eclampsia, Lancet. 387 (2016) 999–1011. [2]
E. Abalos, C. Cuesta, G. Carroli, Z. Qureshi, M. Widmer, J.P. Vogel, J.P. Souza,
WHO Multicountry Survey on Maternal, Pre-eclampsia, eclampsia and adverse maternal and perinatal outcomes: a secondary analysis of the World Health Organization Multicountry Survey on Maternal and Newborn Health, BJOG. 121 Suppl (2014) 14–24. [3]
Instituto Mexicano del Seguro Social, Prevención, diagnóstico y tratamiento de la
preeclampsia en segundo y tercer nivel de atención, Guía Práctica Clínica. http://www.cenetec-difusion.com/CMGPC/S-020-08/ER.pdf, 2017 (accessed 25 January 2018). [4]
World Health Organization, WHO recommendations for prevention and treatment
of pre-eclampsia and eclampsia, Geneva World Health Organization. http://apps.who.int/iris/bitstream/10665/44703/1/9789241548335_eng.pdf, 2011 (accessed 25 January 2018). [5]
NICE, Hypertension in pregnancy: quality standard, NICE Guidelines.
https://www.nice.org.uk/guidance/qs35/resources/hypertension-in-pregnancy-pdf2098607923141, 2013 (accessed 25 January 2018). [6]
American College of Obstetricians and Gynecologists, Task Force on Hypertension
in Pregnancy, Hypertension in pregnancy. Report of the American College of Obstetricians
13
and Gynecologists’ Task Force on Hypertension in Pregnancy, Obstet. Gynecol. 122 (2013) 1122–1131. [7]
Secretaría de Salud, Diagnóstico y detección de enfermedades hipertensivas del
embarazo. http://www.cenetec.salud.gob.mx/descargas/gpc/CatalogoMaestro/058_GPC_Enf.Hipertdel Emb/HIPERTENSION_EMBARAZADAS_EVR_CENETEC.pdf , 2010 (accessed 7 February 2018). [8]
F.A. English, L.C. Kenny, F.P. McCarthy, Risk factors and effective management
of preeclampsia, Integr. Blood Press. Control. 8 (2015) 7–12. [9]
D. Wright, A. Syngelaki, R. Akolekar, L.C. Poon, K.H. Nicolaides, Competing risks
model in screening for preeclampsia by maternal characteristics and medical history, Am. J. Obstet. Gynecol. 213 (2015) 62.e1-62.e10. [10]
D.M. Gallo, D. Wright, C. Casanova, M. Campanero, K.H. Nicolaides, Competing
risks model in screening for preeclampsia by maternal factors and biomarkers at 19-24 weeks’ gestation., Am. J. Obstet. Gynecol. 214 (2016) 619.e1-619.e17. [11]
N. O’Gorman, D. Wright, L.C. Poon, D.L. Rolnik, A. Syngelaki, M. de Alvarado,
I.F. Carbone, V. Dutemeyer, M. Fiolna, A. Frick, N. Karagiotis, S. Mastrodima, C. de Paco Matallana, G. Papaioannou, A. Pazos, W. Plasencia, K.H. Nicolaides, Multicenter screening for pre-eclampsia by maternal factors and biomarkers at 11–13 weeks’ gestation: comparison with NICE guidelines and ACOG recommendations, Ultrasound Obstet. Gynecol. 49 (2017) 756–760.
14
[12]
M. Rodriguez-Lopez, P. Wagner, R. Perez-Vicente, F. Crispi, J. Merlo, Revisiting
the discriminatory accuracy of traditional risk factors in preeclampsia screening, PLoS One. 12 (2017) e0178528. [13]
R.S. Rocha, J.A. Gurgel Alves, S. Bezerra Maia E Holanda Moura, E. Araujo
Junior, W.P. Martins, C.T.M. Vasconcelos, F. Da Silva Costa, M.O.B. Oria, Comparison of three algorithms for prediction preeclampsia in the first trimester of pregnancy, Pregnancy Hypertens. 10 (2017) 113–117. [14]
P. Eusebi, Diagnostic accuracy measures, Cerebrovasc. Dis. 36 (2013) 267–272.
[15]
D.A. Grimes, K.F. Schulz, Refining clinical diagnosis with likelihood ratios,
Lancet. 365 (2005) 1500–1505. [16]
A. Conde-Agudelo, J. Villar, M. Lindheimer, World Health Organization systematic
review of screening tests for preeclampsia, Obstet. Gynecol. 104 (2004) 1367–1391. [17]
J.J. Deeks, Diagnostic tests 4: likelihood ratios, BMJ. 329 (2004) 168–169.
[18]
A.S. Glas, J.G. Lijmer, M.H. Prins, G.J. Bonsel, P.M.M. Bossuyt, The diagnostic
odds ratio: A single indicator of test performance, J. Clin. Epidemiol. 56 (2003) 1129– 1135. [19]
A.K. Akobeng, Understanding diagnostic tests 3: receiver operating characteristic
curves, Acta Paediatr. 96 (2007) 644–647. [20]
NH Wacher, Clinimetría, in: L. Moreno-Altamirano (Ed), Epidemiología Clínica,
McGraw-Hill, México, 2007, pp. 109-139.
15
[21]
A. López, D.U. Galparsoro, P. Fernández, Medidas de concordancia : el índice de
Kappa, Cad Aten Primaria. (2001) 2–6. [22]
K. Leslie, B. Thilaganathan, A. Papageorghiou, Early prediction and prevention of
pre-eclampsia, Best Pract. Res. Clin. Obstet. Gynaecol. 25 (2011) 343–354.
16
Table 1 WHO, NICE, ACOG, and CENETEC risk criteria for preeclampsia [4-7] Risk factor Previous hypertensive disease or preeclampsia Diabetes mellitus Chronic hypertension Autoimmune disease Chronic renal disease Conception by in vitro fertilization Age ≥ 40 years Age < 18 years BMI ≥ 35 kg/m2 BMI ≥ 30 kg/m2 First pregnancy Multiple gestation Family history of preeclampsia Inter-pregnancy interval > 10 years Inter-pregnancy interval < 2 DBP 80-89 mmHg at the beginning of pregnancy
WHO a HRF HRF HRF HRF HRF NTA NTA NTA NTA NTA NTA NTA NTA NTA NTA NTA
NICEb HRF HRF HRF HRF HRF NTA MRF NTA MRF NTA MRF NTA MRF MRF NTA NTA
a
CENETECb HRF HRF HRF HRF HRF NTA HRF MRF HRF MRF MRF HRF MRF MRF MRF MRF
ACOGc No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF NTA NTA No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF NTA NTA NTA
≥1 high-risk factor; b ≥1 high-risk factor or ≥ 2 moderate-risk factors; c ≥1 risk factor. WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); HRF: high-risk factor; MRF: moderate-risk factor; NTA: not taken into account as a risk factor; BMI: body mass index; DBP: diastolic blood pressure.
17
Table 2 Clinical and obstetrical characteristics, and perinatal outcomes of the study population Variable Age (years) Menarche (years) Age of onset of sexual activity (years) Height (cm) Previous weight (kg) Final weight (kg) Gestational age at the first prenatal appointment (weeks) Gestational age at the last prenatal appointment (weeks) Total prenatal visits SBP at the first prenatal appointment (mmHg) DBP at the first prenatal appointment (mmHg) MAP at the first prenatal appointment (mmHg) Gestational age at delivery (weeks) Birth weight (g) Apgar score at 1st minute Apgar score at 5th minute
Preeclampsia Yes (n = 160) No (n = 430) 28.0 ± 5.8 28.2 ± 5.7 12.7 ± 1.7 12.7 ± 1.7 18.9 ± 3.6 19.2 ± 3.7 157.9 ± 6.1 157.9 ± 6.5 69.9 ± 14.4 65.9 ± 14.6 79.8 ± 14.1 74.2 ± 14.1 13.2 ± 6.1 13.8 ± 7.5 34.0 ± 3.2 32.2 ± 7.0 8.5 ± 3.6 8.4 ± 4.5 107.0 ± 13.4 100.8 ± 8.7 67.4 ± 10.1 63.8 ± 6.1 80.6 ± 10.8 76.1 ± 6.2 36.6 ± 3.3 39.1 ± 2.1 2570.3 ± 960.1 3141.0 ± 544.7 7.7 ± 1.5 7.8 ± 0.9 8.7 ± 1.1 8.8 ± 0.7
SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial pressure.
18
p value 0.550 0.437 0.223 0.956 < 0.001 0.002 0.739 0.565 0.458 < 0.001 < 0.001 < 0.001 0.002 0.045 0.885 0.992
Table 3 Comparison of the discriminatory accuracy of the WHO, NICE, ACOG, and CENETEC criteria for categorizing preeclampsia risk [47], according to the type of reference population Organization WHO NICE ACOGc CENETECb Reference population: women with no risk factors under analysis (as defined by each institution) TP (95% CI) 28.8% (21.4-36.1) 30.6% (23.2-38.1) 69.4% (61.9-76.8) 45.0% (37.0-53.0) FP (95% CI) 7.0% (4.6-9.4) 10.9% (8.0-13.9) 58.1% (53.5-62.8) 25.8% (21.7-30.0) PPV (95% CI) 60.5% (48.9-72.2) 51.0% (40.5-61.5) 30.8% (25.9-35.7) 39.3% (32.0-46.7) NPV (95% CI) 77.8% (74.1-81.5) 77.5% (73.7-81.3) 78.6% (73.1-84.1) 78.4% (74.3-82.5) LR+ (95% CI) 4.1 (2.7-6.3) 2.8 (2.0-4.0) 1.2 (1.0-1.4) 1.7 (1.4-2.2) LR- (95% CI) 0.8 (0.7-0.9) 0.8 (0.7-0.9) 0.7 (0.5-0.9) 0.7 (0.6-0.9) AUROC (95% CI) 0.61 (0.57-0.65) 0.60 (0.56-0.64) 0.56 (0.51-0.60) 0.59 (0.55-0.64) DOR (95% CI) 5.4 (3.2-8.9) 3.6 (2.3-5.6) 1.6 (1.1-2.4) 2.4 (1.6-3.4) Reference population: women without any known risk factor TP (95% CI) 83.6% (72.9-94.3) 84.5% (74.3-94.7) 92.5% (87.4-97.6) 88.9% (81.4-96.3) FP (95% CI) 26.3% (18.2-34.4) 35.9% (27.7-44.1) 74.9% (70.2-79.5) 56.9% (50.0-63.9) PPV (95% CI) 60.5% (48.9-72.2) 51.0% (40.5-61.6) 30.8% (25.8-35.6) 39.3% (32.0-46.7) NPV (95% CI) 90.3% (83.8-96.9) 90.3% (83.8-96.9) 90.3% (83.8-96.9) 90.3% (83.8-96.9) LR+ (95% CI) 3.2 (2.3-4.4) 2.4 (1.8-3.0) 1.2 (1.1-1.3) 1.6 (1.3-1.8) LR- (95% CI) 0.2 (0.1-0.4) 0.2 (0.1-0.5) 0.3 (0.2-0.6) 0.3 (0.1-0.5) AUROC (95% CI) 0.79 (0.72-0.85) 0.74 (0.68-0.80) 0.59 (0.56-0.62) 0.66 (0.61-0.71) DOR (95% CI) 14.3 (6.2-32.7) 9.7 (4.4-21.5) 4.1 (2.0-8.5) 6.0 (2.9-12.8)
Indicator of discriminatory accuracy
Indicator of discriminatory accuracy
a
b
a
≥1 high-risk factor; b ≥1 high-risk factor or ≥ 2 moderate-risk factors; c ≥1 risk factor. WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); TP: true positive; FP: false positive; PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio; LR-: negative likelihood ratio; AUROC: area under the receiver-operating characteristic curve; DOR: diagnostic odds ratio; 95% CI: 95% confidence interval. 19
Table 4 Comparison of the level of discriminatory accuracy of the WHO, NICE, ACOG, and CENETEC criteria for categorizing preeclampsia risk [4-7], according to type of reference population
Indicator of discriminatory accuracy
Indicator of discriminatory accuracy
Organization WHO NICE ACOG CENETEC Reference population: women with no risk factors under analysis (as defined by each institution) TP – – + – FP ++ – – – PPV + – – – NPV + + + + LR+ – – – – LR– – – – AUROC – – – – DOR – – – – Reference population: women without any known risk factor TP ++ ++ ++ ++ FP – – – – PPV + – – – NPV ++ ++ ++ ++ LR+ – – – – LR+ + + + AUROC + + – – DOR + + – – WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); TP: true positive; FP: false positive; PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio; LR-: negative likelihood ratio; AUROC: area under the receiver-operating characteristic curve; DOR: diagnostic odds ratio; 95% CI: 95% confidence interval. Symbols: ++: ideal accuracy; + acceptable accuracy; –: unacceptable accuracy.
20
Table 5 Concordance between the preeclampsia risk criteria Organization
Organization NICE
WHO
ACOG
Κappa (95% CI) WHO NICE ACOG CENETEC
0.78 (0.71-0.85) 0.17 (0.13-0.21) 0.50 (0.42-0.57)
0.22 (0.18-0.26) 0.60 (0.53-0.67)
0.43 (0.37-0.48)
WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); CI: confidence interval.
21
HIGHLIGHTS OF THE STUDY COMPARISON OF THE DISCRIMINATORY ACCURACY OF FOUR RISK CRITERIA FOR PREECLAMPSIA -
The criteria for categorizing risk for preeclampsia differ among organizations.
-
The WHO criteria for classifying preeclampsia risk demonstrated the best discriminatory accuracy compared to the NICE, ACOG, and CENETEC criteria
-
Only the WHO and NICE criteria achieved a good level of agreement for categorizing preeclampsia risk
-
WHO and NICE may be considered the most appropriate for preeclampsia screening in primary care.
22
S2210-7789(18)30040-0 https://doi.org/10.1016/j.preghy.2018.06.007 PREGHY 457
To appear in:
Pregnancy Hypertension: An International Journal of Women's Cardiovascular Health
Received Date: Revised Date: Accepted Date:
12 February 2018 13 April 2018 9 June 2018
Please cite this article as: Felizardo Cordero-Franco, H., María Salinas-Martínez, A., Abigail García-Alvarez, T., Estefanía Medina-Franco, G., Javier Guzmán-de la Garza, F., Díaz-Sánchez, O., Ramírez-Sandoval, G., Comparison of the discriminatory accuracy of four risk criteria for preeclampsia, Pregnancy Hypertension: An International Journal of Women's Cardiovascular Health (2018), doi: https://doi.org/10.1016/j.preghy.2018.06.007
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
COMPARISON OF THE DISCRIMINATORY ACCURACY OF FOUR RISK CRITERIA FOR PREECLAMPSIA Hid Felizardo Cordero-Francoa,b*, Ana María Salinas-Martíneza,c, Tania Abigail GarcíaAlvarezd, Gloria Estefanía Medina-Francoa, Francisco Javier Guzmán-de la Garzaa,b, Oscar Díaz-Sánchezd, Gerardo Ramírez-Sandovald. a
Unidad de Investigación Epidemiológica y en Servicios de Salud/CIBIN, Delegación Nuevo León, Instituto Mexicano del Seguro Social. Monterrey, Mexico. b
Universidad Autónoma de Nuevo León, Facultad de Medicina, Monterrey, Mexico.
c
Universidad Autónoma de Nuevo León, Facultad de Salud Pública y Nutrición, Monterrey, Mexico. d
Unidad de Medicina Familiar No. 31, Instituto Mexicano del Seguro Social, San Nicolás de los Garza, Mexico. *Corresponding author: Hid Felizardo Cordero-Franco, Conjunto Lincoln (Contiguo Servicio de Urgencias Hospital No. 34), María de Jesús Candia and Ave Lincoln s/n, Monterrey, Nuevo León, Mexico. Zip code 64730. Phone: (+5281) 1257-3125. E-mail: [email protected]
Abbreviations: WHO: World Health Organization; NICE: National Institute for Health and Care Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health; TP: true positives; FP: false positives; PPV: positive predictive value; NPV: negative predictive value; LR +: positive likelihood ratio; LR-: negative likelihood ratio; DOR: diagnostic odds ratio; AUROC: area under the receiver operating characteristics curve; 95% CI: 95% confidence interval; HRF: high risk factor; MRF: moderate risk factor; NTA: not taken into account as a risk factor; BMI: body mass index; SBP: systolic blood pressure DBP: diastolic blood pressure; MAP: mean arterial pressure.
1
ABSTRACT Objectives: Several criteria have been proposed to categorize the risk of preeclampsia, with notable differences between these criteria. We compared the discriminatory accuracy of criteria for categorizing preeclampsia risk established by four institutions, namely, the World Health Organization (WHO), National Institute for Health and Care Excellence (NICE), American College of Obstetricians and Gynecologists (ACOG), and National Center for Technological Excellence in Health (CENETEC), and estimated the concordance between these criteria. Study design: We performed a secondary data analysis of 590 Mexican obstetric patients who received prenatal care in primary care between 2016 and 2017; 160 had a diagnosis of preeclampsia. Main outcome measures: We estimated the true (TP) and false positive (FP) fractions, positive (PPV) and negative predictive values (NPV), positive (LR+) and negative (LR-) likelihood ratios, diagnostic odds ratio (DOR), area under the receiver operating characteristic curve (AUROC), and Kappa coefficient with corresponding 95% confidence intervals (CIs). Results: Only the WHO criteria, followed by the NICE criteria, had the greatest number of accuracy indicators with ideal or acceptable results: TP 83.6%, PPV 60.5%, NPV 90.3%, DOR 14.3, and AUROC 0.79 and TP 84.5%, PPV 51.0 %, NPV 90.3%, DOR 9.7, and AUROC 0.74, respectively. The Kappa coefficient between WHO and NICE criteria was 0.78 (95% CI 0.71-0.85).
2
Conclusions: The discriminatory accuracies of the WHO and NICE criteria were superior to those of the ACOG and CENETEC criteria for classifying preeclampsia risk. Their concordance was good; thus, both criteria seem appropriate for screening preeclampsia in primary care. Keywords: preeclampsia; guideline; risk; sensitivity; specificity; predictive value
3
INTRODUCTION Preeclampsia is a syndrome characterized by arterial hypertension and proteinuria that appears for the first time after week 20 of gestation [1]. It is considered a serious public health problem; the World Health Organization (WHO) estimates an overall preeclampsia incidence of 2.2%, and the incidence is higher in the Americas and Mexico (3.9% and 3.8%, respectively) [2]. Preeclampsia leads to a significant increase in maternal and perinatal morbidity and mortality [2,3]. The origin of preeclampsia is multifactorial. The association between preeclampsia and the pre-existing diseases hypertension, nephropathy, diabetes, antiphospholipid antibody syndrome, thrombophilia, systemic lupus erythematosus, and obesity has long been recognized. Moreover, a personal or family history of preeclampsia, age, first pregnancy, pregnancy interval >10 years or < 2 years, multiple pregnancy, and a diastolic blood pressure of 80-89 mmHg at the beginning of gestation have been recognized as risk factors of preeclampsia [4-7]. However, the strength of association between each of these factors and preeclampsia varies [8]; thus, several risk categorization criteria have been proposed. Noticeably, important differences exist between the several proposed criteria. For example, WHO does not consider age, pregnancy interval or multiple pregnancy as risk factors. Additionally, the National Institute for Health and Care Excellence (NICE) and the National Center for Technological Excellence in Health (CENETEC) distinguish high from moderate-risk factors while WHO and the American College of Obstetricians and Gynecologists (ACOG) do not [4-7]. Such discrepancies create confusion and precaution is required when comparing data because variations can be explained by the risk criteria applied. Moreover, the primary care clinician must choose one for screening purposes. 4
Preventive measures are needed for reducing preeclampsia incidence in primary care. The literature regarding the evaluation of discriminatory accuracy criteria for preeclampsia risk classification is scarce and mostly focuses on NICE and ACOG [9-13]. Studies indicate a higher sensitivity and higher specificity for the ACOG and NICE criteria, respectively [1113]. Comparing discriminatory accuracy criteria seems meaningful for identifying the criteria with the highest precision, allowing less unnecessary exposure to acetylsalicylic acid and fewer unjustified secondary care referrals due to misclassification. The main objective of the present study was to compare the discriminatory accuracy of four risk criteria for preeclampsia in primary care: WHO, NICE, ACOG, and CENETEC criteria. We also estimated the concordance between these criteria.
5
MATERIALS AND METHODS Design, patients, and procedures We performed a secondary data analysis of 590 Mexican obstetric patients who received prenatal care in primary care between 2016 and 2017; 160 had a diagnosis of preeclampsia (systolic and/or diastolic blood pressure ≥ 140/90 mmHg after week 20 of gestation, along with proteinuria [1,2]). The study was approved by the Local Committee of Ethics and Institutional Research and the confidentiality of patients’ data was ensured. Study variables We selected variables that represented known preeclampsia risk factors for categorizing preeclampsia risk according to four different criteria: WHO, ≥1 high-risk factor [4]; NICE, ≥1 high-risk factor or ≥ 2 moderate-risk factors [5]; ACOG, ≥1 risk factor [6], and CENETEC, ≥1 high-risk factor or ≥ 2 moderate-risk factors [7] (Table 1). We also included sociodemographic variables such as age, weight, and height; gynecological and obstetrical variables such as menarche, age of onset of sexual life, gestational age at first and last prenatal visit, and number of prenatal visits; and blood pressure at first prenatal care. Data analysis The analysis consisted of descriptive statistics and the Mann–Whitney U-test test was used to compare clinical and obstetrical characteristics between patients with and without preeclampsia; a p value <0.05 was considered significant. We estimated the true and false positive fractions (TP and FP, respectively), positive and negative predictive values (PPV and NPV, respectively), positive and negative likelihood ratios (LR + and LR-,
6
respectively), diagnostic odds ratio (DOR), and the area under the receiver operating characteristics curve (AUROC) [14-19]. Two reference populations were used: (1) women with no risk factors under analysis (as defined by each institution) and (2) women without any known risk factor, that is, women between 18 and 39 years without diabetes, hypertension, or a history of preeclampsia, with a body mass index <25 kg/m2, non-first pregnancy, without multiple pregnancies, with a diastolic blood pressure <80 mmHg at the first prenatal visit, and with a pregnancy interval between 2 and 10 years. The latter was comprised of 93 women: 9 with preeclampsia and 84 without preeclampsia. The discriminatory accuracy indicators were labeled as ideal, acceptable, and unacceptable as follows. For TP, PPV, and NPV: ideal ≥80%, acceptable 60-79%, and unacceptable <60%. For FP: ideal ≤5%, acceptable 5.1-10%, and unacceptable >10%. For LR +: ideal >10, acceptable 5-10, and unacceptable <5. For LR-: ideal <0.1, acceptable 0.1-0.4, and unacceptable >0.4. For DOR: ideal >20, acceptable 9-20, and - <9; and for AUROC: ideal >0.90, acceptable 0.70-0.90, and unacceptable <0.70 [15-17, 19]. The Kappa coefficient was used to determine the concordance between the criteria; a value of 0.00-0.20 was considered poor; 0.21-0.40, weak; 0.40- 0.60, moderate; 0.61-0.80, good; and 0.81-1.00, very good [20,21]. EPIDAT® version 3.1 was used for accuracy calculations (Xunta de Galicia / Pan-American Health Organization, A Coruña, Spain).
7
RESULTS The mean maternal age was 28.1 ± 5.7 years; 36.8% of the patients were in their first pregnancy. Patients with preeclampsia were characterized by a greater weight at the beginning and end of the index pregnancy, and by a higher blood pressure at the first prenatal care appointment (Table 2). Comparison of discriminatory accuracy based on women with no risk factors under analysis as the reference population (as defined by each institution) NPV was the only indicator with an acceptable result for categorizing preeclampsia risk between the four criteria. The WHO criteria had the greatest number of accuracy indicators with ideal or acceptable values (Tables 3 and 4). Comparison of discriminatory accuracy based on women without any known risk factor as the reference population Improvement in discriminatory accuracy was noticed in this reference population. Again, the WHO criteria had the greatest number of indicators with ideal or acceptable values, followed by the NICE, ACOG, and CENETEC criteria. TP and NPV demonstrated optimal results in the four criteria (Tables 3 and 4). Concordance between risk categorization criteria The mean Kappa between risk categorization criteria was 0.45 (95% CI 0.27-0.60). Only the WHO and NICE criteria achieved a good level of agreement for categorizing preeclampsia risk (Table 5).
8
DISCUSSION In this study, the WHO criteria for classifying preeclampsia risk demonstrated the best discriminatory accuracy compared to the NICE, ACOG, and CENETEC criteria. Particularly, the use of a reference population with no known risk factors improved the results of all four criteria, and a moderate concordance between the criteria was observed. The main outcomes of this study and their implications are discussed below. Reports have shown that the NICE criteria are less sensitive than the ACOG criteria but are more specific for categorizing preeclampsia risk [9-13], and our results support this assertion. A possible reason is that the NICE criteria differentiate high from moderate-risk factors [4]. A previous Swedish study showed that the use of "any factor" or "one or more factors” was not enough for discriminating preeclampsia risk, but rather the use of a particular single factor or a particular combination of factors [12]. It is noteworthy that accuracy was low when calculations were made using women without the risk factors under analysis as the reference population. This implied that concurrent potential risk factors could be present, and were ignored. For this reason, we also evaluated accuracy using women without any known risk factors as the reference population in order to properly clarify discriminatory ability. Several accuracy results changed; some improved while others did not. Particularly, the FP rate increased and reached unacceptable values, evidencing poor specificity results, especially for ACOG criteria. We could not identify WHO or CENETEC discriminatory accuracy information in the literature or statistics on the concordance between the classification criteria for preeclampsia risk. In this study, a moderate mean concordance was found. Pairwise, only
9
two institutions’ criteria showed a good agreement: the WHO and NICE criteria. This outcome, together with the previous best discriminatory accuracy results, suggests that the WHO or NICE criteria can be recommended for screening preeclampsia in primary care, with a reasonable level of confidence for deciding in favor of prophylactic and/or therapeutic actions. Limitations of the study One limitation of this study was the lack of patients with less common risk factors such as antiphospholipid antibody syndrome and chronic kidney diseases. Patients with these types of conditions do not remain in primary care but are referred to secondary or tertiary care. In the future, women with such factors should be included in order to explore the effect of these factors on accuracy. The likelihood ratios provide greater certainty about the ability of criteria to classify a disease [14]. Unfortunately, no criteria had LR + >10 or LR- <0.1, which are the ideal values for a screening test [22]. Our study also had its strengths. The use of various accuracy measures such as LR +, LR-, and DOR added a wider-ranging assessment. In this study, the WHO criteria had the highest DOR, with several main practical advantages. Specifically, DOR is a measure easily interpretable by physicians. It is also a good indicator of a test performance [18] indicating higher capacity for a correct risk classification, which gives the opportunity of disease prevention from the first appointments of prenatal care. Therefore, it could contribute to reduction of maternal and perinatal morbidity and mortality.
10
CONCLUSIONS The WHO and NICE criteria were superior to the ACOG and CENETEC criteria for accurately categorizing preeclampsia risk. They performed even better when women without any known risk factors were used as the reference population. Moreover, these two criteria demonstrated a good concordance and may be considered the most appropriate for preeclampsia screening in primary care. There is a need to redefine preeclampsia risk criteria, based on discriminatory accuracy. A higher precision of categorization would contribute to increased favorable outcomes in pregnancy.
Acknowledgments We want to thank the personnel of the Clinical File Department of the General Hospital of Zone No. 6 of the Mexican Institute of Social Security. Funding information The authors received no funding from an external source. List of contributions -
Hid Felizardo Cordero-Franco. I declare that I participated in the conception and design of the research, the analysis and interpretation of the data, the writing and 11
critical revision of the manuscript and the approval of its final version. I have no conflict of interest. -
Ana María Salinas-Martínez. I declare that I participated in the design of the research, the analysis and interpretation of the data, the writing and critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Tania Abigail García-Alvarez. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Gloria Estefanía Medina-Franco. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Francisco Javier Guzmán-de la Garza. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Oscar Díaz-Sánchez. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
-
Gerardo Ramírez-Sandoval. I declare that I participated in the acquisition and interpretation of the data, the critical revision of the manuscript and the approval of its final version. I have no conflict of interest.
12
REFERENCES [1]
B.W.J. Mol, C.T. Roberts, S. Thangaratinam, L.A. Magee, C.J.M. de Groot, G.J.
Hofmeyr, Pre-eclampsia, Lancet. 387 (2016) 999–1011. [2]
E. Abalos, C. Cuesta, G. Carroli, Z. Qureshi, M. Widmer, J.P. Vogel, J.P. Souza,
WHO Multicountry Survey on Maternal, Pre-eclampsia, eclampsia and adverse maternal and perinatal outcomes: a secondary analysis of the World Health Organization Multicountry Survey on Maternal and Newborn Health, BJOG. 121 Suppl (2014) 14–24. [3]
Instituto Mexicano del Seguro Social, Prevención, diagnóstico y tratamiento de la
preeclampsia en segundo y tercer nivel de atención, Guía Práctica Clínica. http://www.cenetec-difusion.com/CMGPC/S-020-08/ER.pdf, 2017 (accessed 25 January 2018). [4]
World Health Organization, WHO recommendations for prevention and treatment
of pre-eclampsia and eclampsia, Geneva World Health Organization. http://apps.who.int/iris/bitstream/10665/44703/1/9789241548335_eng.pdf, 2011 (accessed 25 January 2018). [5]
NICE, Hypertension in pregnancy: quality standard, NICE Guidelines.
https://www.nice.org.uk/guidance/qs35/resources/hypertension-in-pregnancy-pdf2098607923141, 2013 (accessed 25 January 2018). [6]
American College of Obstetricians and Gynecologists, Task Force on Hypertension
in Pregnancy, Hypertension in pregnancy. Report of the American College of Obstetricians
13
and Gynecologists’ Task Force on Hypertension in Pregnancy, Obstet. Gynecol. 122 (2013) 1122–1131. [7]
Secretaría de Salud, Diagnóstico y detección de enfermedades hipertensivas del
embarazo. http://www.cenetec.salud.gob.mx/descargas/gpc/CatalogoMaestro/058_GPC_Enf.Hipertdel Emb/HIPERTENSION_EMBARAZADAS_EVR_CENETEC.pdf , 2010 (accessed 7 February 2018). [8]
F.A. English, L.C. Kenny, F.P. McCarthy, Risk factors and effective management
of preeclampsia, Integr. Blood Press. Control. 8 (2015) 7–12. [9]
D. Wright, A. Syngelaki, R. Akolekar, L.C. Poon, K.H. Nicolaides, Competing risks
model in screening for preeclampsia by maternal characteristics and medical history, Am. J. Obstet. Gynecol. 213 (2015) 62.e1-62.e10. [10]
D.M. Gallo, D. Wright, C. Casanova, M. Campanero, K.H. Nicolaides, Competing
risks model in screening for preeclampsia by maternal factors and biomarkers at 19-24 weeks’ gestation., Am. J. Obstet. Gynecol. 214 (2016) 619.e1-619.e17. [11]
N. O’Gorman, D. Wright, L.C. Poon, D.L. Rolnik, A. Syngelaki, M. de Alvarado,
I.F. Carbone, V. Dutemeyer, M. Fiolna, A. Frick, N. Karagiotis, S. Mastrodima, C. de Paco Matallana, G. Papaioannou, A. Pazos, W. Plasencia, K.H. Nicolaides, Multicenter screening for pre-eclampsia by maternal factors and biomarkers at 11–13 weeks’ gestation: comparison with NICE guidelines and ACOG recommendations, Ultrasound Obstet. Gynecol. 49 (2017) 756–760.
14
[12]
M. Rodriguez-Lopez, P. Wagner, R. Perez-Vicente, F. Crispi, J. Merlo, Revisiting
the discriminatory accuracy of traditional risk factors in preeclampsia screening, PLoS One. 12 (2017) e0178528. [13]
R.S. Rocha, J.A. Gurgel Alves, S. Bezerra Maia E Holanda Moura, E. Araujo
Junior, W.P. Martins, C.T.M. Vasconcelos, F. Da Silva Costa, M.O.B. Oria, Comparison of three algorithms for prediction preeclampsia in the first trimester of pregnancy, Pregnancy Hypertens. 10 (2017) 113–117. [14]
P. Eusebi, Diagnostic accuracy measures, Cerebrovasc. Dis. 36 (2013) 267–272.
[15]
D.A. Grimes, K.F. Schulz, Refining clinical diagnosis with likelihood ratios,
Lancet. 365 (2005) 1500–1505. [16]
A. Conde-Agudelo, J. Villar, M. Lindheimer, World Health Organization systematic
review of screening tests for preeclampsia, Obstet. Gynecol. 104 (2004) 1367–1391. [17]
J.J. Deeks, Diagnostic tests 4: likelihood ratios, BMJ. 329 (2004) 168–169.
[18]
A.S. Glas, J.G. Lijmer, M.H. Prins, G.J. Bonsel, P.M.M. Bossuyt, The diagnostic
odds ratio: A single indicator of test performance, J. Clin. Epidemiol. 56 (2003) 1129– 1135. [19]
A.K. Akobeng, Understanding diagnostic tests 3: receiver operating characteristic
curves, Acta Paediatr. 96 (2007) 644–647. [20]
NH Wacher, Clinimetría, in: L. Moreno-Altamirano (Ed), Epidemiología Clínica,
McGraw-Hill, México, 2007, pp. 109-139.
15
[21]
A. López, D.U. Galparsoro, P. Fernández, Medidas de concordancia : el índice de
Kappa, Cad Aten Primaria. (2001) 2–6. [22]
K. Leslie, B. Thilaganathan, A. Papageorghiou, Early prediction and prevention of
pre-eclampsia, Best Pract. Res. Clin. Obstet. Gynaecol. 25 (2011) 343–354.
16
Table 1 WHO, NICE, ACOG, and CENETEC risk criteria for preeclampsia [4-7] Risk factor Previous hypertensive disease or preeclampsia Diabetes mellitus Chronic hypertension Autoimmune disease Chronic renal disease Conception by in vitro fertilization Age ≥ 40 years Age < 18 years BMI ≥ 35 kg/m2 BMI ≥ 30 kg/m2 First pregnancy Multiple gestation Family history of preeclampsia Inter-pregnancy interval > 10 years Inter-pregnancy interval < 2 DBP 80-89 mmHg at the beginning of pregnancy
WHO a HRF HRF HRF HRF HRF NTA NTA NTA NTA NTA NTA NTA NTA NTA NTA NTA
NICEb HRF HRF HRF HRF HRF NTA MRF NTA MRF NTA MRF NTA MRF MRF NTA NTA
a
CENETECb HRF HRF HRF HRF HRF NTA HRF MRF HRF MRF MRF HRF MRF MRF MRF MRF
ACOGc No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF NTA NTA No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF No difference between HRF and MRF NTA NTA NTA
≥1 high-risk factor; b ≥1 high-risk factor or ≥ 2 moderate-risk factors; c ≥1 risk factor. WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); HRF: high-risk factor; MRF: moderate-risk factor; NTA: not taken into account as a risk factor; BMI: body mass index; DBP: diastolic blood pressure.
17
Table 2 Clinical and obstetrical characteristics, and perinatal outcomes of the study population Variable Age (years) Menarche (years) Age of onset of sexual activity (years) Height (cm) Previous weight (kg) Final weight (kg) Gestational age at the first prenatal appointment (weeks) Gestational age at the last prenatal appointment (weeks) Total prenatal visits SBP at the first prenatal appointment (mmHg) DBP at the first prenatal appointment (mmHg) MAP at the first prenatal appointment (mmHg) Gestational age at delivery (weeks) Birth weight (g) Apgar score at 1st minute Apgar score at 5th minute
Preeclampsia Yes (n = 160) No (n = 430) 28.0 ± 5.8 28.2 ± 5.7 12.7 ± 1.7 12.7 ± 1.7 18.9 ± 3.6 19.2 ± 3.7 157.9 ± 6.1 157.9 ± 6.5 69.9 ± 14.4 65.9 ± 14.6 79.8 ± 14.1 74.2 ± 14.1 13.2 ± 6.1 13.8 ± 7.5 34.0 ± 3.2 32.2 ± 7.0 8.5 ± 3.6 8.4 ± 4.5 107.0 ± 13.4 100.8 ± 8.7 67.4 ± 10.1 63.8 ± 6.1 80.6 ± 10.8 76.1 ± 6.2 36.6 ± 3.3 39.1 ± 2.1 2570.3 ± 960.1 3141.0 ± 544.7 7.7 ± 1.5 7.8 ± 0.9 8.7 ± 1.1 8.8 ± 0.7
SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial pressure.
18
p value 0.550 0.437 0.223 0.956 < 0.001 0.002 0.739 0.565 0.458 < 0.001 < 0.001 < 0.001 0.002 0.045 0.885 0.992
Table 3 Comparison of the discriminatory accuracy of the WHO, NICE, ACOG, and CENETEC criteria for categorizing preeclampsia risk [47], according to the type of reference population Organization WHO NICE ACOGc CENETECb Reference population: women with no risk factors under analysis (as defined by each institution) TP (95% CI) 28.8% (21.4-36.1) 30.6% (23.2-38.1) 69.4% (61.9-76.8) 45.0% (37.0-53.0) FP (95% CI) 7.0% (4.6-9.4) 10.9% (8.0-13.9) 58.1% (53.5-62.8) 25.8% (21.7-30.0) PPV (95% CI) 60.5% (48.9-72.2) 51.0% (40.5-61.5) 30.8% (25.9-35.7) 39.3% (32.0-46.7) NPV (95% CI) 77.8% (74.1-81.5) 77.5% (73.7-81.3) 78.6% (73.1-84.1) 78.4% (74.3-82.5) LR+ (95% CI) 4.1 (2.7-6.3) 2.8 (2.0-4.0) 1.2 (1.0-1.4) 1.7 (1.4-2.2) LR- (95% CI) 0.8 (0.7-0.9) 0.8 (0.7-0.9) 0.7 (0.5-0.9) 0.7 (0.6-0.9) AUROC (95% CI) 0.61 (0.57-0.65) 0.60 (0.56-0.64) 0.56 (0.51-0.60) 0.59 (0.55-0.64) DOR (95% CI) 5.4 (3.2-8.9) 3.6 (2.3-5.6) 1.6 (1.1-2.4) 2.4 (1.6-3.4) Reference population: women without any known risk factor TP (95% CI) 83.6% (72.9-94.3) 84.5% (74.3-94.7) 92.5% (87.4-97.6) 88.9% (81.4-96.3) FP (95% CI) 26.3% (18.2-34.4) 35.9% (27.7-44.1) 74.9% (70.2-79.5) 56.9% (50.0-63.9) PPV (95% CI) 60.5% (48.9-72.2) 51.0% (40.5-61.6) 30.8% (25.8-35.6) 39.3% (32.0-46.7) NPV (95% CI) 90.3% (83.8-96.9) 90.3% (83.8-96.9) 90.3% (83.8-96.9) 90.3% (83.8-96.9) LR+ (95% CI) 3.2 (2.3-4.4) 2.4 (1.8-3.0) 1.2 (1.1-1.3) 1.6 (1.3-1.8) LR- (95% CI) 0.2 (0.1-0.4) 0.2 (0.1-0.5) 0.3 (0.2-0.6) 0.3 (0.1-0.5) AUROC (95% CI) 0.79 (0.72-0.85) 0.74 (0.68-0.80) 0.59 (0.56-0.62) 0.66 (0.61-0.71) DOR (95% CI) 14.3 (6.2-32.7) 9.7 (4.4-21.5) 4.1 (2.0-8.5) 6.0 (2.9-12.8)
Indicator of discriminatory accuracy
Indicator of discriminatory accuracy
a
b
a
≥1 high-risk factor; b ≥1 high-risk factor or ≥ 2 moderate-risk factors; c ≥1 risk factor. WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); TP: true positive; FP: false positive; PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio; LR-: negative likelihood ratio; AUROC: area under the receiver-operating characteristic curve; DOR: diagnostic odds ratio; 95% CI: 95% confidence interval. 19
Table 4 Comparison of the level of discriminatory accuracy of the WHO, NICE, ACOG, and CENETEC criteria for categorizing preeclampsia risk [4-7], according to type of reference population
Indicator of discriminatory accuracy
Indicator of discriminatory accuracy
Organization WHO NICE ACOG CENETEC Reference population: women with no risk factors under analysis (as defined by each institution) TP – – + – FP ++ – – – PPV + – – – NPV + + + + LR+ – – – – LR– – – – AUROC – – – – DOR – – – – Reference population: women without any known risk factor TP ++ ++ ++ ++ FP – – – – PPV + – – – NPV ++ ++ ++ ++ LR+ – – – – LR+ + + + AUROC + + – – DOR + + – – WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); TP: true positive; FP: false positive; PPV: positive predictive value; NPV: negative predictive value; LR+: positive likelihood ratio; LR-: negative likelihood ratio; AUROC: area under the receiver-operating characteristic curve; DOR: diagnostic odds ratio; 95% CI: 95% confidence interval. Symbols: ++: ideal accuracy; + acceptable accuracy; –: unacceptable accuracy.
20
Table 5 Concordance between the preeclampsia risk criteria Organization
Organization NICE
WHO
ACOG
Κappa (95% CI) WHO NICE ACOG CENETEC
0.78 (0.71-0.85) 0.17 (0.13-0.21) 0.50 (0.42-0.57)
0.22 (0.18-0.26) 0.60 (0.53-0.67)
0.43 (0.37-0.48)
WHO: World Health Organization; NICE: National Institute for Clinical Excellence; ACOG: American College of Obstetricians and Gynecologists; CENETEC: National Center for Technological Excellence in Health (Centro Nacional de Excelencia Tecnológica en Salud); CI: confidence interval.
21
HIGHLIGHTS OF THE STUDY COMPARISON OF THE DISCRIMINATORY ACCURACY OF FOUR RISK CRITERIA FOR PREECLAMPSIA -
The criteria for categorizing risk for preeclampsia differ among organizations.
-
The WHO criteria for classifying preeclampsia risk demonstrated the best discriminatory accuracy compared to the NICE, ACOG, and CENETEC criteria
-
Only the WHO and NICE criteria achieved a good level of agreement for categorizing preeclampsia risk
-
WHO and NICE may be considered the most appropriate for preeclampsia screening in primary care.
22