- Email: [email protected]
Combining clinical assessment and the Risk of Ovarian Malignancy Algorithm for the prediction of ovarian cancer
Gynecologic Oncology 135 (2014) 547–551
Contents lists available at ScienceDirect
Gynecologic Oncology journal homepage: www.elsevier.com/locate/ygyno
Combining clinical assessment and the Risk of Ovarian Malignancy Algorithm for the prediction of ovarian cancer Richard G. Moore a,⁎, Douglas M. Hawkins b, M. Craig Miller a, Lisa M. Landrum c, Walter Gajewski d, John J. Ball e, W. Jeffery Allard f, Steven J. Skates g a
Program in Women's Oncology, Department of Obstetrics and Gynecology, Women and Infants Hospital, Alpert Medical School, Brown University, Providence, RI, USA School of Statistics, University of Minnesota, Minneapolis, MN, USA Section of Gynecology Oncology, Department of Obstetrics and Gynecology, Oklahoma University Health Science Center, Oklahoma City, OK, USA d Zimmer Cancer Center, New Hanover Regional Medical Center, Wilmington, NC, USA e Jackson Clinic, Jackson, TN, USA f Medivice Consulting, New Durham, NH, USA g Cancer Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA b c
a r t i c l e
i n f o
Article history: Received 12 August 2014 Accepted 19 October 2014 Available online 23 October 2014 Keywords: ROMA Ovarian cancer Pelvic mass Risk assessment
a b s t r a c t Objectives. ACOG guidelines for the evaluation of women with a pelvic mass employ a combination of physical exam, imaging, and CA125 to guide physicians in the triage of women to gynecologic oncologists. We studied the use of ROMA with clinical assessment for cancer risk assessment in women with a pelvic mass. Methods. This was a prospective, multicenter trial evaluating women with a pelvic mass who had an initial clinical risk assessment (ICRA) performed by a generalist. ROMA scores were calculated and sensitivity, specificity, PPV and NPV were determined for ICRA and ICRA + ROMA. Results. A total of 461 women were entered into the study. There were 375 benign tumors, 48 EOC, 18 LMP tumors and 20 non-ovarian malignancies. For detection of ovarian cancer alone, ICRA had a sensitivity of 85.4%, a specificity of 84.3%, and a NPV of 97.8%. Adding ROMA to ICRA produced a significant improvement of 8.4% in sensitivity, achieving a sensitivity of 93.8% with a specificity of 67.2% and a NPV of 98.8%. Examination of all malignancies (ovarian & non-ovarian) provided a sensitivity of 89.7% for ROMA + ICRA in comparison to 77.9% for ICRA alone, a significant increase in sensitivity of 11.8%. The NPV also significantly increased from 95.5% to 97.3%. Overall, ROMA detected 13 additional malignancies missed by ICRA. Conclusions. Adjunctive use of ROMA with clinical assessment improves the stratification of women with a pelvic mass into low and high risk groups for ovarian cancer. The combination is particularly effective in ruling out malignant disease. © 2014 Elsevier Inc. All rights reserved.
Introduction Women who present with an adnexal mass usually require surgical intervention to determine if the mass is benign or malignant. The presence of an adnexal mass is relatively common with 5–10% of all U.S. women affected in their lifetimes [1,2]. However, only a small proportion of women that present with an adnexal mass will be found to have ovarian cancer. It is critical to accurately assess the risk for malignancy in women presenting with an adnexal mass prior to surgery since outcomes such as survival and risk of reoperation are significantly improved when women with ovarian cancer have surgery performed by a gynecologic oncologist at high volume institutions [3–6]. Diagnosis ⁎ Corresponding author at: Program in Women's Oncology, Department of Obstetrics and Gynecology, Women and Infants Hospital, Alpert Medical School, Brown University, Providence, RI 02925, USA. Fax: +1 401 453 7529. E-mail address: [email protected] (R.G. Moore).
http://dx.doi.org/10.1016/j.ygyno.2014.10.017 0090-8258/© 2014 Elsevier Inc. All rights reserved.
and triage of women with an adnexal mass has traditionally relied on pelvic examination, imaging, and serum CA125 testing but these suffer from low sensitivity and specificity [7]. Advances in imaging technologies have provided modest improvement but transvaginal ultrasound remains the imaging method recommended by guideline committees [8]. Using these tools, only 30%–50% of U.S. women presenting with an ovarian cancer are referred to gynecologic oncologists and institutions that are best prepared to manage women diagnosed with ovarian cancer [9,10]. Improved biomarkers are needed to effectively triage women diagnosed with an ovarian cyst or pelvic mass to appropriate physician specialists. Genomic and proteomic approaches using tissue and serum have led to the discovery of novel biomarkers to improve the accuracy of ovarian cancer detection, and HE4 has been shown in multiple studies to provide improved sensitivity when combined with CA125 [11–14]. HE4 is a member of a family of protease inhibitors [15] and provides improved specificity because it is elevated less frequently than CA125 in benign
548
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551
gynecologic conditions such as endometriosis, and is unaffected by the menstrual cycle and hormonal treatment [16,17]. We evaluated nine serum biomarkers in a pilot study, and showed that only HE4 added sensitivity to CA125 at a set specificity [13]. In addition, we found that HE4 and CA125 combined with menopausal status in a logistic model, Risk of Ovarian Malignancy Algorithm (ROMA), provided improved sensitivity at a set specificity [18]. We validated these findings in a multicenter prospective study that enrolled 531 patients at gynecologic oncology centers and found that at 75% specificity, ROMA sensitivity was 92.3% in postmenopausal women and 76.5% in premenopausal women [18]. These findings were further validated in a subsequent study of 472 patients presenting with an adnexal mass to generalist physicians including obstetrician/gynecologists, internists, family practitioners and general surgeons [19]. While the incidence of ovarian cancer was 10% in this cohort, the sensitivity of ROMA remained 92.3% for postmenopausal women, and 100% for premenopausal women. Similar findings were reported in a recent study where the sensitivity for ovarian cancer detection among women presenting with an adnexal mass, as well as the area under the Receiver Operating Curves (ROC), were found to be greater for ROMA than for CA125 or HE4 alone [20]. An algorithm that combines CA125, menopausal status and ultrasound findings, termed the risk of malignancy index (RMI), is widely used in the United Kingdom to triage women with an adnexal mass [21,22]. We previously compared ROMA directly with RMI and found that ROMA provided 94.3% sensitivity at a specificity of 75% compared with a sensitivity of 84.6% for RMI. To investigate the relationship between ROMA and patient examination and triage by generalists, we compared ROMA performance with a clinical risk assessment using a combination of physical examination, serum CA125 measurement and imaging [8,23]. This clinical evaluation, which we have termed initial clinical risk assessment (ICRA), is the current standard of care in the U.S. In this study, we evaluated the addition of ROMA to the performance of ICRA in a population of women presenting with a pelvic mass to generalist physicians. Methods This was a prospective multicenter blinded clinical trial that enrolled premenopausal and postmenopausal women at 13 geographically dispersed sites across the United States between October 2009 and August 2010, as previously described in detail [19]. Briefly, all participating sites obtained institutional review board (IRB) approval from their respective institutions, and the trial was registered with the National Institutes of Health clinical trial registry (ClinicalTrial.gov identifier NCT00987649). Patients were 18 years of age or older, had provided informed consent, had been diagnosed with an ovarian cyst or pelvic mass by their general gynecologist, family practitioner, gastroenterologist, general surgeon or other non-oncology specialist and were scheduled to undergo surgical intervention. As part of their work-up all patients had radiologic imaging either by pelvic ultrasound (US), computed tomography scanning (CT) and/or magnetic resonance imaging (MRI) within six weeks prior to surgery to document the presence of an ovarian cyst or adnexal mass. Each patient was evaluated by their primary physician at presentation, and, using their clinical assessment, serum CA125 biomarker levels and imaging results, each clinician provided an initial clinical risk assessment (ICRA) categorizing the patient as either high risk or low risk for having a malignancy. The physicians performing ICRA were blinded to serum HE4 values and ROMA scores and this data was not part of the ICRA evaluation. Physicians were instructed to use methods of risk assessment that they commonly used in their everyday clinical practice including the use of serum CA125 levels, imaging studies including ultrasound, CT scans, MRI or any combination thereof. No gynecologic oncologists were used to provide an ICRA. Menopausal status was determined either by physicians through history and physical examination, or using age and serum FSH levels as described in a prior publication [19]. Blood was drawn into serum separator tubes within
30 days prior to surgery and prior to the administration of anesthesia. Serum CA125 concentrations were measured by trained operators using the ARCHITECT CA125II assay (Abbott Diagnostics, Abbott Park, IL) and serum HE4 levels were determined using the HE4 EIA assay (Fujirebio Diagnostics, Inc., Malvern, PA). All assays were run in duplicate according to manufacturers' instructions, and appropriate controls were within the ranges provided by the manufacturer for all runs. Laboratory personnel were masked to the clinical outcomes, and clinicians were masked to serum levels for HE4 and the ROMA value. All pathology was reviewed by a gynecologic pathologist. Statistical analysis ICRA provided by the primary physicians was used to place patients into high and low risk subgroups and performance measures of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were determined by comparing to the outcomes of benign versus malignant disease. In addition to ICRA determination, the Risk of Ovarian Malignancy Algorithm (ROMA) was calculated for all patients using menopausal status and serum concentrations of HE4 and CA125. ROMA was initially described in a previous study for stratification of women into groups with low risk of malignancy and high risk of malignancy [18]. Cut points of 13.1% or greater to define high risk in premenopausal women and 27.7% or greater to define high risk in postmenopausal women were chosen in the original pilot study, based upon a set specificity of 75%. These cut points were validated in subsequent prospective multicenter studies and the same cut points were used in this study. Performance measures of sensitivity, specificity, PPV and NPV were determined for ROMA. ICRA alone and the combination of ROMA and ICRA (with either one being positive constituting high risk) were compared for their ability to differentiate benign versus malignant disease using the performance measures of sensitivity, specificity, PPV and NPV. The change in sensitivity and NPV resulting from the adjunctive use of ROMA was calculated, and a confidence interval generated using the jack-knife method. The statistical analysis of the basic performance measurements consisted of the calculation of exact binomial confidence intervals for each. The change in sensitivity and NPV was considered significant if the 95% confidence interval did not cross zero. Results A total of 512 patients were enrolled at 13 centers in the United States. There were 51 patients excluded from the study; 18 because they did not undergo surgery, 6 with no blood or tissue specimens collected, 5 who had no pathology results available, 5 screen failures, 5 patients to whom ICRA was performed by a gynecologic oncologist, 4 patients to whom ICRA was not completed, 3 patients who withdrew consent and 5 for other reasons. The remaining evaluable cohort of 461 patients included 240 premenopausal women and 221 postmenopausal women (Table 1). The racial and ethnic make-up included 84.8% Caucasian women, and the remaining 15.2% included African-American, Hispanic, Asian, Native American and other racial and ethnic groups. The initial clinical risk assessment was performed for 353 (76.6%) patients by general obstetrician/gynecologists, 66 (14.3%) patients by family practice/internal medicine physicians, and 42 (9.1%) patients by other physicians. ICRA was not performed in this study by gynecologic oncologists in order to evaluate ICRA + ROMA as a triage tool used by generalist physicians. The distribution of benign diseases was representative of those expected in a cohort of this type [19]. There were 375 benign tumors, 48 epithelial ovarian cancers (EOC), 18 tumors of low malignant potential (LMP) and 20 non-ovarian malignancies. For patients with benign disease the most common tumors were serous cystadenomas (24.8%), endometriosis (17.3%), functional cysts (11.7%), mucinous tumors
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551 Table 1 Patient demographics.a
Ethnicity
Caucasian Black Hispanic Asian Native American Other Total Nonmalignant EOC LMP tumors Other cancers Total I II III IV Unstaged Total 1 2 3 Total
Premenopausal
Postmenopausal
Total
Mean (%)
Mean (%)
Mean (%)
193 (80.4)b 20 (8.3) 7 (2.9) 12 (5.0) 2 (0.8) 6 (2.4) 240 220 (91.7) 9 (3.8) 7 (2.9) 4 (1.7) 240 2 (22.2) 1 (11.1) 5 (55.6) 0 (0.0) 1 (11.1) 9 5 (55.6) 0 (0.0) 4 (44.4) 9
198 (89.6)b 11 (5.0) 7 (3.2) 1 (0.5) 2 (0.9) 2 (0.9) 221 155 (70.1) 39 (17.6) 11 (5.0) 16 (7.2) 221 6 (15.4) 3 (7.7) 27 (69.2) 2 (5.1) 1 (2.6) 39 5 (12.8) 9 (23.1) 25 (64.1) 39
391 (84.8)b 31 (6.7) 14 (3.0) 13 (2.8) 4 (0.9) 8 (1.7) 461 375 (81.3) 48 (10.4) 18 (3.9) 20 (4.3) 461 8 (16.7) 4 (8.3) 32 (66.7) 2 (4.2) 2 (4.2) 48 10 (20.8) 9 (18.8) 29 (60.4) 48
549
Table 3 Diagnostic Performance of ICRA vs ICRA + ROMA for EOC and LMP tumors in pre- and post-menopausal women.
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
77.3% (65.3–86.7%) 84.3% (80.2–87.8%) 46.4% (36.8–56.1%) 95.5% (92.6–97.4%)
90.9% (81.3–96.6%) 67.2% (62.2–71.9%) 32.8% (26.0–40.1%) 97.7% (95.0–99.1%)
(8.0%) and teratomas (8.0%). For patients with EOC, 12 (25.0%) had stage I/II, 34 (71.9%) had stage III/IV, and 2 (4.2%) were unstaged. The majority of epithelial ovarian cancers were grade 3 (N = 29), with 10 found to be grade 1 and 9 grade 2 (Table 1). Using cut-points of 13.1% for premenopausal women and 27.7% for postmenopausal women to stratify low and high risk subgroups for the detection of EOC only, we found that ROMA provided a sensitivity of 93.8% (95% CI: 82.8%–98.7%), a specificity of 75.5% (95% CI: 70.8– 79.7%) and a NPV of 99% (95% CI: 97.0–99.8%), as was predicted from previously published studies [18,19]. Analysis for the detection of EOC only in pre- and post-menopausal women combined are displayed in Table 2. ICRA had a sensitivity of 85.4% (95% CI: 72.2–93.9%), a specificity of 84.3% (95% CI: 80.2–87.8%), a NPV of 97.8% (95% CI: 95.6–99.1%) and a PPV of 41.0% (95% CI 31.3– 51.3%). The addition of ROMA to ICRA (ICRA + ROMA) produced an improvement in sensitivity of 8.4% (95% CI: 2.3–20.0%), a statistically significant increase at the 5% level, achieving a sensitivity of 93.8% (95% CI: 82.8–98.7%), a specificity of 67.2% (95% CI: 62.2–71.9%), a NPV of 98.8% (95% CI: 96.6–99.8%) and a PPV of 26.8% (95% CI: 20.3– 34.2%). The increase in sensitivity was particularly striking in the premenopausal subgroup, where sensitivity for ICRA alone was 55.6% (95% CI: 21.2–86.3%), with 4 of 9 patients with EOC stratified to the low risk group. ROMA correctly classified all premenopausal women with EOC, including 2 patients with early stage disease, providing a significant increase in sensitivity of 44.4% (95% CI: 13.7–78.8%), resulting in a sensitivity of 100% (95% CI: 66.4–100%) for ICRA + ROMA. In addition, ICRA + ROMA provided a significant increase in a NPV of 1.98% (95% CI: 0.1–3.9%) over ICRA alone. Consideration of pre- and post-menopausal women diagnosed with EOC including LMP tumors is presented in Table 3. For this group, ICRA
had a sensitivity of 77.3% (95% CI: 65.3–86.7%), a specificity of 84.3% (95% CI: 80.2–87.8%), a PPV of 46.4% (95% CI: 36.8–56.1%) and a NPV of 95.5% (95% CI: 92.6–97.4%). The addition of ROMA to ICRA significantly increased the sensitivity by 13.6% (95% CI: 6.4–24.3%) to 90.9% (95% CI: 81.3–96.6%) and significantly increased the NPV by 2.2% (95% CI: 0.4–4.0%) to 97.7% (95% CI: 95.0–99.1%), with a specificity of 67.2% (95% CI: 62.2–71.9%) and a PPV of 32.8% (95% CI: 26.0–40.1%). Considering premenopausal women alone with EOC including LMP tumors, the addition of ROMA to ICRA significantly increased the sensitivity from 43.8% to 81.3%, an increase of 37.5% (95% CI: 15.2–64.6%). The NPV was also significantly increased by 2.3% (95% CI: 0.1–4.6%) from 95.7% to 98.1%. Examination of the addition of ROMA to ICRA in postmenopausal women showed that the sensitivity was significantly improved by 6.0% (95% CI: 1.3–16.5%). When considering all invasive malignancies (EOC & non-ovarian cancer excluding LMP tumors) in pre- and post-menopausal women (Table 4) the sensitivity of ICRA + ROMA was 89.7% (95% CI: 79.9–95.8%) in comparison to 77.9% (95% CI: 66.2–87.1%) for ICRA alone, providing a significant increase in sensitivity of 11.8% (95% CI: 5.2–21.9%) at the 5% level. The NPV also significantly increased from 95.5% (95% CI: 92.6–97.4%) to 97.3% (95% CI: 94.5–98.9%), an increase of 1.83% (95% CI: 0.2–3.5%). For premenopausal women, ICRA + ROMA significantly increased the sensitivity by 46.2% (95% CI: 19.2– 74.9%) and the NPV by 2.58% (95% CI: 0.3–4.9%) to 84.6% and 98.7%, respectively. In postmenopausal women, sensitivity was significantly increased by 3.6% (95% CI: 0.4–12.5%) to 95.3%, however the NPV did not significantly change and was 95.3%. Consideration of all invasive malignancies and LMP tumors in pre- and post-menopausal women are presented in Table 5. The addition of ROMA to ICRA significantly increased sensitivity and NPV by 15.1% (95% CI: 8.3–24.5%) and 3.0% (95% CI: 0.9–5.0%), respectively. Of the 12 patients diagnosed with early stage EOC, ICRA had a sensitivity of 58.3% (95% CI: 27.7–84.8%) and a specificity of 84.3% (95% CI: 80.2–87.8%). The addition of ROMA to ICRA significantly increased the sensitivity by 16.7% (95% CI: 2.1–48.4%) to 75.0% (95% CI: 42.8–94.5%), with a specificity of 67.2% (95% CI: 62.2–71.9%). The NPV was not significantly changed at 98.8% (95% CI: 96.6–99.8%). Examination of the 12 patients with early stage EOC and 18 patients with LMP tumors showed that ICRA had a sensitivity of 56.7% (95% CI: 37.4–74.5%) and a specificity of 84.3% (95% CI: 80.2–87.8%). The addition of ROMA to ICRA significantly increased the sensitivity by 23.2% (95% CI: 9.9–42.3%) to 80.0% (95% CI: 61.4–92.3%), with a specificity of 67.2% (95% CI: 62.2–71.9%). The NPV significantly increased by 1.75 (95% CI: 0.0–3.2%) to 97.7% (95% CI: 95.0–99.1%). Overall, ROMA detected 13 additional malignancies that were missed by ICRA alone (Table 6).
Table 2 Diagnostic performance of ICRA vs ICRA + ROMA for EOC in pre- and post-menopausal women.
Table 4 Diagnostic performance of ICRA vs ICRA + ROMA for all invasive cancers including EOC and all other cancers in pre- and post-menopausal women.
Pathology
Stage
Grade
a b
Includes 461 evaluable patients with ICRA and ROMA testing performed Percent of total within each subgroup by column
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
85.4% (72.2–93.9%) 84.3% (80.2–87.8%) 41.0% (31.3–51.3%) 97.8% (95.6–99.1%)
93.8% (82.8–98.7%) 67.2% (62.2–71.9%) 26.8% (20.3–34.2%) 98.8% (96.6–99.8%)
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
77.9% (66.2–87.1%) 84.3% (80.2–87.8%) 47.3% (37.8–57.0%) 95.5% (92.6–97.4%)
89.7% (79.9–95.8%) 67.2% (62.2–71.9%) 33.2% (26.4–40.5%) 97.3% (94.5–98.9%)
550
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551
Table 5 Diagnostic performance of ICRA vs ICRA + ROMA for all invasive cancersa and LMP tumors in pre- and post-menopausal women.
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
73.3% (62.6–82.2%) 84.3% (80.2–87.8%) 51.6% (42.4–60.8%) 93.2% (90.0–95.7%)
88.4% (79.7–94.3%) 67.2% (62.2–71.9%) 38.2% (31.4–45.3%) 96.2% (93.1–98.2%)
a Includes 48 EOC, 9 endometrial, 2 sex cord stromal, 1 leiomyosarcoma, 8 other nongynecologic cancers.
Discussion CA125 has been used since the early 1980s to monitor response to therapy in patients under treatment for ovarian cancer and as part of a comprehensive examination to stratify women with an adnexal mass into high and low risk groups. The sensitivity of CA125 is low for stage I ovarian cancer, and the specificity is low as well due to nonspecific elevations in common benign gynecologic tumors and diseases such as endometriosis, pelvic inflammatory disease, and in patients with ascites and other common medical conditions [7,17,24]. Because of the limitations of current diagnostic tools, only 30–50% of women with ovarian cancer have their surgery performed by gynecologic oncologists at high volume institutions experienced in the management of women with ovarian cancer [9,10], which is associated with improved survival rates and decreased risk of reoperation [3–6]. Many studies have evaluated genomic and proteomic approaches as a strategy to define a multi-analyte panel for better identification of patients with ovarian cancer [12–14,25]. The only novel biomarker to gain widespread attention however, has been the whey acidic protein, human epididymis protein 4 (HE4). While HE4 has been found to be expressed in a variety of benign gynecologic tissues including endometriosis, inclusion cysts, benign ovarian surface epithelium and mesothelium, HE4 is not elevated in the serum of patients with endometriosis or a wide variety of benign gynecologic diseases and is therefore an ideal candidate for a serum biomarker for ovarian cancer [17]. Ovarian cancer has a modest incidence, with 22,280 new cases estimated in 2012; however mortality is high, with an estimated 15,500 ovarian cancer deaths annually [26]. Optimal cytoreductive surgery and complete surgical staging procedures contribute to improved survival. It is important that women at high risk for ovarian cancer be referred to gynecologic oncologists and to centers with expertise in the management of women diagnosed with ovarian cancer in order to assure the best outcomes for these patients. Conversely, it is important to accurately identify women at low risk for cancer who can have their surgery performed by their gynecologist and in their community without the stress and increased expense of treatment at a tertiary care center. Additionally, accurate assessment of low risk for malignancy may allow some asymptomatic patients to avoid surgery and proceed with conservative management. We evaluated multiple candidate serum biomarkers for their ability to stratify women with ovarian cancer and benign gynecologic conditions. We found that HE4 provided a high level of sensitivity and specificity, and that the combination of HE4 and CA125 provided better performance than either target alone [13]. No other biomarker combinations were found to improve the performance of CA125 and HE4.
Table 6 Additional cancers identified using ICRA + ROMA. Cancers
Premenopausal N = 240
Postmenopausal N = 221
Total N = 461
EOC LMP Other invasive cancers All cancers + LMP
4 2 2 8
0 3 2 5
4 5 4 13
We then developed the Risk of Ovarian Malignancy Algorithm, ROMA, which uses HE4, CA125 and menopausal status to discriminate malignant from benign gynecologic diseases [18]. We validated the algorithm first in a population of high risk women presenting with adnexal mass to the gynecologic oncologist, and found that at 75% specificity, ROMA provided 92.3% sensitivity for detection of all ovarian cancers, which was higher than the sensitivity associated with either HE4 or CA125 alone [18]. We further validated ROMA performance in a population of women presenting with adnexal mass to generalist physicians including general gynecologists, family physicians, internists and general surgeons and found that at 75% specificity, ROMA provided 92.3% sensitivity, exactly as in the previous trial [19]. Similar results have been found in other studies. Nolen et al. evaluated 65 biomarker targets and found that the combination of HE4 and CA125 provided the highest level of discrimination between women with ovarian cancer and benign diseases, and no other biomarker combination was identified with improved performance [27]. Several studies have shown that HE4 combined with CA125 has improved sensitivity and resulted in a higher area under the ROC curve compared to either biomarker alone [27–29]. In addition, ROMA has been studied in a number of European populations and in some studies, ROMA was found to provide improved sensitivity, area under the ROC curve, and negative predictive value [11,20], with similar levels of sensitivity and area under the ROC curves (AUC) as was found in our previous studies. There have been few reports of contradictory results. Van Gorp and colleagues measured preoperative serum levels of HE4 and CA125 in 228 women with benign diseases and 161 women with cancer, and found that ROMA had an AUC of 0.898 as compared to AUCs of 0.857 for HE4 and 0.877 for CA125 [30]. Further, the AUCs for ROMA and CA125 were not statistically significantly different. In contrast, Montagnana et al. evaluated serum CA125 and HE4 in 55 women with EOC and 49 benign cases and found AUCs for ROMA and HE4 of 0.77 as compared to an AUC of 0.64 for CA125 alone [31]. Thus, both studies found that ROMA gave higher or equivalent AUCs, but one found that ROMA was no better than CA125 alone, and the other found that ROMA was no better than HE4 alone. It is not clear why these studies differed from each other and from our results; however, the sample sizes were small and the incidence of ovarian cancer in these two populations was high, at 40% and 53%, respectively, as compared to 24.3% and 10% in our validation studies [18,19]. One explanation for these findings may be that the high incidence of ovarian cancers in these two studies biased results toward clinical assessment and/or a single marker approach as in the case of CA125 testing, and therefore may not be able to detect a difference between the various tests. The lower incidence of ovarian cancer found in our validation trials more closely mirrors the incidence rate that would be seen in the general population of women presenting with a pelvic mass. We report here that the combination of ROMA and ICRA provides improved performance compared to ICRA alone in a population of women with adnexal mass presenting to a generalist physician population. We compared ROMA combined with ICRA to the current standard of care, where the evaluating physician employed history, clinical exam, CA125 and imaging including US, CT scan and MRI or any combination thereof, to form an initial clinical risk assessment (ICRA). Currently the SGO and ACOG guidelines employ clinical exam, CA125 and imaging to triage women with pelvic masses into high and low risk groups for malignancy. Dearking et al. evaluated the ACOG/SGO guidelines in a study examining 837 women presenting with a pelvic mass and achieved a sensitivity ranging from 84.0 to 91.2% with a NPV of 91.7% to 94.5% for postmenopausal patients and a sensitivity of 75.0% to 79.2% with a NPV of 93.1% to 97.7% for premenopausal patients [32]. We reported in a previous study examining the risk of malignancy index which employs menopausal status, CA125 and US imaging to assign a risk for ovarian cancer and found a sensitivity of 84.6% at a specificity of 75% [33]. In the current study ICRA provided a sensitivity of 85.4% with a specificity of 84.3%. The finding of similar sensitivities
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551
and specificities with the use of either the ACOG/SGO guidelines or RMI suggests that ICRA used in this study is consistent with those methods of ovarian cancer risk assessment. However, ICRA used in this study is more subjective than the other assessments and may therefore vary across U.S. obstetrician-gynecology practices. We found that the addition of ROMA to ICRA provided improved cancer detection in all groups studied, resulting in a 15.1% increase in sensitivity for all cancers including LMP tumors, and a corresponding increase of 8.4% in sensitivity for detection of EOC alone. For all cancers including LMP tumors, the increased sensitivity was seen in both premenopausal women, where the sensitivity increased from 35% for ICRA alone to 75% for the combination of ROMA + ICRA, and in postmenopausal women, where sensitivity increased from 84.8% for ICRA to 92.4% for ICRA + ROMA. The negative predictive value increased from 97.8% to 98.8%, and this high level of NPV will provide strong assurance to generalist physicians that a negative ROMA is predictive of benign disease. The positive predictive value, however, decreased from 51.6% to 38.2% as a consequence of increased numbers of benign disease patients identified as positive by ROMA. The increase in sensitivity resulting from the combination of ROMA and ICRA came from the re-categorization of 13 cancers that ICRA alone had predicted as low risk, whereas the decrease in specificity associated with the combination of ROMA and ICRA came from the re-categorization of 64 benign disease patients that ICRA alone had predicted as low risk. This provides a ratio of increased high risk patients to ovarian cancers detected of 5.9, which represents an acceptable increase in surgeries performed by specialists at high volume centers for the increase in cancers detected, and the corresponding decrease in cancers found by generalists in lower volume centers. These data should translate to U.S. centers because the patient population for this study was enrolled in generalist physician practices, where patients with adnexal masses are most commonly identified and triaged for further work-up and treatment, and ICRA is recommended by ACOG/SGO and is standard of care in many institutions. We conclude that the inclusion of ROMA as part of the initial workup of women diagnosed with a pelvic mass will significantly improve stratification of women into high and low risk groups for malignancy, allowing for more effective triage of these patients. The sensitivity and positive predictive values achieved with ROMA will provide a high degree of confidence that women with positive test results should be referred to a gynecologic oncologist and to centers experienced in the care of women diagnosed with ovarian cancer, and that the high negative predictive value provided by ROMA will give confidence to patients and their physicians that there is a low likelihood that a cancer will be discovered at the time of surgery and that these patents can best be managed in their community setting. Conflict of interest statement Richard G. Moore has received research funding from Fujirebio Diagnostics, Inc. and Abbott Diagnostics, Inc. Doug M. Hawkin is a Statistical Consultant for Fujirebio Diagnostics, Inc. M. Craig Miller is a Database design consultant for Fujirebio Diagnostics, Inc. W. Jeffrey Allard is a consultant for Fujirebio Diagnostics, Inc. Lisa M. Landrum, Walter Gajewski, John J. Ball and Steven J. Skates have no conflicts to report.
References [1] Pejovic T, Nezhat F. Laparoscopic management of adnexal masses the opportunities and the risks. Ann N Y Acad Sci Sep 2001;943:255–68. [2] Curtin JP. Management of the adnexal mass. Gynecol Oncol Dec 1994;55(3 Pt 2): S42–6. [3] Aune G, Torp SH, Syversen U, Hagen B, Tingulstad S. Ten years' experience with centralized surgery of ovarian cancer in one health region in Norway. Int J Gynecol Cancer Feb 2012;22(2):226–31. [4] Bristow RE, Palis BE, Chi DS, Cliby WA. The National Cancer Database report on advanced-stage epithelial ovarian cancer: impact of hospital surgical case volume on overall survival and surgical treatment paradigm. Gynecol Oncol Sep 2010; 118(3):262–7. [5] Paulsen T, Kjaerheim K, Kaern J, Tretli S, Trope C. Improved short-term survival for advanced ovarian, tubal, and peritoneal cancer patients operated at teaching hospitals. Int J Gynecol Cancer Jan 2006;16(Suppl. 1):11–7.
551
[6] Tingulstad S, Skjeldestad FE, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol Sep 2003;102(3):499–505. [7] Bast Jr RC, Xu FJ, Yu YH, Barnhill S, Zhang Z, Mills GB. CA 125: the past and the future. Int J Biol Markers Oct 1998;13(4):179–87. [8] Committee ACOG. Opinion: number 280, December 2002. The role of the generalist obstetrician-gynecologist in the early detection of ovarian cancer. Obstet Gynecol Dec 2002;100(6):1413–6. [9] Carney ME, Lancaster JM, Ford C, Tsodikov A, Wiggins CL. A population-based study of patterns of care for ovarian cancer: who is seen by a gynecologic oncologist and who is not? Gynecol Oncol Jan 2002;84(1):36–42. [10] Gershenson DM. Why American women are not receiving state-of-the-art gynecologic cancer care. Cancer J Nov 2001;7(6):450–7. [11] Karlsen MA, Sandhu N, Hogdall C, Christensen IJ, Nedergaard L, Lundvall L, et al. Evaluation of HE4, CA125, Risk of Ovarian Malignancy Algorithm (ROMA) and risk of malignancy index (RMI) as diagnostic tools of epithelial ovarian cancer in patients with a pelvic mass. Gynecol Oncol Nov 2012;127(2):379–83. [12] Lu KH, Patterson AP, Wang L, Marquez RT, Atkinson EN, Baggerly KA, et al. Selection of potential markers for epithelial ovarian cancer with gene expression arrays and recursive descent partition analysis. Clin Cancer Res May 15 2004; 10(10):3291–300. [13] Moore RG, Brown AK, Miller MC, Skates S, Allard WJ, Verch T, et al. The use of multiple novel tumor biomarkers for the detection of ovarian carcinoma in patients with a pelvic mass. Gynecol Oncol Nov 30 2007;108:402–8. [14] Yurkovetsky Z, Skates S, Lomakin A, Nolen B, Pulsipher T, Modugno F, et al. Development of a multimarker assay for early detection of ovarian cancer. J Clin Oncol May 1 2010;28(13):2159–66. [15] Bouchard D, Morisset D, Bourbonnais Y, Tremblay GM. Proteins with whey-acidicprotein motifs and cancer. Lancet Oncol Feb 2006;7(2):167–74. [16] Bordin L, Fiore C, Dona G, Andrisani A, Ambrosini G, Faggian D, et al. Evaluation of erythrocyte band 3 phosphotyrosine level, glutathione content, CA-125, and human epididymal secretory protein E4 as combined parameters in endometriosis. Fertil Steril Oct 2010;94(5):1616–21. [17] Moore RG, Miller MC, Steinhoff MM, Skates SJ, Lu KH, Lambert-Messerlian G, et al. Serum HE4 levels are less frequently elevated than CA125 in women with benign gynecologic disorders. Am J Obstet Gynecol Apr 2012;206(4):351–8. [18] Moore RG, McMeekin DS, Brown AK, Disilvestro P, Miller MC, Allard WJ, et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecol Oncol Jan 2009;112(1): 40–6. [19] Moore RG, Miller MC, Disilvestro P, Landrum LM, Gajewski W, Ball JJ, et al. Evaluation of the diagnostic accuracy of the Risk of Ovarian Malignancy Algorithm in women with a pelvic mass. Obstet Gynecol Aug 2011;118(2, Part 1):280–8. [20] Molina R, Escudero JM, Auge JM, Filella X, Foj L, Torne A, et al. HE4 a novel tumour marker for ovarian cancer: comparison with CA 125 and ROMA algorithm in patients with gynaecological diseases. Tumour Biol Dec 2011;32(6):1087–95. [21] Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol Oct 1990;97(10): 922–9. [22] Hakansson F, Hogdall EV, Nedergaard L, Lundvall L, Engelholm SA, Pedersen AT, et al. Risk of malignancy index used as a diagnostic tool in a tertiary centre for patients with a pelvic mass. Acta Obstet Gynecol Scand Apr 2012;91(4):496–502. [23] Committee Opinion No. 477: the role of the obstetrician-gynecologist in the early detection of epithelial ovarian cancer. Obstet Gynecol Mar 2011;117(3):742–6. [24] Woolas RP, Xu FJ, Jacobs IJ, Yu YH, Daly L, Berchuck A, et al. Elevation of multiple serum markers in patients with stage I ovarian cancer. J Natl Cancer Inst Nov 3 1993;85(21):1748–51. [25] Husseinzadeh N. Status of tumor markers in epithelial ovarian cancer has there been any progress? A review. Gynecol Oncol Jan 2011;120(1):152–7. [26] Siegel R, Naishadham D, Jemal A. Cancer statistics, 2012. CA Cancer J Clin Jan 2012; 62(1):10–29. [27] Nolen B, Velikokhatnaya L, Marrangoni A, De GK, Lomakin A, Bast Jr RC, et al. Serum biomarker panels for the discrimination of benign from malignant cases in patients with an adnexal mass. Gynecol Oncol Jun 2010;117(3):440–5. [28] Huhtinen K, Suvitie P, Hiissa J, Junnila J, Huvila J, Kujari H, et al. Serum HE4 concentration differentiates malignant ovarian tumours from ovarian endometriotic cysts. Br J Cancer Apr 21 2009;100(8):1315–9. [29] Partheen K, Kristjansdottir B, Sundfeldt K. Evaluation of ovarian cancer biomarkers HE4 and CA-125 in women presenting with a suspicious cystic ovarian mass. J Gynecol Oncol Dec 2011;22(4):244–52. [30] Van GT, Cadron I, Despierre E, Daemen A, Leunen K, Amant F, et al. HE4 and CA125 as a diagnostic test in ovarian cancer: prospective validation of the Risk of Ovarian Malignancy Algorithm. Br J Cancer Mar 1 2011;104(5):863–70. [31] Montagnana M, Danese E, Ruzzenente O, Bresciani V, Nuzzo T, Gelati M, et al. The ROMA (Risk of Ovarian Malignancy Algorithm) for estimating the risk of epithelial ovarian cancer in women presenting with pelvic mass: is it really useful? Clin Chem Lab Med Feb 3 2011;49(3):521–5. [32] Dearking AC, Aletti GD, McGree ME, Weaver AL, Sommerfield MK, Cliby WA. How relevant are ACOG and SGO guidelines for referral of adnexal mass? Obstet Gynecol Oct 2007;110(4):841–8. [33] Moore RG, Jabre-Raughley M, Brown AK, Robison KM, Miller MC, Allard WJ, et al. Comparison of a novel multiple marker assay vs the Risk of Malignancy Index for the prediction of epithelial ovarian cancer in patients with a pelvic mass. Am J Obstet Gynecol May 13 2010;203(3):228–34.
Contents lists available at ScienceDirect
Gynecologic Oncology journal homepage: www.elsevier.com/locate/ygyno
Combining clinical assessment and the Risk of Ovarian Malignancy Algorithm for the prediction of ovarian cancer Richard G. Moore a,⁎, Douglas M. Hawkins b, M. Craig Miller a, Lisa M. Landrum c, Walter Gajewski d, John J. Ball e, W. Jeffery Allard f, Steven J. Skates g a
Program in Women's Oncology, Department of Obstetrics and Gynecology, Women and Infants Hospital, Alpert Medical School, Brown University, Providence, RI, USA School of Statistics, University of Minnesota, Minneapolis, MN, USA Section of Gynecology Oncology, Department of Obstetrics and Gynecology, Oklahoma University Health Science Center, Oklahoma City, OK, USA d Zimmer Cancer Center, New Hanover Regional Medical Center, Wilmington, NC, USA e Jackson Clinic, Jackson, TN, USA f Medivice Consulting, New Durham, NH, USA g Cancer Center, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA b c
a r t i c l e
i n f o
Article history: Received 12 August 2014 Accepted 19 October 2014 Available online 23 October 2014 Keywords: ROMA Ovarian cancer Pelvic mass Risk assessment
a b s t r a c t Objectives. ACOG guidelines for the evaluation of women with a pelvic mass employ a combination of physical exam, imaging, and CA125 to guide physicians in the triage of women to gynecologic oncologists. We studied the use of ROMA with clinical assessment for cancer risk assessment in women with a pelvic mass. Methods. This was a prospective, multicenter trial evaluating women with a pelvic mass who had an initial clinical risk assessment (ICRA) performed by a generalist. ROMA scores were calculated and sensitivity, specificity, PPV and NPV were determined for ICRA and ICRA + ROMA. Results. A total of 461 women were entered into the study. There were 375 benign tumors, 48 EOC, 18 LMP tumors and 20 non-ovarian malignancies. For detection of ovarian cancer alone, ICRA had a sensitivity of 85.4%, a specificity of 84.3%, and a NPV of 97.8%. Adding ROMA to ICRA produced a significant improvement of 8.4% in sensitivity, achieving a sensitivity of 93.8% with a specificity of 67.2% and a NPV of 98.8%. Examination of all malignancies (ovarian & non-ovarian) provided a sensitivity of 89.7% for ROMA + ICRA in comparison to 77.9% for ICRA alone, a significant increase in sensitivity of 11.8%. The NPV also significantly increased from 95.5% to 97.3%. Overall, ROMA detected 13 additional malignancies missed by ICRA. Conclusions. Adjunctive use of ROMA with clinical assessment improves the stratification of women with a pelvic mass into low and high risk groups for ovarian cancer. The combination is particularly effective in ruling out malignant disease. © 2014 Elsevier Inc. All rights reserved.
Introduction Women who present with an adnexal mass usually require surgical intervention to determine if the mass is benign or malignant. The presence of an adnexal mass is relatively common with 5–10% of all U.S. women affected in their lifetimes [1,2]. However, only a small proportion of women that present with an adnexal mass will be found to have ovarian cancer. It is critical to accurately assess the risk for malignancy in women presenting with an adnexal mass prior to surgery since outcomes such as survival and risk of reoperation are significantly improved when women with ovarian cancer have surgery performed by a gynecologic oncologist at high volume institutions [3–6]. Diagnosis ⁎ Corresponding author at: Program in Women's Oncology, Department of Obstetrics and Gynecology, Women and Infants Hospital, Alpert Medical School, Brown University, Providence, RI 02925, USA. Fax: +1 401 453 7529. E-mail address: [email protected] (R.G. Moore).
http://dx.doi.org/10.1016/j.ygyno.2014.10.017 0090-8258/© 2014 Elsevier Inc. All rights reserved.
and triage of women with an adnexal mass has traditionally relied on pelvic examination, imaging, and serum CA125 testing but these suffer from low sensitivity and specificity [7]. Advances in imaging technologies have provided modest improvement but transvaginal ultrasound remains the imaging method recommended by guideline committees [8]. Using these tools, only 30%–50% of U.S. women presenting with an ovarian cancer are referred to gynecologic oncologists and institutions that are best prepared to manage women diagnosed with ovarian cancer [9,10]. Improved biomarkers are needed to effectively triage women diagnosed with an ovarian cyst or pelvic mass to appropriate physician specialists. Genomic and proteomic approaches using tissue and serum have led to the discovery of novel biomarkers to improve the accuracy of ovarian cancer detection, and HE4 has been shown in multiple studies to provide improved sensitivity when combined with CA125 [11–14]. HE4 is a member of a family of protease inhibitors [15] and provides improved specificity because it is elevated less frequently than CA125 in benign
548
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551
gynecologic conditions such as endometriosis, and is unaffected by the menstrual cycle and hormonal treatment [16,17]. We evaluated nine serum biomarkers in a pilot study, and showed that only HE4 added sensitivity to CA125 at a set specificity [13]. In addition, we found that HE4 and CA125 combined with menopausal status in a logistic model, Risk of Ovarian Malignancy Algorithm (ROMA), provided improved sensitivity at a set specificity [18]. We validated these findings in a multicenter prospective study that enrolled 531 patients at gynecologic oncology centers and found that at 75% specificity, ROMA sensitivity was 92.3% in postmenopausal women and 76.5% in premenopausal women [18]. These findings were further validated in a subsequent study of 472 patients presenting with an adnexal mass to generalist physicians including obstetrician/gynecologists, internists, family practitioners and general surgeons [19]. While the incidence of ovarian cancer was 10% in this cohort, the sensitivity of ROMA remained 92.3% for postmenopausal women, and 100% for premenopausal women. Similar findings were reported in a recent study where the sensitivity for ovarian cancer detection among women presenting with an adnexal mass, as well as the area under the Receiver Operating Curves (ROC), were found to be greater for ROMA than for CA125 or HE4 alone [20]. An algorithm that combines CA125, menopausal status and ultrasound findings, termed the risk of malignancy index (RMI), is widely used in the United Kingdom to triage women with an adnexal mass [21,22]. We previously compared ROMA directly with RMI and found that ROMA provided 94.3% sensitivity at a specificity of 75% compared with a sensitivity of 84.6% for RMI. To investigate the relationship between ROMA and patient examination and triage by generalists, we compared ROMA performance with a clinical risk assessment using a combination of physical examination, serum CA125 measurement and imaging [8,23]. This clinical evaluation, which we have termed initial clinical risk assessment (ICRA), is the current standard of care in the U.S. In this study, we evaluated the addition of ROMA to the performance of ICRA in a population of women presenting with a pelvic mass to generalist physicians. Methods This was a prospective multicenter blinded clinical trial that enrolled premenopausal and postmenopausal women at 13 geographically dispersed sites across the United States between October 2009 and August 2010, as previously described in detail [19]. Briefly, all participating sites obtained institutional review board (IRB) approval from their respective institutions, and the trial was registered with the National Institutes of Health clinical trial registry (ClinicalTrial.gov identifier NCT00987649). Patients were 18 years of age or older, had provided informed consent, had been diagnosed with an ovarian cyst or pelvic mass by their general gynecologist, family practitioner, gastroenterologist, general surgeon or other non-oncology specialist and were scheduled to undergo surgical intervention. As part of their work-up all patients had radiologic imaging either by pelvic ultrasound (US), computed tomography scanning (CT) and/or magnetic resonance imaging (MRI) within six weeks prior to surgery to document the presence of an ovarian cyst or adnexal mass. Each patient was evaluated by their primary physician at presentation, and, using their clinical assessment, serum CA125 biomarker levels and imaging results, each clinician provided an initial clinical risk assessment (ICRA) categorizing the patient as either high risk or low risk for having a malignancy. The physicians performing ICRA were blinded to serum HE4 values and ROMA scores and this data was not part of the ICRA evaluation. Physicians were instructed to use methods of risk assessment that they commonly used in their everyday clinical practice including the use of serum CA125 levels, imaging studies including ultrasound, CT scans, MRI or any combination thereof. No gynecologic oncologists were used to provide an ICRA. Menopausal status was determined either by physicians through history and physical examination, or using age and serum FSH levels as described in a prior publication [19]. Blood was drawn into serum separator tubes within
30 days prior to surgery and prior to the administration of anesthesia. Serum CA125 concentrations were measured by trained operators using the ARCHITECT CA125II assay (Abbott Diagnostics, Abbott Park, IL) and serum HE4 levels were determined using the HE4 EIA assay (Fujirebio Diagnostics, Inc., Malvern, PA). All assays were run in duplicate according to manufacturers' instructions, and appropriate controls were within the ranges provided by the manufacturer for all runs. Laboratory personnel were masked to the clinical outcomes, and clinicians were masked to serum levels for HE4 and the ROMA value. All pathology was reviewed by a gynecologic pathologist. Statistical analysis ICRA provided by the primary physicians was used to place patients into high and low risk subgroups and performance measures of sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were determined by comparing to the outcomes of benign versus malignant disease. In addition to ICRA determination, the Risk of Ovarian Malignancy Algorithm (ROMA) was calculated for all patients using menopausal status and serum concentrations of HE4 and CA125. ROMA was initially described in a previous study for stratification of women into groups with low risk of malignancy and high risk of malignancy [18]. Cut points of 13.1% or greater to define high risk in premenopausal women and 27.7% or greater to define high risk in postmenopausal women were chosen in the original pilot study, based upon a set specificity of 75%. These cut points were validated in subsequent prospective multicenter studies and the same cut points were used in this study. Performance measures of sensitivity, specificity, PPV and NPV were determined for ROMA. ICRA alone and the combination of ROMA and ICRA (with either one being positive constituting high risk) were compared for their ability to differentiate benign versus malignant disease using the performance measures of sensitivity, specificity, PPV and NPV. The change in sensitivity and NPV resulting from the adjunctive use of ROMA was calculated, and a confidence interval generated using the jack-knife method. The statistical analysis of the basic performance measurements consisted of the calculation of exact binomial confidence intervals for each. The change in sensitivity and NPV was considered significant if the 95% confidence interval did not cross zero. Results A total of 512 patients were enrolled at 13 centers in the United States. There were 51 patients excluded from the study; 18 because they did not undergo surgery, 6 with no blood or tissue specimens collected, 5 who had no pathology results available, 5 screen failures, 5 patients to whom ICRA was performed by a gynecologic oncologist, 4 patients to whom ICRA was not completed, 3 patients who withdrew consent and 5 for other reasons. The remaining evaluable cohort of 461 patients included 240 premenopausal women and 221 postmenopausal women (Table 1). The racial and ethnic make-up included 84.8% Caucasian women, and the remaining 15.2% included African-American, Hispanic, Asian, Native American and other racial and ethnic groups. The initial clinical risk assessment was performed for 353 (76.6%) patients by general obstetrician/gynecologists, 66 (14.3%) patients by family practice/internal medicine physicians, and 42 (9.1%) patients by other physicians. ICRA was not performed in this study by gynecologic oncologists in order to evaluate ICRA + ROMA as a triage tool used by generalist physicians. The distribution of benign diseases was representative of those expected in a cohort of this type [19]. There were 375 benign tumors, 48 epithelial ovarian cancers (EOC), 18 tumors of low malignant potential (LMP) and 20 non-ovarian malignancies. For patients with benign disease the most common tumors were serous cystadenomas (24.8%), endometriosis (17.3%), functional cysts (11.7%), mucinous tumors
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551 Table 1 Patient demographics.a
Ethnicity
Caucasian Black Hispanic Asian Native American Other Total Nonmalignant EOC LMP tumors Other cancers Total I II III IV Unstaged Total 1 2 3 Total
Premenopausal
Postmenopausal
Total
Mean (%)
Mean (%)
Mean (%)
193 (80.4)b 20 (8.3) 7 (2.9) 12 (5.0) 2 (0.8) 6 (2.4) 240 220 (91.7) 9 (3.8) 7 (2.9) 4 (1.7) 240 2 (22.2) 1 (11.1) 5 (55.6) 0 (0.0) 1 (11.1) 9 5 (55.6) 0 (0.0) 4 (44.4) 9
198 (89.6)b 11 (5.0) 7 (3.2) 1 (0.5) 2 (0.9) 2 (0.9) 221 155 (70.1) 39 (17.6) 11 (5.0) 16 (7.2) 221 6 (15.4) 3 (7.7) 27 (69.2) 2 (5.1) 1 (2.6) 39 5 (12.8) 9 (23.1) 25 (64.1) 39
391 (84.8)b 31 (6.7) 14 (3.0) 13 (2.8) 4 (0.9) 8 (1.7) 461 375 (81.3) 48 (10.4) 18 (3.9) 20 (4.3) 461 8 (16.7) 4 (8.3) 32 (66.7) 2 (4.2) 2 (4.2) 48 10 (20.8) 9 (18.8) 29 (60.4) 48
549
Table 3 Diagnostic Performance of ICRA vs ICRA + ROMA for EOC and LMP tumors in pre- and post-menopausal women.
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
77.3% (65.3–86.7%) 84.3% (80.2–87.8%) 46.4% (36.8–56.1%) 95.5% (92.6–97.4%)
90.9% (81.3–96.6%) 67.2% (62.2–71.9%) 32.8% (26.0–40.1%) 97.7% (95.0–99.1%)
(8.0%) and teratomas (8.0%). For patients with EOC, 12 (25.0%) had stage I/II, 34 (71.9%) had stage III/IV, and 2 (4.2%) were unstaged. The majority of epithelial ovarian cancers were grade 3 (N = 29), with 10 found to be grade 1 and 9 grade 2 (Table 1). Using cut-points of 13.1% for premenopausal women and 27.7% for postmenopausal women to stratify low and high risk subgroups for the detection of EOC only, we found that ROMA provided a sensitivity of 93.8% (95% CI: 82.8%–98.7%), a specificity of 75.5% (95% CI: 70.8– 79.7%) and a NPV of 99% (95% CI: 97.0–99.8%), as was predicted from previously published studies [18,19]. Analysis for the detection of EOC only in pre- and post-menopausal women combined are displayed in Table 2. ICRA had a sensitivity of 85.4% (95% CI: 72.2–93.9%), a specificity of 84.3% (95% CI: 80.2–87.8%), a NPV of 97.8% (95% CI: 95.6–99.1%) and a PPV of 41.0% (95% CI 31.3– 51.3%). The addition of ROMA to ICRA (ICRA + ROMA) produced an improvement in sensitivity of 8.4% (95% CI: 2.3–20.0%), a statistically significant increase at the 5% level, achieving a sensitivity of 93.8% (95% CI: 82.8–98.7%), a specificity of 67.2% (95% CI: 62.2–71.9%), a NPV of 98.8% (95% CI: 96.6–99.8%) and a PPV of 26.8% (95% CI: 20.3– 34.2%). The increase in sensitivity was particularly striking in the premenopausal subgroup, where sensitivity for ICRA alone was 55.6% (95% CI: 21.2–86.3%), with 4 of 9 patients with EOC stratified to the low risk group. ROMA correctly classified all premenopausal women with EOC, including 2 patients with early stage disease, providing a significant increase in sensitivity of 44.4% (95% CI: 13.7–78.8%), resulting in a sensitivity of 100% (95% CI: 66.4–100%) for ICRA + ROMA. In addition, ICRA + ROMA provided a significant increase in a NPV of 1.98% (95% CI: 0.1–3.9%) over ICRA alone. Consideration of pre- and post-menopausal women diagnosed with EOC including LMP tumors is presented in Table 3. For this group, ICRA
had a sensitivity of 77.3% (95% CI: 65.3–86.7%), a specificity of 84.3% (95% CI: 80.2–87.8%), a PPV of 46.4% (95% CI: 36.8–56.1%) and a NPV of 95.5% (95% CI: 92.6–97.4%). The addition of ROMA to ICRA significantly increased the sensitivity by 13.6% (95% CI: 6.4–24.3%) to 90.9% (95% CI: 81.3–96.6%) and significantly increased the NPV by 2.2% (95% CI: 0.4–4.0%) to 97.7% (95% CI: 95.0–99.1%), with a specificity of 67.2% (95% CI: 62.2–71.9%) and a PPV of 32.8% (95% CI: 26.0–40.1%). Considering premenopausal women alone with EOC including LMP tumors, the addition of ROMA to ICRA significantly increased the sensitivity from 43.8% to 81.3%, an increase of 37.5% (95% CI: 15.2–64.6%). The NPV was also significantly increased by 2.3% (95% CI: 0.1–4.6%) from 95.7% to 98.1%. Examination of the addition of ROMA to ICRA in postmenopausal women showed that the sensitivity was significantly improved by 6.0% (95% CI: 1.3–16.5%). When considering all invasive malignancies (EOC & non-ovarian cancer excluding LMP tumors) in pre- and post-menopausal women (Table 4) the sensitivity of ICRA + ROMA was 89.7% (95% CI: 79.9–95.8%) in comparison to 77.9% (95% CI: 66.2–87.1%) for ICRA alone, providing a significant increase in sensitivity of 11.8% (95% CI: 5.2–21.9%) at the 5% level. The NPV also significantly increased from 95.5% (95% CI: 92.6–97.4%) to 97.3% (95% CI: 94.5–98.9%), an increase of 1.83% (95% CI: 0.2–3.5%). For premenopausal women, ICRA + ROMA significantly increased the sensitivity by 46.2% (95% CI: 19.2– 74.9%) and the NPV by 2.58% (95% CI: 0.3–4.9%) to 84.6% and 98.7%, respectively. In postmenopausal women, sensitivity was significantly increased by 3.6% (95% CI: 0.4–12.5%) to 95.3%, however the NPV did not significantly change and was 95.3%. Consideration of all invasive malignancies and LMP tumors in pre- and post-menopausal women are presented in Table 5. The addition of ROMA to ICRA significantly increased sensitivity and NPV by 15.1% (95% CI: 8.3–24.5%) and 3.0% (95% CI: 0.9–5.0%), respectively. Of the 12 patients diagnosed with early stage EOC, ICRA had a sensitivity of 58.3% (95% CI: 27.7–84.8%) and a specificity of 84.3% (95% CI: 80.2–87.8%). The addition of ROMA to ICRA significantly increased the sensitivity by 16.7% (95% CI: 2.1–48.4%) to 75.0% (95% CI: 42.8–94.5%), with a specificity of 67.2% (95% CI: 62.2–71.9%). The NPV was not significantly changed at 98.8% (95% CI: 96.6–99.8%). Examination of the 12 patients with early stage EOC and 18 patients with LMP tumors showed that ICRA had a sensitivity of 56.7% (95% CI: 37.4–74.5%) and a specificity of 84.3% (95% CI: 80.2–87.8%). The addition of ROMA to ICRA significantly increased the sensitivity by 23.2% (95% CI: 9.9–42.3%) to 80.0% (95% CI: 61.4–92.3%), with a specificity of 67.2% (95% CI: 62.2–71.9%). The NPV significantly increased by 1.75 (95% CI: 0.0–3.2%) to 97.7% (95% CI: 95.0–99.1%). Overall, ROMA detected 13 additional malignancies that were missed by ICRA alone (Table 6).
Table 2 Diagnostic performance of ICRA vs ICRA + ROMA for EOC in pre- and post-menopausal women.
Table 4 Diagnostic performance of ICRA vs ICRA + ROMA for all invasive cancers including EOC and all other cancers in pre- and post-menopausal women.
Pathology
Stage
Grade
a b
Includes 461 evaluable patients with ICRA and ROMA testing performed Percent of total within each subgroup by column
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
85.4% (72.2–93.9%) 84.3% (80.2–87.8%) 41.0% (31.3–51.3%) 97.8% (95.6–99.1%)
93.8% (82.8–98.7%) 67.2% (62.2–71.9%) 26.8% (20.3–34.2%) 98.8% (96.6–99.8%)
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
77.9% (66.2–87.1%) 84.3% (80.2–87.8%) 47.3% (37.8–57.0%) 95.5% (92.6–97.4%)
89.7% (79.9–95.8%) 67.2% (62.2–71.9%) 33.2% (26.4–40.5%) 97.3% (94.5–98.9%)
550
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551
Table 5 Diagnostic performance of ICRA vs ICRA + ROMA for all invasive cancersa and LMP tumors in pre- and post-menopausal women.
Sensitivity Specificity PPV NPV
ICRA (95% CI)
ICRA + ROMA (95% CI)
73.3% (62.6–82.2%) 84.3% (80.2–87.8%) 51.6% (42.4–60.8%) 93.2% (90.0–95.7%)
88.4% (79.7–94.3%) 67.2% (62.2–71.9%) 38.2% (31.4–45.3%) 96.2% (93.1–98.2%)
a Includes 48 EOC, 9 endometrial, 2 sex cord stromal, 1 leiomyosarcoma, 8 other nongynecologic cancers.
Discussion CA125 has been used since the early 1980s to monitor response to therapy in patients under treatment for ovarian cancer and as part of a comprehensive examination to stratify women with an adnexal mass into high and low risk groups. The sensitivity of CA125 is low for stage I ovarian cancer, and the specificity is low as well due to nonspecific elevations in common benign gynecologic tumors and diseases such as endometriosis, pelvic inflammatory disease, and in patients with ascites and other common medical conditions [7,17,24]. Because of the limitations of current diagnostic tools, only 30–50% of women with ovarian cancer have their surgery performed by gynecologic oncologists at high volume institutions experienced in the management of women with ovarian cancer [9,10], which is associated with improved survival rates and decreased risk of reoperation [3–6]. Many studies have evaluated genomic and proteomic approaches as a strategy to define a multi-analyte panel for better identification of patients with ovarian cancer [12–14,25]. The only novel biomarker to gain widespread attention however, has been the whey acidic protein, human epididymis protein 4 (HE4). While HE4 has been found to be expressed in a variety of benign gynecologic tissues including endometriosis, inclusion cysts, benign ovarian surface epithelium and mesothelium, HE4 is not elevated in the serum of patients with endometriosis or a wide variety of benign gynecologic diseases and is therefore an ideal candidate for a serum biomarker for ovarian cancer [17]. Ovarian cancer has a modest incidence, with 22,280 new cases estimated in 2012; however mortality is high, with an estimated 15,500 ovarian cancer deaths annually [26]. Optimal cytoreductive surgery and complete surgical staging procedures contribute to improved survival. It is important that women at high risk for ovarian cancer be referred to gynecologic oncologists and to centers with expertise in the management of women diagnosed with ovarian cancer in order to assure the best outcomes for these patients. Conversely, it is important to accurately identify women at low risk for cancer who can have their surgery performed by their gynecologist and in their community without the stress and increased expense of treatment at a tertiary care center. Additionally, accurate assessment of low risk for malignancy may allow some asymptomatic patients to avoid surgery and proceed with conservative management. We evaluated multiple candidate serum biomarkers for their ability to stratify women with ovarian cancer and benign gynecologic conditions. We found that HE4 provided a high level of sensitivity and specificity, and that the combination of HE4 and CA125 provided better performance than either target alone [13]. No other biomarker combinations were found to improve the performance of CA125 and HE4.
Table 6 Additional cancers identified using ICRA + ROMA. Cancers
Premenopausal N = 240
Postmenopausal N = 221
Total N = 461
EOC LMP Other invasive cancers All cancers + LMP
4 2 2 8
0 3 2 5
4 5 4 13
We then developed the Risk of Ovarian Malignancy Algorithm, ROMA, which uses HE4, CA125 and menopausal status to discriminate malignant from benign gynecologic diseases [18]. We validated the algorithm first in a population of high risk women presenting with adnexal mass to the gynecologic oncologist, and found that at 75% specificity, ROMA provided 92.3% sensitivity for detection of all ovarian cancers, which was higher than the sensitivity associated with either HE4 or CA125 alone [18]. We further validated ROMA performance in a population of women presenting with adnexal mass to generalist physicians including general gynecologists, family physicians, internists and general surgeons and found that at 75% specificity, ROMA provided 92.3% sensitivity, exactly as in the previous trial [19]. Similar results have been found in other studies. Nolen et al. evaluated 65 biomarker targets and found that the combination of HE4 and CA125 provided the highest level of discrimination between women with ovarian cancer and benign diseases, and no other biomarker combination was identified with improved performance [27]. Several studies have shown that HE4 combined with CA125 has improved sensitivity and resulted in a higher area under the ROC curve compared to either biomarker alone [27–29]. In addition, ROMA has been studied in a number of European populations and in some studies, ROMA was found to provide improved sensitivity, area under the ROC curve, and negative predictive value [11,20], with similar levels of sensitivity and area under the ROC curves (AUC) as was found in our previous studies. There have been few reports of contradictory results. Van Gorp and colleagues measured preoperative serum levels of HE4 and CA125 in 228 women with benign diseases and 161 women with cancer, and found that ROMA had an AUC of 0.898 as compared to AUCs of 0.857 for HE4 and 0.877 for CA125 [30]. Further, the AUCs for ROMA and CA125 were not statistically significantly different. In contrast, Montagnana et al. evaluated serum CA125 and HE4 in 55 women with EOC and 49 benign cases and found AUCs for ROMA and HE4 of 0.77 as compared to an AUC of 0.64 for CA125 alone [31]. Thus, both studies found that ROMA gave higher or equivalent AUCs, but one found that ROMA was no better than CA125 alone, and the other found that ROMA was no better than HE4 alone. It is not clear why these studies differed from each other and from our results; however, the sample sizes were small and the incidence of ovarian cancer in these two populations was high, at 40% and 53%, respectively, as compared to 24.3% and 10% in our validation studies [18,19]. One explanation for these findings may be that the high incidence of ovarian cancers in these two studies biased results toward clinical assessment and/or a single marker approach as in the case of CA125 testing, and therefore may not be able to detect a difference between the various tests. The lower incidence of ovarian cancer found in our validation trials more closely mirrors the incidence rate that would be seen in the general population of women presenting with a pelvic mass. We report here that the combination of ROMA and ICRA provides improved performance compared to ICRA alone in a population of women with adnexal mass presenting to a generalist physician population. We compared ROMA combined with ICRA to the current standard of care, where the evaluating physician employed history, clinical exam, CA125 and imaging including US, CT scan and MRI or any combination thereof, to form an initial clinical risk assessment (ICRA). Currently the SGO and ACOG guidelines employ clinical exam, CA125 and imaging to triage women with pelvic masses into high and low risk groups for malignancy. Dearking et al. evaluated the ACOG/SGO guidelines in a study examining 837 women presenting with a pelvic mass and achieved a sensitivity ranging from 84.0 to 91.2% with a NPV of 91.7% to 94.5% for postmenopausal patients and a sensitivity of 75.0% to 79.2% with a NPV of 93.1% to 97.7% for premenopausal patients [32]. We reported in a previous study examining the risk of malignancy index which employs menopausal status, CA125 and US imaging to assign a risk for ovarian cancer and found a sensitivity of 84.6% at a specificity of 75% [33]. In the current study ICRA provided a sensitivity of 85.4% with a specificity of 84.3%. The finding of similar sensitivities
R.G. Moore et al. / Gynecologic Oncology 135 (2014) 547–551
and specificities with the use of either the ACOG/SGO guidelines or RMI suggests that ICRA used in this study is consistent with those methods of ovarian cancer risk assessment. However, ICRA used in this study is more subjective than the other assessments and may therefore vary across U.S. obstetrician-gynecology practices. We found that the addition of ROMA to ICRA provided improved cancer detection in all groups studied, resulting in a 15.1% increase in sensitivity for all cancers including LMP tumors, and a corresponding increase of 8.4% in sensitivity for detection of EOC alone. For all cancers including LMP tumors, the increased sensitivity was seen in both premenopausal women, where the sensitivity increased from 35% for ICRA alone to 75% for the combination of ROMA + ICRA, and in postmenopausal women, where sensitivity increased from 84.8% for ICRA to 92.4% for ICRA + ROMA. The negative predictive value increased from 97.8% to 98.8%, and this high level of NPV will provide strong assurance to generalist physicians that a negative ROMA is predictive of benign disease. The positive predictive value, however, decreased from 51.6% to 38.2% as a consequence of increased numbers of benign disease patients identified as positive by ROMA. The increase in sensitivity resulting from the combination of ROMA and ICRA came from the re-categorization of 13 cancers that ICRA alone had predicted as low risk, whereas the decrease in specificity associated with the combination of ROMA and ICRA came from the re-categorization of 64 benign disease patients that ICRA alone had predicted as low risk. This provides a ratio of increased high risk patients to ovarian cancers detected of 5.9, which represents an acceptable increase in surgeries performed by specialists at high volume centers for the increase in cancers detected, and the corresponding decrease in cancers found by generalists in lower volume centers. These data should translate to U.S. centers because the patient population for this study was enrolled in generalist physician practices, where patients with adnexal masses are most commonly identified and triaged for further work-up and treatment, and ICRA is recommended by ACOG/SGO and is standard of care in many institutions. We conclude that the inclusion of ROMA as part of the initial workup of women diagnosed with a pelvic mass will significantly improve stratification of women into high and low risk groups for malignancy, allowing for more effective triage of these patients. The sensitivity and positive predictive values achieved with ROMA will provide a high degree of confidence that women with positive test results should be referred to a gynecologic oncologist and to centers experienced in the care of women diagnosed with ovarian cancer, and that the high negative predictive value provided by ROMA will give confidence to patients and their physicians that there is a low likelihood that a cancer will be discovered at the time of surgery and that these patents can best be managed in their community setting. Conflict of interest statement Richard G. Moore has received research funding from Fujirebio Diagnostics, Inc. and Abbott Diagnostics, Inc. Doug M. Hawkin is a Statistical Consultant for Fujirebio Diagnostics, Inc. M. Craig Miller is a Database design consultant for Fujirebio Diagnostics, Inc. W. Jeffrey Allard is a consultant for Fujirebio Diagnostics, Inc. Lisa M. Landrum, Walter Gajewski, John J. Ball and Steven J. Skates have no conflicts to report.
References [1] Pejovic T, Nezhat F. Laparoscopic management of adnexal masses the opportunities and the risks. Ann N Y Acad Sci Sep 2001;943:255–68. [2] Curtin JP. Management of the adnexal mass. Gynecol Oncol Dec 1994;55(3 Pt 2): S42–6. [3] Aune G, Torp SH, Syversen U, Hagen B, Tingulstad S. Ten years' experience with centralized surgery of ovarian cancer in one health region in Norway. Int J Gynecol Cancer Feb 2012;22(2):226–31. [4] Bristow RE, Palis BE, Chi DS, Cliby WA. The National Cancer Database report on advanced-stage epithelial ovarian cancer: impact of hospital surgical case volume on overall survival and surgical treatment paradigm. Gynecol Oncol Sep 2010; 118(3):262–7. [5] Paulsen T, Kjaerheim K, Kaern J, Tretli S, Trope C. Improved short-term survival for advanced ovarian, tubal, and peritoneal cancer patients operated at teaching hospitals. Int J Gynecol Cancer Jan 2006;16(Suppl. 1):11–7.
551
[6] Tingulstad S, Skjeldestad FE, Hagen B. The effect of centralization of primary surgery on survival in ovarian cancer patients. Obstet Gynecol Sep 2003;102(3):499–505. [7] Bast Jr RC, Xu FJ, Yu YH, Barnhill S, Zhang Z, Mills GB. CA 125: the past and the future. Int J Biol Markers Oct 1998;13(4):179–87. [8] Committee ACOG. Opinion: number 280, December 2002. The role of the generalist obstetrician-gynecologist in the early detection of ovarian cancer. Obstet Gynecol Dec 2002;100(6):1413–6. [9] Carney ME, Lancaster JM, Ford C, Tsodikov A, Wiggins CL. A population-based study of patterns of care for ovarian cancer: who is seen by a gynecologic oncologist and who is not? Gynecol Oncol Jan 2002;84(1):36–42. [10] Gershenson DM. Why American women are not receiving state-of-the-art gynecologic cancer care. Cancer J Nov 2001;7(6):450–7. [11] Karlsen MA, Sandhu N, Hogdall C, Christensen IJ, Nedergaard L, Lundvall L, et al. Evaluation of HE4, CA125, Risk of Ovarian Malignancy Algorithm (ROMA) and risk of malignancy index (RMI) as diagnostic tools of epithelial ovarian cancer in patients with a pelvic mass. Gynecol Oncol Nov 2012;127(2):379–83. [12] Lu KH, Patterson AP, Wang L, Marquez RT, Atkinson EN, Baggerly KA, et al. Selection of potential markers for epithelial ovarian cancer with gene expression arrays and recursive descent partition analysis. Clin Cancer Res May 15 2004; 10(10):3291–300. [13] Moore RG, Brown AK, Miller MC, Skates S, Allard WJ, Verch T, et al. The use of multiple novel tumor biomarkers for the detection of ovarian carcinoma in patients with a pelvic mass. Gynecol Oncol Nov 30 2007;108:402–8. [14] Yurkovetsky Z, Skates S, Lomakin A, Nolen B, Pulsipher T, Modugno F, et al. Development of a multimarker assay for early detection of ovarian cancer. J Clin Oncol May 1 2010;28(13):2159–66. [15] Bouchard D, Morisset D, Bourbonnais Y, Tremblay GM. Proteins with whey-acidicprotein motifs and cancer. Lancet Oncol Feb 2006;7(2):167–74. [16] Bordin L, Fiore C, Dona G, Andrisani A, Ambrosini G, Faggian D, et al. Evaluation of erythrocyte band 3 phosphotyrosine level, glutathione content, CA-125, and human epididymal secretory protein E4 as combined parameters in endometriosis. Fertil Steril Oct 2010;94(5):1616–21. [17] Moore RG, Miller MC, Steinhoff MM, Skates SJ, Lu KH, Lambert-Messerlian G, et al. Serum HE4 levels are less frequently elevated than CA125 in women with benign gynecologic disorders. Am J Obstet Gynecol Apr 2012;206(4):351–8. [18] Moore RG, McMeekin DS, Brown AK, Disilvestro P, Miller MC, Allard WJ, et al. A novel multiple marker bioassay utilizing HE4 and CA125 for the prediction of ovarian cancer in patients with a pelvic mass. Gynecol Oncol Jan 2009;112(1): 40–6. [19] Moore RG, Miller MC, Disilvestro P, Landrum LM, Gajewski W, Ball JJ, et al. Evaluation of the diagnostic accuracy of the Risk of Ovarian Malignancy Algorithm in women with a pelvic mass. Obstet Gynecol Aug 2011;118(2, Part 1):280–8. [20] Molina R, Escudero JM, Auge JM, Filella X, Foj L, Torne A, et al. HE4 a novel tumour marker for ovarian cancer: comparison with CA 125 and ROMA algorithm in patients with gynaecological diseases. Tumour Biol Dec 2011;32(6):1087–95. [21] Jacobs I, Oram D, Fairbanks J, Turner J, Frost C, Grudzinskas JG. A risk of malignancy index incorporating CA 125, ultrasound and menopausal status for the accurate preoperative diagnosis of ovarian cancer. Br J Obstet Gynaecol Oct 1990;97(10): 922–9. [22] Hakansson F, Hogdall EV, Nedergaard L, Lundvall L, Engelholm SA, Pedersen AT, et al. Risk of malignancy index used as a diagnostic tool in a tertiary centre for patients with a pelvic mass. Acta Obstet Gynecol Scand Apr 2012;91(4):496–502. [23] Committee Opinion No. 477: the role of the obstetrician-gynecologist in the early detection of epithelial ovarian cancer. Obstet Gynecol Mar 2011;117(3):742–6. [24] Woolas RP, Xu FJ, Jacobs IJ, Yu YH, Daly L, Berchuck A, et al. Elevation of multiple serum markers in patients with stage I ovarian cancer. J Natl Cancer Inst Nov 3 1993;85(21):1748–51. [25] Husseinzadeh N. Status of tumor markers in epithelial ovarian cancer has there been any progress? A review. Gynecol Oncol Jan 2011;120(1):152–7. [26] Siegel R, Naishadham D, Jemal A. Cancer statistics, 2012. CA Cancer J Clin Jan 2012; 62(1):10–29. [27] Nolen B, Velikokhatnaya L, Marrangoni A, De GK, Lomakin A, Bast Jr RC, et al. Serum biomarker panels for the discrimination of benign from malignant cases in patients with an adnexal mass. Gynecol Oncol Jun 2010;117(3):440–5. [28] Huhtinen K, Suvitie P, Hiissa J, Junnila J, Huvila J, Kujari H, et al. Serum HE4 concentration differentiates malignant ovarian tumours from ovarian endometriotic cysts. Br J Cancer Apr 21 2009;100(8):1315–9. [29] Partheen K, Kristjansdottir B, Sundfeldt K. Evaluation of ovarian cancer biomarkers HE4 and CA-125 in women presenting with a suspicious cystic ovarian mass. J Gynecol Oncol Dec 2011;22(4):244–52. [30] Van GT, Cadron I, Despierre E, Daemen A, Leunen K, Amant F, et al. HE4 and CA125 as a diagnostic test in ovarian cancer: prospective validation of the Risk of Ovarian Malignancy Algorithm. Br J Cancer Mar 1 2011;104(5):863–70. [31] Montagnana M, Danese E, Ruzzenente O, Bresciani V, Nuzzo T, Gelati M, et al. The ROMA (Risk of Ovarian Malignancy Algorithm) for estimating the risk of epithelial ovarian cancer in women presenting with pelvic mass: is it really useful? Clin Chem Lab Med Feb 3 2011;49(3):521–5. [32] Dearking AC, Aletti GD, McGree ME, Weaver AL, Sommerfield MK, Cliby WA. How relevant are ACOG and SGO guidelines for referral of adnexal mass? Obstet Gynecol Oct 2007;110(4):841–8. [33] Moore RG, Jabre-Raughley M, Brown AK, Robison KM, Miller MC, Allard WJ, et al. Comparison of a novel multiple marker assay vs the Risk of Malignancy Index for the prediction of epithelial ovarian cancer in patients with a pelvic mass. Am J Obstet Gynecol May 13 2010;203(3):228–34.