A systematic review of the clinical validity of the Cologuard™ genetic test for screening colorectal cancer

+Model

ARTICLE IN PRESS

Rev Clin Esp. 2015;xxx(xx):xxx---xxx

Revista Clínica Española www.elsevier.es/rce

REVIEW

A systematic review of the clinical validity of the CologuardTM genetic test for screening colorectal cancer夽 M.Á. Onieva-García a,∗ , A. Llanos-Méndez b , E. Ba˜ nos-Álvarez b , R. Isabel-Gómez b a b

UGC Interniveles de Salud Pública de Granada, Spain Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, Consejería de Salud de la Junta de Andalucía, Sevilla, Spain

Received 9 April 2015; accepted 19 August 2015

KEYWORDS Systematic review; Colorectal cancer; Screening; Analysis of DNA mutations; Feces

Abstract Objectives: The aim of this study was to assess the available evidence on the validity, diagnostic accuracy and clinical utility of the multitarget DNA test in feces (CologuardTM ) for screening for colorectal cancer (CRC). Material and methods: A systematic review was performed by consulting MedLine, EMBASE and Web of Science to July 2014. Studies on diagnostic tests were selected that evaluated the test in asymptomatic adults who underwent CRC screening. The quality and risk of bias were assessed using the Quality Assessment of Diagnostic Accuracy Studies tool. The level of evidence was defined according to the National Institute for Health and Clinical Excellence. A qualitative synthesis was conducted. Results: A total of 299 literature references were identified, including 1 synthesis report and 5 diagnostic test studies. Three of the five studies had a case-control design in Sackett phase II and were of moderate quality, and two had a prospective design in Sacket phase III and were of high quality. The sensitivity for detecting CRC was greater than 90%, but only 40% for detecting advanced adenomas. The test provided conclusive diagnostic evidence to rule out CRC (negative likelihood ratio, LR−: 0.02---0.09), although it was not useful for ruling out advanced adenoma (LR−: 0.5---0.7). Conclusions: The CologuardTM test is a valid screening test for ruling out cancerous lesions but is suboptimal for ruling out precancerous lesions. There is no evidence in terms of mortality, survival or cost-effectiveness. © 2015 Elsevier Espa˜ na, S.L.U. and Sociedad Espa˜ nola de Medicina Interna (SEMI). All rights reserved.

夽

Please cite this article as: Onieva-García MÁ, Llanos-Méndez A, Ba˜ nos-Álvarez E, Isabel-Gómez R. Validez clínica de la prueba genética CologuardTM para el cribado de cáncer colorrectal: revisión sistemática. Rev Clin Esp. 2015. http://dx.doi.org/10.1016/j.rce.2015.08.002 ∗ Corresponding author. E-mail address: [email protected] (M.Á. Onieva-García). 2254-8874/© 2015 Elsevier Espa˜ na, S.L.U. and Sociedad Espa˜ nola de Medicina Interna (SEMI). All rights reserved.

RCENG-1188; No. of Pages 10

+Model

ARTICLE IN PRESS

2

M.Á. Onieva-García et al.

PALABRAS CLAVE Revisión sistemática; Cáncer colorrectal; Cribado; Análisis de mutaciones de ADN; Heces

Validez clínica de la prueba genética CologuardTM para el cribado de cáncer colorrectal: revisión sistemática Resumen Objetivos: El objetivo fue evaluar la evidencia disponible sobre la validez, precisión diagnóstica y utilidad clínica del test multidiana de ADN en heces (CologuardTM ) en el cribado de cáncer colorrectal (CCR). Material y métodos: Se realizó una revisión sistemática consultando MedLine, EMBASE y Web of Science hasta julio de 2014. Se seleccionaron estudios de pruebas diagnósticas que evaluaran el test en adultos asintomáticos sometidos a cribado de CCR. La calidad y el riesgo de sesgo se evaluaron mediante la herramienta Quality Assessment of Diagnostic Accuracy Studies. El nivel de evidencia se definió según The National Institute for Healthand Clinical Excellence. Se realizó una síntesis cualitativa. Resultados: Se identificaron 299 referencias bibliográficas, incluyéndose un informe de síntesis y cinco estudios de pruebas diagnósticas, tres de ellos con dise˜ no caso-control en fase II de Sackett y de moderada calidad, y dos con dise˜ no prospectivo en fase III de Sacket y de alta calidad. La sensibilidad para detectar CCR fue superior al 90%, pero solo del 40% para la detección de adenoma avanzado. El test proporcionó evidencia diagnóstica concluyente para descartar CCR (cociente de probabilidad negativo, CPN: 0,02-0,09), aunque no fue útil para descartar adenoma avanzado (CPN: 0,5-0,7). Conclusiones: El test CologuardTM es una prueba de cribado válida para descartar lesiones cancerosas, resultando subóptima para descartar lesiones precancerosas. No hay evidencia sobre resultados en términos de mortalidad o supervivencia, ni sobre coste-efectividad. © 2015 Elsevier Espa˜ na, S.L.U. and Sociedad Espa˜ nola de Medicina Interna (SEMI). Todos los derechos reservados.

Background Colorectal cancer (CRC) is the most common malignancy in Europe and Spain and constitutes the second leading cause of overall mortality from cancer.1 It is estimated that the mortality rate from CRC has decreased by 13% in the last two decades in Spain.2 However, this decrease is still well below that achieved in other European countries and the United States, and screening detection rates remain suboptimal, mainly due to the low participation of the population.2,3 Current screening methods targeted toward the population aged 50 years or more at moderate risk include annual high-sensitivity fecal occult blood tests (FOBT) (guaiacbased or immunological [FOBTi]); sigmoidoscopy every 5 years and FOBT every 3 years; or colonoscopy every 10 years.4,5 The method of choice in Spain is FOBTi every 2 years.3 Although there is consensus for recommending any of these alternatives, there is still controversy regarding the cost-effectiveness of deoxyribonucleic acid (DNA) tests in feces due to the lack of data for determining the screening interval.6 Stool DNA tests are designed to detect molecular abnormalities in the epithelial cells of precancerous adenomatous polyps (advanced adenoma [AA] or sessile serrated polyp [SSP]) or sloughed CRC cells into the large intestine lumen.7 To date, only two DNA tests have been marketed (PreGen PlusTM and ColoSureTM ), none of which have been approved by the Food and Drug Administration.6 The multitarget stool DNA test marketed as CologuardTM has recently been approved by the Food and Drug Administration as

a screening test for population at moderate risk of CRC and has been included as a healthcare provision in the Medicare and Medicaid services of the United States.8 The main differences from its predecessors (PreGen PlusTM and ColoSureTM ) lies in the detected genetic markers, the incorporation of fecal occult blood (FOB) detection and the use of QuARTS technologyTM for real-time DNA amplification.6 This emerging technology could be an alternative noninvasive screening, which, with optimal performance and increased adherence rate, might reduce the incidence and mortality of CRC. The aim of this systematic review was to combine the available evidence to analyze the validity, diagnostic accuracy and clinical utility of the multitarget stool DNA test (CologuardTM ) in CRC screening.

Methods A systematic literature review was performed by following the PRISMA recommendations.9 The results were summarized in narrative form. A statistical analysis was not possible due to the heterogeneity of the studies.

Information sources The MedLine, EMBASE, Web of Science and PubMed databases were consulted until July 2014. In addition, we also consulted the Centre for Reviews and Dissemination, the International Information Network on New and Emerging

+Model

ARTICLE IN PRESS

A systematic review of the clinical validity of the CologuardTM genetic test for screening colorectal cancer Health Technologies (EuroScan), the Cochrane Library, the website of agencies not included in the International Network of Agencies for Health Technology Assessment, the Ministry of Health, Social Services and Equality, the platforms of Agencies and Health Technology Assessment Units, the World Health Organization, the Centers for Disease Control and Prevention, the Emergency Care Research Institute (ECRI) and the National Institute for Health and Clinical Excellence. Secondary references were also reviewed. For this purpose, both natural and controlled language was employed, using the terms: ‘‘colorectal neoplasms,’’ ‘‘DNA mutational analysis,’’ ‘‘stool DNA test,’’ ‘‘Cologuard,’’ ‘‘mass screening,’’ ‘‘screening,’’ ‘‘early detection of cancer,’’ ‘‘sensitivity,’’ ‘‘specificity,’’ ‘‘likelihood functions,’’ ‘‘reproducibility of results’’ and ‘‘area under the curve,’’ among others.

Medline (n=162); EMBASE (n=60); WOS (n=25); Others (n=52) Total=299

Duplicates (n=45)

Studies identified as potentially relevant (n=254) Excluded studies (n=239) 23 47 66 41 62

Full-text studies excluded (n=9)

Selection of studies

Data extraction The variables included the characteristics of the study (author, year of publication, country, study period, objectives, phase of the study and funding); the population (sample size, sociodemographic and clinical features, inclusion and exclusion criteria); and the intervention (screening test and reference). In addition, the results relating to diagnostic validity and accuracy (sensitivity, specificity, predictive values, likelihood ratios, ROC curve and intraobserver and interobserver variability) were extracted. When possible, 2 × 2 contingency tables were generated to calculate these parameters from each study’s data.

Assessment of risk of bias and evidence level In addition, two reviewers independently assessed the quality and risk of bias according to the Quality Assessment of Diagnostic Accuracy Studies-1 and Quality Assessment of Diagnostic Accuracy Studies-2 tool, respectively.10,11 Discrepancies were resolved by consensus. The level of evidence was defined according to The National Institute for Health and Clinical Excellence recommendations.12

Results A total of 299 references were identified. After the selection process, 5 primary studies of diagnostic tests (Fig. 1) were included.13---17

Different study topic Different study objective Different intervention from the study intervention Different study population Narrative

Studies included for full texts (n=15)

7

Two reviewers independently selected studies that examined the validity and diagnostic accuracy of the multitarget stool DNA test, comparing it with any method for the diagnosis or screening of CRC in asymptomatic populations. Assessment reports and summaries of emerging technologies were also included, because they contain useful information for corroborating the methodological rigor of our search. Narrative reviews, letters to the editor, editorials, meeting abstracts and preclinical studies were excluded.

3

2

Different intervention from the study intervention Congress abstracts

Studies selected for analysis (n=6) Synthesis report (n=1)

Figure 1

Primary studies (n=5)

Flow diagram for the selection of articles.

Likewise, a summary report published by ECRI that assessed the impact of CologuardTM was included.7 The authors predicted, regarding the current screening tests, an increase in the adherence rate and early detection of lesions that consequently results in greater health benefits but also increased costs.

Characteristics of the primary studies All studies examined the validity of DNA biomarkers in feces, along with the FOB in the detection rate of CRC or advanced precancerous lesions.13---17 The reference test was colonoscopy,13---17 considering the index lesion the most advanced colorectal epithelial lesion13---17 or the larger among two or more similar lesions.13,16 The three studies13---15 that used the DNA CologuardTM biomarker panel assessed aberrant methylations in the promoter region of bone morphogenetic protein 3 (BMP3) and N-Myc downstream-regulated gene 4 (NDRG4), 7-point mutations in the Kirsten rat sarcoma viral oncogene homolog (KRAS) gene and detected FOB by quantitative FOBTi. The remaining two studies evaluated the CologuardTM prototype,16,17 which also detects aberrant methylations in the vimentin (VIM) and tissue factor pathway inhibitor 2 (TFPI2) genes. FOB was detected by the porphyrin method. The cumulative study population for analyzing the performance of the CologuardTM test was 14,235 participants (13,691 apparently asymptomatic and 544 with symptoms suggestive of CRC)13---15 and 825 for the prototype (including asymptomatic individuals and those with suggestive symptoms).16,17 The percentage of women ranged from 50% to 55%, and the mean age ranged from 60 to 65 years (Table 1). The excluded patients were those with syndromes at high risk for CRC (Lynch syndrome or familial adenomatous polyposis), inflammatory bowel disease,13,14,16

+Model

4

Table 1

Characteristics of the study population.

Test

Author and year

N

Women, n (%)a

Type of patient, n Overall

Imperiale 201413

Heigh 201414 Lidgard 201315

CologuardTM prototype

Ahlquist 201217 Ahlquist 201216

With lesions

Overall

Without lesions

With lesions

NC

NC

64 (NC)

NC

NC

61b (52---77)b 65b (38---87)b

NA

NA

65 (50---84)

64b (51---75)b 60 (39---92)

59c (51---66) 57 (41---87)

12,776

Asymptomatic

456

Asymptomatic

138b (53)b

NA

NA

1003

Asymptomatic, 459 Symptomatic, 544 NC

549b (55)b

462b (58)

87b (42)

50b (51)b

26c (57)

24 (46)

Asymptomatic Symptomatic

339b (50)

164b (56)

173b (45)

147 678

5364 (54)

Without lesions

CRC stage, n (%)a I---III

IV

White, 8392 (84) AfricanAmerican, 1068 (11) Other, 523 (5) NC

NA

NA

NC

NC

65 (38---87)

NC

76 (92)

7 (8)

69 (61---75) 63 (39---92)

NC

22 (73)

8 (27)

Blanca, 549 (81)

225 (89)

27 (11)

M.Á. Onieva-García et al.

Abbreviations: N, population size; NA, not applicable; NC, not specified. a Data referring to the assessed population. b Data calculated by the authors of the review based on data from the studies. c Controls paired by age and sex for feces sample (does not include the controls paired by age and sex for plasma sample).

Race, n (%)a

ARTICLE IN PRESS

CologuardTM

Mean age, years (range)a

+Model

ARTICLE IN PRESS

A systematic review of the clinical validity of the CologuardTM genetic test for screening colorectal cancer

Reference test

Study test

Selection of participants

Applicability

Follow-up and losses

Reference test

Study test

Selection of participants

Risk of bias

Ahlquist, Taylor 2012 Ahlquist, Zou 2012 Heigh 2014

5

The time interval between the colonoscopy and the study test was reported in 1 study (90 days at most).13 In the remaining studies,14---17 this interval was not specified, raising doubts about the similarity of the spectrum of patients undergoing each test at two different times. Only the Imperiale et al. study suffered losses (21.8% of participants). The authors found significant differences in age and race between the participants who completed the study and were evaluated and those who did not complete it and were not included in the analysis. The magnitude of these differences was not noticeable except for the participants older than 74 years. The proportion of these participants was 9% in the evaluated group compared to 13.6% in the nonevaluated group. This could have underestimated the detection of proximal cancers, which are more common than distal cancers at the age of 70 or older.13

Imperiale 2014 Lidgard 2013

Results of the primary studies High

Figure 2

Uncertain

Low

Risk of bias. Applicability of each study.

previous colorectal resection,13,14 recent screening test,13,14 a history of gastrointestinal neoplasia or bleeding in the past 30 days13 or who underwent incomplete colonoscopy.14

Risk of bias in the primary studies The methodological quality of the prospective studies was high13,14 (or Sackett phase III18 ) with level Ib evidence. The rest of the studies had moderate quality15---17 and a casecontrol design (Sackett phase II) and level III evidence. The main risk of bias was related to the characteristics of the participants in the case-control studies (selection bias).15---17 The inclusion of patients with symptoms suggestive of the target disease could overestimate the diagnostic validity parameters (predictive values and sensitivity) (Fig. 2). The risk of bias in the interpretation of the study test results was low, because the positivity of the test was objectively defined with a pre-set cutoff value.13---17 In contrast, there was a risk of observer bias in three studies.15---17 The observers did not record whether the colonoscopy (a subjective test) was performed without knowing the test’s positivity or negativity, which could have influenced the interpretation of the colonoscopy results and biased the results. All patients had undergone colonoscopy as the reference test.13---17 Regarding the patient classification, it was considered that there had been no errors in the diagnosis of precancerous or cancerous lesions found during the colonoscopy because, except for 9 lesions in the Heigh et al. study,14 all lesions were confirmed by pathology (gold standard). Moreover, although there is some uncertainty about the correct classification of the healthy participants who did not undergo pathology, this was not considered relevant according to the colonoscopy diagnostic performance (95% sensitivity and 90% specificity).19

Diagnostic accuracy and overall performance of the CologuardTM test The sensitivity ranged from 92.3% to 97.8% for CRC and from 42.4% to 57% for AA.13,14 The sensitivity for SSP was 55.2%.14 The specificity for advanced colorectal neoplasia (including CRC and AA) achieved a range between 86.6% and 90%. The positive likelihood ratio (LR+) ranged from 5.9 to 9.8 for CRC and from 3.2 to 5.7 for AA,13,14 indicating the number of times more likely the test will be positive for a sick patient than for a healthy one. Therefore, a positive test result provides strong diagnostic evidence for CRC, while it is less useful for detecting AA, although its detection could be considered clinically relevant (values above 10 indicate conclusive diagnostic evidence; between 5 and 10 indicates strong evidence; and between 1 and 2 indicates low evidence).20 Furthermore, the negative likelihood ratio (LR−) ranged from 0.02 to 0.09 for CRC and from 0.5 to 0.7 for AA,13,14 indicating the number of times more likely the test will be negative for a sick patient than for a healthy one. Thus, a negative test result provides conclusive diagnostic evidence against CRC, in other words, it is an excellent test for ruling out CRC. A negative test result also provides low evidence for ruling out AA (values below 0.1 indicate conclusive diagnostic evidence; lower than 0.2 indicates strong evidence; and between 0.6 and 1 indicates low evidence).20 The overall test performance was higher for CRC (area under the curve: 0.94) than for advanced colorectal neoplasia (0.73).13 Heigh et al.14 measured the discriminative ability of each marker and found statistically significant values for BMP3, NDRG4 and KRAS (0.87 [95% CI 0.80---0.95], p < .0001; 0.79 [95% CI 0.70---0.88], p < .0001; 0.64 [95% CI 0.53---0.75], p = .0068, respectively) but not for occult hemoglobin (0.50 [95% CI 0.40---0.61], p = .47)13,14 (Table 2).

Diagnostic validity and overall performance of CologuardTM prototype The sensitivity ranged from 84.9% to 86.7% for CRC and from 54.1% to 81.8% for AA. The specificity for advanced colorectal neoplasia was 89.1%. This test provided strong diagnostic evidence to detect and rule out CRC (LR+, 7.7; LR−, 0.2) and was less useful for AA (LR+, 4.9; LR−, 0.5).16,17

+Model

6

Table 2

Efficacy results in terms of diagnostic validity.

Type of lesion CologuardTM Colorectal cancer

Advanced adenoma

Sensitivity (95% CI)

Specificity (95% CI)

PPV (95% CI)

NPV (95% CI)

LR+ (95% CI)

LR− (95% CI)

Imperiale 201413 Lidgard 201315

III

0.923 (0.832---0.967)

II

0.978 (0.924---0.997)a

0.037 (0.029---0.048) NC

0.999 (0.998---0.999) NC

Imperiale 201413 Lidgard 201315

III

0.424 (0.389---0.460)

II

0.570 (0.474---0.663)a

0.207a (0.187---0.228)a NC

0.8657a (0.859---0.872)a NC

Heigh 201414

III

0.552 (0.357---0.736)

Imperiale 201413

III

0.464a (0.429---0.498)a

0.844 (0.836---0.851) 0.900 (0.879---0.919)b 0.866a (0.859---0.872)a 0.900 (0.879---0.919)b 0.909 (0.865---0.943) 0.866 (0.859---0.872)

0.432a (0.287---0.591)a 0.236 (0.216---0.258)

0.942a (0.903---0.965)a 0.947 (0.942---0.952)

5.902a (5.428---6.418)a 9.785b (8.033---11.918)b 3.158a (2.863---3.483)a 5.745b (4.416---7.474)b 6.095a (3.611---10.288)a 3.452a (3.154---3.777)a

0.091a (0.039---0.212)a 0.024b (0.006---0.094)b 0.665a (0.626---0.708)a 0.477b (0.386---0.590)b 0.493a (0.328---0.740)a 0.620a (0.581---0.661)a

Lidgard 201315

II

0.754a (0.689---0.811)a

0.900 (0.879---0.919)b

NC

NC

7.593a (6.074---9.493)a

0.274a (0.214---0.347)a

II

0.867 (0.693---0.962)

ND

NC

NC

ND

ND

II

0.849 (0.799---0.891)

NC

NC

II

0.818 (0.615---0.927)

0.890 (0.856---0.918) ND

NC

NC

7.697a (5.849---10.130)a ND

0.169a (0.126---0.228)a ND

II

0.541 (0.453---0.628)

NC

NC

II

0.846 (0.719---0.931)

0.891 (0.849---0.924) 0.935 (0.821---0.986)

NC

NC

4.957a (3.450---7.122)a 12.974a (4.318---38.981)a

0.515a (0.426---0.622)a 0.165a (0.087---0.313)a

II

0.743a (0.696---0.786)a

0.891a (0.849---0.924)a

NC

NC

6.802a (4.879---9.482)a

0.289a (0.242---0.344)a

CologuardTM Prototype Colorectal Ahlquist 201217 cancer Ahlquist 201216,c Advanced Ahlquist 201217 adenoma Ahlquist 201216,c Advanced Ahlquist 201217 colorectal neoplasmd Ahlquist 201216,c

Abbreviations: LR−, negative likelihood ratio; LR+, positive likelihood ratio; 95% CI, 95% confidence interval; NC, not calculated; ND, no data; NPV, negative predictive value; PPV, positive predictive value. a Values calculated by the authors of this review based on data from the original studies. b Values calculated by the authors of this review based on data from the original study, assuming a specificity of 90%. c Results referring to all patients as a whole. d Including colorectal cancer and advanced adenoma.

M.Á. Onieva-García et al.

Phase

ARTICLE IN PRESS

Sessile serrated polyp Advanced colorectal neoplasmd

Author and year

+Model

ARTICLE IN PRESS

A systematic review of the clinical validity of the CologuardTM genetic test for screening colorectal cancer Ahlquist et al.16 found an area under the curve of 0.90 (95% CI 0.86---0.93) for the full marker panel (BMP3, NDRG4, KRAS, TFPI2, VIM and hemoglobin) versus 0.88 (95% CI 0.84---0.91) for the panel without hemoglobin. The relative contribution of TFPI2 and VIM was minimal, increasing the sensitivity from 1 to 3 percentage points. In contrast, BMP3, NDRG4 and KRAS significantly contributed to the test’s discriminative ability (Table 2).

Accuracy Ahlquist et al.16 found a high degree of agreement in the results among the various laboratories (correlation coefficient of 0.978---0.996).20

Comparison with other screening tests • Comparison of CologuardTM with quantitative FOBTi (OCFIT CHEK, Polymedco). In the study by Heigh et al.,14 the detection rate for SSP ≥1 cm using the CologuardTM BMP3 test was 66% and 63% for the predetermined specificities of 91% and 95%, respectively, compared to the 10% and 0% achieved with FOBTi. These differences were statistically significant (p = .003; p < .001, respectively). In the Imperiale et al. study,13 CologuardTM was more sensitive for detecting CRC in any stage (stages I---III) and for both locations (proximal and distal), with values of approximately 90% or higher. The absolute differences of 24 percentage points for stages I and II, and the proximal location stand out. Although the sensitivity of CologuardTM achieved lower values (approximately 50%) for AA detection, the sensitivity was also greater, with absolute differences ranging from 16 to 26 percentage points and up to 37 points for SSP detection. However, the specificity of FOBTi (94.9---96.4%) was higher than that of CologuardTM (86.6---89.8%), with false positive rates of 3.6---5.1% and 10.2---13.4%, respectively13 (Tables 3 and 4). • Comparison of the CologuardTM prototype with methylated SEPT9 gene in plasma (Septin 9). The sensitivity for detecting CRC and large AA with the CologuardTM prototype was higher than that of Septin 9 (85% vs. 40%, p = .0001). The same occurred with the isolated sensitivity for detecting large AA (82% vs. 14%, p = .0001), CRC (87% vs. 60%, p = .046), stages I---III of CRC (95% vs. 50%, p = .013) and proximal CRC (92% vs. 46%, p = .034).17 The false positive rate of Septin 9 was higher than that of the prototype (21% vs. 7%) (Tables 3 and 4).

Discussion Spain has recently approved the inclusion of CRC screening in the National Health System’s basic portfolio of services, using FOBTi (performed every 2 years) for the population aged 50---69 years.3 Although FOBTi has proven to be costeffective,21---23 the low participation rate (approximately 35%) has led to suboptimal results.3,24,25 The acceptance rate is even lower when colonoscopy is the screening test of choice (approximately 25%).24,25 New genetic tests, such as CologuardTM , represent an alternative screening that could improve the program’s performance as a whole.5

7

Considering that sensitivity is the most important attribute of a diagnostic test designed for mass screening, one could conclude that CologuardTM is a valid test for detecting CRC at any stage, including early stages and both proximal and distal locations, with sensitivities of over 90%. An LR− lower than 0.1 also makes it an excellent test for ruling out CRC. However, its utility in detecting AA is lower, with sensitivities of approximately 40%,13---15 although always higher than those obtained by FOBTi.13 The results for the prototype were similar but with slightly lower LR− and sensitivity values.16,17 There are two remarkable findings that could play a major role in clinical practice. One of these findings is the increased sensitivity of CologuardTM for serrated polyps compared with FOBTi, given that the polyp-cancer sequence appears fastest in the serrated pathway.26 The other important aspect is that proximal cancer detection with CologuardTM was substantially higher than with FOBTi. This finding is particularly relevant because of the lower diagnostic performance when detecting proximal malignancies (when compared with distal malignancies) reported for FOBT and lower endoscopy.27 The articles included in this review could be considered a compendium of consecutive studies of increasing complexity that gradually control the various biases that can affect the validation process, seeking more pragmatic goals that are applicable to clinical practice. Thus, we can see that the Ahlquist et al.16,17 and Lidgard et al.15 (case-control design) studies have methodological deficits (mainly related to the spectrum of patients) to a greater degree than the Heigh et al.14 and Imperiale et al.13 (prospective design) studies. As previously mentioned, the inclusion of patients with similar symptoms to those of the target disease in the Alhquist et al.16,17 and Lidgard et al.15 studies would theoretically increase the prevalence of the disease, overestimating the diagnostic validity parameters related to the post-test probability (predictive values). Thus, an increase in the number of patients would indirectly increase the test sensitivity. For this reason and given that the case-control design is the most critical aspect in the introduction of risk of bias,28 we have assigned more importance to the prospective studies in the conclusions of this systematic review. These studies were able to minimize the risk of bias in each assessed aspect: adequate spectrum of patients, blind interpretation of results, appropriate reference test and adequate reporting of losses. The external validity was limited by the heterogeneity in the test’s maturity, as a number of studies used the marketed test (CologuardTM )13---15 while others used the prototype.16,17 Another aspect of the external validity was the study setting, which was focused primarily on North America. However, the extrapolation of results to a hypothetical Spanish population does not imply, in theory, a lower diagnostic validity for the test (in terms of post-test probability), given that the prevalence of CRC in Spain is higher than in North America.1 It is also noteworthy that the funding for most of the studies came from the company that developed and distributed the test, which could represent a conflict of interest. It has already been stated that CologuardTM is a valid test, a prerequisite for any screening or diagnostic test, but it is not sufficient. Its success in a screening program,

+Model

ARTICLE IN PRESS

8

M.Á. Onieva-García et al. Table 3

Results of sensitivity for detecting colorectal cancer lesions. CologuardTM 13---15

CologuardTM Prototype16,17

Quantitative FOBTi13

Septin 917

Stage I II III IV I-III

0.897a ---0.952 1.000a ---1.000 0.900a ---0.968 0.750a ---1.000 0.933---0.974

0.844---0.857 0.800---0.857 0.949---1.000 0.704---0.750 0.871---0.910

0.655 0.762 0.900 0.750 0.733

0.570 0.570 0.380 0.880 0.500

Location Proximal Distal

0.900a ---1.000 0.940---0.943a

0.874---0.920 0.810---0.828

0.667 0.800

0.460 0.690

Abbreviations: FOBTi, immunologic fecal occult blood test. a Data from the study reports submitted to the Food and Drug Administration for approval of CologuardTM .60

Table 4

Results of Sensitivity for detecting advanced precancerous lesions.

Type Advanced adenoma High-grade dysplasia Sessile serrated polyp Location Proximal Distal Size ≤5 mm 6---9 mm 10---19 mm 20---29 mm ≥30 mm >40 mm

CologuardTM 13---15

CologuardTM prototype16,17

Quantitative FOBTi13

Septin 917

0.424---0.570 0.629---0.833 0.424---0.600b

0.541---0.818 NC NC

0.238 0.462c 0.05c

0.140 NC NC

0.332---0.513 0.545---0.676

0.552 0.533

0.155c 0.348c

NC NC

0.200c 0.321c 0.392c ---0.570 0.646c 0.684c ---0.833 NC

NC NC 0.540d 0.765 0.860e 0.916

0.200c NC 0.209c 0.430c 0.421c NC

NC NC NC NC NC NC

Abbreviations: NC, not specified; FOBTi, immunologic fecal occult blood test. b This range of values also includes the results from the Heigh et al. study. c Data from the study report submitted to the Food and Drug Administration for approval of CologuardTM . d Data regarding AA measuring ≥10 mm. e Data regarding AA measuring >30 mm.

measured in terms of health benefits, will depend on many other factors that have yet to be analyzed. Thus, the possibility of including CologuardTM in the National Health System’s portfolio of services depends on ethical, social and organizational aspects. First, we should assess the acceptance of the screening test by the population. The authors of the summary report from the ECRI7 measured greater adherence compared with the current screening tests. However, there are limiting factors. For example, 36 g of feces are required for the proper processing of CologuardTM , which could hinder both population recruitment and laboratory handling. Moreover, CologuardTM and FOBT share the same sample collection procedure; therefore, from the patient’s point of view, both tests would be similar. In our opinion, these issues could lessen the hypothetical increase in adherence with the use of CologuardTM . We should be aware that the higher adherence mentioned by the ECRI authors was based on expert forecasts and opinions.

Second, the rate at which CologuardTM should be performed to ensure lesion detection is not yet known. The company that marketed the test has proposed a 3-year interval between two consecutive screenings,29 but the studies included in this review do not provide sufficient data on this issue. It is essential to determine this interval to measure the effects of CologuardTM in reducing CRC incidence and mortality and to compare it with other screening methods, given that the literature shows that periodicity determines, along with other factors, the achievement (or not) of optimum results.30 It would therefore be advisable to have studies that compare the frequency of the test to that of current tests and compare the overall performance of the screening program for various strategies. Third, the specificity of the test should be considered. Although the sensitivity is necessary for the screening tests, it is not sufficient. The lower specificity found for CologuardTM compared with FOBT increases the probability of finding false positives, thereby resulting in the

+Model

ARTICLE IN PRESS

A systematic review of the clinical validity of the CologuardTM genetic test for screening colorectal cancer need for confirmation by colonoscopy. Therefore, although CologuardTM is a valid test in terms of efficiency, it might be inefficient for most patients who end up needing a colonoscopy, with the resulting consumption of resources (cost increase) and side effects such as overdiagnosis, anxiety and risk of complications from invasive tests such as colonoscopy. The estimated cost for CologuardTM is also 30 times higher than that of FOBT,31 which significantly increases the cost of the screening program. In conclusion, this systematic review shows that the multitarget stool DNA test CologuardTM is an effective (and valid) screening test for ruling out cancerous lesions and, to a lesser extent, precancerous lesions. However, there are no data for determining whether the CologuardTM screening program could obtain greater health benefits compared with current programs; that is, whether it would be more effective in terms of reducing CRC incidence and mortality. To determine to what extent this test could provide improvements, future studies should delve deeper into issues such as the impact on population participation (adherence), the time interval between two consecutive screenings to ensure early lesion detection and test efficiency through costeffectiveness studies.

4.

5.

6.

7.

8.

Funding This document was developed under the collaboration agreement signed by the Institute of Health Carlos III, an autonomous body of the Ministry of Economy and Competitiveness and the Department of Equality, Health and Social Policy of the Andalusian Government as part of the development of activities of the Spanish Network of Agencies for Health Technology Assessment and Services of the NHS, funded by the Ministry of Health, Social Services and Equality.

Conflict of interests

9.

10.

11.

12.

The authors declare that they have no conflicts of interest.

Acknowledgements

13.

This work has benefited from the contributions of Dr. Manuel Romero Gomez, as well as from the planning and documentation work of Dr. Antonio Romero-Tabares.

14.

References 15. 1. GLOBOCAN 2012. Colorectal cancer. Estimated incidence, mortality and prevalence worldwide in 2012 [Base de datos en Internet]. Lyon: International Agency for Research on Cancer. World Health Organization; 2014 [accessed 16.09.14]. Available from: http://globocan.iarc.fr/Pages/fact sheets cancer.aspx. 2. Sociedad Espa˜ nola de Oncología Médica. Las Cifras del Cáncer na 2014 [Internet]. Madrid: Sociedad Espa˜ nola de en Espa˜ Oncología Médica; 2014 [accessed 16.09.14]. Available from: http://www.seom.org/en/prensa/el-cancer-en-espanyacom/ 104582-el-cancer-en-espana-2014. 3. Sistema Nacional de Salud. Informe del grupo de expertos sobre concreción de cartera común de servicios para cribado de cáncer [Internet]. Madrid: Ministerio de Sanidad, Servicios

16.

17.

18.

9

Sociales e igualdad; 2013 [accessed 30.07.14]. Available from: http://www.msssi.gob.es/profesionales/saludPublica/ prevPromocion/docs/ResumenEjecutivoCribadoCancer.pdf. National Comprehensive Cancer Network Guidelines (NCCN Guidelines). Colorectal Cancer Screening. Version 1.2014 [Internet]. Washington: NCCN Guidelines; 2014 [accessed 29.07.14]. Clinical Practice Guidelines in Oncology (NCCN Guidelines). Available from: http://www.nccn.org/professionals/physician gls/pdf/colorectal screening.pdf. Segnan, N., Patnick, J., von Karsa (eds). European guidelines for quality assurance in colorectal cancer screening and diagnosis [Internet]. Luxemburgo: Publications Office of the European Union; 2010 [accessed 28.09.14]. Available from: http://www. kolorektum.cz/res/file/guidelines/CRC-screening-guidelinesEC-2011-02-03.pdf. Lin, J.S., Webber, E.M., Beil, T.L., Goddard, K.A., Whitlock, E.P. Fecal DNA testing in screening for colorectal cancer in average risk adults [Internet]. Rockville (MD): Agency for Healthcare Research and Quality; 2012 [accessed 11.07.14]. 52 p. Available from: http://www.ncbi.nlm.nih.gov/pubmedhealth/ PMH0041016/pdf/TOC.pdf. ECRI. Cologuard Stool DNA Test for Colorectal Cancer Screening; 2014 [accessed 12.07.14]. Available from: https://www.ecri. org/components/Forecast/Pages/12954.aspx?tab=2. Food and Drug Administration (FDA) [Internet]. United States: FDA. FDA approves first non-invasive DNA screening test for colorectal cancer; 2014 [updated 08.08.14; accessed 16.09.14]. Available from: http://www.fda.gov/NewsEvents/Newsroom/ PressAnnouncements/ucm409021.htm. Urrutia G, Bonfill X. Declaración PRISMA: una propuesta para mejorar la publicación de revisiones sistemáticas y metaanálisis. Med Clin. 2010;135:507---11. Withing P, Rutjes A, Reitsma J, Bossuyt P, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol. 2003;3:25. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. Quadas-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155:529---36. National Institute for Health and Clinical Excellence (NICE). The guidelines manual. Manchester: NICE; 2012 [accessed 30.07.14]. Available from: http://www.nice.org.uk/ article/pmg6/resources/non-guidance-the-guidelines-manualpdf. Imperiale TF, Ransohoff DF, Itzkowitz SH, Levin TR, Lavin P, Lidgard GP, et al. Multitarget stool DNA testing for colorectalcancer screening. N Engl J Med. 2014;370:1287---97. Heigh RI, Yab TC, Taylor WR, Hussain FTN, Smyrk TC, Mahoney DW, et al. Detection of colorectal serrated polyps by stool DNA testing: comparison with fecal immunochemical testing for occult blood (FIT) [Internet]. PLOS ONE. 2014;9:e85659. Available from: http://www.ncbi.nlm.nih.gov/ pmc/articles/PMC3896420/ [accessed 12.12.14]. Lidgard GP, Domanico MJ, Bruinsma JJ, Light J, Gagrat ZD, Oldham-Haltom RL, et al. Clinical performance of an automated stool DNA assay for detection of colorectal neoplasia. Clin Gastroenterol Hepatol. 2013;11:1313---8. Ahlquist DA, Zou H, Domanico M, Mahoney DW, Yab TC, Taylor WR, et al. Next-generation stool DNA test accurately detects colorectal cancer and large adenomas. Gastroenterology. 2012;142:248---56. Ahlquist DA, Taylor WR, Mahoney DW, Zou H, Domanico M, Thibodeau SN, et al. The stool DNA test is more accurate than the plasma septin 9 test in detecting colorectal neoplasia. Clin Gastroenterol Hepatol. 2012;10:272---7. Sackett D, Haynes R. Evidence base of clinical diagnosis. The architecture of diagnostic research. BMJ. 2002;324:539---41.

+Model

ARTICLE IN PRESS

10 19. Zauber, A.G., Lansdorp-Vogelaar, I., Knudsen, A.B., Wilschut, J., van Ballegooijen, M., Kuntz, K.M. Evaluating test strategies for colorectal cancer screening----age to begin, age to stop, and timing of screening intervals: a decision analysis of colorectal cancer screening for the U.S. Preventive Services Task Force from the Cancer Intervention and Surveillance Modeling Network (CISNET) [Internet]. United States: Agency for Healthcare Research and Quality; 2009 [accessed 30.07.14]. 61 p. Available from: http://www.uspreventiveservicestaskforce. org/uspstf08/colocancer/colcanes2.pdf. 20. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989;45:255---68. 21. Agency for Healthcare Research and Quality (AHRQ). The guide to clinical preventive services 2014. Recommendations of the U.S. Preventive services task force. United States: AHRQ; 2014 [accessed 30.07.14]. 123 p. Available from: http:// www.ahrq.gov/professionals/clinicians-providers/guidelinesrecommendations/guide/cpsguide.pdf. 22. Hewitson P, Glasziou P, Irwig L, Towler B, Watson E. Screening for colorectal cancer using the faecal occult blood test, Hemoccult. Cochrane database Syst Rev. 2007;1. CD001216. 23. Sánchez D. Cribado del cáncer colorrectal. Rev Clin Esp. 2014;214:275---6. nez J, Molina-Barceló A, Hernán24. Salas D, Vanaclocha M, Ibᘠdez V, Cubiella J, et al. Participation and detection rates by age and sex for colonoscopy versus fecal immunochemical testing in colorectal cancer screening. Cancer Causes Control. 2014;25:985---97.

M.Á. Onieva-García et al. 25. García J. Colonoscopia versus sangre oculta en heces en el cribado del cáncer colorrectal. Rev Clin Esp. 2012;212:407. 26. Bacchiddu S, Álvarez-Urturri A, Bessa-Caserras X. Pólipos colorrectales. FMC. 2012;19:472---80. 27. Castells A, Quintero E, Álvarez C, Bujanda L, Cubiella J, Salas D, et al. Rate of detection of advanced neoplasms in proximal colon by simulated sigmoidoscopy vs fecal immunochemical tests. Clin Gastroenterol Hepatol. 2014;12:1708---16. 28. Lijmer JG, Mol BW, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JH, et al. Empirical evidence of design-related bias in studies of diagnostic tests. JAMA. 1999;282:1061---6. 29. ECRI Institute AHRQ Healthcare Horizon Scanning System---Potential high impact interventions report. Priority area 02: cancer [Internet]. Plymouth: ECRI Institute; 2014 [accessed 25.10.14]. 144 p. Available from: http://effective healthcare.ahrq.gov/ehc/assets/File/Cancer-Horizon-ScanHigh-Impact-1406.pdf. 30. Zauber AG, Lansdorp-Vogelaar I, Knudsen AB, Wilschut J, van Ballegooijen M, Kuntz KM. Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med. 2008;149: 659---69. 31. Exact Sciences [Internet]. Madison, Wisconsin: Exact Sciences Corporation; 2014. FDA Approves Exact Sciences’ Cologuard®; First and Only Stool DNA Noninvasive Colorectal Cancer Screening Test. [updated 12.08.14; accessed 25.10.14]. Available from: http://investor.exactsciences.com/releasedetail. cfm?ReleaseID=865698.