Journal of Hepatology 46 (2007) 543–545 www.elsevier.com/locate/jhep
Editorial
Technology evaluation: A critical step in the clinical utilization of novel diagnostic tests for liver fibrosis q Nezam H. Afdhal*, Michael Curry Liver Center, Beth Israel Deaconess Medical Center, 110 Francis St., Boston, MA 02215, USA
See Article, pages 628–634
Novel technologies are appearing throughout medicine and this is equally true for Hepatology. In particular there has been an explosion of new serological and radiological tests for the non-invasive diagnosis of liver fibrosis related to HCV [1–10]. The need for non-invasive tests and the performance characteristics of such tests have been extensively described and putative ideal biomarkers have been delineated [3]. However, the clinical reality is markedly different from many of the research studies reported and the clinician needs more validation of these biomarkers and their performance characteristics before they can be accepted into routine clinical practice [11,12]. Additionally, there has also been a significant regional difference in the routine uptake of biomarkers for fibrosis, with the European countries much more accepting of their clinical utilization than in the United States. Much of this discrepancy is related to the lack of validation of these tests in the US, the lack of FDA approval and the reluctance of 3rd party payers to reimburse for these tests. Current utilization of serological tests in the US is low but anticipated to increase with the commercial availability of three tests; HepaScore, FibroSpect and FibroTest. The cost of these tests is certainly acceptable at approximately $300 and more reimbursement is becoming available. FibroScan is neither FDA approved nor widely available outside of research in the US. The first critical step in increasing utilization is large scale,
q Dr. Afdhal has received grant support and acted as a consultant for Prometheus, Quest Diagnostics and EchoSens. He is the sponsor of the FDA clinical trial of FibroScan in the United States. * Corresponding author. Tel.: +1 617 632 1118; fax: +1 617 632 1054. E-mail address:
[email protected] (N.H. Afdhal).
vigorous scientific validation of the performance characteristics of novel tests and procedures. The first issue in the evaluation of a novel diagnostic test is the validation of the test against the current clinical gold standard to determine the sensitivity, specificity, and predictive value of the new test. The standard expression of the effectiveness of a test is to look at the area under the curve (AUC) of the receiver operator characteristic (ROC), which plots the sensitivity over 1 – specificity, and a perfect test will score 1.0. This is usually performed in large-scale clinical trials and has been done for both serological fibrosis markers and for the measurement of liver stiffness with FibroScan. The results for the majority of these tests all fall within the same range, with an AUC for the ROC, which varies from 0.75 to 0.88 with the majority of tests all clustering at around 0.85. The very comparable clustering for the AUC is hard to explain when many of the tests measure different biomarkers or different physical properties [7]. The explanation for this diagnostic equivalency of many tests may lie in the inaccuracy of liver biopsy. There are extensive clinical data suggesting that liver biopsy is only accurate for staging fibrosis in 80–90% of patients and that in the remainder there may be up to a 1 stage difference in accuracy [13–16]. Optimal biopsies are 25–40 mm in length and non-fragmented, characteristics that are difficult to achieve routinely in practice [13]. Recent mathematical modeling by our group has suggested that assuming either 80% or 90% diagnostic accuracy of liver biopsy, that non-invasive markers cannot achieve an AUC better than 0.9 and are likely to perform between 0.75 and 0.9, exactly where they are today. We may already have a perfect non-invasive surrogate for liver biopsy but we may never be able to prove it secondary to the limitations of percutaneous liver biopsy.
0168-8278/$32.00 Ó 2007 European Association for the Study of the Liver. Published by Elsevier B.V. All rights reserved. doi:10.1016/j.jhep.2007.01.008
544
N.H. Afdhal, M. Curry / Journal of Hepatology 46 (2007) 543–545
Perhaps the more important clinical issue is the stages at which inaccuracy for both biopsies and biomarkers is most frequent. Biomarkers have been predominantly evaluated for hepatitis C but may perform equally well for fibrosis staging in other diseases, particularly when based on a physical principle such as liver stiffness. Both serum studies and stiffness perform well for the diagnosis of no fibrosis (Metavir Stage 0) and for the diagnosis of cirrhosis. However the real clinical issue is the differentiation between metavir Stage 1 and 2 disease [7]. Here both biopsy and biomarkers have an indeterminate range; biopsy secondary to interobserver variability and sampling error and biomarkers because of the spectrum of disease seen at this stage where there may be co-existence of both stages in different areas of the liver which is reflected in the more global nature of biomarkers. The importance of this lies in the current clinical recommendations for interferon therapy which state that stage 1 patients can be observed and stage 2 or greater patients should be treated. This has led to biomarkers dividing their cutoffs to differentiate stage 0/1 from stage 2 to 4. The individual clinician may be uncomfortable using a single biomarker to make such an important treatment decision without good clinical evaluation and guidelines. In reality, assuming a prevalence of significant fibrosis of 30%, approximately 40% of results for serological tests fall into an indeterminate range where the probability of accuracy of the test is low. This has certainly been an issue in the acceptance of these tests in clinical practice in the US. However, the reality is that these tests are complementary and all add to our diagnostic armamentarium. The complimentary nature of serological tests and liver stiffness was confirmed in recent studies showing that combining FibroScan and serological tests may increase the diagnostic accuracy for liver fibrosis [17]. There is probably no best single test, but combinations of serological tests, stiffness measurement and biopsy need to be evaluated for both efficacy and cost effectiveness. We have recently
Elevated ALT / Suspected Liver Disease
Biomarkers / FibroScan
Low value excludes significant fibrosis @ 30% patients
No biopsy Follow or treat
Gray zone @ 40% patients
Proceed to biopsy only if treatment not planned
High value suggests liver disease F3/4 @ 30% patients
No biopsy HCC + varices screening
Fig. 1. Proposed pathway for the non-invasive evaluation of HCV fibrosis. [This figure appears in colour on the web.]
proposed a scheme for the utilization of these novel diagnostic tests in practice (Fig. 1). However, amongst all these confusing choices, there is some good news for the clinician. As we are beginning to see improving HCV therapies, the need for staging and biopsy is diminishing – for example, the 90% success rate of interferon based therapies for HCV genotype 2 and 3 has made biopsy almost redundant in those patients. We may see the same trend as our therapies improve for genotype 1 patients. This will however leave one important diagnostic question, which is ‘‘does the patient have cirrhosis?’’ The presence of cirrhosis is important both for potentially duration of therapy but more importantly so that the clinician can institute screening for hepatocellular carcinoma and varices. Recent studies with biomarkers have addressed the issue of diagnosing cirrhosis, portal hypertension and clinical outcomes [18–22]. So now we begin to see the importance of the study by Kettaneh et al. in this issue of the Journal [23]. These investigators have evaluated the parameters associated with the successful performance of a liver stiffness measurement with FibroScan as part of the evaluation of this technology in diagnosing cirrhosis. FibroScan is becoming more widely used in Europe and the initial clinical trials and product information suggested that the median of 10 measurements or ‘‘shots’’ was needed for accurate stiffness measurement. In this study, we see the importance of continuing evaluations of new technologies and identifying factors such as patient and operator characteristics that may influence accurate stiffness measurement. The need for trained operators is apparent and a minimum of 50 cases is necessary before the operator is skilled in the use of FibroScan. Patient characteristics such as obesity also affect the ability to get a measurement – however, this is a technical issue and not related to an inaccuracy of stiffness caused by hepatic steatosis. In fact, there has been excellent correlation of liver stiffness with fibrosis secondary to NASH in a recent large US–French study, which included patient with BMI up to 47. We can deduce that FibroScan appears to be accurate in patients with NASH steatosis and also in obese HCV patients, but that this group represents a technical challenge with the need for refinements of the ultrasonic probe for stiffness measurement. Interestingly gender and fasting had no effect on stiffness measurements. Although not evaluated by Kettaneh, we have found in our experience that there may be a difference depending on which intercostal space is chosen and in particular the lower margin of the liver can give higher results. There also has been no difference based on inspiration or expiration but we recommend performance in expiration, similar to that done for biopsy. The investigators also specifically looked at the accuracy for diagnosis of cirrhosis, where even 3 valid ‘‘shots’’ appeared to have an equivalent accuracy to 10 ‘‘shots’’. The simplicity of the FibroScan
N.H. Afdhal, M. Curry / Journal of Hepatology 46 (2007) 543–545
and the ease of use and training appear to make this an ideal tool for screening of patients with potential liver disease. However, some caution is necessary since the interpretation of the findings must be placed in the correct clinical context and we cannot advocate broad use of this technology outside of gastroenterology and liver practices without further study. As we have noted with the so-called ‘‘normal ALT’’, there is unlikely to be any one test that can screen for all liver diseases. The evaluation of new technologies should be continuous and scientifically rigorous so that the medical community can integrate these technologies into clinical practice. The studies of Kettaneh et al. represent a step forward in finding the correct place for FibroScan in the practice of hepatology. However, clearly much remains to be done to further refine our use of biomarkers. The focus should now shift from cross-sectional diagnosis to utilization of these tests in longitudinal studies to look at disease progression, regression and clinical outcomes. In order to achieve these goals we will need to include biomarkers in our future clinical trials and cohort studies before we can really define their ultimate role in practice. References [1] Adams LA, Bulsara M, Rossi E, DeBoer B, Speers D, George J, et al. Hepascore: an accurate validated predictor of liver fibrosis in chronic hepatitis C infection. Clin Chem 2005;51:1867–1873. [2] Afdhal N. Can serum markers be used to reliably detect liver fibrosis?. Nat Clin Pract Gastroenterol Hepatol 2005;2:132–133. [3] Afdhal NH, Nunes D. Evaluation of liver fibrosis: a concise review. Am J Gastroenterol 2004;99:1160–1174. [4] Fontana RJ, Kleiner DE, Bilonick R, Terrault N, Afdhal N, Belle SH, et al. Modeling hepatic fibrosis in African American and Caucasian American patients with chronic hepatitis C virus infection. Hepatology 2006;44:925–935. [5] Imbert-Bismut F, Ratziu V, Pieroni L, Charlotte F, Benhamou Y, Poynard T. Biochemical markers of liver fibrosis in patients with hepatitis C virus infection: a prospective study. Lancet 2001;357:1069–1075. [6] Kelleher TB, Afdhal N. Assessment of liver fibrosis in co-infected patients. J Hepatol 2006;44:S126–S131. [7] Parkes J, Guha IN, Roderick P, Rosenberg W. Performance of serum marker panels for liver fibrosis in chronic hepatitis C. J Hepatol 2006;44:462–474. [8] Patel K, Gordon SC, Jacobson I, Hezode C, Oh E, Smith KM, et al. Evaluation of a panel of non-invasive serum markers to differentiate mild from moderate-to-advanced liver fibrosis in chronic hepatitis C patients. J Hepatol 2004;41:935–942.
545
[9] Rosenberg WM, Voelker M, Thiel R, Becka M, Burt A, Schuppan D, et al. Serum markers detect the presence of liver fibrosis: a cohort study. Gastroenterology 2004;127:1704–1713. [10] Ziol M, Handra-Luca A, Kettaneh A, Christidis C, Mal F, Kazemi F, et al. Noninvasive assessment of liver fibrosis by measurement of stiffness in patients with chronic hepatitis C. Hepatology 2005;41:48–54. [11] Friedman SL, Rockey DC, Bissell DM. Hepatic fibrosis 2006: Report of the third AASLD Single Topic Conference. Hepatology 2007;45:242–249. [12] McHutchison J, Poynard T, Afdhal N. Fibrosis as an end point for clinical trials in liver disease: a report of the international fibrosis group. Clin Gastroenterol Hepatol 2006;4:1214–1220. [13] Bedossa P, Dargere D, Paradis V. Sampling variability of liver fibrosis in chronic hepatitis C. Hepatology 2003;38:1449–1457. [14] Ratziu V, Charlotte F, Heurtier A, Gombert S, Giral P, Bruckert E, et al. Sampling variability of liver biopsy in nonalcoholic fatty liver disease. Gastroenterology 2005;128:1898–1906. [15] Regev A, Berho M, Jeffers LJ, Milikowski C, Molina EG, Pyrsopoulos NT, et al. Sampling error and intraobserver variation in liver biopsy in patients with chronic HCV infection. Am J Gastroenterol 2002;97:2614–2618. [16] Rousselet MC, Michalak S, Dupre F, Croue A, Bedossa P, SaintAndre JP, et al. Sources of variability in histological scoring of chronic viral hepatitis. Hepatology 2005;41:257–264. [17] Castera L, Vergniol J, Foucher J, Le Bail B, Chanteloup E, Haaser M, et al. Prospective comparison of transient elastography, Fibrotest, APRI, and liver biopsy for the assessment of fibrosis in chronic hepatitis C. Gastroenterology 2005;128:343–350. [18] Ganne-Carrie N, Ziol M, de Ledinghen V, Douvin C, Marcellin P, Castera L, et al. Accuracy of liver stiffness measurement for the diagnosis of cirrhosis in patients with chronic liver diseases. Hepatology 2006;44:1511–1517. [19] Kazemi F, Kettaneh A, N’Kontchou G, Pinto E, Ganne-Carrie N, Trinchet JC, et al. Liver stiffness measurement selects patients with cirrhosis at risk of bearing large oesophageal varices. J Hepatol 2006;45:230–235. [20] Ngo Y, Munteanu M, Messous D, Charlotte F, Imbert-Bismut F, Thabut D, et al. A prospective analysis of the prognostic value of biomarkers (FibroTest) in patients with chronic hepatitis C. Clin Chem 2006;52:1887–1896. [21] Thabut D, Trabut JB, Massard J, Rudler M, Muntenau M, Messous D, et al. Non-invasive diagnosis of large oesophageal varices with FibroTest in patients with cirrhosis: a preliminary retrospective study. Liver Int 2006;26:271–278. [22] Yu ML, Lin SM, Lee CM, Dai CY, Chang WY, Chen SC, et al. A simple noninvasive index for predicting long-term outcome of chronic hepatitis C after interferon-based therapy. Hepatology 2006;44:1086–1097. [23] Kettaneh A, Marcellin P, Douvin C, Poupon R, Ziol M, Beaugrand M, et al. Features associated wth success rate and performance of fibroscan measurements for the diagnosis of cirrhosis in HCV patients: A prospective study of 935 patients. J Hepatol 2007;46:628–634.