- Email: [email protected]
Four challenges in medical image analysis from an industrial perspective
Accepted Manuscript
Four Challenges in Medical Image Analysis from an Industrial Perspective Jurgen Weese, Cristian Lorenz ¨ PII: DOI: Reference:
S1361-8415(16)30099-8 10.1016/j.media.2016.06.023 MEDIMA 1142
To appear in:
Medical Image Analysis
Received date: Revised date: Accepted date:
28 March 2016 14 June 2016 15 June 2016
Please cite this article as: Jurgen Weese, Cristian Lorenz, Four Challenges in Medical Image Analysis ¨ from an Industrial Perspective, Medical Image Analysis (2016), doi: 10.1016/j.media.2016.06.023
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Highlights • Medical image analysis related challenges are presented from an indus-
CR IP T
trial perspective.
• First challenge: Adaptable image analysis technologies enabling efficient development.
• Second challenge: Tools and approaches for the efficient generation of
AN US
ground truth data.
• Third challenge: Medical image analysis algorithms for heterogeneous image data.
AC
CE
PT
ED
and organ models.
M
• Fourth challenge: Efficient construction of detailed personalized anatomy
1
ACCEPTED MANUSCRIPT
AC
CE
PT
ED
M
AN US
CR IP T
Graphical Abstract
2
ACCEPTED MANUSCRIPT
Four Challenges in Medical Image Analysis from an Industrial Perspective
CR IP T
J¨ urgen Weese∗, Cristian Lorenz
Philips Research Hamburg, R¨ ontgenstrasse 22 - 24, D-22335 Hamburg, Germany
Abstract
AN US
Today’s medical imaging systems produce a huge amount of images contain-
ing a wealth of information. However, the information is hidden in the data and image analysis algorithms are needed to extract it, to make it readily available for medical decisions and to enable an efficient work flow. Ad-
M
vances in medical image analysis over the past 20 years mean there are now many algorithms and ideas available that allow to address medical image
ED
analysis tasks in commercial solutions with sufficient performance in terms of accuracy, reliability and speed. At the same time new challenges have arisen. Firstly, there is a need for more generic image analysis technologies
PT
that can be efficiently adapted for a specific clinical task. Secondly, efficient approaches for ground truth generation are needed to match the increas-
CE
ing demands regarding validation and machine learning. Thirdly, algorithms for analyzing heterogeneous image data are needed. Finally, anatomical and
AC
organ models play a crucial role in many applications, and algorithms to construct patient-specific models from medical images with a minimum of user ∗
Corresponding author Email addresses: [email protected] (J¨ urgen Weese), [email protected] (Cristian Lorenz) Preprint submitted to Medical Image Analysis
June 16, 2016
ACCEPTED MANUSCRIPT
interaction are needed. These challenges are complementary to the on-going need for more accurate, more reliable and faster algorithms, and dedicated
CR IP T
algorithmic solutions for specific applications. Keywords: medical image analysis technologies, ground truth generation, anatomical models, heterogeneous data 1. Introduction
AN US
Over the last 20 years, medical imaging systems have evolved considerably. Advances in acquisition technology such as multi-slice Computed Tomography (CT), digital Photon Emission Tomography (PET), parallel Magnetic Resonance Imaging (MRI) or Ultrasound (US) transducer technology have not only enabled the acquisition of much better images with
M
higher resolution, they enabled also the generation of many more images. In addition, the diversity of imaging protocols in usage has increased. 2D
ED
imaging techniques such as 2D US and 2D X-ray remain important and are complemented by (rather than replaced by) 3D techniques. In MRI, the abil-
PT
ity to flexibly design pulse sequences, and thereby exploit a range of different physical properties, has enabled a wide spectrum of acquisition protocols
CE
with dedicated applications. While today’s medical images contain a wealth of information, the rel-
AC
evant information is hidden in the pixels or voxels and often not readily available. Therefore, manufacturers of medical imaging systems also provide software, workstations and solutions for archiving, visualizing and analyzing images in the context of disease areas such as neurology, oncology, or cardiology with the goal to support screening, diagnosis, therapy planning, 4
ACCEPTED MANUSCRIPT
treatment and follow-up examinations. The overall scope of medical image analysis is broader still. It covers further imaging modalities such as en-
CR IP T
doscopy or microscopy, additional application areas such as ophthalmology, pathology, pre-clinical image analysis as well as basic research, for instance targeting an understanding of the structure and function of organs.
Many algorithmic approaches have been investigated and advanced over the past 20 years. Strong emphasis has been put by the community on a clear
AN US
motivation of the addressed task and a related validation. In parallel to very
specific processing pipelines for dedicated tasks, more generic approaches have also evolved. For segmentation, deformable models (Kass et al., 1988; McInerney and Terzopoulos, 1996) and subsequently level-set methods had a large impact. In order to make segmentation more robust, statistical shape
M
models of organs have been built and used for segmentation (Cootes et al., 1995; Heimann and Meinzer, 2009). Specific approaches have been developed
ED
for vessel segmentation (Lesage et al., 2009). Image registration techniques have been boosted by the discovery of mutual information as a generic and
PT
broadly applicable similarity measure (Wells III et al., 1996; Maes et al., 1997; Pluim et al., 2003). Registration approaches have not only been used
CE
to align images, but also for segmentation by registering an atlas to an image. Multi-atlas segmentation techniques are today a very widespread approach (Iglesias and Sabuncu, 2015). Of course, there are many more important
AC
ideas and approaches that have been used in medical image analysis like scale-space approaches, wavelets, fuzzy clustering, graph cuts, genetic algorithms, sparse representations, support vector machines, boosted approaches, random forests or more recently deep learning - to name just a few.
5
ACCEPTED MANUSCRIPT
For a number of image analysis tasks, several algorithms have been published together with validation results. However, it is often difficult to un-
CR IP T
derstand whether one algorithmic approach is superior to another one. This is due to varying imaging properties, differences in the considered patient cohort, (slight) variations in the addressed task and differences in the metric
used for validation. This issue has been recognized by the community and
benchmarking challenges have been organized (Consortium for Open Medi-
AN US
cal Image Computing, 2012 – 2016) over the last 10 years. Examples include benchmarks for liver segmentation in CT images, multiple sclerosis lesion segmentation in MRI images, cardiac MRI left ventricle segmentation, coronary artery segmentation, thoracic CT image registration and pulmonary nodule detection in chest CT images.
M
While many algorithmic approaches and ideas are now available which allow medical image analysis tasks in commercial products to be addressed,
ED
new challenges are arising.
PT
2. Efficient development
An increasing number of commercial products rely on medical image anal-
CE
ysis techniques. Examples are lung packages supporting chronic obstructive pulmonary disease assessment and lung nodule detection in CT images, com-
AC
puter assisted diagnosis solutions for mammography, cardiac packages supporting inspection of the coronaries in CT angiography images and the functional assessment of the heart in MRI or US images, and radiation therapy planning solutions supporting the efficient segmentation of risk organs in CT images. Recently introduced solutions for trans-catheter aortic valve replace6
ACCEPTED MANUSCRIPT
ment (TAVR) support intervention planning via image-based measurements on CT images and intervention guidance by overlaying aortic valve models
CR IP T
onto X-ray images in the CathLab. Considering the large number and variety of medical applications which exist, it is not sustainable commercially to develop a new algorithm for each application. For industry, it is important that algorithmic solutions can be
developed and validated very efficiently (meaning in a short time period with
AN US
minimum effort). Next to demonstrating new algorithmic ideas in the context
of a specific application, it is therefore very important to identify algorithmic approaches and develop algorithmic frameworks that can efficiently be adapted to a large number of applications.
Making algorithms available as public domain software and integrating
M
algorithms into larger libraries like the Insight Segmentation and Registration Toolkit (ITK) is an important step forward in this context. However, it would
ED
be even better to have image analysis technologies capable of addressing all of the following three pillars:
PT
• an algorithmic framework that can be adapted to a large number of image analysis tasks
CE
• a well-defined process supported by suitable tools to adapt the frame-
AC
work to a specific task, and
• a validation environment with (several) metrics which allows testing of the algorithm on an image database.
The model-based segmentation framework as described by Ecabert et al. (2011) comes close to such a technology. The framework can be adapted 7
ACCEPTED MANUSCRIPT
for the automatic segmentation of anatomical structures with well-defined shape such as brain structures, the heart or bones in images acquired with
CR IP T
different imaging modalities (see Fig. 1). Adaptation of the framework to a specific segmentation task starts with the construction of a geometric mesh
of the target anatomy, the compilation of a representative image database and its annotation by (manually) adapting the geometric mesh to the images in the database. The annotated images are used to train a General-
AN US
ized Hough Transformation that automatically localizes the target anatomy, to train boundary detectors for model adaptation and to define parameters characterizing the (typical) shape variability. In addition, the pipeline for adapting the model to an image is configured. Within this step, parametric and shape-constrained deformable model adaptation can be used and the or-
M
der for adapting different model parts can be defined. Technical validation is done using surface-to-surface distances or overlap metrics either for the en-
ED
tire model or sub-parts. Of course, the technical validation does not replace a clinical evaluation assessing e.g. the quality of measurements or clinical
PT
indices derived from the segmentation. It is also important for the resulting algorithms and their implementation
CE
to satisfy the major requirements for the subsequent development process regarding accuracy, reliability, robustness, memory consumption, and computation time. These requirements are often tightly linked to the algorithmic
AC
approach leading to a high risk for a complete redesign of the algorithm or the selection of a different algorithmic approach if the requirements are not met. The integration of image analysis algorithms into commercial products has proven particularly successful when algorithms have been advanced
8
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 1: Model of various brain structures and MRI image with adapted brain model,
CE
heart model and CT angiography image with adapted heart model, and model of the hip, femurs and sacrum and DIXON MRI images with adapted model. The latter example uses several extensions of the model-based segmentation framework of Ecabert et al. (2011) such
AC
as the possibility to use two images (Dixon water and fat image) for model adaptation.
9
ACCEPTED MANUSCRIPT
beyond feasibility demonstrated on a limited test set, towards a software component tested on a large image database which can be integrated within
CR IP T
the development process into a solution for a clinical application. 3. Ground Truth for Validation and Machine Learning
The increasing use of image analysis algorithms in products leads to con-
tinuously increasing requirements with respect to accuracy, robustness and
AN US
reliability. While, for instance, for typical segmentation tasks success rates of 70 − 90% might have been sufficient in the past, users start to expect a correct and accurate result in 98 − 99% of the cases. This trend increases the demands on the size of the image database with reference or ground truth (GT) annotations for validation. While 20 − 50 annotated image data sets
M
have often been sufficient in the past, high reliability and robustness requires testing on a few hundred or many more annotated data sets in the future.
ED
For some tasks such as the localization of an anatomical landmark, the generation of GT annotations can be done quickly and even for a larger
PT
database the effort is limited. For other tasks like the segmentation of anatomical structures in 3D images or for non-rigid registration of 3D im-
CE
ages, the generation of GT annotations is a difficult and time consuming task. For 3D segmentation the relevant boundaries must be annotated in
AC
image volumes. This task becomes particularly time consuming for complex anatomical structures or organs composed of multiple parts. For non-rigid registration, it would be desirable (although usually not feasible) to define a GT deformation for the entire image volume. This task is even more complex and time consuming. Combined with the request for robustness and 10
ACCEPTED MANUSCRIPT
reliability, GT annotations for more than 100 data sets must be generated showing that GT generation becomes a task in itself.
CR IP T
Efficient interaction approaches (Olabarriaga and Smeulders, 2001) are, therefore, not only required for correcting imperfections of automatic algorithms, but become important for the efficient development of algorithmic
solutions. To speed up GT annotation, an iterative bootstrapping approach may be applied and initial versions of an algorithm may be used to generate
AN US
approximate GT annotations for new data sets that are manually corrected and used in the next iteration of algorithm development. While this approach
can help to obtain consistent GT annotations, it may introduce a bias in the validation. Collecting GT annotations via crowd sourcing or from interactively corrected results of clinical applications in the field is an alternative
M
strategy. Both approaches help to collect a large number of GT annotations, but it is difficult to control their quality and consistency. Databases with
ED
highly realistic images may also be generated synthetically as described by Alessandrini et al. (2015) for cardiac US images. While this approach re-
PT
sults in a well-defined GT, the image database must be carefully designed to be representative. Each approach for GT generation has advantages and
CE
disadvantages, and further advances in the generation of GT will help with the design and development of highly robust, reliable and accurate image analysis algorithms.
AC
A closely related topic is the generation of annotated data for algorithm
training and (supervised) machine learning. Using techniques like crossvalidation, the annotated image data sets can be used for both, validation and algorithm training or machine learning. Training or machine learning
11
ACCEPTED MANUSCRIPT
techniques may require, however, many more data sets than validation. Especially in the context of deep learning, advances have been reported which
CR IP T
require a considerably increased number of training data sets. The success of these approaches in medical image analysis goes hand in hand with the possi-
bility to generate or collect sufficient numbers of data sets together with GT annotations. As this can be difficult, time consuming or simply expensive, approaches which optimally exploit the available images and GT annotations
AN US
are becoming more interesting. Accordingly, training and machine learning approaches are gaining importance which for instance combine supervised with unsupervised learning, use a small database with detailed GT annotations (e.g. detailed 3D delineation of anatomical structures) together with a large database with weak annotations (e.g. a sparse set of organ bound-
ED
image properties.
M
ary locations) or synthetically extend the GT database by exploiting known
4. Heterogeneous Image Data
PT
As mentioned before, strong emphasis has been put over the last 20 years in medical image analysis on addressing well-defined medical applications.
CE
As a result, many algorithms have been developed for and validated on welldefined types of images with well-defined acquisition protocol. However,
AC
there seems to be an increasing demand for algorithms working with more heterogeneous image data. One example is MRI image analysis. MRI offers a huge flexibility to de-
sign acquisition protocols and to generate images with considerably differing image appearance and sampling properties. While image analysis algorithms 12
ACCEPTED MANUSCRIPT
work usually well when optimized for a specific scanner type and protocol, clinical sites like to exploit the flexibility and versatility of MRI imaging and
CR IP T
adapt scanning protocols to their local needs and preferences. As a result, automated image analysis algorithms either perform sub-optimally or need
to be adapted. Algorithmic approaches that work accurately, reliably and robustly for varying MRI protocols and related MRI characteristics would
resolve this issue and facilitate a much more widespread support of MRI
AN US
post-processing applications in commercial products.
Another example is the radiology reading and reporting environment. A considerable diversity of images must be inspected and a wide variety of different diseases is diagnosed in this context. The approach of developing algorithms for a well-defined clinical application with well-defined types of
M
images only supports a fraction of the cases with sufficiently high occurrence. The reading work flow could be made still more efficient, e.g. by providing
ED
organ related functionality in the interface for reporting, measurements and the comparison with previous examinations and findings. Algorithms that
PT
recognize the anatomy covered by an image and find the location of the relevant organs for images acquired with different imaging modalities and
CE
showing different anatomical regions could support this process. A last example relates to big data and analytics. Today there are huge
image databases in hospitals and image analysis may be used to discover
AC
knowledge from them. Examples could be statistics on the organ geometry in relation to patient information (e.g. age, sex) and disease. Quantitative imaging bio-markers could be better correlated to the disease status and new imaging bio-markers could be discovered. The image data is even more het-
13
ACCEPTED MANUSCRIPT
erogeneous, and simply the task of retrieving a distinct class of images can require image analysis, since DICOM header information may be mislead-
CR IP T
ing, incomplete or contain private tags. When looking at a class of images retrieved from a PACS database such as chest CT images, images may have
been taken with or without contrast agent, low as well as high dose images may be included and the field-of-view can vary considerably.
AN US
5. Anatomical Models
For centuries, anatomical sketches have played a crucial role in medicine for teaching of anatomy and the understanding of disease and pathologies. 3D medical images provide a wealth of relevant information, but it is difficult to mentally reconstruct the anatomy from cross-sectional image slices.
M
Proper segmentation and annotation of 3D images is essential to visualize the data in an easy understandable way. Voxel-Man (Pommert et al., 2001)
ED
(Fig. 2) is one of the early examples providing a 3D anatomical atlas generated from medical images that can be interactively inspected. In general, the
PT
importance of generating patient-specific anatomical models from 3D image data was recognized early as a key topic in medical image analysis with many
CE
applications (H¨ohne et al., 1995). Surgery and interventions are an area where patient-specific anatomical
AC
models are frequently used. During planning, their visualization helps to understand the anatomy and anomalies. They are also used to plan procedures and perform measurements for device sizing or selection. With the advances in 3D printing technology, it is not only possible to generate physical models that provide a better 3D appreciation of pathological structure, but also 14
CR IP T
ACCEPTED MANUSCRIPT
Figure 2: Anatomical Model of the inner organs with 650 constituents (image courtesy of
M
AN US
K.H. H¨ ohne; reprinted from Pommert et al. (2001) with the permission of Elsevier).
Figure 3: Aortic valve model overlaid onto an X-ray image to support TAVR guidance
ED
and simulated blood flow through an aortic valve model (image courtesy of D.R. Hose).
PT
to design customized prostheses or implants (Rengier et al., 2010). During surgery and interventions patient-specific anatomical models are used to
CE
complement intra-procedural images with additional information (Fig. 3a). Increasingly, anatomical models are being combined with simulation ap-
proaches with the goal of deriving physiologic measurements from anatomical
AC
images or predicting the outcome of surgeries or interventions. Organizations like the Virtual Physiologic Human (VPH) institute have been set up that put this vision (Hunter et al., 2010) forward. Cardiovascular simulations have been advanced ranging from blood flow simulations (Morris et al., 2016) (see
15
CR IP T
ACCEPTED MANUSCRIPT
Figure 4: Musculoskeletal model after femur reconstruction in an intermediate frame of
AN US
a gait cycle (image courtesy of M. Viceconti; reprinted from Taddei et al. (2012) with the permission of Elsevier). The X-ray images show the reconstructed femur immediately after surgery (a) and after removal of distal screws (b). The lower limbs CT image with the gait analysis markers has been acquired at month 31 of follow-up.
M
Fig. 3b for an example) to complex patient-specific simulations of the heart including cardiac mechanics, electrophysiology and blood flow (Smith et al.,
ED
2011). As another example, the musculoskeletal system has been modeled integrating cell, tissue, organ and body models (Fig. 4, (Taddei et al., 2012)). These simulations require often various inputs ranging from images to phys-
PT
iologic parameters and offer an approach to the integration of clinical data. They bring together several disciplines such as medical image analysis for
CE
generating anatomical models from images, computer assisted design and simulation technologies, biophysics and physiology.
AC
Another interesting direction is the integration of anatomical models with
ontologies such as the foundational model of anatomy (FMA) (Rosse and Mejino Jr., 2003). This idea is, for instance, pursued by the concept of My Corporis Fabrica (MyCF) (Palombi et al., 2014). This or similar approaches
16
ACCEPTED MANUSCRIPT
may develop in the future into an interesting link between medical image analysis and decision support systems on the basis of semantic and reasoning
CR IP T
technologies. 6. Summary
The advances in medical image analysis observed over the past 20 years boost the use of image analysis algorithms in commercial solutions. At the
AN US
same time new challenges arise from an industrial perspective. Firstly, there
is a need for more generic image analysis technologies that can efficiently be adapted for a specific clinical task. The second, closely related challenge concerns the efficient generation of GT annotations to match the increasing requirements on robustness and reliability in commercial solutions and the
M
increased demands when using machine learning. Thirdly, algorithms for analyzing more heterogeneous image data will enable the more widespread
ED
adoption of MRI post-processing applications, as well as new applications, for instance related to big data and analytics. The fourth challenge is related
PT
to the construction of detailed anatomical and organ models with minimum user interaction. Besides their direct relevance in many applications, these
CE
models are key to developments like the Virtual Physiologic Human, and could link medical image analysis to semantic and reasoning technologies.
AC
These challenges are complementary to the on-going need for more accurate, more reliable and faster algorithms and dedicated algorithms for specific applications.
17
ACCEPTED MANUSCRIPT
Acknowledgment The authors would like to thank K.H. H¨ohne, D.R. Hose and M. Viceconti
CR IP T
for providing images. In addition, we would like to acknowledge C. Buerger and S. Young for supporting the preparation of the manuscript. References
Alessandrini, M., Craene, M. D., Bernard, O., Giffard-Roisin, S., Allain,
AN US
P., Waechter-Stehle, I., Weese, J., Saloux, E., Delingette, H., Sermesant, M., D’hooge, J., 2015. A pipeline for the generation of realistic 3D synthetic echocardiographic sequences: Methodology and open-access database. IEEE Trans. Med. Imag. 34 (7), 1436–1451.
M
Consortium for Open Medical Image Computing, 2012 – 2016. Grand challenges in biomedical image analysis (accessed 25.03.16).
ED
URL http://grand-challenge.org Cootes, T. F., Taylor, C. J., Cooper, D. H., Graham, J., 1995. Active shape
CE
38–59.
PT
models – their training and application. Comput. Vis. Image Und. 61 (1),
Ecabert, O., Peters, J., Walker, M. J., Ivanc, T., Lorenz, C., von Berg, J., Lessick, J., Vembar, M., Weese, J., 2011. Segmentation of the heart and
AC
great vessels in CT images using a model-based adaptation framework. Med. Image Anal. 15 (6), 863 – 876.
Heimann, T., Meinzer, H.-P., 2009. Statistical shape models for 3D medical image segmentation: A review. Med. Image Anal. 13 (4), 543 – 563. 18
ACCEPTED MANUSCRIPT
H¨ohne, K. H., Pflesser, B., Pommert, A., Riemer, M., Schiemann, T., Schubert, R., Tiede, U., 1995. A new representation of knowledge concerning
CR IP T
human anatomy and function. Nat. Med. 1 (6), 506511. Hunter, P., Coveney, P. V., de Bono, B., Diaz, V., Fenner, J., Frangi, A. F., Harris, P., Hose, R., Kohl, P., Lawford, P., McCormack, K., Mendes, M.,
Omholt, S., Quarteroni, A., Sk˚ ar, J., Tegner, J., Randall Thomas, S., Tollis, I., Tsamardinos, I., van Beek, J. H. G. M., Viceconti, M., 2010. A
AN US
vision and strategy for the virtual physiological human in 2010 and beyond. Phil. Trans. R. Soc. A 368 (1920), 2595–2614.
Iglesias, J. E., Sabuncu, M. R., 2015. Multi-atlas segmentation of biomedical images: A survey. Med. Image Anal. 24 (1), 205 – 219.
M
Kass, M., Witkin, A., Terzopoulos, D., 1988. Snakes: Active contour models. Int. J. Comput. Vision 1 (4), 321–331.
ED
Lesage, D., Angelini, E. D., Bloch, I., Funka-Lea, G., 2009. A review of 3D vessel lumen segmentation techniques: Models, features and extraction
PT
schemes. Med. Image Anal. 13 (6), 819 – 845.
CE
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P., 1997. Multimodality image registration by maximization of mutual information.
AC
IEEE Trans. Med. Imag. 16 (2), 187–198.
McInerney, T., Terzopoulos, D., 1996. Deformable models in medical image analysis: a survey. Med. Image Anal. 1 (2), 91 – 108.
Morris, P. D., Narracott, A., von Tengg-Kobligk, H., Silva Soto, D. A., Hsiao, S., Lungu, A., Evans, P., Bressloff, N. W., Lawford, P. V., Hose, D. R., 19
ACCEPTED MANUSCRIPT
Gunn, J. P., 2016. Computational fluid dynamics modelling in cardiovascular medicine. Heart 102 (1), 18–28.
CR IP T
Olabarriaga, S., Smeulders, A., 2001. Interaction in the segmentation of medical images: A survey. Med. Image Anal. 5 (2), 127 – 142.
Palombi, O., Ulliana, F., Favier, V., L´eon, J.-C., Rousset, M.-C., 2014. My corporis fabrica: an ontology-based tool for reasoning and querying on
AN US
complex anatomical models. J. Biomed. Semant. 5 (1), 1–13.
Pluim, J. P. W., Maintz, J. B. A., Viergever, M. A., 2003. Mutualinformation-based registration of medical images: a survey. IEEE Trans. Med. Imag. 22 (8), 986–1004.
Pommert, A., H¨ohne, K. H., Pflesser, B., Richter, E., Riemer, M., Schie-
M
mann, T., Schubert, R., Schumacher, U., Tiede, U., 2001. Creating a highresolution spatial/symbolic model of the inner organs based on the visible
ED
human. Med. Image Anal. 5 (3), 221 – 228.
PT
Rengier, F., Mehndiratta, A., Tengg-Kobligk, H., Zechmann, C. M., Unterhinninghofen, R., Kauczor, H.-U., Giesel, F. L., 2010. 3D printing based
CE
on imaging data: review of medical applications. Int. J. Comput. Assist. Radiol. Surg. 5 (4), 335–341.
AC
Rosse, C., Mejino Jr., J. L. V., 2003. A reference ontology for biomedical informatics: the foundational model of anatomy. J. Biomed. Inform. 36 (6), 478 – 500.
Smith, N., de Vecchi, A., McCormick, M., Nordsletten, D., Camara, O., Frangi, A. F., Delingette, H., Sermesant, M., Relan, J., Ayache, N., 20
ACCEPTED MANUSCRIPT
Krueger, M. W., Schulze, W. H. W., Hose, R., Valverde, I., Beerbaum, P., Staicu, C., Siebes, M., Spaan, J., Hunter, P., Weese, J., Lehmann, H.,
CR IP T
Chapelle, D., Rezavi, R., 2011. euHeart: personalized and integrated cardiac care using patient-specific cardiovascular modelling. Interface Focus 1 (3), 349–364.
Taddei, F., Martelli, S., Valente, G., Leardini, A., Benedetti, M. G., Manfrini, M., Viceconti, M., 2012. Femoral loads during gait in a patient with
AN US
massive skeletal reconstruction. Clin. Biomech. 27 (3), 273 – 280.
Wells III, W. M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R., 1996. Multi-modal volume registration by maximization of mutual information.
AC
CE
PT
ED
M
Med. Image Anal. 1 (1), 35 – 51.
21
Four Challenges in Medical Image Analysis from an Industrial Perspective Jurgen Weese, Cristian Lorenz ¨ PII: DOI: Reference:
S1361-8415(16)30099-8 10.1016/j.media.2016.06.023 MEDIMA 1142
To appear in:
Medical Image Analysis
Received date: Revised date: Accepted date:
28 March 2016 14 June 2016 15 June 2016
Please cite this article as: Jurgen Weese, Cristian Lorenz, Four Challenges in Medical Image Analysis ¨ from an Industrial Perspective, Medical Image Analysis (2016), doi: 10.1016/j.media.2016.06.023
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT
Highlights • Medical image analysis related challenges are presented from an indus-
CR IP T
trial perspective.
• First challenge: Adaptable image analysis technologies enabling efficient development.
• Second challenge: Tools and approaches for the efficient generation of
AN US
ground truth data.
• Third challenge: Medical image analysis algorithms for heterogeneous image data.
AC
CE
PT
ED
and organ models.
M
• Fourth challenge: Efficient construction of detailed personalized anatomy
1
ACCEPTED MANUSCRIPT
AC
CE
PT
ED
M
AN US
CR IP T
Graphical Abstract
2
ACCEPTED MANUSCRIPT
Four Challenges in Medical Image Analysis from an Industrial Perspective
CR IP T
J¨ urgen Weese∗, Cristian Lorenz
Philips Research Hamburg, R¨ ontgenstrasse 22 - 24, D-22335 Hamburg, Germany
Abstract
AN US
Today’s medical imaging systems produce a huge amount of images contain-
ing a wealth of information. However, the information is hidden in the data and image analysis algorithms are needed to extract it, to make it readily available for medical decisions and to enable an efficient work flow. Ad-
M
vances in medical image analysis over the past 20 years mean there are now many algorithms and ideas available that allow to address medical image
ED
analysis tasks in commercial solutions with sufficient performance in terms of accuracy, reliability and speed. At the same time new challenges have arisen. Firstly, there is a need for more generic image analysis technologies
PT
that can be efficiently adapted for a specific clinical task. Secondly, efficient approaches for ground truth generation are needed to match the increas-
CE
ing demands regarding validation and machine learning. Thirdly, algorithms for analyzing heterogeneous image data are needed. Finally, anatomical and
AC
organ models play a crucial role in many applications, and algorithms to construct patient-specific models from medical images with a minimum of user ∗
Corresponding author Email addresses: [email protected] (J¨ urgen Weese), [email protected] (Cristian Lorenz) Preprint submitted to Medical Image Analysis
June 16, 2016
ACCEPTED MANUSCRIPT
interaction are needed. These challenges are complementary to the on-going need for more accurate, more reliable and faster algorithms, and dedicated
CR IP T
algorithmic solutions for specific applications. Keywords: medical image analysis technologies, ground truth generation, anatomical models, heterogeneous data 1. Introduction
AN US
Over the last 20 years, medical imaging systems have evolved considerably. Advances in acquisition technology such as multi-slice Computed Tomography (CT), digital Photon Emission Tomography (PET), parallel Magnetic Resonance Imaging (MRI) or Ultrasound (US) transducer technology have not only enabled the acquisition of much better images with
M
higher resolution, they enabled also the generation of many more images. In addition, the diversity of imaging protocols in usage has increased. 2D
ED
imaging techniques such as 2D US and 2D X-ray remain important and are complemented by (rather than replaced by) 3D techniques. In MRI, the abil-
PT
ity to flexibly design pulse sequences, and thereby exploit a range of different physical properties, has enabled a wide spectrum of acquisition protocols
CE
with dedicated applications. While today’s medical images contain a wealth of information, the rel-
AC
evant information is hidden in the pixels or voxels and often not readily available. Therefore, manufacturers of medical imaging systems also provide software, workstations and solutions for archiving, visualizing and analyzing images in the context of disease areas such as neurology, oncology, or cardiology with the goal to support screening, diagnosis, therapy planning, 4
ACCEPTED MANUSCRIPT
treatment and follow-up examinations. The overall scope of medical image analysis is broader still. It covers further imaging modalities such as en-
CR IP T
doscopy or microscopy, additional application areas such as ophthalmology, pathology, pre-clinical image analysis as well as basic research, for instance targeting an understanding of the structure and function of organs.
Many algorithmic approaches have been investigated and advanced over the past 20 years. Strong emphasis has been put by the community on a clear
AN US
motivation of the addressed task and a related validation. In parallel to very
specific processing pipelines for dedicated tasks, more generic approaches have also evolved. For segmentation, deformable models (Kass et al., 1988; McInerney and Terzopoulos, 1996) and subsequently level-set methods had a large impact. In order to make segmentation more robust, statistical shape
M
models of organs have been built and used for segmentation (Cootes et al., 1995; Heimann and Meinzer, 2009). Specific approaches have been developed
ED
for vessel segmentation (Lesage et al., 2009). Image registration techniques have been boosted by the discovery of mutual information as a generic and
PT
broadly applicable similarity measure (Wells III et al., 1996; Maes et al., 1997; Pluim et al., 2003). Registration approaches have not only been used
CE
to align images, but also for segmentation by registering an atlas to an image. Multi-atlas segmentation techniques are today a very widespread approach (Iglesias and Sabuncu, 2015). Of course, there are many more important
AC
ideas and approaches that have been used in medical image analysis like scale-space approaches, wavelets, fuzzy clustering, graph cuts, genetic algorithms, sparse representations, support vector machines, boosted approaches, random forests or more recently deep learning - to name just a few.
5
ACCEPTED MANUSCRIPT
For a number of image analysis tasks, several algorithms have been published together with validation results. However, it is often difficult to un-
CR IP T
derstand whether one algorithmic approach is superior to another one. This is due to varying imaging properties, differences in the considered patient cohort, (slight) variations in the addressed task and differences in the metric
used for validation. This issue has been recognized by the community and
benchmarking challenges have been organized (Consortium for Open Medi-
AN US
cal Image Computing, 2012 – 2016) over the last 10 years. Examples include benchmarks for liver segmentation in CT images, multiple sclerosis lesion segmentation in MRI images, cardiac MRI left ventricle segmentation, coronary artery segmentation, thoracic CT image registration and pulmonary nodule detection in chest CT images.
M
While many algorithmic approaches and ideas are now available which allow medical image analysis tasks in commercial products to be addressed,
ED
new challenges are arising.
PT
2. Efficient development
An increasing number of commercial products rely on medical image anal-
CE
ysis techniques. Examples are lung packages supporting chronic obstructive pulmonary disease assessment and lung nodule detection in CT images, com-
AC
puter assisted diagnosis solutions for mammography, cardiac packages supporting inspection of the coronaries in CT angiography images and the functional assessment of the heart in MRI or US images, and radiation therapy planning solutions supporting the efficient segmentation of risk organs in CT images. Recently introduced solutions for trans-catheter aortic valve replace6
ACCEPTED MANUSCRIPT
ment (TAVR) support intervention planning via image-based measurements on CT images and intervention guidance by overlaying aortic valve models
CR IP T
onto X-ray images in the CathLab. Considering the large number and variety of medical applications which exist, it is not sustainable commercially to develop a new algorithm for each application. For industry, it is important that algorithmic solutions can be
developed and validated very efficiently (meaning in a short time period with
AN US
minimum effort). Next to demonstrating new algorithmic ideas in the context
of a specific application, it is therefore very important to identify algorithmic approaches and develop algorithmic frameworks that can efficiently be adapted to a large number of applications.
Making algorithms available as public domain software and integrating
M
algorithms into larger libraries like the Insight Segmentation and Registration Toolkit (ITK) is an important step forward in this context. However, it would
ED
be even better to have image analysis technologies capable of addressing all of the following three pillars:
PT
• an algorithmic framework that can be adapted to a large number of image analysis tasks
CE
• a well-defined process supported by suitable tools to adapt the frame-
AC
work to a specific task, and
• a validation environment with (several) metrics which allows testing of the algorithm on an image database.
The model-based segmentation framework as described by Ecabert et al. (2011) comes close to such a technology. The framework can be adapted 7
ACCEPTED MANUSCRIPT
for the automatic segmentation of anatomical structures with well-defined shape such as brain structures, the heart or bones in images acquired with
CR IP T
different imaging modalities (see Fig. 1). Adaptation of the framework to a specific segmentation task starts with the construction of a geometric mesh
of the target anatomy, the compilation of a representative image database and its annotation by (manually) adapting the geometric mesh to the images in the database. The annotated images are used to train a General-
AN US
ized Hough Transformation that automatically localizes the target anatomy, to train boundary detectors for model adaptation and to define parameters characterizing the (typical) shape variability. In addition, the pipeline for adapting the model to an image is configured. Within this step, parametric and shape-constrained deformable model adaptation can be used and the or-
M
der for adapting different model parts can be defined. Technical validation is done using surface-to-surface distances or overlap metrics either for the en-
ED
tire model or sub-parts. Of course, the technical validation does not replace a clinical evaluation assessing e.g. the quality of measurements or clinical
PT
indices derived from the segmentation. It is also important for the resulting algorithms and their implementation
CE
to satisfy the major requirements for the subsequent development process regarding accuracy, reliability, robustness, memory consumption, and computation time. These requirements are often tightly linked to the algorithmic
AC
approach leading to a high risk for a complete redesign of the algorithm or the selection of a different algorithmic approach if the requirements are not met. The integration of image analysis algorithms into commercial products has proven particularly successful when algorithms have been advanced
8
PT
ED
M
AN US
CR IP T
ACCEPTED MANUSCRIPT
Figure 1: Model of various brain structures and MRI image with adapted brain model,
CE
heart model and CT angiography image with adapted heart model, and model of the hip, femurs and sacrum and DIXON MRI images with adapted model. The latter example uses several extensions of the model-based segmentation framework of Ecabert et al. (2011) such
AC
as the possibility to use two images (Dixon water and fat image) for model adaptation.
9
ACCEPTED MANUSCRIPT
beyond feasibility demonstrated on a limited test set, towards a software component tested on a large image database which can be integrated within
CR IP T
the development process into a solution for a clinical application. 3. Ground Truth for Validation and Machine Learning
The increasing use of image analysis algorithms in products leads to con-
tinuously increasing requirements with respect to accuracy, robustness and
AN US
reliability. While, for instance, for typical segmentation tasks success rates of 70 − 90% might have been sufficient in the past, users start to expect a correct and accurate result in 98 − 99% of the cases. This trend increases the demands on the size of the image database with reference or ground truth (GT) annotations for validation. While 20 − 50 annotated image data sets
M
have often been sufficient in the past, high reliability and robustness requires testing on a few hundred or many more annotated data sets in the future.
ED
For some tasks such as the localization of an anatomical landmark, the generation of GT annotations can be done quickly and even for a larger
PT
database the effort is limited. For other tasks like the segmentation of anatomical structures in 3D images or for non-rigid registration of 3D im-
CE
ages, the generation of GT annotations is a difficult and time consuming task. For 3D segmentation the relevant boundaries must be annotated in
AC
image volumes. This task becomes particularly time consuming for complex anatomical structures or organs composed of multiple parts. For non-rigid registration, it would be desirable (although usually not feasible) to define a GT deformation for the entire image volume. This task is even more complex and time consuming. Combined with the request for robustness and 10
ACCEPTED MANUSCRIPT
reliability, GT annotations for more than 100 data sets must be generated showing that GT generation becomes a task in itself.
CR IP T
Efficient interaction approaches (Olabarriaga and Smeulders, 2001) are, therefore, not only required for correcting imperfections of automatic algorithms, but become important for the efficient development of algorithmic
solutions. To speed up GT annotation, an iterative bootstrapping approach may be applied and initial versions of an algorithm may be used to generate
AN US
approximate GT annotations for new data sets that are manually corrected and used in the next iteration of algorithm development. While this approach
can help to obtain consistent GT annotations, it may introduce a bias in the validation. Collecting GT annotations via crowd sourcing or from interactively corrected results of clinical applications in the field is an alternative
M
strategy. Both approaches help to collect a large number of GT annotations, but it is difficult to control their quality and consistency. Databases with
ED
highly realistic images may also be generated synthetically as described by Alessandrini et al. (2015) for cardiac US images. While this approach re-
PT
sults in a well-defined GT, the image database must be carefully designed to be representative. Each approach for GT generation has advantages and
CE
disadvantages, and further advances in the generation of GT will help with the design and development of highly robust, reliable and accurate image analysis algorithms.
AC
A closely related topic is the generation of annotated data for algorithm
training and (supervised) machine learning. Using techniques like crossvalidation, the annotated image data sets can be used for both, validation and algorithm training or machine learning. Training or machine learning
11
ACCEPTED MANUSCRIPT
techniques may require, however, many more data sets than validation. Especially in the context of deep learning, advances have been reported which
CR IP T
require a considerably increased number of training data sets. The success of these approaches in medical image analysis goes hand in hand with the possi-
bility to generate or collect sufficient numbers of data sets together with GT annotations. As this can be difficult, time consuming or simply expensive, approaches which optimally exploit the available images and GT annotations
AN US
are becoming more interesting. Accordingly, training and machine learning approaches are gaining importance which for instance combine supervised with unsupervised learning, use a small database with detailed GT annotations (e.g. detailed 3D delineation of anatomical structures) together with a large database with weak annotations (e.g. a sparse set of organ bound-
ED
image properties.
M
ary locations) or synthetically extend the GT database by exploiting known
4. Heterogeneous Image Data
PT
As mentioned before, strong emphasis has been put over the last 20 years in medical image analysis on addressing well-defined medical applications.
CE
As a result, many algorithms have been developed for and validated on welldefined types of images with well-defined acquisition protocol. However,
AC
there seems to be an increasing demand for algorithms working with more heterogeneous image data. One example is MRI image analysis. MRI offers a huge flexibility to de-
sign acquisition protocols and to generate images with considerably differing image appearance and sampling properties. While image analysis algorithms 12
ACCEPTED MANUSCRIPT
work usually well when optimized for a specific scanner type and protocol, clinical sites like to exploit the flexibility and versatility of MRI imaging and
CR IP T
adapt scanning protocols to their local needs and preferences. As a result, automated image analysis algorithms either perform sub-optimally or need
to be adapted. Algorithmic approaches that work accurately, reliably and robustly for varying MRI protocols and related MRI characteristics would
resolve this issue and facilitate a much more widespread support of MRI
AN US
post-processing applications in commercial products.
Another example is the radiology reading and reporting environment. A considerable diversity of images must be inspected and a wide variety of different diseases is diagnosed in this context. The approach of developing algorithms for a well-defined clinical application with well-defined types of
M
images only supports a fraction of the cases with sufficiently high occurrence. The reading work flow could be made still more efficient, e.g. by providing
ED
organ related functionality in the interface for reporting, measurements and the comparison with previous examinations and findings. Algorithms that
PT
recognize the anatomy covered by an image and find the location of the relevant organs for images acquired with different imaging modalities and
CE
showing different anatomical regions could support this process. A last example relates to big data and analytics. Today there are huge
image databases in hospitals and image analysis may be used to discover
AC
knowledge from them. Examples could be statistics on the organ geometry in relation to patient information (e.g. age, sex) and disease. Quantitative imaging bio-markers could be better correlated to the disease status and new imaging bio-markers could be discovered. The image data is even more het-
13
ACCEPTED MANUSCRIPT
erogeneous, and simply the task of retrieving a distinct class of images can require image analysis, since DICOM header information may be mislead-
CR IP T
ing, incomplete or contain private tags. When looking at a class of images retrieved from a PACS database such as chest CT images, images may have
been taken with or without contrast agent, low as well as high dose images may be included and the field-of-view can vary considerably.
AN US
5. Anatomical Models
For centuries, anatomical sketches have played a crucial role in medicine for teaching of anatomy and the understanding of disease and pathologies. 3D medical images provide a wealth of relevant information, but it is difficult to mentally reconstruct the anatomy from cross-sectional image slices.
M
Proper segmentation and annotation of 3D images is essential to visualize the data in an easy understandable way. Voxel-Man (Pommert et al., 2001)
ED
(Fig. 2) is one of the early examples providing a 3D anatomical atlas generated from medical images that can be interactively inspected. In general, the
PT
importance of generating patient-specific anatomical models from 3D image data was recognized early as a key topic in medical image analysis with many
CE
applications (H¨ohne et al., 1995). Surgery and interventions are an area where patient-specific anatomical
AC
models are frequently used. During planning, their visualization helps to understand the anatomy and anomalies. They are also used to plan procedures and perform measurements for device sizing or selection. With the advances in 3D printing technology, it is not only possible to generate physical models that provide a better 3D appreciation of pathological structure, but also 14
CR IP T
ACCEPTED MANUSCRIPT
Figure 2: Anatomical Model of the inner organs with 650 constituents (image courtesy of
M
AN US
K.H. H¨ ohne; reprinted from Pommert et al. (2001) with the permission of Elsevier).
Figure 3: Aortic valve model overlaid onto an X-ray image to support TAVR guidance
ED
and simulated blood flow through an aortic valve model (image courtesy of D.R. Hose).
PT
to design customized prostheses or implants (Rengier et al., 2010). During surgery and interventions patient-specific anatomical models are used to
CE
complement intra-procedural images with additional information (Fig. 3a). Increasingly, anatomical models are being combined with simulation ap-
proaches with the goal of deriving physiologic measurements from anatomical
AC
images or predicting the outcome of surgeries or interventions. Organizations like the Virtual Physiologic Human (VPH) institute have been set up that put this vision (Hunter et al., 2010) forward. Cardiovascular simulations have been advanced ranging from blood flow simulations (Morris et al., 2016) (see
15
CR IP T
ACCEPTED MANUSCRIPT
Figure 4: Musculoskeletal model after femur reconstruction in an intermediate frame of
AN US
a gait cycle (image courtesy of M. Viceconti; reprinted from Taddei et al. (2012) with the permission of Elsevier). The X-ray images show the reconstructed femur immediately after surgery (a) and after removal of distal screws (b). The lower limbs CT image with the gait analysis markers has been acquired at month 31 of follow-up.
M
Fig. 3b for an example) to complex patient-specific simulations of the heart including cardiac mechanics, electrophysiology and blood flow (Smith et al.,
ED
2011). As another example, the musculoskeletal system has been modeled integrating cell, tissue, organ and body models (Fig. 4, (Taddei et al., 2012)). These simulations require often various inputs ranging from images to phys-
PT
iologic parameters and offer an approach to the integration of clinical data. They bring together several disciplines such as medical image analysis for
CE
generating anatomical models from images, computer assisted design and simulation technologies, biophysics and physiology.
AC
Another interesting direction is the integration of anatomical models with
ontologies such as the foundational model of anatomy (FMA) (Rosse and Mejino Jr., 2003). This idea is, for instance, pursued by the concept of My Corporis Fabrica (MyCF) (Palombi et al., 2014). This or similar approaches
16
ACCEPTED MANUSCRIPT
may develop in the future into an interesting link between medical image analysis and decision support systems on the basis of semantic and reasoning
CR IP T
technologies. 6. Summary
The advances in medical image analysis observed over the past 20 years boost the use of image analysis algorithms in commercial solutions. At the
AN US
same time new challenges arise from an industrial perspective. Firstly, there
is a need for more generic image analysis technologies that can efficiently be adapted for a specific clinical task. The second, closely related challenge concerns the efficient generation of GT annotations to match the increasing requirements on robustness and reliability in commercial solutions and the
M
increased demands when using machine learning. Thirdly, algorithms for analyzing more heterogeneous image data will enable the more widespread
ED
adoption of MRI post-processing applications, as well as new applications, for instance related to big data and analytics. The fourth challenge is related
PT
to the construction of detailed anatomical and organ models with minimum user interaction. Besides their direct relevance in many applications, these
CE
models are key to developments like the Virtual Physiologic Human, and could link medical image analysis to semantic and reasoning technologies.
AC
These challenges are complementary to the on-going need for more accurate, more reliable and faster algorithms and dedicated algorithms for specific applications.
17
ACCEPTED MANUSCRIPT
Acknowledgment The authors would like to thank K.H. H¨ohne, D.R. Hose and M. Viceconti
CR IP T
for providing images. In addition, we would like to acknowledge C. Buerger and S. Young for supporting the preparation of the manuscript. References
Alessandrini, M., Craene, M. D., Bernard, O., Giffard-Roisin, S., Allain,
AN US
P., Waechter-Stehle, I., Weese, J., Saloux, E., Delingette, H., Sermesant, M., D’hooge, J., 2015. A pipeline for the generation of realistic 3D synthetic echocardiographic sequences: Methodology and open-access database. IEEE Trans. Med. Imag. 34 (7), 1436–1451.
M
Consortium for Open Medical Image Computing, 2012 – 2016. Grand challenges in biomedical image analysis (accessed 25.03.16).
ED
URL http://grand-challenge.org Cootes, T. F., Taylor, C. J., Cooper, D. H., Graham, J., 1995. Active shape
CE
38–59.
PT
models – their training and application. Comput. Vis. Image Und. 61 (1),
Ecabert, O., Peters, J., Walker, M. J., Ivanc, T., Lorenz, C., von Berg, J., Lessick, J., Vembar, M., Weese, J., 2011. Segmentation of the heart and
AC
great vessels in CT images using a model-based adaptation framework. Med. Image Anal. 15 (6), 863 – 876.
Heimann, T., Meinzer, H.-P., 2009. Statistical shape models for 3D medical image segmentation: A review. Med. Image Anal. 13 (4), 543 – 563. 18
ACCEPTED MANUSCRIPT
H¨ohne, K. H., Pflesser, B., Pommert, A., Riemer, M., Schiemann, T., Schubert, R., Tiede, U., 1995. A new representation of knowledge concerning
CR IP T
human anatomy and function. Nat. Med. 1 (6), 506511. Hunter, P., Coveney, P. V., de Bono, B., Diaz, V., Fenner, J., Frangi, A. F., Harris, P., Hose, R., Kohl, P., Lawford, P., McCormack, K., Mendes, M.,
Omholt, S., Quarteroni, A., Sk˚ ar, J., Tegner, J., Randall Thomas, S., Tollis, I., Tsamardinos, I., van Beek, J. H. G. M., Viceconti, M., 2010. A
AN US
vision and strategy for the virtual physiological human in 2010 and beyond. Phil. Trans. R. Soc. A 368 (1920), 2595–2614.
Iglesias, J. E., Sabuncu, M. R., 2015. Multi-atlas segmentation of biomedical images: A survey. Med. Image Anal. 24 (1), 205 – 219.
M
Kass, M., Witkin, A., Terzopoulos, D., 1988. Snakes: Active contour models. Int. J. Comput. Vision 1 (4), 321–331.
ED
Lesage, D., Angelini, E. D., Bloch, I., Funka-Lea, G., 2009. A review of 3D vessel lumen segmentation techniques: Models, features and extraction
PT
schemes. Med. Image Anal. 13 (6), 819 – 845.
CE
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P., 1997. Multimodality image registration by maximization of mutual information.
AC
IEEE Trans. Med. Imag. 16 (2), 187–198.
McInerney, T., Terzopoulos, D., 1996. Deformable models in medical image analysis: a survey. Med. Image Anal. 1 (2), 91 – 108.
Morris, P. D., Narracott, A., von Tengg-Kobligk, H., Silva Soto, D. A., Hsiao, S., Lungu, A., Evans, P., Bressloff, N. W., Lawford, P. V., Hose, D. R., 19
ACCEPTED MANUSCRIPT
Gunn, J. P., 2016. Computational fluid dynamics modelling in cardiovascular medicine. Heart 102 (1), 18–28.
CR IP T
Olabarriaga, S., Smeulders, A., 2001. Interaction in the segmentation of medical images: A survey. Med. Image Anal. 5 (2), 127 – 142.
Palombi, O., Ulliana, F., Favier, V., L´eon, J.-C., Rousset, M.-C., 2014. My corporis fabrica: an ontology-based tool for reasoning and querying on
AN US
complex anatomical models. J. Biomed. Semant. 5 (1), 1–13.
Pluim, J. P. W., Maintz, J. B. A., Viergever, M. A., 2003. Mutualinformation-based registration of medical images: a survey. IEEE Trans. Med. Imag. 22 (8), 986–1004.
Pommert, A., H¨ohne, K. H., Pflesser, B., Richter, E., Riemer, M., Schie-
M
mann, T., Schubert, R., Schumacher, U., Tiede, U., 2001. Creating a highresolution spatial/symbolic model of the inner organs based on the visible
ED
human. Med. Image Anal. 5 (3), 221 – 228.
PT
Rengier, F., Mehndiratta, A., Tengg-Kobligk, H., Zechmann, C. M., Unterhinninghofen, R., Kauczor, H.-U., Giesel, F. L., 2010. 3D printing based
CE
on imaging data: review of medical applications. Int. J. Comput. Assist. Radiol. Surg. 5 (4), 335–341.
AC
Rosse, C., Mejino Jr., J. L. V., 2003. A reference ontology for biomedical informatics: the foundational model of anatomy. J. Biomed. Inform. 36 (6), 478 – 500.
Smith, N., de Vecchi, A., McCormick, M., Nordsletten, D., Camara, O., Frangi, A. F., Delingette, H., Sermesant, M., Relan, J., Ayache, N., 20
ACCEPTED MANUSCRIPT
Krueger, M. W., Schulze, W. H. W., Hose, R., Valverde, I., Beerbaum, P., Staicu, C., Siebes, M., Spaan, J., Hunter, P., Weese, J., Lehmann, H.,
CR IP T
Chapelle, D., Rezavi, R., 2011. euHeart: personalized and integrated cardiac care using patient-specific cardiovascular modelling. Interface Focus 1 (3), 349–364.
Taddei, F., Martelli, S., Valente, G., Leardini, A., Benedetti, M. G., Manfrini, M., Viceconti, M., 2012. Femoral loads during gait in a patient with
AN US
massive skeletal reconstruction. Clin. Biomech. 27 (3), 273 – 280.
Wells III, W. M., Viola, P., Atsumi, H., Nakajima, S., Kikinis, R., 1996. Multi-modal volume registration by maximization of mutual information.
AC
CE
PT
ED
M
Med. Image Anal. 1 (1), 35 – 51.
21