ARTICLE IN PRESS
JID: MEDIMA
[m5G;June 21, 2016;6:58]
Medical Image Analysis 0 0 0 (2016) 1–3
Contents lists available at ScienceDirect
Medical Image Analysis journal homepage: www.elsevier.com/locate/media
Editorial
Medical Image Analysis – past, present, and future William M. Wells III∗ Harvard Medical School and Brigham and Women’s Hospital, MIT CSAIL, United States
a r t i c l e
i n f o
Article history: Received 25 April 2016 Revised 8 June 2016 Accepted 13 June 2016 Available online xxx
a b s t r a c t In this editorial I summarize, against the backdrop of the research disciplines, meetings and journals of the time, the emergence in the early 1990s of the field that is eponymous with the present journal. I briefly summarize the current status of the field, and outline some possible future directions. © 2016 Elsevier B.V. All rights reserved.
Keywords: Medical Image Analysis Editorial
1. Introduction
2.2. Journals
I was delighted to receive an invitation from the editors to contribute a guest editorial on the 20th anniversary of the founding of the journal Medical Image Analysis, which is now the flagship publication venue for original contributions in the eponymous field. In the following I summarize my view of the origins of the field, its current status, and possible future directions.
There was at that time an established discipline that may be called Medical Image Processing that was represented by, e.g., the Institute of Electrical and Electronics Engineers (IEEE) Transactions on Medical Imaging, however, the emphasis was different; primary foci included tomographic reconstruction, and the heritage was somewhat more in the realm of Electrical Engineering than Computer Science. Much of the methodological developments that were applied in MIA, primarily from the field of Computer Vision, appeared in the IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), and the International Journal of Computer Vision (IJCV).
2. Origins 2.1. Computer Vision The field of Medical Image Analysis (MIA), is a spin-off from the field of Computer Vision, which historically has been a subdiscipline of Artificial Intelligence or Computer Science. The field emerged in the early 1990s when a group of young researchers, including our esteemed editors, and others with backgrounds in Computer Vision, began to apply to medical images the applied mathematical methods that had gained traction in solving nonmedical image analysis problems. As an enthusiast of probabilistic methods, I was happy to discover that the medical community had no problems with probability theory or statistics. At that time, there was some antipathy to probability in Computer Vision, perhaps stemming from excessive enthusiasm about predicate logic in the parent field of Artificial Intelligence. Thankfully, that antipathy has dissipated.
∗
Corresponding author. Fax: + 1 617 732 7963. E-mail address:
[email protected]
2.2.1. Meetings In the early 1990s Computer Vision also had vibrant established meetings, including the US-based Computer Vision and Pattern Recognition (CVPR) (Wikipedia, 2016a) as well as the International Conference on Computer Vision (ICCV) (Wikipedia, 2016b); both are IEEE meetings. The meetings were single track, and contributions were in the form of complete small papers that were rigorously peer-reviewed with acceptance rates of approximately one fourth to one third. There were also meetings that covered computation with medical images, including the Visualization in Biomedical Computing (VBC) meetings of 1990, 92, 94, and 96 (Höhne and Kikinis, 1996). These meetings touched on many of the problems MIA, though the emphasis was on visualization and graphics. In 1994 I hosted a workshop at the annual meeting of the American Association of Artificial Intelligence (AAAI) at Stanford University entitled “Applications of Computer Vision in Medical Image Processing (Wells, 1994).” Owing to fears of embarrassment
http://dx.doi.org/10.1016/j.media.2016.06.013 1361-8415/© 2016 Elsevier B.V. All rights reserved.
Please cite this article as: W.M. Wells III, W.M. Wells III Medical Image Analysis – past, present, and future, Medical Image Analysis (2016), http://dx.doi.org/10.1016/j.media.2016.06.013
JID: MEDIMA 2
ARTICLE IN PRESS
[m5G;June 21, 2016;6:58]
W.M. Wells III / Medical Image Analysis 000 (2016) 1–3
by limited attendance, I sent a fair amount of spam promoting the meeting, and much to my surprise, the attendance was about 75. I think this meeting may have been the real tipping point for the formation of our field; there was an impressive critical mass of young researchers, including our esteemed editors, who found they had much in common, to the extent that they wanted to start their own meeting. This led to Prof. Nicholas Ayache hosting the Computer Vision, Virtual Reality, and Robotics in Medicine (CVRMed) meeting in 1995 in Nice (Ayache, 1995). I was happy to attend the associated program committee meeting; there I was greatly relieved when I discovered that the hotel bill was paid by the organization, as at that time, my credit card was not working, and my cash was limited; such was life as a fresh Instructor in Radiology!
technical discussion, small size, and its nurturing of young scientists is unique; in my view, IPMI is the methodological heart and soul of MIA – much of the success here is due to the tireless efforts of Prof. Stephen Pizer at the University of North Carolina at Chapel Hill to integrate MIA into the IPMI meeting to integrate IPMI into the evolving MIA field. 3. Current status The field has certainly come a very long way since the humble beginnings represented at the AAAI meeting of 1994. The research has advanced a tremendous amount with many powerful capabilities emerging, e.g., in segmentation and registration of medical images, and representations of shapes of anatomical structures in individuals and populations, to name just a few.
2.3. Medical Image Analysis, the journal The new field was in need of an archival journal, and in 1996, Profs. Ayache and James Duncan obliged by founding the present journal Medical Image Analysis; published originally by Oxford University Press and currently by Elsevier. Medical Image Analysis is now the leading journal of the eponymous field. The journal gained traction immediately – I am happy to say that an article I published in the first issue has been well cited. 2.4. Robotics Like Computer Vision, the field of Robotics was seen as a subdiscipline of Artificial Intelligence, and in that context it seemed natural that sensing and action should go together. In the early 1990s there was similar ferment in the nascent field of Medical Robotics that led to the founding of the Medical Robotics and Computer-Assisted Surgery (MRCAS) meeting in 1994 in Pittsburgh, with a subsequent meeting in 1995 in Baltimore; Prof. Russell Taylor at John’s Hopkins University was a driving force behind these meetings. A joint meeting of CVRMed and MRCAS was held in Grenoble in 1997 (Troccaz et al., 1997). 2.5. MICCAI The VBC, CVRMed and MRCAS meetings were progenitors of a new international meeting, Medical Image Computing and Computer Assisted Intervention (MICCAI) that was launched after a series of discussions by an organizing group at the joint CVRMed/MRCAS meeting and subsequently by email; it was essentially a merger of VBC CVRMed and MRCAS. Eric Grimson of MIT and Ron Kikinis of Harvard Medical School took on organizing the first meeting in Boston in 1998. MICCAI was structured after the Computer Vision meetings of the time. To some, MICCAI was seen as the offspring of a “shotgun wedding” among the medical image analysts and medical roboticists. The union has led to impressive inter-disciplinary accomplishments, however, as is typical of siblings, there has been some healthy squabbling. MICCAI has evolved into the flagship meeting of Medical Image Analysis and, to some extent, of Medical Robotics; recent meetings have attracted more than 10 0 0 attendees. 2.6. IPMI The Information Processing in Medical Imaging (IPMI) workshop has also become an important part of the MIA culture. This semi-annual meeting which began in 1969 and has its roots in scintigrapraphic reconstruction, has become the primary venue for methodological innovations in our field. Its format of unlimited
3.1. Academic home Along with the successes there are a few challenging aspects of the field. One stems from the following question: what is the academic home for MIA? One possibility is Computer Science departments – many of them are happy to house researchers in Computer Vision, but it seems to be a more difficult sell for MIA, despite the fact that the methodologies are often very similar. The methodology can fit into Electrical Engineering, but again, placing junior faculty in those departments seems challenging. From my discussions with department chairs, it seems that there are not many junior hires in MIA in recent years, at least in the US. Another potential home is in Radiology departments of medical schools–there may be interest in facilitating, e.g., disease related image analysis projects, that are typically funded by the National Institutes of Health (NIH). While my own career is proof that this arrangement can work, there are cultural differences between Medicine and Computer Science or Electrical Engineering that complicate the enterprise. A significant advantage here is that it is easier to focus on real problems by way of close proximity to medical personnel. Put another way: there is a tendency for researchers in Computer Science and Electrical Engineering departments to become diverted working on toy problems. 3.2. Funding A significant current difficulty of the field stems from the prevailing climate for obtaining research grants from the NIH. The NIH budgets have been flat for some time, and the prospect of funding a career from such grants may seem quite daunting to young researchers entering the field, with the result that many, making a rational decision, choose to leave the field for greener pastures. 3.3. Tribalism I think it is fair to say that much of the low hanging fruit in MIA has already been picked, and I think the same is true in other fields, for example, Magnetic Resonance Imaging Physics. However, there are remaining opportunities at the interface of the two fields that require expertise from both. There can also be difficulties in working at the interface, e.g., there is a smaller number of reviewers that are qualified in both areas, which can make it more difficult to obtain good quality reviews of manuscripts. Because of differences in their fields of study and the departments they were trained in, there can be tensions among MIA researchers and their counterparts in MR Physics, sometimes these are evident in grant review panels. Some of this stems from unfamiliarity with the style of applied math used by the respective fields, which makes accepting its use more difficult.
Please cite this article as: W.M. Wells III, W.M. Wells III Medical Image Analysis – past, present, and future, Medical Image Analysis (2016), http://dx.doi.org/10.1016/j.media.2016.06.013
JID: MEDIMA
ARTICLE IN PRESS
[m5G;June 21, 2016;6:58]
W.M. Wells III / Medical Image Analysis 000 (2016) 1–3
3.4. Potential solutions Having more graduate training programs that run the gamut from imaging Physics to, e.g., Bayesian solutions of inverse problems could help with tribalism; hostilities might be less among graduates trained in the same program. In addition, forming a meeting that sits squarely at the interface, could help. It may be that some of the difficulties summarized above could be alleviated by way of an effective advocacy, or lobbying, group that promoted MIA to universities and the government. This might be an activity of, e.g., the MICCAI Society, though these issues may be best organized at a national level, while the MICCAI society has an international orientation. 4. Future directions Despite my polemical tone above, I think the field is in great shape and remains vibrant and exciting. “Big data” is becoming a reality with very large scale imaging projects underway or planned. This new scale of data is enabling the solution of challenging problems where the simplicity of methods can offset by the quantity of data available. There are very exciting opportunities at the interface of MIA and the field of Medical Informatics; however there a very few people currently working in both areas. The recent successes of “deep learning” in problem solving in our field is another trend to be reckoned with. Results in the field of Computer Vision have been astonishing; progress in MIA has been somewhat slower, perhaps because of practical difficulties in making the convolutional network structures work well on 3D data, within the constraints of current hardware; another challenge is the lack of the very large labeled data sets that are available in Computer Vision. Despite these limitations we are beginning to see very impressive results that challenge our understanding of the intrinsic difficulty of these problems. To some extent this trend represents a movement away from other technologies of machine learning, e.g., estimation using generative models. In a sense, earlier work strove to extract the best results from limited data by carefully handcrafting models; this my not be needed if enough training data is available. As the field has become lager and somewhat mature, it has also become a bit staid, particularly with respect to meetings. My impression is that the field of Computer Vision is currently leading
us in terms of innovation in the review process. Another shift is that in Computer Vision and machine learning there is a trend of “pre-publishing” work early and publicly in, e.g., ArXiv, with subsequent appearance in conference proceedings and journals. Practitioners have reported to me that this has substantially increased the velocity of research in those fields. It also seems that the purpose and formats of meetings and journals is in flux, and interesting hybrids are a possibility. The proceedings of some meetings, for example, CVPR, ICCV, NIPS, and MICCAI, have publication impact that is somewhat like journal articles, and the MICCAI proceedings are indexed by Medline, which is important in the medical arena; there is also a trend for conference papers to appear in special issues of journals. At the same time, the greatly reduced cost of electronic distribution has changed the landscape of journals, with a proliferation of non-profit and / or open electronic venues; the Journal of Machine Learning Research is notable example. Perhaps in the future the review process could take on aspects of social networking, with an independent system of endorsements by experts that establish the creditability of contributions. 5. Closing In my view the formation of MIA was revolutionary in nature, and my sense is that the time is approaching for another major paradigm shift that could be very exciting indeed. References Ayache, N., 1995. Computer Vision, Virtual Reality and Robotics in Medicine. Springer Science & Business Media. Höhne, K.H., Kikinis, R., 1996. Visualization in Biomedical Computing: 4th International Conference, VBC’96, Hamburg, Germany, September 22–25, 1996, Proceedings, 1131. Springer Science & Business Media. Troccaz, J., Grimson, E., Mösges, R., 1997. CVRMed-MRCAS’97: First Joint Conference Computer Vision, Virtual Reality and Robotics in Medicine and Medical Robotics and Computer-Assisted Surgery, Grenoble, France, March 19–22, 1997: Proceedings. Springer Science & Business Media. Wells, W. (Ed.), 1994, Proceedings of the AAAI Symposium on Medical Applications of Computer Vision. AAAI Press, Menlo Park. Technical Report SS-94-05 Wikipedia, 2016a. Conference on computer vision and pattern recognition — Wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/Conference_on_ Computer_Vision_and_Pattern_Recognition. [Online; accessed June 7, 2016]. Wikipedia, 2016b. International conference on computer vision — Wikipedia, the free encyclopedia. https://en.wikipedia.org/wiki/International_Conference_ on_Computer_Vision. [Online; accessed June 7, 2016].
Please cite this article as: W.M. Wells III, W.M. Wells III Medical Image Analysis – past, present, and future, Medical Image Analysis (2016), http://dx.doi.org/10.1016/j.media.2016.06.013
3