International Congress Series 1230 (2001) 372 – 378
Representing and sharing numeric and symbolic knowledge of brain cortex anatomy using web technology B. Gibaud*, O. Dameron, X. Morandi Laboratoire IDM, Faculte´ de Me´decine, Universite´ de Rennes I, France
Abstract We present a means to represent and share knowledge of brain cortex anatomy using web technologies and general standards and tools (e.g. XML, VRML, Java). This knowledge encompasses both symbolic features, i.e. descriptions based on language, and numeric features obtained from brain images such as models of the shapes of brain sulci. D 2001 Elsevier Science B.V. All rights reserved. Keywords: Brain atlases; Ontologies; Knowledge management
1. Purpose A priori knowledge in brain cortex anatomy and function provides precious help when planning and performing brain surgery. For many years now, brain atlases [1 –3] have been a very valuable knowledge source, contributing to overcome the limitations of brain imaging techniques, and assisting surgeons in the identification of anatomical and functional areas within the patient’s brain. Nowadays, neuro-imaging has made tremendous progress, and offers high-resolution images of brain anatomy (CT, MR) and function (PET, fMRI, MEG). Moreover, data fusion techniques help surgeons to better appreciate the functional environment of pathology. However, clinical requirements are ever greater, prompting surgeons to remove increasingly smaller lesions and to operate in the vicinity of high-risk brain areas. The use of atlases is therefore still highly relevant, helping surgeons * Corresponding author. Laboratoire IDM, Faculte de Medecine, UPRES-EA 3192, 2 Av. Prof. Leon Bernard, 35043 Rennes Cedex, France. Tel.: +33-2-99-33-68-66; fax +33-2-99-33-68-64. E-mail address:
[email protected] (B. Gibaud).
0531-5131/01/$ – see front matter D 2001 Elsevier Science B.V. All rights reserved. PII: S 0 5 3 1 - 5 1 3 1 ( 0 1 ) 0 0 0 7 8 - 4
B. Gibaud et al. / International Congress Series 1230 (2001) 372–378
373
understand the functional organisation of a brain area and keep a critical outlook on information provided by functional imaging techniques, not yet 100% reliable. Computerised systems significantly extend the capacities of classical paper-based atlases [4]. Great achievements have been made in this field over the past 5 years, in particular, by the teams of Nowinski [5], Ho¨hne [6,7], and Kikinis [8]. The work presented here focuses on sulco-gyral anatomy, described both symbolically (using description primitives based on language) and numerically (using description primitives extracted from anatomical images of actual brains). Our approach assumes that in the future, the web will be the main vector to disseminate digital knowledge. It also assumes that the utilisation of knowledge will no longer be limited to human actors (as it is now) but will increasingly involve software components (of decision support systems) as well. Consequently, knowledge structuring and representation must take syntactical and semantical interoperability issues into account. For this reason, we deemed it important to rely on well-known and well-accepted terminology such as UMLS (Unified Medical Language System, National Library of Medicine), and universal data syntaxes such as XML (eXtensible Markup Language [9]). Our work is a first step in this direction, its object being to demonstrate the relevance and feasibility of the approach. It consisted in representing both symbolic and numeric structured knowledge on sulco-gyral anatomy in the parietal, frontal and temporal regions, and showing how it can be shared and reused over the web.
2. Methods 2.1. Symbolic aspect Our objective was to define and describe abstractions of the most significant anatomical entities, and associate them with various properties such as nature of tissues, shape, location in brain space, or variability. Some of these properties can be represented by simple attributes but others involve relationships among several conceptual entities—‘‘is – a’’ relationship, partitive relationship, spatial relationships, etc. It is both important and difficult to formalise these entities and relationships soundly because coherence is critical with regards to future maintainability and extendibility of the knowledge corpus, and absolutely required in the perspective of computational processing and reasoning in decision support systems. It is worth noting that the conceptualisation stage is highly related to the analysis of existing medical vocabularies. Consequently, we paid particular attention to anatomy-related aspects of the UMLS project led by NLM [10]. The UMLS system aims at facilitating the search for and integration of biomedical data by grouping together different vocabularies, and articulating them around a common semantic network and metathesaurus. Since 1992, UMLS integrates the ‘‘Neuronames’’ system developed by Bowden and Martin at Seattle [11]. This terminology is composed of mutually exclusive entities, organised into a hierarchy defined upon a partitive relationship. It now includes about 850 English terms and around 4000 synonyms [12]. That is why we have formulated a conceptual model of the major entities related to sulcal and gyral anatomy using conceptual entities mentioned in ‘‘Neuronames,’’ on one
374
B. Gibaud et al. / International Congress Series 1230 (2001) 372–378
hand, and in the ‘‘Digital Anatomist’’ project, on the other hand. The latter project is led by Rosse and Brinkley at the University of Washington in Seattle, and aims at building an ontology (in the sense defined by Chandrasekaran et al. [13]) of the domain of macroscopic anatomy, as an enhancement and extension of the semantic network and metathesaurus of UMLS [14]. We used UML (Unified Modeling Language) as a modeling technique and formalism because it is increasingly considered as a standard. Entities and relations were first represented in UML class diagrams, and then translated into an XML DTD (Document Type Definition). A symbolic knowledge base was then produced by instantiating the previous models, with examples of sulci and gyri described in the Ono [15] and Szikla [1] atlases (with sulci and gyri belonging to the parietal, frontal and temporal regions). 2.2. Numeric aspect What we mean here by numeric anatomical knowledge (as opposed to symbolic) is information that describes the shape and/or the position in space of cerebral anatomical features extensively. It is numeric because it is generally extracted from 3D image data sets of real brain instances, using in vivo imaging (e.g. MRI). This information may describe a prototypical example (i.e. representative of a population of individuals) or provide statistical information, such as 3D probability maps (cf ICBM project—International Consortium for Brain Mapping [16]). Of course, symbolic and numeric knowledge are highly complementary. In this work, we used a numeric description model representing the 3D shape of cortical sulci in what was considered a prototypical individual. In this model, each sulcus is modeled by a 3D surface, which approximates the median surface of the sulcal fold. This surface was extracted automatically from an MRI 3D data set by means of a method called ‘‘active ribbon method,’’ developed by Le Goualher et al. [17]. The 3D scene containing the sulci is encoded according to the VRML standard (Virtual Reality Markup Language).
3. Results The results of this work are: (1) a set of UML class diagrams showing an ontology of the field of brain cortex anatomy (Fig. 1), and (2) demonstration software including a knowledge base and navigation software to browse and display it. 3.1. Ontology Our model defines conceptual entities such as Concept (with two subclasses: AnatomicalConcept and FunctionalConcept), which is a supertype of all anatomical and physical entities. With the Concept entity, various designations from different terminology systems can be associated with any anatomical entity. The model then characterises anatomical structures according to their anatomical type (i.e. substance, such as cerebrospinal fluid or grey matter) and their ‘‘arity’’ (single structure or paired structure, i.e. present in both brain hemispheres). Finally, the model defines entities such as Hemisphere, Lobe, Gyrus, Pars
375
Fig. 1. UML Class diagrams of brain cortex anatomy entities.
B. Gibaud et al. / International Congress Series 1230 (2001) 372–378
376
B. Gibaud et al. / International Congress Series 1230 (2001) 372–378
(i.e. part of a gyrus), or Sulcus, and represents various relationships associating them (fractionating a Lobe into Gyri, spatial relationships such as Gyri instances ‘‘separated-by’’ a Sulcus, for example). Compatibility with the UMLS and ‘‘Digital Anatomist’’ models is achieved through the PhysicalAnatomicalEntity, AnatomicalSpatialEntity and OrganSubdivision entities (entities from the ‘‘Digital Anatomist’’ anatomical ontology are denoted in italics). Sulcus appears in our model as a subclass of AnatomicalSpatialEntity, one of the three subtypes of PhysicalAnatomicalEntity. Similarly, entities like Hemisphere, Lobe, Gyrus, Pars are subclasses of OrganSubdivision. 3.2. Demonstration software Its achievement first consisted in instantiating the previous model with instances of gyri and sulci. We focused on a specific subset of brain gyri and sulci, having particular relevance with respect to pre-operative functional mapping, namely gyri and sulci of the parietal, frontal and temporal regions, corresponding to the motor, sensori-motor and language areas. This knowledge base was encoded as a valid XML file, i.e. conformant to the DTD derived from our conceptual model. We then implemented browsing software, initially in JAVA. This software presents the knowledge base as a hierarchy built upon the partitive relationship (Fig. 2a). Contextual menus enable the user to display the properties and relationships of a concept (designations according to various vocabularies, spatial relationships, points of view from which the entity can be displayed, etc.). The software performs simple inferences based on the model. Thus, lobe contiguity is deduced from gyri contiguity and the ‘‘anatomical part – of’’ relationship. The software manages the link between the symbolic and numeric descriptions of an entity. This lets the user visually identify each sulcus in the 3D space relatively to surrounding sulci. Each sulcus within the VRML scene is a selectable entity and is linked to the symbolic representation of this entity (properties). 3D display and user interaction possibilities (rotation, translation, zoom, pan) are implemented with JAVA 3D (Fig. 2b).
Fig. 2. (a) User interface of the application; (b) representation of 3D shapes of several brain sulci.
B. Gibaud et al. / International Congress Series 1230 (2001) 372–378
377
A second version of this software dedicated to browsing and visualisation was later developed using usual web browsers. It enabled us to translate the original XML knowledge base into a series of HTML files, showing the major properties of the knowledge base entities and performing hypertext links between concepts, based upon various relation types. This translation was very easy to implement, thanks to XSL files (eXtensible Style Language).
4. Conclusion We present a model to represent numeric and symbolic knowledge of cerebral cortex anatomy, model that is easy to extend and capable of referring to existing vocabularies and terminology systems such as UMLS. The use of web technology (in particular, XML) makes it possible to represent structured knowledge, with very low development costs resulting from reuse of existing software. The perspectives of this work are manifold: enhancement of the symbolic knowledge corpus, representation of other kinds of numeric knowledge (in particular, probability maps associated with anatomical features, statistical models of shape), links to instantiated data (descriptions of actual patient cases), and finally, use and assessment of added value in the context of planning surgery.
References [1] G. Szikla, G. Bouvier, T. Hori, Angiography of the Human Cortex, Springer-Verlag, Berlin, 1977. [2] G. Schaltenbrand, W. Wahren, Atlas for Stereotaxy of the Human Brain, Georg Thieme Verlag, Stuttgart, 1977. [3] J. Talairach, P. Tournoux, in: G.T. Verlag (Ed.), Co-Planar Stereotactic Atlas of the Human Brain, Thieme Medical Publishers, New York, 1988. [4] B. Gibaud, S. Garlatti, C. Barillot, E. Faure, Computerized brain atlases as decision support systems: a methodological approach, Artif. Intell. Med. 14 (1998) 83 – 100. [5] W.L. Nowinski, A. Fang, B.T. Nguyen, J.K. Raphel, L. Jagannathan, R. Raghavan, R.N. Brian, G.A. Miller, Multiple brain atlas database and atlas-based neuroimaging system, Comput.-Aided Surg. 2 (1997) 42 – 66. [6] A. Pommert, R. Schubert, M. Riemer, T. Schiemann, U. Tiede, K.H. Ho¨hne, Symbolic modelling of human anatomy for visualization and simulation, in: R.A. Robb (Ed.), Visualization in Biomedical Computing, Proc. SPIE 2359, SPIE, Rochester, MN, (1994) 412 – 423. [7] K.H. Ho¨hne, B. Pflesser, A. Pommert, M. Riemer, T. Schiemann, R. Schubert, U. Tiede, A ‘virtual body’ model for surgical education and rehearsal, Computer, (1996) 25 – 31 (Jan). [8] R. Kikinis, M.E. Shenton, D.V. Iosifescu, R.W. McCarley, P. Saiviroonporn, H.H. Hokama, A. Robatino, D. Metclaf, C.G. Wible, C.M. Portas, R.M. Donnino, F.A. Jolesz, A digital brain atlas for surgical planning, model-driven segmentation, and teaching, IEEE Trans. Visualization Comput. Graphics 2 (3) (1996) 232 – 241 (Sept). [9] World Wide Web Consortium, XML Homepage, http://www.w3.org/XML/. [10] National Library of Medicine, Overview of the UMLS Project, In: CDROM Documentation, 2 – 16. [11] D.M. Bowden, R.F. Martin, NeuroNames brain hierarchy, Neuroimage 2 (1995) 63 – 83. [12] RPRC Neurosciences Division, University of Washington, Seattle (WA), NeuroNames Template Atlas, http://rprcsgi.rprc.washington.edu/neuronames/. [13] B. Chandrasekaran, J.R. Josephson, V.R. Benjamins, What are ontologies and why do we need them, IEEE Intell. Syst. 1 (1999) 20 – 26, Jan – Feb, 1094-7167/99.
378
B. Gibaud et al. / International Congress Series 1230 (2001) 372–378
[14] C. Rosse, J.L. Mejino, B.R. Modayur, R. Jakobovitz, K.P. Hinshaw, J.F. Brinkley, Motivation and organizational principles for anatomical knowledge representation, JAMIA 5 (1) (1998) 17 – 40 (Jan – Feb). [15] M. Ono, S. Kubik, Abernathey, in: G.T. Verlag (Ed.), Atlas of the Cerebral Sulci, Thieme Medical Publishers, New York, 1990. [16] ICBM project, ICBM Consortium, http://nessus.loni.ucla.edu/icbm/index0.html. [17] G. Le Goualher, C. Barillot, Y. Bizais, Three-dimensional segmentation and representation of cortical sulci using active ribbons, Int. J. Pattern Recognit. Artif. Intell. 11 (8) (1997).