International Journal of Medical Informatics 47 (1997) 69 – 73
WWW creates new interactive 3D graphics and colIaborative environments for medical research and education S. Samothrakis a, T.N. Arvanitis b,*, A. Plataniotis a, M.D.J. McNeill a, P.F. Lister a
Uni6ersity of Sussex, Centre for VLSI and Computer Graphics, School of Engineering, Falmer, Brighton BN1 9QT, UK b Uni6ersity of Sussex, School of Cogniti6e and Computing Sciences, Falmer, Brighton BN1 9QH, UK
Abstract Virtual Reality Modelling Language (VRML) is the start of a new era for medicine and the World Wide Web (WWW). Scientists can use VRML across the Internet to explore new three-dimensional (3D) worlds, share concepts and collaborate together in a virtual environment. VRML enables the generation of virtual environments through the use of geometric, spatial and colour data structures to represent 3D objects and scenes. In medicine, researchers often want to interact with scientific data, which in several instances may also be dynamic (e.g. MRI data). This data is often very large and is difficult to visualise. A 3D graphical representation can make the information contained in such large data sets more understandable and easier to interpret. Fast networks and satellites can reliably transfer large data sets from computer to computer. This has led to the adoption of remote tale-working in many applications including medical applications. Radiology experts, for example, can view and inspect in near real-time a 3D data set acquired from a patient who is in another part of the world. Such technology is destined to improve the quality of life for many people. This paper introduces VRML (including some technical details) and discusses the advantages of VRML in application developing. © 1997 Elsevier Science B.V. Keywords: Virtual Reality Modelling Language; Interactive graphical environments; Medical research; Medical education
1. Introduction In the past the exchange of medical information between hospitals was time consuming, with a reliance on traditional mail (surface or air mail). High speed computer * Corresponding author.
networks not only, provide the means of high-speed electronic transfer of information but offer the possibility of user interactivity with information displayed in a three-dimensional (3D) graphical format [1]. VRML is a world-wide de facto standard for 3D software developers in the Internet. The language [2] supports animation for moving objects and
1386-5056/97/$17.00 © 1997 Elsevier Science B.V. All rights reserved. PII S 1 3 8 6 - 5 0 5 6 ( 9 7 ) 0 0 0 7 8 - 6
70
S. Samothrakis et al. / International Journal of Medical Informatics 47 (1997) 69–73
describes complex movements. The latter makes the language a powerful tool for virtual reality applications in medicine. Magnetic Resonance Imaging (MRI) data is an example of data which can be represented in a VRML format. MRI is one of the most advanced methods for non-invasively diagnostic imaging. Volume data set from MRI scans are scalar data of the form f(x, y, z).
1.1. 3D computer graphics and VRML VRML is a language which allows to build 3D worlds. An object is typically defined using 3D geometry. Basic primitives of objects are polygons. Each polygon is defined (in Cartesian space) by a set a set of (x, y, z)triplets representing the vertices of the polygon [1]. A polygon can have characteristics such as colour, material and texture. The colour is typically defined by a red, green, blue triple. The material can be wood, metal, glass etc. and is applied using a texture function. The properties of material can be reflective, diffuse, transparent. A 3D scene is composed of 3D objects and lights. Rendering functions create photo-realistic environments by simulating how light reflect on surfaces. A 3D scene can be seen from any angle and any position in 3D. The viewpoint and the view direction define the user position in 3D space [1]. In order to explore or interact with a 3D scene it is necessary to use a 3D viewer program. This program will read the VRML file and will create graphically a 3D scene. Typically it will also provide controls (such as thumbnails, scrollbars or mouse movements) for navigating in 3D. It is usual to be able to move and rotate objects, change the viewing position, or add lights. There are two types of VRML viewers—plug-in and the stand-alone viewers. Plug-ins which are programs integrated with others. An example
will be an integration of a VRML viewer with an Internet browser. You can still explore the Internet as usual but when a VRML file is detected it loads the plug in (the 3D viewer associated with the Internet browser) and additional controls appear on the screen for interacting in 3D. The standalone applications are separate programs which allow you to open VRML files locally or in the Internet. A VRML world can contain embedded links to other (distributed) worlds. This is one of the most important features of VRML in terms of storage efficiency. The VRML file can exist anywhere in the Internet and loaded on demand. The files can be stored in different computers. The links specify the location of each of the VRML files. Hence when a huge file is loaded the program will only load only a portion of it. The user does not know where the files are stored. While you view the 3D world the program identifies the links and transfers the data from one place to another.
1.2. Education and consultancy using multi-user en6ironments VRML servers present and manage multiuser VRML environments. These servers manage dynamically 3D data in a VRML format creating a site into a 3D world. It lets the user build or inspect 3D data. In this way visitors to the server site automatically visualise 3D data. The server data can exist locally or remotely and can be viewed and inspected by many clients simultaneously. VRML models can be used in lectures and training for students. The language provides animation, which can simulate, for example, human moves or rotate objects. In electronic journals text documents can contain links to VRML data. 3D representations can be viewed from any angle [3]. This makes it unnecessary to have many images in 2D for
S. Samothrakis et al. / International Journal of Medical Informatics 47 (1997) 69–73
viewing the same data from any angle. Links can be set anywhere in the Internet making the storage of huge databases efficient. It is necessary sometimes to ask for a specialist diagnosis in a distant place. Transferring 3D images gives better information and analysis of data. The high interest for medicine and Internet has already a variety of medical data such as brain scans which can be converted to a VRML representation. The use of a WWW graphical standard format, such as VRML, will motivate developers to expand their software for all 3D medical applications like never before.
1.3. Collaborati6e en6ironments Scientists can collaborate by interacting and visualising 3D data sets from any geographical position [7]. In single user machines any application can run remotely somewhere else but there is no interaction between users. The goal of a collaborative system is to share data, views and manipulate 3D data interactively. There is a review of a recent collaboration project in [8]. Since VRML is a relatively new language, we have not seen collaborative environments using VRML. Collaboration is difficult to implement and there are some fundamental problems to solve: The creation of workgroups and the communication. There are many requirements for developing such a system: Synchronisation, private and shared workspaces, data protection, sharing and permissions, independence in some collaboration tasks, and the tools which perform these tasks. The following part of the paper, will introduce advanced features of VRML and the part entitled ‘Using VRML in applications’ will discuss the advantages of VRML and the capabilities offered for application development.
71
2. Advanced features of VRML for moving worlds VRML specifications change from year to year. New proposals are frequently submitted in an effort to make this language more powerful. The current version VRML 2.0 adds capabilities for animation which was not supported in the first version. There are two ways for moving worlds in VRML. The first is the interpolators, a set of values which specifies the path of animation in time. The second way is to use a programming language. Some actions are too complex for interpolators, shapes, etc. The computation of animation paths, algorithmic shapes and the creation of collaborative environments require to write algorithms in a language. Programs can be written in Java or Javascript which accept events as inputs and generate events as outputs. The Script languages provide an interface between VRML languages. The communication between the programs and VRML is done using events. An event is a message that contains a data value.
2.1. Network distribution using VRML and COBRA Because this paper is focused in the Internet and VRML which supports Java scripting we will discuss object oriented network distribution supported by the Java language. Java recently integrated object oriented network distribution. The Common Object Request Broker Architecture (CORBA) it is the union of object oriented technology with network distribution. CORBA defines an architecture for communication between objects. Older technologies involved the use of sockets or RPC for client server communication. The goal is to create a standard infrastructure for interaction between objects which are
72
S. Samothrakis et al. / International Journal of Medical Informatics 47 (1997) 69–73
reusable, efficient, transparent, reliable and scalable. In CORBA the components of distributed programming are objects. The objects have well defined interfaces. Each object is associated with a server and has a unique identifier which is used to locate the object in the network. Each server can have any number of objects for communication with its clients. In a distributed software system there are objects running on client and servers. When an object calls an operation on another then the first is the client and the second is the server. A server can make operation calls and thereby act as a client. CORBA have the characteristics of Object Oriented Programming (OOP) encapsulation, abstraction, inheritance and polymorphism. In contrast in a network procedural programming the user has to partition the program data and the functions then distribute to a client – server system and finally use appropriate function calls.
2.2. Using VRML in applications Volume rendering is a typical example of many 3D applications in medicine. We emphasise the object oriented network distribution and the advantages of using VRML.
2.3. Background for 6olume rendering Volume rendering [5] takes scalar or vector data sampled in three dimensions. The methods for rendering can be categorized into surface rendering and direct volume rendering. Geometric surface rendering gets data in a 3D scalar or vector polygons and then renders a scene with a rendering algorithm. Direct volume rendering uses the original data points for the final rendering. Ray tracing provides high quality rendering but is very time consuming. Ray tracing fires rays
for all pixels on the screen from a viewpoint towards the 3D model (collection of polygons). Each ray reflects to surface and the final colour is the average of all these colours. Volume data is a 3D data set of scalar or vector values representing cube areas called voxels.
2.4. Database design The Intranet offers a flexible way for storing data on a network. Intranet is accessed by people of one organisation such as hospitals. An Intranet connects a large number of computers. Each one can provide storage facilities for different parts of database. Many megabytes of data can be transferred from one computer to another in seconds or a few minutes. The database may contain Gigabytes of volumetric data stored transparently on the Intranet. Transparency means that the user do not know where the data is stored but only access them by its name. VRML will automatically transfer a large data set by using its ability to link 3D data. Each machine requires part of the database for rendering. Since one copy of the database exists in a server, all clients access server memory. Using a distributed model Like CORBA the objects are called by their name. The application is built on top of CORBA. The application needs only to specify the objects for distribution. Programming with CORBA is easy and robust because it is a well defined interface providing many methods for network distribution. It has also all the advantages of object oriented programming discussed in the previous section.
2.5. Application for rendering A program written in any language (C, Pascal etc.) can take as input these data in a
S. Samothrakis et al. / International Journal of Medical Informatics 47 (1997) 69–73
VRML format. VRML 2.0 supports scripting environments. An Internet language can be used for writing software for a volume rendering application. Java and Javascript is supported today by the most Internet browsers in the market. This program will process the VRML file and store it in memory. Then the rendering algorithm will produce the final image on the screen.
2.6. Resources required SuperJanet is a large academic network which connects the universities in UK and other research institutions like hospitals. Network speeds start at hundreds of megabytes/ s. Current developments in communications can achieve data transfer of gigabytes/s. Medical visualisation [4,6] data ranges from kilobytes to gigabytes. The size of these data is a challenge both for hardware architectures as also for software techniques. Better and faster algorithms are presented as more researchers use visualisation data. Massively scalable architectures provide a solution for rendering large sets of data. It would be ideal for researchers to work with personal computers. Today powerful processors, hardware graphics accelerators and larger memory capabilities bring 3D graphics to PCs. The Internet provides a huge database for most 3D medical applications. Furthermore its structure allows us to combine the computation power of all computers connected.
3. Conclusions In this paper we introduced concepts of new computer technologies to people with a .
73
medical background. For example languages integrated with VRML, such as Java. Java supports an object oriented network distribution—a new way of programming for the Internet distributed applications. Demanding applications in computation time like volume rendering can benefit by these technologies. We also discussed the use of VRML in education and consultancy using multi-user environments. In the next years we expect high performance collaborative environments to be developed for medical applications.
References [1] J.D. Foley, A. Van Dam, S.K. Feiner, J.F. Hughes. Computer Graphics Principle and Practice, 2nd edn., Addison Wesley, Reading, MA, 1990. [2] A.L. Ames, D.R. Nadeau, J.L. Moreland, The VRML SourceBook 2.0 Canada, Wiley, New York, 1996. [3] O. Casher, C. Leach, C.S. Page, H.S. Rzepa, Advanced VRML Based Chemistry Applications: A 3D Molecular Hyperglossary, Journal of Molecular Structure (Theochem) 368 (1996) 49 – 56. [4] K.W. Brodlie, L.A. Carpenter, R.A. Earnshaw et al. (Eds.), Scientific Visualization: Techniques and Applications, Springer-Verlag, Berlin, 1992. [5] K.-L. Ma, J.S. Painter, C.D. Hansen, M.F. Krogh, A Data Distributed, Parallel Algorithm for RayTraced Volume Rendering 15 – 22 Parallel Rendering Symposium, Proceedings of ACM SIGGRAPH, California, 1993, p. 15 – 22. [6] N. Gershon, J.R. Brown, The Role of Computer Graphics and Visualization in the Global Information Infrastrure, IEEE Computer Graphics and Applications 16 (2) (1996) 60 – 61. [7] N. Gershon, Moving Happily Through the World Wide Web, IEEE Computer Graphics and Applications 16 (2) (1996) 72 – 75. [8] B. Fro¨hlich, M. Agrawala, A. Beers, P. Hanrahan, Collaborative production modelling and planning, IEEE Computer Graphics and Applications 17 (4) (1997) 13 – 15.