- Email: [email protected]
Dynamic Holographic Endoscopy: New Perspectives in Minimally Invasive Diagnostics
Med. Laser Appl. 17: 59–64 (2002) © Urban & Fischer Verlag http://www.urbanfischer.de/journals/lasermed
Dynamic Holographic Endoscopy: New Perspectives in Minimally Invasive Diagnostics GERT VON BALLY, BJÖRN KEMPER, and SABINE KNOCHE* Laboratory of Biophysics, University of Muenster, Germany
Summary An endoscopic holographic camera system based on a dynamic holographic technique, i.e. Electronic-Speckle-Pattern Interferometry (ESPI), is presented which can be applied to in-vitro and in-vivo minimally invasive medical diagnostics as well as to examinations of technical objects. Integration of optical fibers for the guidance of a cwlaser beam and an endoscopic imaging system yield a compact ESPI system which opens up new possibilities for highly sensitive interferometric intra-cavity inspection under hand-held conditions with dynamic on-line (videoframe-rate) information presentation of interferograms. A CCD-camera in combination with a fast frame-grabber system allows dynamic image subtractions up to a frequency of 25 Hz with high fringe contrast. Results from investigations of biological specimen in-vitro and in-vivo in the fields of gastroenterology and coloscopy are obtained, demonstrating the capability for structural differentiation by analysis of tissue elasticity even underneath the visible surface as well as for compensation of the loss of tactile sense impression in endoscopic surgery by dynamic optical information (“endoscopic taction”).
Key words Dynamic holographic endoscopy, ESPI, endoscopic taction
Introduction Endoscopy is a wide spread subjective intra-cavity observation technique routinely used in medical minimally invasive diagnostics and industrial inspection. Combination of holographic interferometric metrology with endoscopic imaging allows the development of a special class of instruments for non-destructive quantitative diagnostics within body cavities (4). Applications include the analysis of structure, form, deformation, and vibration of the object. This opens up new perspectives in minimal-invasive diagnostics, especially to obtain information about structural differ-
ences by means of analysis of tissue elasticity even underneath the visible surface. In addition, the loss of tactile sense impression in endoscopic surgery can be compensated by optical information in this way (“endoscopic taction”). Modern diagnostic techniques require on-line process analysis with video frame rate (i.e. 25 Hz). The development of digital imaging, digital holographic interferometry, Electronic-SpecklePattern Interferometry (ESPI) (3), and micro optics allow to develop new techniques of fringe processing by endoscopy. Here, a compact transportable endoscope ESPI camera system with an external (proximal) holographic camera using standard endoscope optics is pre-
* Clinical investigations were performed in co-operation with the Department of Medicine B of the Medical Centre of the University of Münster (W. Avenhaus, MD; W. Domschke, Prof.) and the Department of Surgery of the Medical Centre of the University of Münster (D. Tübergen, MD; N. Senninger, Prof.) Financial support by the German Ministry for Education and Research (BMBF) and Karl Storz GmbH & Co., Tuttlingen , Germany is gratefully acknowledged. 1615-1615/02/17/01-059 $ 15.00/0
60
G. VON BALLY et al.
sented (1, 2). Using a cw-laser, “on-line” information about motions and deformations obtained by analyzing speckle correlation patterns with the double exposure image subtraction method in video real time under hand-held conditions in-vivo.
The dynamic holographic camera system
to an image processing system (BV/PC) which allows the subtraction of two video frames at a rate up to 25 Hz by a fast arithmetic unit. For double exposure image subtraction of two subsequent images in video real time the light pulses must be placed variably within the signal frames to allow different time delays between two exposures. Furthermore, the exposure times themselves have to be variable. Therefore, a double pulse generator (DG), which is synchronized to the vertical delay signal of the CCD-camera, controls an acousto-optic modulator (AOM) for light modulation. The image frame rate of the speckle correlation patterns is only limited by the camera frame acquisition rate. Fig. 3 shows the compact transportable system as used in a clinical environment (gastroenterology).
Fig. 1 shows the schematic setup of the endoscope ESPI camera system. An Argon-ion-laser (maximum output power 1.5 Watt, green 514.5 nm line) is used as light source. The laser beam is split and coupled into mono-mode optical fibers for object illumination and reference wave guidance. The object beam can be expanded by a microlens system or a rigid optical multimode fiber bundle. The reference beam directly illuminates the CCD-sensor of a progressive-scan grayscale camera (Sony XC-8500CE) with a resolution of 782 × 582 pixels. The size of the pixels is 8.3 µm × 8.3 µm. For endoscopic imaging a standard Hopkins rod lens system (e.g. Storz otoscope, Storz laparoscope) or a flexible fibrescope is connected to an adapter, which includes a beam splitter for joining the object and reference beams. A polarisation filter in front of the camera objective reduces the scattered light of the object, which is especially important while investigating biological specimens. Fig. 2 shows a photograph of a prototype developed at the Laboratory of Biophysics in cooperation with Karl Storz GmbH & Co., Tuttlingen, Germany. The endoscope ESPI camera system is linked
Fig. 2. Holographic camera system as developed at the Laboratory of Biophysics in cooperation with Karl Storz GmbH & Co., Tuttlingen, Germany (here attached to an otoscope with a Hopkins rod lens system).
Fig. 1. Setup for dynamic holographic endoscopy (for details see text).
Fig. 3. Compact transportable dynamic holographic endoscope system as used in gastroenterology.
Dynamic Holographic Endoscopy
Applications and Results First in-vitro experiments with the presented system have been carried out on mucous membranes. These specimens represent typical measuring conditions in organ cavities especially with respect to difficulties caused by wet surfaces, spatial inhomogeneous surface roughness, and low interferometric stability in time resulting in speckle decorrelation. Fig. 4 shows the results of endoscopic investigations of a porcine stomach in-vitro. An important test for the endoscope ESPI system is to verify the ability to differentiate between areas of tissue with different elasticity. Therefore, tissue elasticity was altered by a rigid implantation underneath the visible surface into the muscularis
61
propria layer and locally dissecting the tunica muscularis, while leaving the mucosal surface intact. As shown in Fig. 4 the response to the mechanical stimulation by a test needle allows a clear differentiation between the “healthy” (soft, Fig. 4b) and a manipulated (hard, Fig. 4 d) region underneath the visible surface according to the differences in the fringe patterns. The soft area causes circular patterns while the rigid area results in parallel fringes. Conventional white light endoscopic recordings (Fig. 4a and c) give no indication to such underlaying structural differences (!). A flexible glass fiber endoscope rather than a rigid laparoscope was used in a further series of experiments on porcine stomachs. With the same experimental setup, again gastric wall elasticity was assessed. Repre-
Fig. 4. Inspection of biological specimen in-vitro (porcine stomach, mucous membrane) by a rigid laparoscope: conventional endoscopic white light images of “healthy” tissue (a) and a manipulated rigid area underneath the visible surface (c); correlation fringe patterns of the corresponding stimulated areas (b), (d) (for further details see text).
62
G. VON BALLY et al.
sentative white light images of normally appearing porcine gastric mucosa without and with underlaying alteration are shown in Figs. 5a and c, respectively. The corresponding speckle correlation patterns are shown in Fig. 5b and d. Corresponding to the results obtained with a rigid standard laparoscope, deformations of parts of the unaltered ex-vivo stomach lead to high contrast circular correlation fringes (Fig. 5b), whereas deformation of areas of the gastric wall with focally disturbed elasticity result in parallel fringes (Fig. 5d). In Figures 6a and 6b first in-vivo investigations of a human hand are presented (laparoscope, progressive-scan mode, image repetition rate 12.5 Hz, exposure time t1 = 0.5 ms, delay between two exposures t2 = 5 ms). The endoscope camera system was used hand-held with a single support at 1 cm from the tip of the endoscope simulating
the clinical situation where the tip of the endoscope is inserted into a body cavity. The fingertip as well as the fingernail are stimulated by a test needle. The elasticity of the specimen (fingertip: soft, fingernail: hard) can be clearly differentiated by observing the different fringe patterns. The rigid fingernail is just tilted (parallel fringes) while the fingertip shows circular fringes as response, demonstrating the capability for compensation of the loss of tactile sense impression in endoscopic surgery by dynamic holographic optical information (“endoscopic taction”). Clinical in-vivo tests of the dynamic holographic interferometric endoscopy system were carried out on the human colon. Fig. 7 shows resulting endoscopic interferograms of the colon wall taken during routine coloscopy with a flexible coloscope freely hand-held
Fig. 5. Inspection of biological specimen in-vitro (porcine stomach, mucous membrane) as in Fig. 4 but here by a flexible fiberscope: white light images of “healthy” tissue (a) and a manipulated rigid area underneath the visible surface (c); correlation fringe patterns of the corresponding stimulated areas (b), (d) (for further details see text).
Dynamic Holographic Endoscopy
63
Fig. 6. In-vivo investigation of a human hand: fingertip (a) and fingernail (b) stimulated by a test needle, for further details see text.
Fig. 7. Dynamic holographic endoscopic recordings taken during routine coloscopy.
by the operator. The change in interference patterns shows the movement of a blood vessel in the observation area.
Discussion and conclusion A new compact transportable dynamic holographic endoscope ESPI camera system has been developed which can be used for on-line intra-cavity examinations in minimally invasive medical diagnostics as well as in industrial environments under hand-held conditions. This method adds a new metrological feature to endoscopic minimally invasive diagnostics. It can enhance conventional endoscopic examinations
by highly sensitive optical analysis of local differences in tissue elasticity even underneath the visible surface as well as by substituting the operator’s tactile sense by visual interferometric information (“endoscopic taction”). These results can be transferred also to intra-cavity inspection in a technical environment. Dynamisch holographische Endoskopie: neue Perspektiven in der minimalinvasiven Diagnostik Es wird ein endoskopisch holographisches Kamerasystem basierend auf einer dynamisch holographischen Technik, der sogenannten Electronic-Speckle-Pattern-Interferometrie (ESPI),
64
G. VON BALLY et al.
vorgestellt, das in der in-vitro und in-vivo minimalinvasiven medizinischen Diagnostik sowie bei Untersuchungen technischer Objekte eingesetzt werden kann. Die Integration von optischen Fasern zur CW-Laserstrahlführung und endoskopischen Abbildungssystemen ermöglicht den Aufbau eines kompakten in der Hand zu haltenden ESPI-Systems. Dieses eröffnet neue Möglichkeiten für die hochempfindliche interferometrische Hohlrauminspektion mit dynamischer on-line (Videowiederholrate) Information aus holographischen Interferogrammen. Eine CCD-Kamera in Kombination mit einem schnellen „Frame Grabber“ ermöglicht eine dynamische Bildsubtraktion bis zu einer Frequenz von 25 Hz mit hohem Streifenkontrast. Ergebnisse von in-vitro und in-vivo Untersuchungen in den Bereichen Gastroenterologie und Koloskopie zeigen die Fähigkeit zur Strukturdifferenzierung durch Analyse der lokalen Gewebeelastizität sogar unterhalb der sichtbaren Oberfläche als auch zur Kompensation des Tasteindruckverlustes in der endoskopischen Chirurgie durch dynamisch holographische optische Information („Endoskopisches Fühlen“).
Schlüsselwörter Dynamisch holographische Endoskopie, ESPI, endoskopisches Fühlen
References 1. AVENHAUS W, KEMPER B, VON BALLY G, DOMSCHKE W: Gastric wall elasticity assessed by dynamic holographic endoscopy: ex vivo investigations in the porcine stomach. Gastrointestinal Endoscopy 54: 496–500 (2001) 2. KEMPER B, DIRKSEN D, AVENHAUS W, MERKER A, VON BALLY G: Endoscopic double-pulse electronic-speckle-pattern interferometer for technical and medical intracavity inspection. Appl. Opt. 39: 3899–3905 (2000) 3. KREIS T: Holographic Interferometry: Principles and Methods. Vol. 1 of Series in Optical Metrology, Akademie Verlag, 1996 4. VON BALLY G, BJELKHAGEN HI: Techniques for endoscopic holography, in Optics in Medicine, Biology and Environmental Research, Optics Within Life Sciences (OWLS I), G. von Bally and S. Khanna, eds., Elsevier Science, Amsterdam, 1993, pp. 13–21 Correspondence address: Prof. h.c. Gert von Bally, Laboratory of Biophysics, University of Münster, Robert-Koch-Str. 45, D-48129 Münster, Germany Tel.: ++49-251-8356888; Fax: ++49-251-835 8536; e-mail: [email protected]
Dynamic Holographic Endoscopy: New Perspectives in Minimally Invasive Diagnostics GERT VON BALLY, BJÖRN KEMPER, and SABINE KNOCHE* Laboratory of Biophysics, University of Muenster, Germany
Summary An endoscopic holographic camera system based on a dynamic holographic technique, i.e. Electronic-Speckle-Pattern Interferometry (ESPI), is presented which can be applied to in-vitro and in-vivo minimally invasive medical diagnostics as well as to examinations of technical objects. Integration of optical fibers for the guidance of a cwlaser beam and an endoscopic imaging system yield a compact ESPI system which opens up new possibilities for highly sensitive interferometric intra-cavity inspection under hand-held conditions with dynamic on-line (videoframe-rate) information presentation of interferograms. A CCD-camera in combination with a fast frame-grabber system allows dynamic image subtractions up to a frequency of 25 Hz with high fringe contrast. Results from investigations of biological specimen in-vitro and in-vivo in the fields of gastroenterology and coloscopy are obtained, demonstrating the capability for structural differentiation by analysis of tissue elasticity even underneath the visible surface as well as for compensation of the loss of tactile sense impression in endoscopic surgery by dynamic optical information (“endoscopic taction”).
Key words Dynamic holographic endoscopy, ESPI, endoscopic taction
Introduction Endoscopy is a wide spread subjective intra-cavity observation technique routinely used in medical minimally invasive diagnostics and industrial inspection. Combination of holographic interferometric metrology with endoscopic imaging allows the development of a special class of instruments for non-destructive quantitative diagnostics within body cavities (4). Applications include the analysis of structure, form, deformation, and vibration of the object. This opens up new perspectives in minimal-invasive diagnostics, especially to obtain information about structural differ-
ences by means of analysis of tissue elasticity even underneath the visible surface. In addition, the loss of tactile sense impression in endoscopic surgery can be compensated by optical information in this way (“endoscopic taction”). Modern diagnostic techniques require on-line process analysis with video frame rate (i.e. 25 Hz). The development of digital imaging, digital holographic interferometry, Electronic-SpecklePattern Interferometry (ESPI) (3), and micro optics allow to develop new techniques of fringe processing by endoscopy. Here, a compact transportable endoscope ESPI camera system with an external (proximal) holographic camera using standard endoscope optics is pre-
* Clinical investigations were performed in co-operation with the Department of Medicine B of the Medical Centre of the University of Münster (W. Avenhaus, MD; W. Domschke, Prof.) and the Department of Surgery of the Medical Centre of the University of Münster (D. Tübergen, MD; N. Senninger, Prof.) Financial support by the German Ministry for Education and Research (BMBF) and Karl Storz GmbH & Co., Tuttlingen , Germany is gratefully acknowledged. 1615-1615/02/17/01-059 $ 15.00/0
60
G. VON BALLY et al.
sented (1, 2). Using a cw-laser, “on-line” information about motions and deformations obtained by analyzing speckle correlation patterns with the double exposure image subtraction method in video real time under hand-held conditions in-vivo.
The dynamic holographic camera system
to an image processing system (BV/PC) which allows the subtraction of two video frames at a rate up to 25 Hz by a fast arithmetic unit. For double exposure image subtraction of two subsequent images in video real time the light pulses must be placed variably within the signal frames to allow different time delays between two exposures. Furthermore, the exposure times themselves have to be variable. Therefore, a double pulse generator (DG), which is synchronized to the vertical delay signal of the CCD-camera, controls an acousto-optic modulator (AOM) for light modulation. The image frame rate of the speckle correlation patterns is only limited by the camera frame acquisition rate. Fig. 3 shows the compact transportable system as used in a clinical environment (gastroenterology).
Fig. 1 shows the schematic setup of the endoscope ESPI camera system. An Argon-ion-laser (maximum output power 1.5 Watt, green 514.5 nm line) is used as light source. The laser beam is split and coupled into mono-mode optical fibers for object illumination and reference wave guidance. The object beam can be expanded by a microlens system or a rigid optical multimode fiber bundle. The reference beam directly illuminates the CCD-sensor of a progressive-scan grayscale camera (Sony XC-8500CE) with a resolution of 782 × 582 pixels. The size of the pixels is 8.3 µm × 8.3 µm. For endoscopic imaging a standard Hopkins rod lens system (e.g. Storz otoscope, Storz laparoscope) or a flexible fibrescope is connected to an adapter, which includes a beam splitter for joining the object and reference beams. A polarisation filter in front of the camera objective reduces the scattered light of the object, which is especially important while investigating biological specimens. Fig. 2 shows a photograph of a prototype developed at the Laboratory of Biophysics in cooperation with Karl Storz GmbH & Co., Tuttlingen, Germany. The endoscope ESPI camera system is linked
Fig. 2. Holographic camera system as developed at the Laboratory of Biophysics in cooperation with Karl Storz GmbH & Co., Tuttlingen, Germany (here attached to an otoscope with a Hopkins rod lens system).
Fig. 1. Setup for dynamic holographic endoscopy (for details see text).
Fig. 3. Compact transportable dynamic holographic endoscope system as used in gastroenterology.
Dynamic Holographic Endoscopy
Applications and Results First in-vitro experiments with the presented system have been carried out on mucous membranes. These specimens represent typical measuring conditions in organ cavities especially with respect to difficulties caused by wet surfaces, spatial inhomogeneous surface roughness, and low interferometric stability in time resulting in speckle decorrelation. Fig. 4 shows the results of endoscopic investigations of a porcine stomach in-vitro. An important test for the endoscope ESPI system is to verify the ability to differentiate between areas of tissue with different elasticity. Therefore, tissue elasticity was altered by a rigid implantation underneath the visible surface into the muscularis
61
propria layer and locally dissecting the tunica muscularis, while leaving the mucosal surface intact. As shown in Fig. 4 the response to the mechanical stimulation by a test needle allows a clear differentiation between the “healthy” (soft, Fig. 4b) and a manipulated (hard, Fig. 4 d) region underneath the visible surface according to the differences in the fringe patterns. The soft area causes circular patterns while the rigid area results in parallel fringes. Conventional white light endoscopic recordings (Fig. 4a and c) give no indication to such underlaying structural differences (!). A flexible glass fiber endoscope rather than a rigid laparoscope was used in a further series of experiments on porcine stomachs. With the same experimental setup, again gastric wall elasticity was assessed. Repre-
Fig. 4. Inspection of biological specimen in-vitro (porcine stomach, mucous membrane) by a rigid laparoscope: conventional endoscopic white light images of “healthy” tissue (a) and a manipulated rigid area underneath the visible surface (c); correlation fringe patterns of the corresponding stimulated areas (b), (d) (for further details see text).
62
G. VON BALLY et al.
sentative white light images of normally appearing porcine gastric mucosa without and with underlaying alteration are shown in Figs. 5a and c, respectively. The corresponding speckle correlation patterns are shown in Fig. 5b and d. Corresponding to the results obtained with a rigid standard laparoscope, deformations of parts of the unaltered ex-vivo stomach lead to high contrast circular correlation fringes (Fig. 5b), whereas deformation of areas of the gastric wall with focally disturbed elasticity result in parallel fringes (Fig. 5d). In Figures 6a and 6b first in-vivo investigations of a human hand are presented (laparoscope, progressive-scan mode, image repetition rate 12.5 Hz, exposure time t1 = 0.5 ms, delay between two exposures t2 = 5 ms). The endoscope camera system was used hand-held with a single support at 1 cm from the tip of the endoscope simulating
the clinical situation where the tip of the endoscope is inserted into a body cavity. The fingertip as well as the fingernail are stimulated by a test needle. The elasticity of the specimen (fingertip: soft, fingernail: hard) can be clearly differentiated by observing the different fringe patterns. The rigid fingernail is just tilted (parallel fringes) while the fingertip shows circular fringes as response, demonstrating the capability for compensation of the loss of tactile sense impression in endoscopic surgery by dynamic holographic optical information (“endoscopic taction”). Clinical in-vivo tests of the dynamic holographic interferometric endoscopy system were carried out on the human colon. Fig. 7 shows resulting endoscopic interferograms of the colon wall taken during routine coloscopy with a flexible coloscope freely hand-held
Fig. 5. Inspection of biological specimen in-vitro (porcine stomach, mucous membrane) as in Fig. 4 but here by a flexible fiberscope: white light images of “healthy” tissue (a) and a manipulated rigid area underneath the visible surface (c); correlation fringe patterns of the corresponding stimulated areas (b), (d) (for further details see text).
Dynamic Holographic Endoscopy
63
Fig. 6. In-vivo investigation of a human hand: fingertip (a) and fingernail (b) stimulated by a test needle, for further details see text.
Fig. 7. Dynamic holographic endoscopic recordings taken during routine coloscopy.
by the operator. The change in interference patterns shows the movement of a blood vessel in the observation area.
Discussion and conclusion A new compact transportable dynamic holographic endoscope ESPI camera system has been developed which can be used for on-line intra-cavity examinations in minimally invasive medical diagnostics as well as in industrial environments under hand-held conditions. This method adds a new metrological feature to endoscopic minimally invasive diagnostics. It can enhance conventional endoscopic examinations
by highly sensitive optical analysis of local differences in tissue elasticity even underneath the visible surface as well as by substituting the operator’s tactile sense by visual interferometric information (“endoscopic taction”). These results can be transferred also to intra-cavity inspection in a technical environment. Dynamisch holographische Endoskopie: neue Perspektiven in der minimalinvasiven Diagnostik Es wird ein endoskopisch holographisches Kamerasystem basierend auf einer dynamisch holographischen Technik, der sogenannten Electronic-Speckle-Pattern-Interferometrie (ESPI),
64
G. VON BALLY et al.
vorgestellt, das in der in-vitro und in-vivo minimalinvasiven medizinischen Diagnostik sowie bei Untersuchungen technischer Objekte eingesetzt werden kann. Die Integration von optischen Fasern zur CW-Laserstrahlführung und endoskopischen Abbildungssystemen ermöglicht den Aufbau eines kompakten in der Hand zu haltenden ESPI-Systems. Dieses eröffnet neue Möglichkeiten für die hochempfindliche interferometrische Hohlrauminspektion mit dynamischer on-line (Videowiederholrate) Information aus holographischen Interferogrammen. Eine CCD-Kamera in Kombination mit einem schnellen „Frame Grabber“ ermöglicht eine dynamische Bildsubtraktion bis zu einer Frequenz von 25 Hz mit hohem Streifenkontrast. Ergebnisse von in-vitro und in-vivo Untersuchungen in den Bereichen Gastroenterologie und Koloskopie zeigen die Fähigkeit zur Strukturdifferenzierung durch Analyse der lokalen Gewebeelastizität sogar unterhalb der sichtbaren Oberfläche als auch zur Kompensation des Tasteindruckverlustes in der endoskopischen Chirurgie durch dynamisch holographische optische Information („Endoskopisches Fühlen“).
Schlüsselwörter Dynamisch holographische Endoskopie, ESPI, endoskopisches Fühlen
References 1. AVENHAUS W, KEMPER B, VON BALLY G, DOMSCHKE W: Gastric wall elasticity assessed by dynamic holographic endoscopy: ex vivo investigations in the porcine stomach. Gastrointestinal Endoscopy 54: 496–500 (2001) 2. KEMPER B, DIRKSEN D, AVENHAUS W, MERKER A, VON BALLY G: Endoscopic double-pulse electronic-speckle-pattern interferometer for technical and medical intracavity inspection. Appl. Opt. 39: 3899–3905 (2000) 3. KREIS T: Holographic Interferometry: Principles and Methods. Vol. 1 of Series in Optical Metrology, Akademie Verlag, 1996 4. VON BALLY G, BJELKHAGEN HI: Techniques for endoscopic holography, in Optics in Medicine, Biology and Environmental Research, Optics Within Life Sciences (OWLS I), G. von Bally and S. Khanna, eds., Elsevier Science, Amsterdam, 1993, pp. 13–21 Correspondence address: Prof. h.c. Gert von Bally, Laboratory of Biophysics, University of Münster, Robert-Koch-Str. 45, D-48129 Münster, Germany Tel.: ++49-251-8356888; Fax: ++49-251-835 8536; e-mail: [email protected]