- Email: [email protected]
Digital picture analysis of the renal tubulus
COMPUTERS
AND
BIOMEDICAL
Digital
RESEARCH
Picture
12,291-298 (1979)
Analysis
of the Renal Tubulus
G. ZAJICEK Department
of Experimental Medicine and Cancer Research, The Hebrew Medical School, P.O. Box II 72, Jerusalem, Israel
University-Hadassah
AND CH. MAAYAN Department
of Pediatrics,
The Hadassah
University
Hospital,
Mount
Scopus, Jerusalem,
Israel
Mount
Scopus, Jerusalem,
Israel
AND
E. ROSENMANN Department
ofPathology,
The Hadassah
University
Hospital,
Received July 10, 1978 Image analysis of tissues has to cope with two difficulties: (a) Most tissue elements are indistinct and their border appears fuzzy, and (b) The vast amount of information to be dealt with. These call for special procedures which are applied here to analysis of the histological image of the renal tubulus. Kidney histological sections have been photographed on a black and white negative transparency which was then scanned under computer control by an image dissector. The software utilized consists of routines for on-line image analysis such as feature detection of nuclei and area measurement of connected shapes. To store the relevant pictures in the computer the digitized image was transformed into isophotes. Each isophote was then encoded into an octal digit chain and stored as such. This procedure reduces drastically the image information content while preserving its relevant features. INTRODUCTION
The major effort of digital picture analysis in pathology is directed today toward the analysis of isolated objects such as cells or chromosomes(1, 2). Yet most of the material studied by the pathologist consists of tissuesand organs in which cells are aggregatedtogether. Tissue structure analysis differs from cytological analysis in two major aspects. First, the tissue elements are not distinct. Cells tend to overlap and their borders appear fuzzy. However the major difficulty in digital tissue analysis originates from the multitude of pattern and structural elementswhich contribute to the pathological diagnosis. All have to be separated, classified, and sorted out. Unfortunately most of the strategies and algorithms utilized in cytological analysis remain inadequate to cope with theseproblems. 291
oulo~~xoY,7Y/o3o?Yl-ox$o~.~~~ CopyrIght@ lY7Y by Academic Press. Inc. All rights of reproduction in any form reserved. Prmted in Great Britam
292
ZAJICEK, MAAYAN,
AND ROSENMANN
Renal pathology, which forms the subject of the present study deals with tissues which exhibit many features. The differentiation between the various glomerular diseases for instance has to take into account about 50 structural variables all observed in the glomerulus (3). The renal tubulus which appears to be somewhat easier to deal with, serves in this study as a convenient object on which the above arguments are to be illustrated. MATERIALS
AND METHODS
The histological sections to be analyzed have been photographed on a black and white transparency. The image was then projected onto an EMR image dissector (Model 658A, Schlumberger) digitized and fed into a PDP-15/76 computer. The dissector consists of a sensor in which the optical image is transformed into an electrical signal which is then magnetically focused onto a plate which contains an apperture in its center. Magnetic deflection of this image positions the desired X-Y location at the apperture. The addressing capability of the Model 658A is 4096-X by 4096-Y locations. Scanning may be made in any mode under computer software control. The input lines to the digitizer consist of 12.bit parallel binary X or Y position information. The output consists of an &bit parallel binary intensity word. The computer provides the X or Y position information. This information is then strobed within 150 nsec into the digitizer input storage register and initiates conversion in the appropriate digital to analog converter. At the end of the conversion period, the intensity of the desired point is held in the output register and is read by the computer. The whole cycle lasts on the average 50 psec. For routine scanns the dissector was set so as to convert every tenth point on the image. In this manner a 2-cm* transparency was digitized into 400 x 400 points. A more detailed digitation has been applied to special areas of interest such as nuclei. Over such areas the dissector sampled every second point. The histological material consists of a normal renal tubulus stained with Haematoxylin-Eosin photographed on a Kodak panatomic-x, 35-mm black and white negative transparency. The magnification of the image on the transparency was 200 times. RESULTS
The tubulus (Fig. 1 in the center) consists of a gray epithelial lining which surrounds a white lumen. Its nuclei appear as black dots. Since in the computer the digitized image originates from the negative transparency its gray tones are reversed. The nuclei are white while the lumen appears black. The gray levels are represented in the computer by numbers ranging between 0 and 255. In this scale 0 stands for black and 255 for white, while the numbers in between stand for the various gray levels.
ANALYSIS
OF THE RENAL
FIG. 1. The histological image of the normal human kidney. digitized (marked by a T). Above it lies the glomerulus (G).
TUBULUS
In the center the tubulus
293
which had been
Figure 2 depicts the gray level relief of the digitized tubulus. Each curve represents the gray level variation of one digitized line. The bottom line for instance depicts the gray level variation of the lowest digitized line in the transparency. Proceeding from left to right, it starts with a peak, further on it follows closely along the abscissa, then it gradually climbs to a brighter region whereupon it descends again and so on. Several curves together outline hills and valleys. The hills represent the bright nuclei which on the transparency appear white. The narrow valleys represent the cytoplasm while the crater in the center depicts the black tubular lumen. The hills surrounding the crater are actually epithelial nuclei. The interstitial nuclei form the second and third hill chains. Such a relief forms the starting point for tissue scene analysis. it is apparent that in this scene nuclear area coincides with the hill base which is marked by a certain gray level value. One may therefore study structure according to its gray level value, and depict only objects which are brighter than a desired gray level (Fig. 3). In this representation objects above the threshold appear white, everything else remains
294
ZAJICEK, MAAYAN,
AND ROSENMANN
FIG. 2. Gray level relief of the tubular scene. Each curve represents the gray level valUf% measure:d in a line. The higher the curve the brighter the scanned object. The nuclei are reprr :sellted by hills while the tubular lumen appears as a crater. Since the images were digitized directly fn 3m the negativr: transparency, the nuclei appear as bright objects while the tubular lumen is black.
black. The white areas may now be measuredclassified,and presentedin form of an area distribution (Fig. 4). Many features may be similarly treated. For each feature type a gray level window in which the desired feature becomesprominent may be defined. All features outside the window remain black while the desiredfeatures are depicted in white, sincein the sceneall tubuli appear as the darkest objects. The window in which the tubuli appear most prominent should extend over the gray level values between0 and 20. STORAGE
AND RETRIEVAL.
Since each picture is digitizable into 1.6 x lo’numbers, one is faced with a serious storage problem. This difficulty could be partially resolved by regarding the original image as primary storage with an average access time of 50 psec. This seems adequate for routine searching procedures like those described above, which are generally performed on line. Such a storage however does not meet a demand for accumulation of interesting features for future reference which could be stored in a data base.To meet this, objective data reduction procedures have to be designedso asto preserve the relevant features of the original image.
ANALYSIS OF THE RENAL TUBULUS
295
FIG. 3 Nuclei viewed through a gray level window which excludes all points below a given glray level value.
FIG. 4. Nuclear area distribution in the tubulus and neighborhood. The abscissa represents the area in relative units. The ordinate stands for frequency.
296
ZAJICEK,
MAAYAN,
AND
ROSENMANN
The following representation of the digitized image, known as “isophote represen tation” meets the above objectives adequately. An isophote connects all points which share the same gray level value. Figure 5 depicts the tubulus of Fig. 2 in isophotes. Each curve there stands for a gray level increment of f0 units, i.e., each two adjacent isophotes differ by 10 gray level units. The nuclear hills appear here as concentric curves embedded each in the other. These are separated by the “cytoplasm valleys.‘* The crater which in Fig. 2 appeared homogeneous exhibits substructures which
Fl G. 2i. Isophote representation of the tubular neighborhood. Each isophote connects pointis ot‘equal gray level value. All gray level values in this scan were separated into 25 gray level categcxies Two adjac:ent isophotes differ by 10 gray level units. (The scene represents the mirror image of Fig. 2.1
indicate protein remnants in the tubular lumen. Figure 6 depicts the nuclear scenery under higher magnifictation which could be achieved by digitizing every second point instead of every tenth, as in Fig. 5. Each isophote is represented in the computer by a series of coordinate couples; aX and a-Y coordinates. By adopting Freeman’s encoding procedure (4) the amount of space required to store isophotes could be reduced. This procedure encodes isophotes into chains of digits. Each chain starts with three entries. The first describes the chain gray level value. The following two entries describe the X.Y coordinates in the original image where the isophote starts. All subsequent chain elements are expressed in relation to the starting point in the following way. From a
ANALYSIS
FIG. 6. A detailed
differs
from its neighbor
OF THE RENAL
297
TUBULUS
view of two nuclei which belong to the epithelial by 15 gray level units.
tubular
lining.
Here each isophote
given image point one may proceed only in eight directions (Table I). Each direction may now be encoded by an octal digit. In this form, each chain entry from the fourth chain element and onward indicates the direction one has to proceed to reach the subsequent point on the original image. Generally, the nth element contains the encoded direction which brings it to the (n + 1)th element. Each number chain TABLE RECTANGULAR WHICH FREEMAN’S
I ARRAY DESCRIBES CODES
LI Each octal digit stands for a direction one may proceed in from the central point.
298
ZAJICEK, MAAYAN.
AND ROSENMANN
represents an isolated feature which may now be stored in a data base. It serves also for computations such as nuclear area estimation, which has been attempted previously utilizing a different computational procedure (Fig. 4). Nuclear area may be computed from closed chains which share two properties: (a) Their isophote value has to exceed a given threshold and (b) since nuclear isophotes are concentric only the peripheral isophote in such a complex should serve for nuclear area estimation. Isophote representation of the original image preserves its relevant features. Its information content depends upon two parameters: (a) The number of digitized points in the original image, and (b) the gray level increment represented by each isophote. Both parameters may be varied so as to meet the pathologist’s objective without overburdening the computer storage capacity. REFERENCES K., JR. Digital picture analysis in cytology. Top. Appl. Phys. 1 I, 209 (1976). J. M. S. Parametric and non parametric recognition by computer. An application 10 leukocyte image processing. Advan. Computers 12,285 (19721. ZAJICEK, G., MAAYAN, CH.. AND ROSENMANN, E. The application of cluster analysis to glomerular histopathology. Comput. Biomed. Res. lo,47 1 (1977). FREEMAN, H. On the encoding of arbitrary geometric configurations. IRE Trans. h/ecrro~r. Computers EC-IO, 260 (196 1).
1. PRESTON, 2. PREWIT. 3. 4.
AND
BIOMEDICAL
Digital
RESEARCH
Picture
12,291-298 (1979)
Analysis
of the Renal Tubulus
G. ZAJICEK Department
of Experimental Medicine and Cancer Research, The Hebrew Medical School, P.O. Box II 72, Jerusalem, Israel
University-Hadassah
AND CH. MAAYAN Department
of Pediatrics,
The Hadassah
University
Hospital,
Mount
Scopus, Jerusalem,
Israel
Mount
Scopus, Jerusalem,
Israel
AND
E. ROSENMANN Department
ofPathology,
The Hadassah
University
Hospital,
Received July 10, 1978 Image analysis of tissues has to cope with two difficulties: (a) Most tissue elements are indistinct and their border appears fuzzy, and (b) The vast amount of information to be dealt with. These call for special procedures which are applied here to analysis of the histological image of the renal tubulus. Kidney histological sections have been photographed on a black and white negative transparency which was then scanned under computer control by an image dissector. The software utilized consists of routines for on-line image analysis such as feature detection of nuclei and area measurement of connected shapes. To store the relevant pictures in the computer the digitized image was transformed into isophotes. Each isophote was then encoded into an octal digit chain and stored as such. This procedure reduces drastically the image information content while preserving its relevant features. INTRODUCTION
The major effort of digital picture analysis in pathology is directed today toward the analysis of isolated objects such as cells or chromosomes(1, 2). Yet most of the material studied by the pathologist consists of tissuesand organs in which cells are aggregatedtogether. Tissue structure analysis differs from cytological analysis in two major aspects. First, the tissue elements are not distinct. Cells tend to overlap and their borders appear fuzzy. However the major difficulty in digital tissue analysis originates from the multitude of pattern and structural elementswhich contribute to the pathological diagnosis. All have to be separated, classified, and sorted out. Unfortunately most of the strategies and algorithms utilized in cytological analysis remain inadequate to cope with theseproblems. 291
oulo~~xoY,7Y/o3o?Yl-ox$o~.~~~ CopyrIght@ lY7Y by Academic Press. Inc. All rights of reproduction in any form reserved. Prmted in Great Britam
292
ZAJICEK, MAAYAN,
AND ROSENMANN
Renal pathology, which forms the subject of the present study deals with tissues which exhibit many features. The differentiation between the various glomerular diseases for instance has to take into account about 50 structural variables all observed in the glomerulus (3). The renal tubulus which appears to be somewhat easier to deal with, serves in this study as a convenient object on which the above arguments are to be illustrated. MATERIALS
AND METHODS
The histological sections to be analyzed have been photographed on a black and white transparency. The image was then projected onto an EMR image dissector (Model 658A, Schlumberger) digitized and fed into a PDP-15/76 computer. The dissector consists of a sensor in which the optical image is transformed into an electrical signal which is then magnetically focused onto a plate which contains an apperture in its center. Magnetic deflection of this image positions the desired X-Y location at the apperture. The addressing capability of the Model 658A is 4096-X by 4096-Y locations. Scanning may be made in any mode under computer software control. The input lines to the digitizer consist of 12.bit parallel binary X or Y position information. The output consists of an &bit parallel binary intensity word. The computer provides the X or Y position information. This information is then strobed within 150 nsec into the digitizer input storage register and initiates conversion in the appropriate digital to analog converter. At the end of the conversion period, the intensity of the desired point is held in the output register and is read by the computer. The whole cycle lasts on the average 50 psec. For routine scanns the dissector was set so as to convert every tenth point on the image. In this manner a 2-cm* transparency was digitized into 400 x 400 points. A more detailed digitation has been applied to special areas of interest such as nuclei. Over such areas the dissector sampled every second point. The histological material consists of a normal renal tubulus stained with Haematoxylin-Eosin photographed on a Kodak panatomic-x, 35-mm black and white negative transparency. The magnification of the image on the transparency was 200 times. RESULTS
The tubulus (Fig. 1 in the center) consists of a gray epithelial lining which surrounds a white lumen. Its nuclei appear as black dots. Since in the computer the digitized image originates from the negative transparency its gray tones are reversed. The nuclei are white while the lumen appears black. The gray levels are represented in the computer by numbers ranging between 0 and 255. In this scale 0 stands for black and 255 for white, while the numbers in between stand for the various gray levels.
ANALYSIS
OF THE RENAL
FIG. 1. The histological image of the normal human kidney. digitized (marked by a T). Above it lies the glomerulus (G).
TUBULUS
In the center the tubulus
293
which had been
Figure 2 depicts the gray level relief of the digitized tubulus. Each curve represents the gray level variation of one digitized line. The bottom line for instance depicts the gray level variation of the lowest digitized line in the transparency. Proceeding from left to right, it starts with a peak, further on it follows closely along the abscissa, then it gradually climbs to a brighter region whereupon it descends again and so on. Several curves together outline hills and valleys. The hills represent the bright nuclei which on the transparency appear white. The narrow valleys represent the cytoplasm while the crater in the center depicts the black tubular lumen. The hills surrounding the crater are actually epithelial nuclei. The interstitial nuclei form the second and third hill chains. Such a relief forms the starting point for tissue scene analysis. it is apparent that in this scene nuclear area coincides with the hill base which is marked by a certain gray level value. One may therefore study structure according to its gray level value, and depict only objects which are brighter than a desired gray level (Fig. 3). In this representation objects above the threshold appear white, everything else remains
294
ZAJICEK, MAAYAN,
AND ROSENMANN
FIG. 2. Gray level relief of the tubular scene. Each curve represents the gray level valUf% measure:d in a line. The higher the curve the brighter the scanned object. The nuclei are reprr :sellted by hills while the tubular lumen appears as a crater. Since the images were digitized directly fn 3m the negativr: transparency, the nuclei appear as bright objects while the tubular lumen is black.
black. The white areas may now be measuredclassified,and presentedin form of an area distribution (Fig. 4). Many features may be similarly treated. For each feature type a gray level window in which the desired feature becomesprominent may be defined. All features outside the window remain black while the desiredfeatures are depicted in white, sincein the sceneall tubuli appear as the darkest objects. The window in which the tubuli appear most prominent should extend over the gray level values between0 and 20. STORAGE
AND RETRIEVAL.
Since each picture is digitizable into 1.6 x lo’numbers, one is faced with a serious storage problem. This difficulty could be partially resolved by regarding the original image as primary storage with an average access time of 50 psec. This seems adequate for routine searching procedures like those described above, which are generally performed on line. Such a storage however does not meet a demand for accumulation of interesting features for future reference which could be stored in a data base.To meet this, objective data reduction procedures have to be designedso asto preserve the relevant features of the original image.
ANALYSIS OF THE RENAL TUBULUS
295
FIG. 3 Nuclei viewed through a gray level window which excludes all points below a given glray level value.
FIG. 4. Nuclear area distribution in the tubulus and neighborhood. The abscissa represents the area in relative units. The ordinate stands for frequency.
296
ZAJICEK,
MAAYAN,
AND
ROSENMANN
The following representation of the digitized image, known as “isophote represen tation” meets the above objectives adequately. An isophote connects all points which share the same gray level value. Figure 5 depicts the tubulus of Fig. 2 in isophotes. Each curve there stands for a gray level increment of f0 units, i.e., each two adjacent isophotes differ by 10 gray level units. The nuclear hills appear here as concentric curves embedded each in the other. These are separated by the “cytoplasm valleys.‘* The crater which in Fig. 2 appeared homogeneous exhibits substructures which
Fl G. 2i. Isophote representation of the tubular neighborhood. Each isophote connects pointis ot‘equal gray level value. All gray level values in this scan were separated into 25 gray level categcxies Two adjac:ent isophotes differ by 10 gray level units. (The scene represents the mirror image of Fig. 2.1
indicate protein remnants in the tubular lumen. Figure 6 depicts the nuclear scenery under higher magnifictation which could be achieved by digitizing every second point instead of every tenth, as in Fig. 5. Each isophote is represented in the computer by a series of coordinate couples; aX and a-Y coordinates. By adopting Freeman’s encoding procedure (4) the amount of space required to store isophotes could be reduced. This procedure encodes isophotes into chains of digits. Each chain starts with three entries. The first describes the chain gray level value. The following two entries describe the X.Y coordinates in the original image where the isophote starts. All subsequent chain elements are expressed in relation to the starting point in the following way. From a
ANALYSIS
FIG. 6. A detailed
differs
from its neighbor
OF THE RENAL
297
TUBULUS
view of two nuclei which belong to the epithelial by 15 gray level units.
tubular
lining.
Here each isophote
given image point one may proceed only in eight directions (Table I). Each direction may now be encoded by an octal digit. In this form, each chain entry from the fourth chain element and onward indicates the direction one has to proceed to reach the subsequent point on the original image. Generally, the nth element contains the encoded direction which brings it to the (n + 1)th element. Each number chain TABLE RECTANGULAR WHICH FREEMAN’S
I ARRAY DESCRIBES CODES
LI Each octal digit stands for a direction one may proceed in from the central point.
298
ZAJICEK, MAAYAN.
AND ROSENMANN
represents an isolated feature which may now be stored in a data base. It serves also for computations such as nuclear area estimation, which has been attempted previously utilizing a different computational procedure (Fig. 4). Nuclear area may be computed from closed chains which share two properties: (a) Their isophote value has to exceed a given threshold and (b) since nuclear isophotes are concentric only the peripheral isophote in such a complex should serve for nuclear area estimation. Isophote representation of the original image preserves its relevant features. Its information content depends upon two parameters: (a) The number of digitized points in the original image, and (b) the gray level increment represented by each isophote. Both parameters may be varied so as to meet the pathologist’s objective without overburdening the computer storage capacity. REFERENCES K., JR. Digital picture analysis in cytology. Top. Appl. Phys. 1 I, 209 (1976). J. M. S. Parametric and non parametric recognition by computer. An application 10 leukocyte image processing. Advan. Computers 12,285 (19721. ZAJICEK, G., MAAYAN, CH.. AND ROSENMANN, E. The application of cluster analysis to glomerular histopathology. Comput. Biomed. Res. lo,47 1 (1977). FREEMAN, H. On the encoding of arbitrary geometric configurations. IRE Trans. h/ecrro~r. Computers EC-IO, 260 (196 1).
1. PRESTON, 2. PREWIT. 3. 4.