Automatic fringe analysis for moiré topography

Optics and Lasers in Engineering 3 (1982) 73—83

AUTOMATIC FRINGE ANALYSIS FOR MOIRE TOPOGRAPHY

T. YATAGAI and M.

IDESAWA

The Institute of Physical and Chemical Research, 2—1, Hirosawa, Wako, Saitama 351, Japan (Received: 16 November 1981)

ABSTRACT

Two digital systems are developed for the automatic measurement of 3-D shapes using moire techniques—an automatic 3-D shape measuring system using the scanning moire method and an interactive fringe analysing system for moire fringe photographs. In the automatic 3-D shape measuring system, a deformed grating is scanned and sampled with an electronic image scanning device so that moire fringes are generated in a computer. This technique of electronic fringe generation eliminates ambiguity with regard to the sign of the moire fringes. The interactive fringe analyser provides a flexible and versatile tool for moire fringe analysis. Medical applications of these analysing systems are discussed.

INTRODUCTION

The ease with which diffuse surfaces can be measured has been significantly increased by the development of moire topographic techniques.1 Twodimensional moire fringe patterns are very useful in extracting qualitative information but it is sometimes extremely tedious and time-consuming to evaluate quantitative data from the fringe pattern. Some efforts have been made to analyse fringe data by using digital and analogue techniques. A technique using point source fringe projection and real time video processing combined with electronic filtering to generate moire fringes has been described.2 Recently, we have reported a promising automatic 3-D shape measuring method—the scanning moire method.3 In this method, moire fringes are generated by electronic scanning and sampling of a deformed shadow grating, 73 Optics and Lasers in Engineering Publishers Ltd, England, 1982 Printed in Northern Ireland

0143-8166/82/0003-0073/$0275—--© Applied

Science

74

T. YATAGA! AND M. IDESAWA

instead of the use of reference grating superposition as in conventional moire topography. The electronic scanning and sampling technique automatically eliminates the shape ambiguity problem, which is a serious one in conventional moire topography. A practical measuring system has been developed,4 which is especially useful for medical applications. In order to analyse moire fringe photographs, an interactive fringe analyser has been described for analysing moire fringe patterns. In this method, fringe peaks are automatically detected and false peaks due to noise are visually inspected and corrected manually.5 A simpler method is to manually trace fringe peaks with a digitiser.6 Analogue methods have also been proposed. A phase-locked moire method has been described which uses an oscillating moire fringe pattern and an analogue signal processor in a feedback ioop in order to locate and lock on a fixed phase value of the fringe pattern.7 A heterodyne detection technique is used to measure the moire phase with high resolution and to make sign determination.8~°In general, the sensitivities of analogue methods are of the order of 1/20 of the contour interval. The advantages of analogue approaches are that greater accuracy and faster measurement are obtained compared with digital methods. Novel methods are developed in analogue approaches. A local frequency of a deformed grating applied to the specimen surface is measured by optical diffraction in order to measure the surface strain11 and slope’2 of the object. Recent evolutions of digital image processing techniques leading to enhanced usefulness have permitted the development of comprehensive micro-computerbased fringe pattern analysing systems. In the present paper, we describe two types of digital moire analysing system we have developed—an automatic 3-D shape measurement system using the scanning moire method and an interactive fringe analyser. In conclusion, some of medical applications of these systems are discussed. AUTOMATIC 3-D SHAPE MEASUREMENT SYSTEM USING SCANNING MOIRE METHOD

The scanning moire topography set up is shown in Fig. 1. A grating is projected onto the object to be measured. The shadow cast by the grating on the object is observed with an electronic scanning device, such as a TV camera and a photodiode array. A moire fringe pattern is observed on a TV monitor. The procedure of scanning and sampling in the image input device is similar to the superposition of a reference grating in conventional projection-type moire topography. Thus, we can generate moire contours of the object in the same manner as the conventional method. Using only a set of moire contours, it is not possible to distinguish between a depression and an elevation in an object, since the relative order, or sign,

AUTOMATIC FRINGE ANALYSIS FOR MOIRE TOPOGRAPHY

Scanning Device

Observation System

A

Image

Processing Device

~ 0

75

Projection Refe,erlCe System Grating Object to be Fleasured

Display Monitor 0

Fig. 1.

Schematic of scanning moire method.

between two adjacent contours is unknown. Additional information is necessary in order to remove this ambiguity. With the scanning moire method, the phase, the pitch and the direction of the scanning lines can all be changed and, therefore, various contours can be immediately generated. This means that sign determination and interpolation of moire contours is possible, as described below. In a typical projection-type moire topography arrangement the relationship between a point (x,,, y~,Zn) in the object space and a point (x,~,y~)on the observation plane corresponding to the nth moire fringe is given as follows: Zn =

—

Xn

al/[P(n + d)] = znxm/a

(1) (2)

z~y~/a

(3)

=

where a is a distance from the projection grating and the nodal point of the projection lens and 1 is a distance between the optical axes of the projection and the observation system, and: n=nTh—np d=d,,.~—d,,

(4) (5)

In eqns. (2) to (5), fl,,,, dm, n~and d~are the index number of the observation grating, its phase factor, the index number of the projection grating and its phase factor, respectively. n denotes the absolute order of the moire fringe and d its phase factor. When the phase factor, d,,1, of the observation grating varies, the phase factor, d, of the moire fringe also varies, and the moire contour levels

76~

T. YATAGAI AND M. IDESAWA

IMAGE SENSOR SCANNING SERVO MECHANISM .1

OBSERVATION SYSTEM / ~ r..Li

\_J~iJTiiii1

MOIRE

\ L..j I

~_4T~,

,_.~

CAMERA

TV

CAMERA GRATIN

PROJECTION SYSTEM

SCANNING SERVO ~OLLER

Fig. 2.

IV

SENSITIVITY CONTROLLER A/D CONVERTER

_______________

P~CESSOR

~

MONITOR

LIGHT ~

MICRO—COMPUTER

CMI #0

( LSI

CMI #1

-

11 / 02 )

Block diagram of the automatic measurement system using the scanning moire method.

move accordingly. This means that the direction of the contour displacement depends on whether the slope of the object runs up or down. In the scanning moire method, automatic sign determination of moire fringes can be made by a computer, because the phase of the observation grating is electronically controllable. In practice, we determined the sign of the moire fringes by sampling a shadow image of the projected grating with three different phases. A block diagram of the automatic measuring system using the scanning moire method is shown in Fig. 2. An optical system is configured with the projection system and the observation system. A grating is projected on to the object to be measured. The shadow cast by the grating on the object is read by an electronic scanning device. In order to carry out a stable and reliable measurement in the scanning moire method, it is necessary that an image scanning device has good positional accuracy. We employed a photodiode array, the Reticon RL-1728H, on a precision stage because of its high positional accuracy. This device has 1728 photosensitive elements whose spacing is 15 ~.tm. A scanning servo mechanism moves the photodiode array in the transverse direction so that all the image area can be covered. An AID

AUTOMATIC FRINGE ANALYSIS FOR MOIRE TOPOGRAPHY

77

converter converts the corrected signals to 8 bit digital signals, stored in a moire processor memory. All the system is controlled by a micro-computer, LSI-11I02, with 32k words of IC memory. The software system developed actually provides two modes—an automatic mode and a man—machine interactive mode. The automatic mode measures all the area. Sectional shapes are measured along the measuring grids which are previously set and relative fringe orders are determined according to the method described above. In the interactive mode, an operator can use a light pen as an input device to position the area or the line to be measured. Application software is available for spline interpolation of data, sectional shape plots, 3-D plots on an X-Y plotter, etc.

INTERACTIVE FRINGE ANALYSER

A conceptual confirmation of an interactive fringe analysing system, called RIFRAN (Riken Interactive FRinge ANalyzer) is shown in Fig. 3. A TV camera acquires image data. A TV monitor and a light pen facilitate operator interaction with the system. The 110 controller does not require a large memory but may have a small memory consisting of a line buffer memory for reading in a line to be analysed and a binary refresh memory for displaying information used in interactive control. In RIFRAN, fringe analysis is restricted to one dimension. This leads to the simplification of the system in that the picture buffer memory may be one-dimensional, and the software of RIFRAN can be made smaller, which enables one to employ a micro-computer as the main processor.

TV camera

TV I/O controller microcomputer

~tor light pen Fig. 3.

‘~

Basic configuration of an interactive fringe analysing system using a light pen.

78

T. YATAGAI AND M. IDESAWA

a

e

b C

d Fig. 4. Command areas for ioteracti~eproce~sitie.A TV monitor displays a fringe pattern to be analysed and command areas a. h h). White lines on the TV monitor are analysis lines.

The algorithm in RIFRAN is characterised by the use of man—machine interaction with a light pen and the fringe order matching technique. To achieve man—machine interaction, a TV monitor displays not only the object to be analysed but also some useful information used in the interactive mode, such as the lines to be analysed, the positions of the fringe peaks, the cursor pointers, and so on. When the light pen is pointed at an element on the TV monitor, a computer is interrupted and then reads pertinent commands for system control, or the X and Y co-ordinates of the light pen hit. By directing the light pen at the small areas to the right and left, as shown in Fig. 4, the system responds to the operator’s request. In RIFRAN, whose general flow diagram is shown in Fig. 5, one vertical and some horizontal lines are ordinarily analysed. A vertical analysis line is used for matching fringe orders between adjacent horizontal lines. This system consists of the following seven steps. Input of unit length A unit length is required in the conversion from the measuring space co-ordinate system to that of the object space. It is assumed that photographs

AUTOMATIC FRINGE ANALYSIS FOR MOIRE TOPOGRAPHY

79

INPuT UNIT LENGTH

SET START & END

PoINTS

DETECT FRINGE PEAKS

LIGHT PEN PROCESSING Roui I NE

DELETE OR APPEND FRINGE PEAKS

Ass IGN FRINGE ORDERS

VERTICAL LINE ?

YES

NO FRINGE ORDER hATCHING OF VERTICAL & HORIZONTAL SECTIONS

ALL LINES ANALYZED

NO

?

YES FINAL DATA FORMATTING

Fig. 5.

AUXILIARY DATA FILE

Flow diagram of RIFRAN.

to be analysed involve fiducial patterns whose lengths are known in the object space. Confirmation analysis line When the unit length is completed, a vertical analysis line superimposed on the original fringe pattern is displayed on the TV screen. The operator confirms whether the position of the analysis line and the analysis interval are suitable or not. When the vertical line has been analysed, the first horizontal line to be

80

T. YATAGAI AND M. IDESAWA

analysed is displayed. As with the vertical line, the position and the interval to be analysed are confirmed and changed if necessary. Fringe peak detection This step is automatically performed. Fringe data along the analysis line are read in and then fringe peaks are detected.13 If this step is completed, detected peak points along the analysis line are displayed on the TV screen; these are superimposed on the original fringe pattern. Correction of peaks The operator confirms the positions of the peaks analysed. If errors are found in detected fringe peaks, the operator can insert fringe peaks, delete spurious peaks and correct peak positions using a light pen. Fringe order determination In order to determine fringe orders, the operator should inform a computer of the signs of the moire fringes along the analysis line. Down slope intervals are indicated by pointing the light pen at the two ends of the interval, after which downward and upward slope intervals are displayed on the TV screen. The operator can confirm these intervals and make corrections with the light pen. Matching of fringe order This step adjusts the fringe orders between the vertical analysis line and the horizontal analysis lines. The nearest two peaks, one of which is on the vertical line and the other on the horizontal line, are displayed on the TV screen with small rectangular pointers, whose longer sides are along either the vertical or the horizontal analysis lines. Since the direction of the longer side corresponds to that of the analysis line, the operator can easily distinguish which analysis line the fringe peak is on. The operator indicates the fringe order difference between them by pointing the light pen towards the command area. If fringe order matching is performed, all the detected peak positions and their fringe orders are displayed on a console terminal. If necessary, the measured sectional shape is displayed. If errors are found, the operator can re-enter this fringe order matching procedure. These fringe analysis procedures are repeated until all the horizontal analysis lines have been analysed. Formation of complete data In order that they may be used in further processing, data files are formed comprising the previously analysed data and its auxiliary data. In the case of scoliosis examination, the auxiliary data include the information on the subject, the date of the analysis, the operator’s name, and so on.

AUTOMATIC FRINGE ANALYSIS FOR MOIRE TOPOGRAPHY

Fig. 6.

81

3-D plot of a human back measured by the automatic system.

-

-

Fig. 7. 3-D plot of a reconstructed human back. Solid lines show a detailed structure of a 3-D model which is given by interpolating broken lines of sectional shapes analysed by RIFRAN.

82

T. YATAGAI AND M. IDESAWA MEASUREMENT EXAMPLES

An experiment demonstrating the scanning moire method was performed using the automatic system developed. The object was a human back. Figure 6 shows a 3-D plot of the measured data. Some errors of fringe order determination appear in the auxiliary parts, where the deformed grating lines are discontinuous or their contrast is low. The RIFRAN system developed is mainly employed to analyse moire fringe photographs in scoliosis. Moire contourograms analysed have been taken by a moire camera developed for screening for scoliosis among schoolchildren.14 In this case, ten horizontal lines are analysed, as shown in Fig. 4. The broken lines in Fig. 7 are measured sectional shapes. Two-dimensional interpolation is carried out so that the 3-D model of the object consists of meshed data points as shown in Fig. 7. CONCLUDING REMARKS

We have developed two digital systems to automatically analyse moire fringe information. One is a fully automatic 3-D shape measuring system using the scanning moire method. A portable and real-time measurement system has been developed. An alternative to this is an interactive fringe pattern analysing system. This system is used to analyse moire fringe photographs in scoliosis screening. It is possible to produce a 3-D plot of a human back deformity by evaluating the data analysed by the present system. By analysing around a hundred moire contourograms, we sought the parameters which would indicate the features of scoliotic deformity. For example, parameters of hump angle, lateral derivation, and so on, are investigated in order to search judgement indices for the early detection of scoliosis.-It is found that some parameters have a linear relationship to the degrees of spinal curvature shown on X-ray film.’5 ACKNOWLEDGEMENTS

We would like to express our thanks to Dr H. Saito and Dr F. Goto of IPCR for their advice. We also express our appreciation to Mr Y. Yamaashi and Mr H. Ohshima of Fuji Photo Optical Company for their co-operation.

REFERENCES

1.

H. TAKASAKI, The development and the present status of moire topography. In: Holography in medicine and biology (G. von Bally (Ed)), Springer, Berlin, 1979, 45—59.

AUTOMATIC FRINGE ANALYSIS FOR MOIRE TOPOGRAPHY

2.

3. 4.

5.

6. 7.

8. 9. 10. 11. 12. 13. 14. 15.

83

B. Dessus, J.-P. GERARDIN and P. MOUSSELET, Une methode des franges en temps reel et ses applications industrielles: Deformations, vibrations, Courbes de Niveau, OpI.. Quant. Elec., 7

(1975) 15—45. M. IDESAWA, T. YATAGAI and T. SOMA, Scanning moire method and automatic measurement of 3-D shapes, Appl. Opt., 16 (1977) 2152—62. T. YATAGAI, M. IDESAWA, Fl. OHSHIMA and M. SUZUKI, Automatic measure of 3-D shapes using scanning moire method, In: Moire fringe topography and spinal deformity (M. S. Moreland (Ed)), Pergamon Press, New York, 1981. T. YATAGAI, M. IDESAWA, Y. YAMAASHI and M. SUZUKI, Interactive fringe analyzer: Application to moire topography. In: Moire fringe Iopography and spinal deformity (M. S. Moreland (Ed)), Pergamon Press, New York, 1981. M. IDESAWA and T. YATAGAI, Interactive fringe processing system: RIFRAN II, Proc. Jpn. Infor. Proc. Soc., 1981, 997—8. D. T. Moomis and B. E. TRUAX, Phase-locked moire fringe analysis for automated contouring of diffuse surfaces, Appl. Opt., 18 (1979) 91—6. J. C. PERRIN and A. THOMAS, Electronic processing of moire fringes: Application to moire topography and comparison with photography, AppL Opt., 18 (1979) 563—74. G. INDEBETOUW, Profile measurement using projection of running fringes, Appl. Opt., 17 (1978) 2930—3. R. N. SHAGAM, Heterodyne interferometric method for profiling recorded moire interferograms, Opt. Engr., 19 (1980) 806—9. P. M. BOONE, A method for directly determining surface strain fields using diffraction gratings, Exp. Mech., 11 (1971) 481—9. T. YATAGAI and H. SArro, Measurement of surface slope using laser beam deflection of deformed frating, to be published. T. YATAGAI, 5. NAKADATE and H. SArro, Automatic fringe analysis using digital image processing techniques, Opt. Engr., to be published. M. SUZUKI, N. KANAYA, K. SUZUKI and N. SHOKOUCHI, Projection type moire topography camera system used for early detection of scoliosis. In: Moire fringe topography and spinal deformity (M. S. Moreland (Ed)), Pergamon Press, New York, 1981. A. SHINOTO, Y. Ommuic&, S. IN0UE, M. IDESAWA and T. YATAGAI, Quantitative analysis of scoliosis and kyphosis deformity by moire method. In: Moire fringe topography and spinal deformity (M. S. Moreland (Ed)), Pergamon Press, New York, 1981.