A Portable Stereovision System for Cultural Heritage Monitoring

A Portable Stereovision System for Cultural Heritage Monitoring A. Balsamo' (2), A. Chimienti2, S. Desogus', P. Grattoni2, A. Medal, R. Nerino3, G. Pettiti2, M.L. Rastello3, M. Spertino2 1 lstituto di Metrologia ccG. Colonnetti>>del CNR, Turin, Italy 2 lstituto di Elettronica e d'lngegneria dell'lnformazione e delle Telecomunicazioni del CNR, Turin, Italy 3 lstituto Elettrotecnico Nazionale Galileo Ferraris, Turin, Italy

Abstract In a nation-wide project named SIINDA, a novel instrument was developed and tested, intended for monitoring and diagnosing monuments. The instrument -named AVS- is portable to operate on field, and is capable of measuring 3D and colorimetric coordinates simultaneously. The paper describes the AVS, with a focus on its geometric measurement capability. Particular attention is given to the software compensation of the geometrical errors: the model is given, and the experimental plan to derive the model parameters is described, including the artefact designed and made for this purpose. Experimental results of laboratory and of on field tests are reported. Keywords: Optical, Measuring instrument, Geometric modelling

1 INTRODUCTION Italy is renowned word-wide for its artistic treasures and cultural heritage, their amount and concentration. On one hand, this is a good national opportunity -both culturally and economically-, on the other hand, it requires a continuing large maintenance effort to preserve and restore. As different skills and backgrounds are necessary in this effort, teams of experts are formed, in which engineers are called to provide and improve state of the art equipment for data acquisition of art pieces and monuments, to monitor, analyse and diagnose them. A three-year nation-wide project named SllNDA (Computer-Aided Innovative Systems for Investigations and Diagnosis, Table 1) was carried out in 2002-4, to develop an integrated knowledge system for documenting, analysing and diagnosing monuments. In the project, a need was identified of portable equipment for acquiring on-field images of monuments, able to yield geometrical as well as colorimetric coordinates of individual pixels. Hence, a subproject was devoted to designing, manufacturing and testing a novel instrument called AVS (Active Vision System). The next sections describe it, focusing on its 3D measurement capabilities and software compensation of its geometrical errors

SllNDA acronym Partners

Budget

Sistemi lnnovativi d'lNdagine e Diagnosi Assistita Public institutions: CNR - National Research Council IEN - National Electrotechnical Institute Technical University of Milan Regional Government of Valle d'Aosta Private companies: CM Sistemi S.p.A. FOART S.r.1. Menci Software S.r.1. 2002-2004 € 2.94M, 69% from Ministry of Research Roman Theatre in Aosta (1st century)

Table 1: Headlines of the SllNDA project

2 THE ACTIVE VISION SYSTEM (AVS) Photogrammetry is a sound discipline [1][2][3] for non contacting 3D coordinate measurement. However, some requirements do not meet the needs of this application: 1. Possibly coded artificial targets are usually stuck on the object to measure, to precisely localise points in image pairs for intersection. Art pieces like frescoes do not allow any sticking. Structured light projection combined with photogrammetry [4] is not suitable when the image texture is dense and colourful. 2. The exterior camera parameters are usually unknown as the cameras are positioned on the field with no prior information, and are determined by finding set of correspondent points in images observed from different viewpoints (resection). Either calibrated artefacts or EDMs (Electromagnetic Distance Meters) give the metric. This procedure may be time-consuming and expensive, and requires highly specialised personnel, usually not available in the artistic restoration community. Standard colorimeters usually measure the average colour of surface portions, and cannot image wide areas. Contacting colorimeters are unsuitable, while non contacting ones require known illuminants, available in laboratories and not on field in open sky. A novel instrument was designed, manufactured and tested, to achieve the following functionalities: Automatic acquisition of image pairs, and camera self-centring (fixation) on artificial and natural targets. Simultaneous measurement of 3D and colorimetric coordinates of individual pixels. 3D surface rendering by stereovision. Image mosaicing of high-resolution images. The AVS (Figure 1) is basically a triangulation device. Three cameras are mounted on a horizontal rotating shaft driven by a stepper. A camera is mounted in the midpoint of the shaft and is equipped with wide-angle lenses (WA). Two rotating steppers with vertical axes are mounted at the shaft ends, each driving a camera equipped with high focal-length objectives to frame a small portion of the

Figure 1: Frontal view of the AVS. scene (TLI and TL2). The latter cameras are the pointing devices for triangulation, while the WA camera observes the scene and drives the other two. T L I is equipped with a filter wheel carrying a transparent and three RGB filters, for normal view and tristimulus colorimetry, respectively. Component details are summarised in Table 2. 2.1 AVS operation The triangulation and the stereovision are both based on matching corresponding points in the T L I , TL2 images. When possible, artificial targets are used, otherwise natural targets are detected. The artificial targets are high contrast annuli: an image algorithm detects elliptical features, and distinguishes targets from background fakes by checking the concentricity of the two circular borders of the annulus. The border centroid is then taken as the sought location point: accuracies of 2-3% of a pixel can be achieved, barely affected by illumination and perspective distortion, with a target slant up to 80" [5]. Natural targets are searched for as points showing ' a significant variation of the luminance in more than one direction' [6]. Accuracies of few tenth of a pixel can be achieved with typical surfaces. The WA detects the line of view to a target, and T L I and TL2 are driven along it at increasing distances iteratively, until the target enters the T L I , TL2 fields of view. The two cameras are then driven to centre the target individually. The coordinate measurement is carried out in three consecutive steps: 1. The coordinates of the central point used for fixation are evaluated by triangulation (see next sections)

Cameras T L I ,TL2 BAN cooled digital Type 105 mm Focal length Framed field (33 x 22) cm2 @ 2.5 m (92 x 61) cm2 @7m (1536 x 1024) pixel2 CCD 14 bivpixel Pixel depth (9 x 9) pm2 Pixel size 0.4 plx Sensitivity 1 to 64 oixels Binnina

Stepper resolution Triangulation base Measurement volume

I

WA BAN digital 12.5 mm ( 1 . 7 1.3) ~ m2 (4.7 x 3.6) m2 (756 x 580) pixel2 8 bivpixel (11 x 11) pm2 0.15 Ix No

Figure 2: Photorealistic 3D image of a quoin, coordinates (mm) of the surface in the AVS reference frame. 2. Correspondences among all points in a pair of stereo images are searched for by cross-correlation analysis. The coordinates of the resulting dense series of 3D points are measured in reference to the central point, known by triangulation A 3D surface is evaluated. 3. The surface texture is projected onto this surface. The coordinates of all points in the scene are thus taken in the AVS reference system, borrowing the measurement accuracy of triangulation and the point density of stereovision. The obtained 3D image is made photorealistic by projecting the texture onto the evaluated 3D surface (Figure 2). By rotating the wheel, the RGB filters are inserted in sequence in the optical path of TLI. As its resolution exceeds ordinary colorimetric needs, the three images are taken with a 2 x 2 binning, to speed up the procedure. For each pixel, the RGB coordinates are rotated onto the CIE 1931 space, and eventually transformed into CIELAB coordinates. The former rotation depends on the filter responses; hence, they were calibrated. Both transformations depend on the illuminant, which may vary considerably in open sky, even during a single measurement. Colour temperature fluctuations as large as 3,000 K - 20,000 K were observed in a sunny day with scudding clouds. To overcome this, a reference white tile was put in the scene, just outside the view of interest: from time to time, T L I was driven to aim at it for zeroing itself. The AVS can mosaic a full scene inside its measuring volume from an ordered sequence of partially overlapping full resolution images. The mosaicing algorithm [7] exploits the known AVS geometry, and is based on compensation of acquisition errors and parallax effects rather than on minimizing matching errors between adjacent images, as traditionally done. The algorithm is non-iterative and fast, accurate enough not to reduce the accuracy of original 3D coordinates. Highly dense (14 Mpixel) and deep (14 bit) 3D

0.001" = 17 prad 800 mm xz Y (1.8 x 2.8) m2 2.5 m (10x6)m2 7m

I

Table 2: Summary of AVS component characteristics

Figure 3: Nominal model of the AVS.

and colorimetric images were achieved by mosaicing 70 tiles, with an overall acquisition time of 20 min. 3 GEOMETRIC COORDINATE MEASUREMENT In the triangulation measurement, only the central points of the stereo images are considered. The TLI and TL2 cameras are regarded here as mere pointing devices. Let us call zand rn the two pointing directions when the three steppers are at home, i.e. when a = p = y = 0 (Figure 3). The coordinate system is defined as having the x axis coincident with the yaxis, the origin equidistant to a and p, and the z axis parallel to a and p. When the target lies in the xy plane, ordinary triangulation is performed in it, otherwise the necessary rotation about y introduces a cosine-sine component in the y-z coordinates. The resulting nominal direct kinematic equation of the AVS is given in Equation (1). - sin@

+a) (11

2cosacospsin y

4 COMPENSATION OF THE GEOMETRIC ERRORS The AVS geometric performance is mainly limited by residual misalignment of its axes, also due to transportation shocks and operation in open air, where the temperature may range of some tens of kelvin. On the other hand, the resulting errors are highly repeatable, hence suitable for software compensation. To do so, an error model and a suitable experimental procedure to derive the actual model parameters were developed. 4.1 Error model The following assumptions were made: 1. After proper calibration, the angular errors of the steppers are negligible. 2. The stepper wobbles off axis are negligible, i.e. the a,p,yaxes are still in space during rotation. The diagram in Figure 3 shows five lines in space, namely three rotation and two optical axes. Each line is parameterised with a localisation point p and an orientation unit vector a. The former is taken at the intersection with a coordinate plane (two coordinates), and the latter constrained to unit norm (two direction cosines). zand rn undergo a double rotation about a or p, respectively, and y Even if physically simultaneous, the rotations are in a logically non-commutative sequence, as the a and p axes are affected by the rotation about y but not vice versa. As py = 0, it holds: P:, =R(a,,y)[R(a,,a)(p,

-P,)+P,l

a:, =R(a,,y)R(a,,a)a,

(2)

so that the target always lies on z', while the elevation of d is irrelevant. To implement this in the model, a, is replaced by an orthogonal unit vector b in the xy plane (Figure 3 ) , insensitive to the a, elevation.

The following simultaneous equations result:

(4) t is the distance of the target q to p:, ; the first equation the second the horizontal -but expresses that q lies on z', not vertical- convergence of o'onto the target. The AVS reference system defintion is refined as follows, in view of the geometric errors: ycoincides with the x axis, i.e. is affected by no errors.

a prevails on p i n orienting the z axis about x, i.e. the y component of a, is null. The origin is such that a is at a nominal distance d in the x direction, i.e. the x coordinate of pa is -d. 4.2 ldentifiability of the model parameters The parameter identifiability was investigated both in theory and by simulation: The vertical localisation of cr (z component of p,) cancels out in the product with the horizontal unit vector b in Equation (4). The orientation errors of p are second order, as rotations cause mainly an elevation error of d,which is filtered out in Equation (4) After eliminating the above non identifiable parameters, the final parameterisation in Table 3 results, with a total of 10 parameters. 4.3 Experimental plan Before on field operation, the AVS performs some measurements of a suitable artefact, to derive the actual values of the above parameters by least-squares adjustment. For each pair of artefact feature points J, the linearised distance equation holds: nii(qi T -qj)=lii

+Eii

where nii is the unit vector of the line through the two feature points qi and qj, /ii is the calibrated distance, and E~ is the measurement noise. By setting the artefact in different positions and orientations, as many Equations (5) are written, from which the model parameters are derived. A proper artefact was designed, manufactured and calibrated (Figure 4). It consists of a solid bar with six square plates bearing silk-screened high contrast targets, separated by 400 mm, 800 mm in all. To enable on-field operation with high temperature fluctuations (10-30 " C ) , it

where primes indicate rotated quantities, and R s are rotation matrices about the unit vectors a's. A similar equation holds for 0'. In general, the matrix that operates a rotation of an angle 9about a unit vector a is [8]: R(a,9)= I + sinSL, +(I - cos9)Li

(3)

where I is the identity matrix and La = (a x ) is the matrix corresponding to the external product by a. Due to the geometric errors and to the tilt y being common for the two cameras, z'and d do not intersect. To solve this ambiguity, TLI is chosen arbitrarily to prevail on TL2,

Table 3: Parametrisation of the error model

theodolites, and onto each other. The test was performed in winter, in open air and with an average temperature of O T , partly during the day, and partly in the dark with artificial lights. At that time, the software compensation was not available yet. In the laboratory test, the 800 mm calibrated artefact was measured. The results are summarised in Tables 4 and 5.

Figure 4: CAD view of the artefact and of its support. is all made in invar, including the plates and their rear pins matched into the bar body. The artefact rests on lateral pins at its Airy points, and is mounted on an articulating tripod through a support made of aluminium, to save weight. The differential thermal expansion with the artefact is dealt with by a fully kinematic design: two forks connect the base with the artefact pins, one (rear in Figure 4) free only of rotating horizontally (cylindrical constrain), and the other resting on a single sphere. Contrast springs enable vertical operation of the artefact. The minimum number of artefact measurements used to derive the parameters successfully was 11, including vertical, horizontal (front and diagonal), and spatial diagonal. Seven different more were taken for testing purposes only, and not used to derive the parameters. 5 EXPERIMENTAL RESULTS The AVS was tested both on field at the Roman Theatre in Aosta and in laboratory. In the former case, tens of artificial targets were stuck on a pillar of the monument, and measured by theodolites and by the AVS under three different views surrounding the pillar. To match the different reference systems, the points in each view were rototranslated onto the corresponding ones measured with

AVS vs. theodolites

2 3

2.74 3.93

AVS vs. AVS

1.5 1.8

1-3 2-3

3.66 2.02

2.3 1.3

Table 4: Rototranslation residuals, on field test (mm) Test measurem.

1

7 Mean St. dev.

Before compensation

After compensation

3.40 2.73 2.92 3.26 2.61 3.36 2.91 3.03 0.31

0.57 0.46 0.19 0.11 0.03 0.73 0.05 0.31 0.28

Table 5: Errors of indication, laboratory test (mm). St.dev.

Spectroradiom. Table 6: Colorimetric test results (AELAB)

The colorimetric capability was tested too. In laboratory, 11 ceramic colour standards ranging the whole visible spectrum were measured by the AVS and by a reference spectrophotometer. The resulting AELAB gave a measure of the absolute colorimetric errors. On field, a grey and an orange areas of the pillar were measured by the AVS and by a commercial contact spectroradiometer with in-built illumination. The capability of discriminating colours was investigated, as diagnosis techniques are based on colour mapping. The results are summarised in Table 6. AELAB< 5 is considered the human perception threshold. 6 CONCLUSIONS A novel portable instrument called AVS has been developed and tested, capable of measuring on field 3D and colorimetric coordinates simultaneously, intended for monitoring and diagnosing monuments. The experimental test results proved satisfactory, with mean errors in 800 mm size measurements of 0.3 mm (software compensated), and sub-unit (AELAB)errors in chromatic discrimination. While separated instruments are available with better performances, the AVS achieves them on a single portable instrument.

7

REFERENCES Atkinson, K.B. (editor), 1996, Close range photogrammetry, Whittles Publishing Services, UK. Estler, W.T., Edmundson, K.L., Peggs, G.N., Parker, D.H, 2002, Large-Scale Metrology - An Update, Annals of the CIRP, 51/2:587-609. Schwenke, H., Neuschaefer-Rube, U., Pfeifer, T., Kunzmann, H., 2002, Optical Methods for Dimensional Metrology in Production Engineering, Annals of the CIRP, 51/2:685-699. Reich, C., et al., 2000, 3-D shape measurements of complex objects by combining photogrammetry and fringe projection, Optical Engineering, 39-1 :224-231. Cumani, A,, Grattoni, P., Guiducci, A,, Pettiti, G., 1993, Characterization of Commercial Solid-state TV Cameras for Non-Contact Metrology, SPlE Proceedings, 2067:115-122. Robbins, B., Owens, R., 1987, 2D Feature Detection via Local Energy, Image and Vision Computing,

15:353-368. Grattoni, P., Spertino, M., 2003, A Mosaicing Approach for the Acquisition and Representation of 3D Painted Surfaces for Conservation and Restoration Purposes", MVA, 15/1:1-10. Fillmore, J.P., 1984, A Note on Rotation Matrices, IEEE CG&A. 4/1:30-33.