Automated system for coarse-to-fine pyramidal area correlation stereo matching

ELSEVIER

Automated

Image and Vision Computing 14 (1996) 225-236

system for coarse-to-fine pyramidal area correlation stereo matching Mark O’Neill*, Mia Denos Digital Vision, Edmonds Court, Didcot, Oxfordshire OX1 I 8Q Y, UK

Received 8 March 1995; revised 31 August 1995; accepted 5 September 1995

Abstract

An automatic, general purpose system for the production of dense digital disparity models (DDM) is described. The system applies an area-based correlation stereo matcher via an image resolution pyramid. The theory of this coarse-to-fine technique is described and possible improvements to it are outlined. The results of applying the system to different types of imagery including SPOT-l stereo imagery of Western Cyprus, structured light stereo imagery of faces and knees and MRI scanner imagery of the human brain are presented. Keywords: Coarse-to-fine; Texture-correlation;

Stereo matching

1. Introduction In order to be able to generate topographic maps from stereo imagery, a dense array of conjugate points must be identified by an appropriate stereo matching algorithm. Stereo matching algorithms intended for use within topographic digital mapping systems differ appreciably from those which are designed for other machine vision systems. Typically, mainstream machine vision systems are designed for tasks such as object avoidance or robotic path planning. Tasks of this nature only require a small subset of the available stereo data to be processed. Stereo matching algorithms intended primarily for these purposes generally detect a small set of discrete features rather than generating a dense homogeneous array of stereo correspondences. For example, in the case of the PMF algorithm [l] and the Ohta and Kanade algorithm [2], the discrete features matched are edge1 strings computed from intensity anomalies which demarcate the edges of objects, while Barnard and Thompson’s [3] algorithm matches discrete point features detected by the Moravec [4], Foerstner and Gulch [5] or similar operator. These feature-based stereo matchers are designed for use in a controlled, clutter free environment. As a consequence of this, many mainstream machine vision * Email: [email protected]. 0262~8856/96/$15.00 0 1996 Elsevier Science B.V. All rights reserved SSDZ 0262-8856(95)01061-O

algorithms also make assumptions about camera geometry which do not hold in the case of topographic mapping tasks. For example, the PMF algorithm requires epipolar imagery, i.e. images which are acquired along a common baseline. This condition is clearly not realisable with sensors carried by a platform which is subjected to nonlinear motions during the period of image acquisition. The remit of stereo matching algorithms intended for use in topographic mapping systems is very different. Terrain imagery acquired either via satellite platforms or from aircraft forms a quasi continuous texture function in two dimensions. Due to the nonlinear motions of the platform, the geometry of the sensor acquiring the imagery is relatively poorly defined. As a consequence, the feature-based stereo algorithms intended for machine vision work are inappropriate for use within digital topographic mapping systems. In the digital mapping regime, which is concerned with the extraction of dense height data (typically on a lo-40 m grid in the case of SPOT-l imagery and other small scale imagery) to build topographic maps at scales between 1 : 50000 and 1 : 100000, area correlation stereo matchers such as Gruen’s adaptive least squares correlation (ALSC) algorithm [6-81 fare considerably better than feature-based correlation methods. Area correlation algorithms (ACMs) are designed to correlate continuous texture rather than intensity anomaly features, thus they tend to work very well with satellite and small

226

M. O’Neill, M. Denosllmage

and Vision Computing I4 (1996) 225-236

scale aerial stereo photography, which tend to contain large amounts of texture at an appropriate granularity (between 4-8 pixels) for effective area correlation. With small scale terrain imagery, the only potential problem areas for ACMs are ridge lines in mountainous areas (where the affine distortion model may be inappropriate) and urban areas (where the texture granularity may be too coarse for ACM matching). The Gruen ALSC ACM algorithm uses an affine transformation to accommodate locally the distortions found in non-rectified stereo views. The Alvey MMI-137 (Real Time 2.5D Vision Systems) project at University College London took the basic Gruen algorithm and coupled it with a predictive sheet growing mechanism. The resulting algorithm [9] and its developments [lO,l l] are capable of generating a dense homogeneous grid of stereo correspondences given a small number of seed points (typically between 5 and 10). These seed points are accurately measured stereo correspondences which are used to initiate the sheet growing mechanism. They are required as a consequence of the unimodal nature of the least squares correlation mechanism used by the Gruen ALSC algorithm. The sheet growing mechanism uses the seed points to predict further potential stereo correspondences in their immediate neighbourhood, which are subsequently refined by the adaptive least squares algorithm. The refined stereo correspondences are then used as seed points themselves, in order to generate further potential conjugate points. This cycle is repeated until no further refinable candidate stereo correspondences are available. The work which is reported here describes a set of techniques which enhance the basic area correlation technique extending the types of imagery to which it may be applied. The basic adaptive least squares area correlation matching (ALSC) approach suffers from three basic problems: 1) Breakdown at discontinuities: Denos [12] has shown that blundering is likely if conjugate points lie on a disparity domain boundary (that is, a break line which indicates a discontinuous change in the disparity function). 2) Because ALSC patch correlation is unimodal, initiating the stereo matching process requires a number of approximate stereo correspondences (seedpoints) to be measured. These seedpoints must lie within l-2 pixels of an actual disparity if the ALSC algorithm is to converge. As these points are manually measured, this presents a significant barrier to automating ALSC-based area correlation matching. 3) ALSC breaks down if the imagery to be correlated contains insufficient texture at an appropriate granularity to provide (local) stereo cues for correlation. 4) Because ACM tend to break down in the vicinity of intensity anomaly features and breaklines. We have

a

b Fig. 1. SPOT-l PA imagery of the Isle of Wight showing coverage attained using (a) fixed scale matching, (b) pyramidal matching via Cheops.

shown [12,13] that use of conventional autoseeding methods, for example using the Moravec Interest Operator [4] for Foerstner Interest Operator [5] to select potential seedpoints is not reliable.’ These findings are supported by the poor performance of the autoseeding algorithm devised by Allison et al. [14], which attempted to use semantic feature detection to autoseed the Otto-Chau sheet growing texture correlator which is based on the Gruen ALSC algorithm. To reduce the impact of these problems, a coarse-tofine ACM stereo system has been developed, which consists of a harness that applies fixed scale ALSC ACM algorithms via a coarse-to-fine image resolution pyramid. Correlation via this harness has been found to significantly reduce the blunder rate of generic ’ Even if candidate conjugate points found using semantic interest operators could be processed reliably by ACM algorithms, two objections to their use remain. Firstly, finding out whether the candidate conjugate points in the reference image actually correspond to those in the non-reference image involves an exhaustive search. Secondly, texture images are typically a poor source of semantic features.

M. O’NeiN, M. Denosllmage

and Vision Computing

227

14 (1996) 225-236

FIFO

(Scaled images)

y-w

Rescale points to form seeds for next tier if not at bottomtier

List of conjugate c I*

m points for current tier

FIFO

4

(verified seedpoints in current tier)

PSLAM:

Prototype Least Squares Area Matcher this is the ACM used to match each tier

REDUCE: is the pixel averaging filter

ECOBE Global

CHEOPS: is the coarse to fine matching harness ECOBE:

is the blunder eliminator

FIFO: is a first in first out buffer

Fig. 2. Showing

dataflow

ALSC ACM algorithms when applied to ‘difficult’ texture imagery containing high levels of noise and/or shadows, discontinuities and occlusions. In addition, the system can be fully automated and thus does not require manually measured seed points. The system we have developed is composed of two component subsystems. The Cheops subsystem which is a coarse-to-fine pyramidal stereo matching harness, and the Cascade subsystem which sits on top of Cheops and provides a source of automatically generated approximate stereo correspondences (seedpoints).

2. Cheops pyramidal stereo matching harness The Cheops harness enhances

the performance

of

through

the Cheops

harness

generic fixed scale ALSC ACM implementations by successively correlating stereo imagery at many scales of resolution. This tends to give a greater coverage than that attained if the same algorithm is applied to a fixed scale. The Cheops algorithm is also effective for correlating imagery with low signal to noise ratio for example, synthetic aperture radar (SAR) imagery [ 15,161 and optical imagery containing discontinuities, shadows and occlusions [12,13]. Fig. 1 shows the typical extent of stereo coverage enhancement which may be attained using Cheops. In Fig. l(a) we see the coverage attained when the Otto-Chau ALSC ACM is applied at fixed scale to SPOT- 1 level 1A panchromatic imagery of the Isle of Wight. Fig. l(b) shows the corresponding coverage attained by applying the Otto-Chau ALSC ACM in a coarse to fine manner via Cheops to the

228

M. O’NeiN, M. Denosllmage and Vision Computing 14 (1996) 225-236

#______________________________________--_-________________________________---_-------# SamplePDL script

#____________________________________------__________________________----_------------# Pyramidsizeand scalingtype apex4 Laplacian2.0 no_pscale tierlocalhostnoshakeup pslsm-patchrad2-pthresh0.85 emd rescale etier # Sheetgrowinginthistier tierlocalhostnoshakeup pslam-patchrad2-auto_foveate -auto-patch-pthresh0.85 -grid2 emd rescale etier tierlocalhostnoshakeup pslm-patchrad2-auto_foveate-auto-patch-pthresh0.85 emd rescale etier tierlocalhostnoshakeup pslam-patchrad2-auto_foveate -auto-patch-pthresh0.85 emd etier end nice/bin/csh-cU rescale-rf 8 red.left.636.tmp; rescale-rf8Xdummy.rightI fsw-fs . -block100 >red.right.636.tmp; zcatQaatch.in.636.tmp I pslsm-patchrad2-pthreshO.BS -imagenamesred.left.636.tmpred.right.636.tmp I compressI fsw-fs . -block 100>match.out.636.tmp" nice/bin/csh-cU rescale-rf4 red.left.636.tmp; rescale-rf4 red.right.636.tmp; zcatQnatch.in.636.tmp I pslam-patchrad2-auto_foveate -auto-patch-pthresh0.85 -grid0322 0 322 -image_names red.left.636.tmp red.right.636.tmp I compressI fsw-fs . -block100>match.out.636.tmp" nice /bin/csh-c' rescale-rf 2 red.left.636.tmp;rescale-rf 2Cdummy.rightI fsw-fs . -block100 >red.right.636.tmp; zcatXmatch.in.636.tmp I pslam-patchrad2-auto_foveate -auto-patch-pthresh0.85 -image_names red.left.636.tmpred.right.636.tmp I compressI fsw-fs . -block100>match.out.636.tmp" nice/bin/csh-c"zcatQaatch.in.636.tmp I pslam-patchrad2-auto_foveate -auto_patch-pthresh0.85-image_namesdummy.leftdummy.right I compress I fsw-fs . -block100>match.out.636.tmp" Fig. 3. Showing

PDL script description

of image resolution

same imagery. Visual inspection alone is sufficient to verify that coarse-to-fine correlation via Cheops gives a substantially enhanced coverage. 2.1. Implementation of the Cheops harness Cheops has been implemented on a number of UNIX

pyramid

and corresponding

automatically

compiled

shellscript.

platforms using the ANSI-C programming language and the PUPS (Portable Unix Programming System) libraries [17]. As illustrated in Fig. 2, its principal function is to manage the activities of as a set of programs (UNIX filters) which perform the jobs of preparing the imagery for stereo matching and subsequently correlating it. In addition, it also acts as a glue module,

M. O’NeiN, M. Denosllmage

and Vision Compuring

14 11996) 225-236

229

Rsflner linds false minimum at tier 3 in tier 3 prediisd by scaling the corresponding co-ordinate found at tier 2 lies outside Ihe pull in range of the optlmiser at tier 3 wkhin the pyram

as the conjugate point co-ordinate

Tier 1

‘Ter 4

Fig. 4. Showing a schematic of the cascade process. The minima labelled 1 and 2 represent attractors surrounding a matchable feature appearing in both components of the stereo pair. The edge of the minima represent a maxima1 displacement between the predicted position of a match [computed by scaling the match co-ordinates found in the tier above] above which a correct match will not be located in the current tier due to the limitations of the Gruen ALSC algorithm. In the case of trajectory I the minima all overlap and thus the cascade algorithm can locate a conjugate pair in the basal tier of the pyramid. In the case of the second trajectory the non overlapping minima in tiers 2 and 3 of the pyramid will cause a false match to occur in tier 3. Thus the point will be dropped within this tier as an apparent blunder.

providing appropriate communication channels between the image preparation and correlation programs. 2.2. P_vramid description language The correlation task which is to be accomplished by Cheops is described using a simple interpreted script, PDL. An example of a typical PDL script is shown in Fig. 3. PDL provides a flexible vehicle for specifying the processing operations which are to be applied to a resolution pyramid of stereo imagery in both single processor and distributed computing environments.

3. Cascade: automatic conjugate point generation systems using Cheops.

One of the most powerful aspects of this system is its potential as a automated texture-image correlator. For our purposes, an automated system may be defined as a system which, when presented with a stereo pair and no auxiliary data, is capable of generating dense homogeneous digital disparity models (DDM). With the provision of an appropriate sensor model, the 2D conjugate points within the DDM can be transformed, forming a 3D digital elevation model (DEM). Thus, the remit of Cascade is to permit unimodal correlators such as Gruens ALSC ACM to function as effectively global correlators. The principal advantage

gained is that the algorithms can be made self initiating, and therefore operate in ‘turnkey’ systems without the need for any operator intervention. As we have already stated, in general, unimodal ALSC algorithms require manually measured stereo correspondences which lie within a unimodal ‘pull-in’ range (of about l-2 pixels) of an actual correspondence, in order to start the matching process. The Cascading technique is based on the observation that if the scale of the imagery is reduced sufficiently using pixel grey level averaging, a state will be reached where the ‘pull in’ range of the ALSC is large compared to the dimension of the (resealed) imagery. This is a direct consequence of the ‘pull-in’ range remaining invariant at l-2 pixels irrespective of image scale.’ If matches located in this low resolution imagery are used to seed further stereo matching of the same imagery, but at a resolution that is slightly higher than that at which the seeds were derived, a large proportion of them will still be within range of the correct local minimum at the higher resolution. Thus, as indicated in Fig. 4, with a suitably scaled image resolution pyramid, it is possible to generate n trial seed points stochastically ’ This is an experimental observation. Therefore. there may be some types of texture imagery for in which the ‘pull-in’ range does not remain invariant as the images are scaled. We have not found any examples yet, despite testing the system on a wide range of different types of texture imagery.

230

M. O’Neill, M. Denosllmage and Vision Computing 14 (1996) 225-236

and then progressively prune out those which represent poor stereo correspondences. This is achieved by thresholding the matches produced by the ALSC ACM in each tier of the pyramid using an appropriate match precision measure. The prototypical Gruen ALSC ACM and the Otto-Chau algorithm use a match measure which is related to the largest eigenvalue of the least squares covariance matrix. The magnitude of this value gives a measure of the error in the least squares fit. A full description of this error metric is given by both Otto and Chau [9] and Gruen [6]. In the case of the PSLAM algorithm, the match precision measure is a nonparametric statistic, which is based on the Kendall r statistic. A discussion of the Kendall r statistic and the modifications which were made to it prior to its incorporation in the PSLAM algorithm are described by O’Neill [lo]. Thus, using the Cascade algorithm a set of n trial stereo correspondences may be correlated via an image resolution pyramid to produce m [m <
3.1. Brute force iteration The basic Cascade autoseeding process described above is repeated with successive sets of randomly generated trial conjugate points until the required number of validated conjugate points have been accumulated. 3.2. Use of Otto-Chau sheet growing to amplify the number of conjugate points at selected tiers within the pyramid The sheet growing variant of the ALSC technique described by Otto and Chau [9] or O’Neill [lo] may be used to great effect within the Cheeps/Cascade system. Its use confers two advantages compared to a non-sheet growing ALSC: l

M. O’NeiN, M. Denosllmage and Vision Computing 14 11996) 225-236

a

231

the non-reference image may also be offset relative to the reference image. The size of this offset may be controlled using an annealing mechanism [20], which permits the Cascade harness to adaptively adjust for gross shifts in the positions of corresponding features in the images without operator intervention. Preliminary results have indicated that using seed point gridding significantly improves the yield compared to the brute force method.4 With a randomly positioned set of candidate correspondences and no sheet growing, cascading typically generates between 30 and 40 validated seed points from 1000 initial candidate conjugate points. Using gridding, this can be increased to between 300 and 900 validated seed points from 1000 initial estimates.” The precise number of points produced is, of course, data dependent. A comparison of the yields of the brute force and gridded autoseeding methods is shown in Fig. 6.

4. Implementation

of the Cascade system

The Cascade harness is implemented in the ANSI-C programming language, and is designed to run under the UNIX family of operating systems. A block diagram of the Cascade harness is shown in Fig. 7. This harness consists of three parts:

1) A simulated

b Fig. 5. Showing automatically generated seeds output by the Cascade/ Cheops system. (a) Brute force seeding; (b) using gridded candidate conjugate points, and sheet growing within image pyramid.

enhanced. Since a tool which can scale images by an arbitrary amount was not part of the standard image processing toolkit used in the development of the current system, the validity of this assertion is yet to be tested. 3.4. Generation ofcandidate correspondences on a regular grid which extends uniformly over the apical image set The number of validated seed points may also be increased by generating an initial set of trial stereo correspondences at the apical tier of the resolution pyramid on a regular grid. In the methods described above, the position of a candidate corresponding point, P,, in the reference image is chosen at random. In the case of gridding, the reference image co-ordinates of candidate stereo correspondences occupy regular positions on a Cartesian 2-grid. Their corresponding co-ordinate in the non-reference image, P,,, is constrained to lie within a (user defined) uncertainty radius of the reference grid co-ordinate as shown in Fig. 5. To accommodate gross shifts in the position of a feature, the origin of the grid in

annealer which is used to generate trial sets of candidate seed points at the apex of the image pyramid. This has been chosen in preference to a simple pseudo-random number generator because the distribution of trial corresponding points produced can evolve over time, generating further candidate seed points using grid offset and uncertainty radius parameters which have previously given quasi optimal seed point yields for a given image pair. 2) A Cheops command pipeline preparation/execution module: this builds a Cheops image preparation/ correlation pipeline, which is then invoked to validate the candidate seed point set using coarse to fine ALSC matching. module: this passes candidate 3) A data manipulation seed points to the ALSC pipeline and collects the verified seed points produced, resealing them appropriately as the resolution pyramid is traversed. It also counts the number of verified seed points which have been produced, and if appropriate, controls the number of (brute force or seed-grid) iterations to be performed.

4 Use sheet growing can potentially give a corresponding point yield as large as that attainable using candidate point gridding. However, generating the candidate corresponding points on a random grid can result in a less uniform distribution of validated corresponding points at the basal tier of the pyramid or elimination of all candidate points in the middle tiers of the pyramid. 5 This is without any sheet growing within the pyramid.

232

M. O’Neill, M. Denosllmage and Vision Computing 14 (1996) 225-236

m Reference Image

n

Fig. 6. Showing the cixcept of candidate seedpoint gridding. Note that points in the non reference image are generated stochastically, and lie within the uncertainty radius of the corresponding point position in the reference image, P,. If the annealing feature is being used then the grid in the nonreference image may be offset with respect to the reference image by an offset vector [dqdy]. 0: Seedpoint co-ordinate in reference image; 0: Seedpoint co-ordinate in non reference image.

5. Results applying the Cascade technique to a selection of different classes of stereo texture-imagery The results of Cascade-based autoseeding using seedpoints derived from a regular grid of candidate stereo correspondences are shown in Figs. 8(a-d) for knee

and facial image stereo-pairs acquired in a vision cell [2 11,SPOT- 1 level 1A PAN imagery of Western Cyprus, and SPOT- 1 level 1A PAN imagery of Berkshire, respectively. The result of using sheet growing within the resolution pyramid in the case of the Cyprus imagery is shown in Fig. 9. It should be noted that use of the sheet

(2 Raw

Imagery

Annealer which GeneraW a Set of Trial Seed Points

FIFO

v CHEOPS

I-

List of Verified Se&Match Points

CASCADE CHEOPS: marse to fine stereomatching harness CASCADE: autosseding harness FIFO: first in first out buffer

Fig. 7. Showing data flow through the Cascade and Cheops harne when autoseeding.

233

M. O’Neili, M. Denosllmage and Vision Compuring 14 (1996) 225-236

a

C

Fig. 8. Showing coverage attained using seeds automatically (c) SPOT-l PA (Cyprus); (d) SPOT-l PA (Berkshire, UK).

generated

growing gives better coverage in the top right-hand area of the stereo pair (Fig. 9(a)) where low signal to noise causes a problem for non sheet growing case (Fig. 9(b)). We have also successfully used the technique with non stereo imagery. Fig. 10 shows a typical example of the coverage attained when correlating magnetic resonance image (MRI) slices of the human brain, to correct for motion artifacts in the data prior to its use in functional MRI studies. Because of the statistical nature of these studies, it is often necessary to correlate sets of between 25 and 100 MRI slices. For this sort of task it is clear that autoseeding is essential if much tedious intervention is to be avoided. 5.1. Testing the accuracy of the Cascade system In order to provide a quantitative check of the accuracy of corresponding points generated by Cascade, the conjugate points found for the Western Cyprus SPOT-l were transformed to ground truth using the O’Neill-Dowman SPOT- 1 Camera Modelling System [17,22]. As discussed by O’Neill [17], the ray-ray skewness (misclosure) statistic output by the camera model is

by the Cascade/Cheeps

system.

(a) Knee

imagery

(b) facial

imagery;

a measured of the combined error in the matcher and camera model. Thus, a small mean ray-ray skewness is indicative of accurately matched conjugate points and an accurate camera model. Since the errors in the SPOT-l camera model of Western Cyprus is known (see [ 171) the misclosure statistic can be used to assess the accuracy of Cascade. As we have stated, the misclosure of a ray pair associated with a given conjugate point pair is a measure of the joint positional accuracy of the conjugate point pair on the images. If the correlator has blundered, the rays will emanate from incorrect pixels. This will lead to a misclosure which is much larger than that expected due to camera model error alone (typically 100-200 m in x, y, z as opposed 5-15 m in x, y, z). In the absence of any correlation errors, the misclosure of all ray pairs should be approximately the same.6 Thus, when all ray-pairs associated with a given DEM are considered, a narrow Gaussian distribution of misclosure, whose maximum is centred at the nominal camera model error should be seen if blundering is insignificant. In Fig. 11 we see precisely this situation. The peak in the 6 This constant

skewness is linearly related to the camera

model error.

M. O’NeiN. M. Denosjlmage and Vision Computing 14 (1996) 225-236

a

Fig. 9. Showing the effect of using a Gaussian resolution pyramid on stereo coverage. (a) Fixed scale pslam matching, left image; (b) fixed scale pslam matching, right image; (c) coarse to fine pslam matching using cascade, left image; (d) coarse to fine pslam matching using cascade, right image. Note: non-matched areas arc white.

misclosure distribution is -10.5 m (which is the typical camera model error for the O’Neill-Dowman SPOT-l sensor model). Given that the nominal pixel size for the SPOT-l sensor is 10 m, the 2a range of the misclosme distribution (~10 m) is also to be expected, given that the PSLAM, Otto-Chau and Gruens ACMs can locate a potential match to the nearest 0.25-0.5 pixels in the case of SPOT-1 imagery.

6. Blunder detection In common with many image correlation systems, Cascade and Cheops and their associated ALSC ACMs occasionally produce erroneous correspondences (blunders). Because of its nature and purpose, the Cascade harness is particularly prone to blunders: typically, we have found that about 2-5% of the seed points emitted from the base of the pyramid are blunders. To detect and remove these blunders an inter tier local disparity vector continuity constraint mechanism has been developed. This constraint mechanism is to be applied as

one of the parametric constraints within the sheet growing/conjugate point prediction module of the ACM.7 This local constraint is invoked in the following manner: a conjugate point pair [P, pred, P, pIea]is only considered valid if the associated disparity vector r = Pr pred - Pm pred is similar in direction and magnitude to the disparity vector P, - P,, associated with conjugate pair [P,, P,,] from which was the conjugate pair Wr pred > Pnr predI was predicted via the sheet growing mechanism. This adaptive local surface continuity constraint has been used with some success in the PSLAM [lo] algorithm. It has proved of particular use when attempting to extend texture correlation to areas of imagery adjacent to discontinuities and occlusions, as it tends to suppress blundering occurring at disparity domain boundaries which was observed in the case of the (earlier) Otto-Chau algorithm. In PSLAM, the local continuity constraint described above is combined with novel affine transformation constraints which detect ’ This constraint has also been implemented in the form of a postprocessing filter.

M. O’Neill. M. Denosllmage and Vision Compuring I4 (1996) 225-236

Fig. 10. Showing a set of successive MRI scans of human to functional studies.

brain which have been corrected

for motion

using Cascade/Cheeps

235

seeded ALSC ACM prior

the transformation of the image patch in the nonreference image to a line or singularity.

7. Conclusion

o0

10

30

20

Skewness (m) Fig. 11. Showing

distribution

of skewness

from SPOT 1 DEM.

A sophisticated stereo matching system for textureimaging correlation has been described. It has been demonstrated that this system is capable of generating a dense homogeneous DDM without operator intervention. The coarse-to-fine stereo matching component of the system, Cheops, is particularly useful for matching imagery containing substantial amount of noise, for example, SAR imagery, but it can potentially increase the stereo coverage achievable with most forms of texture-imagery. The Cascade harness autoseeder does not require rectified imagery to function, and in fact it may be used to facilitate the generation of image rectification parameters in the case of non-rectified imagery. The problem of blunder detection and elimination has also been addressed, and to this end, a parametric constraint mechanism has been developed which limits blundering using (local) disparity continuity, and affine transformation parameter constraints.

236

hf. O’NeiN, M. Denosllmage and Vision Computing 14 (1996) 225-236

multiphoto matching, Photogrammetric Engineering and Remote

Acknowledgements We would like to thank Laser-Scan Ltd Cambridge and Defence Research Agency Farnborough for the provision of SPOT-l stereo imagery and executables of the Otto-Chau (GruenS) algorithm. We would also like to thank Thorn EM1 Central Research Laboratories for the provision of the knee and face stereo pairs. Lastly, we would like to thank the Chris Miall and Patrick Haggard of the McDonnell-Pew Centre for Cognitive Neuroscience, Oxford, for the provision of the MRI images.

References ill J.B. Pollard, J.E.W. Mayhew and J.P. Frisby, PMF: A stereo correspondence algorithm using a disparity gradient limit, Perception, 14 (1985) 449-470.

121Y. Ohta and T. Kanade, Stereo by intra- and interscanline search using dynamic programming,

IEEE Trans. PAMI,

7(2) (1985)

83-89.

[31 S.T. Barnard and S.B. Thompson, Disparity analysis of images, IEEE Trans. PAMZ, 2(4) (1980) 333-340.

[41 H.P. Moravec, Towards automatic visual obstacle avoidance, Proc. 5th Joint Conf on Artificial Intelligence, MIT, Cambridge,

MA, 1977. PI W. Foerstner and E. Gulch, A fast operator for detection and precise location of distinct points, corners and centres of circular features, Proc. ZSPRS, 25(3) (1987) 281-305. 161A.W. Gruen, Adaptive least squares correlation - a powerful image matching technique, South African J. Photogrammetry, Remote Sensing and Cartography, 14(3) (1985).

171A.W. Gruen and E.P. Baltsavias, High precision image matching for digital terrain model generation, Proc. ISPRS, 25(3) (1987) 254-278.

PI A.W. Gruen and E.P. Baltsavias, Geometrically

constrained

Sensing, 54(5) (1988) 633-641. [9] G.P. Otto and T.K.W. Chau, Region growing algorithm for the matching of terrain images, Image and Vision Computing, 7(2)

(1989) 83-93. [lo] M.A. O’Neill, Advanced ACM Stereo Matching, Laser-Scan/ DRA SMART Contract Report, January 1993. [ll] M.A. O’Neill, A Comparison of the Otto-Chau and PSLAM ACM Stereo Matchers, Report to Thorn-EM1 Central Research Laboratories, Hayes, February 1994. [12] M. Denos, Experiments in the stereo matching of urban areas, MSc thesis, Department of Photogrammetry and Surveying, University College London, October 1989. [13] M.A. O’Neill and M.I. Denos, A practical approach to the stereo matching of urban imagery, Image and Vision Computing, IO(2) (1992) 89-98. [14] D. Allison, M.J.A. Zemerly and J.-P.A.L. Muller, Automatic seedpoint generation for stereo matching and multi-image registration, Proc. IGARRS 91. Espoo, Helsinki, Finland, June 3-6 1991. [15] M. Denos, An automated approach to stereo matching SEASAT imagery, Proc. BMVCPl, Glasgow, 24-26 September 1991, pp. 335-338. [16] M. Denos, A pyramidal scheme for stereo matching SIR-B SAR imagery, Znt. J. Rem. Sens. (Lets), 13(2) (1992) 397-392. [17] M.A. O’Neill, A kinematic numerical camera model for the SPOT-1 sensor, PhD Thesis, University College London, August 1992. [18] W.E.L. Grimson, Computational experiments with a feature based stereo algorithm, IEEE Trans. PAMI, 7(l) (1985) 17-34. [ 191 D. Marr and T. Poggio, A computational theory of human stereo vision, Proc. R. Sot. Lond. B., 204 (1979) 301-338. [20] W. Metropolis, M. Rosenbluth, A. Teller and E. Teller, J. Chem. Phys., 21 (1953) 1087. [21] M.A. O’Neill, MI. Denos, J.-P.A.L. Muller and S.N. Bhatia, An

automated technique for the generation of facial surface models as an aid in orthodontic and facial orthognastic research, Proc. IEE Colloq. on Machine Storage and Recognition of Faces, 24

January 1992. [22] M.A. O’Neill and I.J. Dowman, A New SPOT-I Camera Model, Proc. OEEPE, 1991.