- Email: [email protected]
Restoration of VOYAGER 1 images of Io
COMPUTER
GRAPHICS
AND IMAGE
PROCESSING
(198
15,79-86
1)
NOTE Restoration of VOYAGER 1 Images of lo M. Physics
and Engineering
Laboratory,
J. MCDONNELL* Department of Scientific Hurt, New Zealand
Received A number
of VOYAGER
February
1 high-resolution
and Industrial
Research,
Lower
29, 1980
images
of IO were
degraded
by nonuniform
motion blur. The motion was caused by a radiation-induced timing error on the spacecraft that caused the scan platform to be slewed during exposure. Usually each blurred image overlies unblurred images, which in general were taken through different color filters. A technique for deriving the point spread function of the blur from such overlapping image pairs has been developed. The resultant point spread function is then applied to restore the blurred
images.
1. INTRODUCTION
Approximately 20 narrow angle VOYAGER 1 image frames of Jupiter’s moon 10 were degraded by a complicated nonuniform motion blur. The blurring was due to Jovian magnetospheric interference, which caused loss of synchronization between image exposure and movement of the scan platform on which the cameras are mounted. This in turn caused the camera to be moving relative to the image scene during all or part of the exposure. Camera motion is controlled by a stepping motor that moves at 0.0099”/ step and 33.63 steps/set. Each image contains 800 x 800 picture elements or pixels, and covers an angular field of view 0.8 X 0.8”. Thus, each stepping motor step covers approximately 10 pixels in 30 msec. Initial steps are usually smaller because slack in the cables must be taken up. The exposure times of the blurred images range from 0.06 to 0.48 set, so that the blurring could be as large as 160 pixels in extent. The motion is complex and highly variable from image to image, and so the point spread function (psf) that characterizes the blurring is different for each blurred image. Fortuitously, the direction of blurring in each image is such that, after rectification to remove camera distortions, the blurring is in the along scan line direction. Thus the problem of restoring these blurred sional. To begin the process it is known only that the blurring
images is one dimenis due to nonuniform
motion of limited extent in the line direction. The exact form of the psf for each blurred
image
is unknown
and must
be deduced
from
the available
blurred
and
unblurred images. Each blurred image partly overlaps two or three other sequential VOYAGER images that have approximately the same orientation and scale. For each blurred image, one of these overlapping images is unblurred or can be restored. Thus, each blurred image can be compared to an image that is not blurred. *Supported by a Study Award from the Physics and Engineering Laboratory, Department of Scientific and Industrial Research, Flagstaff, Arizona 86001.
New
Zealand.
Research
accomplished 79
while
at U.
S. Geological
Survey,
80
M. J. MC DONNELL
FIG.
1. VOYAGER
1 orange
filter
image
5J1 + 000 after
rectification
and smoothing.
Section 3 shows how the psf can be deduced from the overlapping regions of a blurred image. Section 2 describes the preprocessing that was applied to the raw VOYAGER data to produce data suitable for the procedures in Section 3 and 4. The restoration procedure is described in Section 4. VOYAGER image frames with picture numbers 00551 + 000 and 00731 + 000 are used to illustrate the discussion in Sections 3 and 4. For simplicity they are referred to as images 5Jl and 751, respectively. Image 5Jl (Fig. 1) is unblurred and was taken through an orange filter with an exposure time of 0.48 set and a range from the image center of 10 of 155,756 km. Image 751 (Fig. 2) is badly blurred and was taken through a clear filter with an exposure of 0.18 set and a range of 154,375 km. The small scale difference between the images is ignored.
VOYAGER
FIG.
2. VOYAGER
1 RESTORATION
81
1 clear filter image 7J1 + 000 after rectification and smoothing.
2. PREPROCESSING
Each blurred or unblurred image used in Section 3 was preprocessed as follows. The first step was to run a standard program that calibrated the raw VOYAGER data. This program performed a two-dimensional, spatially varying calibration resulting in an image that is linear in intensity, and for which each pixel value is within the range O-255. It should be noted here that, as well as causing the blurring, the magnetospheric interference also partially exposed the image while it was being read out from the vidicon tube. This caused calibration errors and in particular a gradual increase in recorded intensity towards the bottom of the image. This increase was compensated to a certain extent by the d.c. offset described in Section 3.
82
M. J. MC
DONNELL
Secondly, a program was run to find and remove the camera reseau marks. At this stage the image contained a significant amount of high-frequency noise. In order to obtain satisfactory results in Sections 2 and 3, it was desirable to reduce this bit error noise by an algorithm that smoothed the image but did not blur it significantly. This algorithm replaced nonboundary pixels, which had values differing by more than a threshold of 10 from the average of their four nearest neighbors, by that average. Next, a histogram of the image intensity distribution was obtained. This was used to find a suitable d.c. offset for use in Section 3, and to find suitable pixel values to use in a linear stretch when writing the image out on an Optronics Photowrite machine. The image was then rectified to remove the effect of camera distortion. The resultant full rectified image had curved edges and was contained within a 1000 x 1000 pixel image frame. A border of 40 pixels was then removed to give a 920 x 920 pixel subimage that was completely within the full rectified image. This 920 x 920 pixel image was then ready for restoration. The 1000 X 1000 pixel rectified image was finally overlaid with a grid and written out on a Optronics Photowrite machine. 3. IDENTIFYING
POINT
SPREAD
FUNCTION
Figure 1 is the unblurred orange filter image 5Jl after the preprocessing described in Section 2. For clarity the grid has not been included Figure 2 is the clear filter blurred image 751 after preprocessing. By simply overlaying the two gridded transparencies, images 5J1 and 751 were registered and a suitable 848 X 512 pixel overlapping region was found. The top left pixel coordinate of this image is (41,261) in 5Jl and (113,223) in 751. The psf was then deduced as follows. The first part of this procedure is based on a technique used by Stockham et al. [l]. Let f(x) and p(x) be corresponding lines of the blurred and unblurred images, respectively, after appropriate d.c. correction has been applied. Then, assuming that the images have been taken through the same filter, and that n(x) is the noise,
f(x) =p(x) Q9h(x) + n(x),
(1)
where @I denotes convolution and h(x) is the psf, an estimate of which is to be found. Let W(X) be a Harming window given by: w(x) = f [ 1 - cos(2sx/l)]
)
Olxi:L,
where L is the length of an image line. Then, multiplying w(x) to reduce edge effects gives
0) both f(x)
and p(x) by
4(x) = P(XMX) and
4x1 =f(xMx).
(4)
VOYAGER
Normalizing
I RESTORATION
83
q(x) to a d.c. term of 1 gives
s(x) = q(x)/Q(Oh
(5)
where upper case letters denote Fourier transformation of the function denoted by the lower case letters. Then, Wiener filtering [2] r(x) by s(x) gives
A(u) =
NuP(u) s(u)s*(u) + cp2’
where Ei
84
M. J. MC DONNELL
FIG.
3. Estimate h^(x) of h(x). (a) After Wiener filtering, (b) final estimate.
After the iterations are complete, the i(x) for use in Section 4 is obtained by repeating the first step of the above iteration, and setting values of 6(x) that are below a threshold of 20 to 0. The final i(x), scaled to have a maximum of 255, is shown in Fig. 3b. It is 56 pixels in extent. In this example, the effect of the above iteration has been fairly small. However, the effect of the iteration increases as the quality of the h(x) given by Wiener filtering decreases. Note that if the extent of the overlapping subimages had not been 512 pixels in the along scan line direction, they would have been initially scaled to be of this extent, and the final x(x) would have been resealed. This is because image size is required to be a power of 2 for the use with the FFT. The iterative procedure above is related to techniques previously used by McDonnell and Bates [3] and Gerchberg [4].
VOYAGER
1 RESTORATION
4. RESTORATION
85
PROCEDURE
The 920 x 920 pixel image 751 was restored using the technique described in [2]. First the edges of the image were extended by the length of the psf to give a 920 x 975 pixel image. Simple linear interpolation was used. The purpose of this edge extension is to force the blurred image to approximate a circular convolution of the psf w&h the original image that contributed to the given blurred image. Edge extension using cubic polynomials as described in [2] was also attempted but seemed to make little difference to the restored image. The reason for this is probably the high value of @ that was necessary in the restoration. The blurred image was then resampled to be 920 X 1024 pixels for compatibility with the FFT. Nearest neighbor interpolation was used. The psf was similarly resampled. The image was then restored using Wiener filtering and constant Q, of 0.06. The image was convolved with a 3 X 1 vertical box filter to introduce some smoothing across scan lines. The resulting restored image is shown in Fig. 4.
FIG. 4. Restored
version
of Fig. 2.
86
M. J. MC
DONNELL
Residual edge effects are visible on the left side of the image. The same linear contrast stretches have been used in Figs. 2 and 4 to enable fair comparison. The resolution of Fig. 4 is significantly improved over that of Fig. 2. Furthermore, Fig. 4, which is taken through a clear filter, is significantly different from the orange filter image in Fig. 1. This restoration was carried out on PDP 1 l/45 computer. An array processor was used to carry out the FFTs. The restoration took 19 min of computer time. 5. CONCLUSIONS
A practical technique has been described for restoring an image which has been degraded by an unknown psf in the case when an overlapping unblurred image, which may have been taken through a different color filter, is available. The technique has been applied to a motion blurred VOYAGER 1 image, and its application to corresponding two-dimensional restoration problems is straightforward. ACKNOWLEDGMENTS
Helpful discussions with S. Andrew Collins of the VOYAGER Imaging Science team at the Jet Propulsion Laboratory, on technical details of VOYAGER, and with Eric M. Eliason of the U. S. Geological Survey, Flagstaff, on calibration requirements are gratefully acknowledged. REFERENCES 1. T. G. Stockham, T. M. Cannon, and R. B. Ingebretson, Blind deconvolution through digital signal processing, Proc. IEEE 63, 1975,678-690. 2. M. J. McDonnell and R. H. T. Bates, Preprocessing of degraded images to augment existing restoration methods, Computer Grqphics and Image Processing 4, 1975, 25-39. 3. M. J. McDonnelI and R. H. T. Bates, Digital restoration of an image of Betelgeuse, Astrophys. J. ##I, 1976, 443-452. 4. R. W. Gerchberg, Superresolution through error energy reduction, Opt. Acf4 21, 1974, 709-715.
GRAPHICS
AND IMAGE
PROCESSING
(198
15,79-86
1)
NOTE Restoration of VOYAGER 1 Images of lo M. Physics
and Engineering
Laboratory,
J. MCDONNELL* Department of Scientific Hurt, New Zealand
Received A number
of VOYAGER
February
1 high-resolution
and Industrial
Research,
Lower
29, 1980
images
of IO were
degraded
by nonuniform
motion blur. The motion was caused by a radiation-induced timing error on the spacecraft that caused the scan platform to be slewed during exposure. Usually each blurred image overlies unblurred images, which in general were taken through different color filters. A technique for deriving the point spread function of the blur from such overlapping image pairs has been developed. The resultant point spread function is then applied to restore the blurred
images.
1. INTRODUCTION
Approximately 20 narrow angle VOYAGER 1 image frames of Jupiter’s moon 10 were degraded by a complicated nonuniform motion blur. The blurring was due to Jovian magnetospheric interference, which caused loss of synchronization between image exposure and movement of the scan platform on which the cameras are mounted. This in turn caused the camera to be moving relative to the image scene during all or part of the exposure. Camera motion is controlled by a stepping motor that moves at 0.0099”/ step and 33.63 steps/set. Each image contains 800 x 800 picture elements or pixels, and covers an angular field of view 0.8 X 0.8”. Thus, each stepping motor step covers approximately 10 pixels in 30 msec. Initial steps are usually smaller because slack in the cables must be taken up. The exposure times of the blurred images range from 0.06 to 0.48 set, so that the blurring could be as large as 160 pixels in extent. The motion is complex and highly variable from image to image, and so the point spread function (psf) that characterizes the blurring is different for each blurred image. Fortuitously, the direction of blurring in each image is such that, after rectification to remove camera distortions, the blurring is in the along scan line direction. Thus the problem of restoring these blurred sional. To begin the process it is known only that the blurring
images is one dimenis due to nonuniform
motion of limited extent in the line direction. The exact form of the psf for each blurred
image
is unknown
and must
be deduced
from
the available
blurred
and
unblurred images. Each blurred image partly overlaps two or three other sequential VOYAGER images that have approximately the same orientation and scale. For each blurred image, one of these overlapping images is unblurred or can be restored. Thus, each blurred image can be compared to an image that is not blurred. *Supported by a Study Award from the Physics and Engineering Laboratory, Department of Scientific and Industrial Research, Flagstaff, Arizona 86001.
New
Zealand.
Research
accomplished 79
while
at U.
S. Geological
Survey,
80
M. J. MC DONNELL
FIG.
1. VOYAGER
1 orange
filter
image
5J1 + 000 after
rectification
and smoothing.
Section 3 shows how the psf can be deduced from the overlapping regions of a blurred image. Section 2 describes the preprocessing that was applied to the raw VOYAGER data to produce data suitable for the procedures in Section 3 and 4. The restoration procedure is described in Section 4. VOYAGER image frames with picture numbers 00551 + 000 and 00731 + 000 are used to illustrate the discussion in Sections 3 and 4. For simplicity they are referred to as images 5Jl and 751, respectively. Image 5Jl (Fig. 1) is unblurred and was taken through an orange filter with an exposure time of 0.48 set and a range from the image center of 10 of 155,756 km. Image 751 (Fig. 2) is badly blurred and was taken through a clear filter with an exposure of 0.18 set and a range of 154,375 km. The small scale difference between the images is ignored.
VOYAGER
FIG.
2. VOYAGER
1 RESTORATION
81
1 clear filter image 7J1 + 000 after rectification and smoothing.
2. PREPROCESSING
Each blurred or unblurred image used in Section 3 was preprocessed as follows. The first step was to run a standard program that calibrated the raw VOYAGER data. This program performed a two-dimensional, spatially varying calibration resulting in an image that is linear in intensity, and for which each pixel value is within the range O-255. It should be noted here that, as well as causing the blurring, the magnetospheric interference also partially exposed the image while it was being read out from the vidicon tube. This caused calibration errors and in particular a gradual increase in recorded intensity towards the bottom of the image. This increase was compensated to a certain extent by the d.c. offset described in Section 3.
82
M. J. MC
DONNELL
Secondly, a program was run to find and remove the camera reseau marks. At this stage the image contained a significant amount of high-frequency noise. In order to obtain satisfactory results in Sections 2 and 3, it was desirable to reduce this bit error noise by an algorithm that smoothed the image but did not blur it significantly. This algorithm replaced nonboundary pixels, which had values differing by more than a threshold of 10 from the average of their four nearest neighbors, by that average. Next, a histogram of the image intensity distribution was obtained. This was used to find a suitable d.c. offset for use in Section 3, and to find suitable pixel values to use in a linear stretch when writing the image out on an Optronics Photowrite machine. The image was then rectified to remove the effect of camera distortion. The resultant full rectified image had curved edges and was contained within a 1000 x 1000 pixel image frame. A border of 40 pixels was then removed to give a 920 x 920 pixel subimage that was completely within the full rectified image. This 920 x 920 pixel image was then ready for restoration. The 1000 X 1000 pixel rectified image was finally overlaid with a grid and written out on a Optronics Photowrite machine. 3. IDENTIFYING
POINT
SPREAD
FUNCTION
Figure 1 is the unblurred orange filter image 5Jl after the preprocessing described in Section 2. For clarity the grid has not been included Figure 2 is the clear filter blurred image 751 after preprocessing. By simply overlaying the two gridded transparencies, images 5J1 and 751 were registered and a suitable 848 X 512 pixel overlapping region was found. The top left pixel coordinate of this image is (41,261) in 5Jl and (113,223) in 751. The psf was then deduced as follows. The first part of this procedure is based on a technique used by Stockham et al. [l]. Let f(x) and p(x) be corresponding lines of the blurred and unblurred images, respectively, after appropriate d.c. correction has been applied. Then, assuming that the images have been taken through the same filter, and that n(x) is the noise,
f(x) =p(x) Q9h(x) + n(x),
(1)
where @I denotes convolution and h(x) is the psf, an estimate of which is to be found. Let W(X) be a Harming window given by: w(x) = f [ 1 - cos(2sx/l)]
)
Olxi:L,
where L is the length of an image line. Then, multiplying w(x) to reduce edge effects gives
0) both f(x)
and p(x) by
4(x) = P(XMX) and
4x1 =f(xMx).
(4)
VOYAGER
Normalizing
I RESTORATION
83
q(x) to a d.c. term of 1 gives
s(x) = q(x)/Q(Oh
(5)
where upper case letters denote Fourier transformation of the function denoted by the lower case letters. Then, Wiener filtering [2] r(x) by s(x) gives
A(u) =
NuP(u) s(u)s*(u) + cp2’
where Ei
84
M. J. MC DONNELL
FIG.
3. Estimate h^(x) of h(x). (a) After Wiener filtering, (b) final estimate.
After the iterations are complete, the i(x) for use in Section 4 is obtained by repeating the first step of the above iteration, and setting values of 6(x) that are below a threshold of 20 to 0. The final i(x), scaled to have a maximum of 255, is shown in Fig. 3b. It is 56 pixels in extent. In this example, the effect of the above iteration has been fairly small. However, the effect of the iteration increases as the quality of the h(x) given by Wiener filtering decreases. Note that if the extent of the overlapping subimages had not been 512 pixels in the along scan line direction, they would have been initially scaled to be of this extent, and the final x(x) would have been resealed. This is because image size is required to be a power of 2 for the use with the FFT. The iterative procedure above is related to techniques previously used by McDonnell and Bates [3] and Gerchberg [4].
VOYAGER
1 RESTORATION
4. RESTORATION
85
PROCEDURE
The 920 x 920 pixel image 751 was restored using the technique described in [2]. First the edges of the image were extended by the length of the psf to give a 920 x 975 pixel image. Simple linear interpolation was used. The purpose of this edge extension is to force the blurred image to approximate a circular convolution of the psf w&h the original image that contributed to the given blurred image. Edge extension using cubic polynomials as described in [2] was also attempted but seemed to make little difference to the restored image. The reason for this is probably the high value of @ that was necessary in the restoration. The blurred image was then resampled to be 920 X 1024 pixels for compatibility with the FFT. Nearest neighbor interpolation was used. The psf was similarly resampled. The image was then restored using Wiener filtering and constant Q, of 0.06. The image was convolved with a 3 X 1 vertical box filter to introduce some smoothing across scan lines. The resulting restored image is shown in Fig. 4.
FIG. 4. Restored
version
of Fig. 2.
86
M. J. MC
DONNELL
Residual edge effects are visible on the left side of the image. The same linear contrast stretches have been used in Figs. 2 and 4 to enable fair comparison. The resolution of Fig. 4 is significantly improved over that of Fig. 2. Furthermore, Fig. 4, which is taken through a clear filter, is significantly different from the orange filter image in Fig. 1. This restoration was carried out on PDP 1 l/45 computer. An array processor was used to carry out the FFTs. The restoration took 19 min of computer time. 5. CONCLUSIONS
A practical technique has been described for restoring an image which has been degraded by an unknown psf in the case when an overlapping unblurred image, which may have been taken through a different color filter, is available. The technique has been applied to a motion blurred VOYAGER 1 image, and its application to corresponding two-dimensional restoration problems is straightforward. ACKNOWLEDGMENTS
Helpful discussions with S. Andrew Collins of the VOYAGER Imaging Science team at the Jet Propulsion Laboratory, on technical details of VOYAGER, and with Eric M. Eliason of the U. S. Geological Survey, Flagstaff, on calibration requirements are gratefully acknowledged. REFERENCES 1. T. G. Stockham, T. M. Cannon, and R. B. Ingebretson, Blind deconvolution through digital signal processing, Proc. IEEE 63, 1975,678-690. 2. M. J. McDonnell and R. H. T. Bates, Preprocessing of degraded images to augment existing restoration methods, Computer Grqphics and Image Processing 4, 1975, 25-39. 3. M. J. McDonnelI and R. H. T. Bates, Digital restoration of an image of Betelgeuse, Astrophys. J. ##I, 1976, 443-452. 4. R. W. Gerchberg, Superresolution through error energy reduction, Opt. Acf4 21, 1974, 709-715.