A search engine for auroral forms

Pergamon www.elsevier.com/locate/asr

Adv. Space Rex Vol. 28, No. 11, pp. 161 l-1616,2001 0 2001 COSPAR. Published by Elsevier Science Ltd. All rights reserved Printed in Great Britain 0273-l 177/01 $20.00 + 0.00 PII: SO273-1177(01)00492-6

A SEARCH ENGINE FOR AURORAL FORMS M.T. Syrjbuo’ , K. Kauristiel,

and T.I. Pulkkinen’

‘Finnish Meteorological In&it&e, Geophysical Research, Vuorikatu 15 A, FIN-00100

Helsinki, Finland

ABSTRACT The Finnish Meteorological Institute operates five digital all-sky camel=, which routinely monitor the auroral emissions in Northern Finland, Sweden, and Svalbard; each camera records a31image of the full sky at 20-s interval at a resolution of 512 x 512 pixels. As a result, over 2.5 million images are recorded each winter. We demonstrate a syntactic pattern recognition algorithm for searching auroral arcs in the all-sky images. The algorithm operates in two phases: a quicksearch and a detection phase. The quicksearch is based on simple correlation to a model arc and is utilised to scan through the database to locate potential auroras. In the detection, an auroral image skeleton is determined, and, based on the shape skeleton, auroral forms known as %rcs” are located. A quality estimate is calculated for each arc. aial runs indicate that the algorithms developed can be implemented in real-time at the auroral all-sky stations. 0 2001 COSPAR.Published by Elsevier Science Ltd. All rights reserved. INTRODUCTION Finnish Meteorological Institute operates five digital all-sky cameras, which routinely monitor the auroral emissions in Northern Fennoscandia and S&bard (Syrjbuo et al., 1998). The cameras record all-sky images at 20-s cadence resulting in over 2.5 million images every year. The data are, however, mainly used in case studies, and the analysis is performed visually examining the images one by one. Our aim is to develop methods that can be used to automatically locate and classify auroraI events in a huge database. We demonstrate the current status our auroral search engine by searching auroral arcs. The actual search is performed in two phases: the quicksearch and the detection. The quicksearch examines each auroral image effectively classifying the images into those containing aurora and those without aurora. The detection of arcs is based on shape skeletons that are a commonly used shape representation in machine vision. THE ALL-SKY CAMERAS Each all-sky station operates autonomously: the start and end of imaging is determined by calculating the elevation of the Sun. The station computer controls the all-sky imager mounted in a transparent dome and captures images at regular intervals. Weak auroral luminosity is amplified in the image intensifier and exposed to the CCD-camera. The nominal imaging interval is 20 seconds for a 557.7nm image and 60 seconds for a 63O.Onm image. An unfiltered image is acquired once an hour to provide star info required in geometrical calibration of the instrument. A typical exposure is one second. SCANNING THE DATABASE: THE QUICKSEARCH The aim of the quicksearch is to quickly determine whether an image requires further examination or not. In each image, three vertical brightness profiles as seen in Figure 2 are extracted and if the correlation to a model auroral arc exceeds a fixed threshold, the image is marked for later skeletonisation. The correlation template length is fixed to 11 pixels. The position of the moon in the image is calculated and the horizontal location of a colliding vertical profile is modified to avoid the moon.

1611

1612

M. T.Syjlsuo et al Auroral all-sky corneros

Fig. 1. MIRACLE

Fig. 2. Three

all-sky

vertical

cameras

profiles (left)

in operation:

three

and the model

-

1999/01/09

21:18UT

aurora1 images

arc brightness

on a map.

profile (right).

Aurora1 Search Engine

Fig. 3. Input-output

relations of the subprocesses. One should note that the original

1613

image from which the moon

is removed is used in several subprocesses.

DETECTION OF ARCS Overview All images located by the quicksearch are analysed. An overview of the relationships of the subprocesses is given in Figure 3. The detection is completely automatic and produces (1) geographical locations of the arcs, (2) the longitudinal span of each arc, (3) an average luminosity within each arc, and (4) an estimate of how well the aurora can be represented by using a skeleton. In the next sections, each subprocess is explained in more detail. Contrast Enhancement and Moon Removal A fixed grey-level transformation is performed to enhance the contrast in the all-sky images. This operation emphasises the difference between the (visually) faint and bright auroras. The moon is so bright an object that it saturates the camera, which affects the arc detection algorithm. The location of the moon is, however, a priori information and it can be “blocked” by filling the moon area with background intensity. In practice, a square is placed on top of the moon, and the inner pixel values of the square are interpolated from the corner points. Vertical Correlation In Syrjbuo et al. (2000), separate edge information was fused to the original all-sky image, which improved the convergence of the skeleton. The drawback of that, in case of a reasonably faint arc, the skeletoniser converges to both edges of the arc instead of the middle resulting in two arcs instead of one arc. In this work, we have replaced the edge information by a vertical correlation in each pixel column. If the correlation to the model arc introduced in the quicksearch phase exceeds a threshold, artificial “‘brightness” is added to the image. This emphasises arc-like features in the image and the skeleton correctly converges on top of the arc. Determining the Aurora1 Skeleton The shape skeleton is based on a minimum spanning-tree self-organising map (Singh et al., 1998, Kangas et al., 1990). A grid of skeleton nodes is initialised over the aurora1 image, and the nodes are updated towards the brightest aurora in an organised manner. As the iteration proceeds, new nodes can be added or two

M. T.

1614

Fig. 4. The flow of iteration:

the initial

state

(left),

SyrjSisuo et al.

after

the first iteration

(centre)

and the final configuration

(right)

EIReconstructedshape

N Aurora

I/

l!iiffl Fig. 5. The concept a skeleton

overlaid

of reconstruction: with the aurora

the aurora

in all-sky

image

\

(left)

and the area which

is reconstructed

from

(right).

nearby nodes can be merged if necessary. Once the skeleton is formed, syntactic pattern recognition can be utilised to locate the arcs in the image. Details of the algorithm can be found in Syrj&uo and Pulkkinen (1999) and in SyrjLsuo et al. (2000). An example of the iteration is shown in Figure 4. Quality Estimate It is not a trivial task to evaluate the quality of the skeletons. Even determining what is auroral luminosity in the all-sky image is ambiguous: the level of precipitation varies constantly and the definition of a “faint” or (‘bright” aurora1 arc depends not only on the context but also on the instrument used. The reconstruction from a skeleton, however, can provide information about the quality. If the reconstruction does not correspond “well” to the auroras in the image, then ~~~quite intuitively ~-~ the skeleton is not a good one. To determine an approximative shape of the aurora, we first threshold the original image from which the moon is removed. The threshold values for 64 subimages are determined by using optimal thresholding (Gonzales and Woods, 1993). A threshold level is doubled if the mean or the variance in the subimage is small. The centre pixel of each subimage is associated with the subimage threshold level and from these values individual threshold levels for all image pixels are interpolated. This is illustrated in Figure 6. One should note that the thresholding clearly does not produce contiguous areas even though the aurora appear as contiguous in the original image. shape and Nthres the number of pixels in thresholded Let NM be the number of pixels in the reconstructed

Aurora1 Search Engine

1615

Threshdd level (nonnalised)

Fig. 6. The original result of thresholding

Fig. 7. Columns:

all-sky image with overlaid threshold (right)

original

(KEV 1997/11/22

(left),

with a good reconstruction

reconstructed

(Q = 0.6).

the original image (Q = 0). Bottom

Middle

area grid (left),

(centre) row:

and thresholded

No aurora,

threshold

(centre)

and the

(right)

image.

Top row: An aurora1 arc

note the covered moon in lower right corner of

row: active aurora which cannot

shape. The number of pixels included in both the reconstructed number of pixels in the thresholded but not in the reconstructed estimate Q = [0,11: Q++!!$@)

interpolated

15:20:00 UT)

be described as an arc (Q x 0).

and thresholded shape is N,,fit.

shapes is Nf;t, and the Now we can derive an

(1)

skel

So, Q + 0 when the aurora in the image cannot be represented as an east-west aligned arc-like structure. There can be other structures such as spirals or no aurora at all. Examples of the quality estimate are shown in Figure 7. EMPIRICAL

RESULTS

Test Period

For testing purposes, two sets of all-sky images from Kevo (N69.76, E27.01) in Northern Finland were selected. The first set covered 19 days in November 6.-28., 1997, whereas the second set contained all images

1616

between issue).

M. T. Syjlsuo

1997/12/22

and 1999/01/03.

Results

et al.

from the second test set are described

in Kauristie

et al. (this

Operating Environment and Performance All the search tests were performed on a normal workstation (Pentium-II 400MHz) running Linux. On average, the quicksearch consumed less than three seconds per image (including reading and decompressing). The skeletoniser consumed typically 7 to 10 seconds per image, and the number of iterations was fixed to seven. Arcs with a good contrast to background were ususlly easily located in images. The detection of weaker arcs would probably have benefitted from a larger number of iterations. Realistically, it is not possible to measure the width of an auroral arc in an ASC image except in such cases where the camera is perfectly under the arc in sense of magnetic field lines. Otherwise, the viewing geometry distorts the arc appearance. It is, however, possible to calculate the average auroral luminosity, which is the approach chosen in our work. DISCUSSION We have demonstrated an automatic auroral search engine that detects and locates aurora1 arcs in all-sky camera images. As noted earlier by Syrjijsuo et al., 2000, the engine can significantly reduce the manual labour required in a search by using traditional methods. By introducing a quality estimate, the results of both the quicksearch and the arc detection phases can be significantly improved: false alarms due to, for example, the moon or reflections from the moon can be rejected. The simple correlation performed in the quicksearch is computationally very light, and most modern computers can easily perform the arc detection at the nominal 10 to 20 imaging interval used at auroral all-sky stations. REFERENCES Gonzales, R. and Woods, R., Digital Image Processing, Addison-Wesley, 1993. Kauristie, K., Syrjbuo, M. T., Amm, O., Viljanen A., Pulkkinen, T. I. and Opgenoorth, H. J., A Statistical Study of Evening Sector Arcs and Electrojets, in this, 2001. Singh, R., Papanikolopoulos, N. P., and Cherkassky, V., Object skeletons from sparse shapes in industrial image settings, in Proceedings of the IEEE International Conference on Robotics and Automation, volume 4, 1998. Syrjbuo, M. T. and Pulkkinen, T. I., Determining the Skeletons of Aurora, in Proceedings of the 16th International Conference on Image Analysis and Processing, ICIAP-99, pp. 1063-1066, IEEE Computer Society, Printing House, 1999. Syrjasuo, M. T., Pulkkinen, T. I., Janhunen, P., Viljanen, A., Pellinen, R., et al., Observations of Substorm Electrodynamics by Using the MIRACLE Network, in Proceedings of the Fourth International Conference on Substorms, eds. S. Kokobun and Y. Kamide, 1998. Syrjtiuo, M.T., Kauristie, K., and Pulkkinen, T.1, Searching for Aurora, in Proceedings of the IASTED International Conference on Signal and Image Processing, SIP-2000, pp. 1063-1066, Las Vegas, USA,2000.