Representation and processing of multispectral satellite images and sequences

Available online at www.sciencedirect.com

ScienceDirect Available online at www.sciencedirect.com

Available online at www.sciencedirect.com

ScienceDirect ScienceDirect

Procedia Computer Science 00 (2018) 000–000

Procedia Computer Science 126 (2018) 49–58 Procedia Computer Science 00 (2018) 000–000

www.elsevier.com/locate/procedia

www.elsevier.com/locate/procedia

International Conference on Knowledge Based and Intelligent Information and Engineering Systems, KES2018, 3-5 September 2018, Belgrade, Serbia International Conference on Knowledge Based and Intelligent Information and Engineering Representation and KES2018, processing of multispectral satellite Systems, 3-5 September 2018, Belgrade, Serbia images and sequences Representation and processing of multispectral satellite images and Konstantin Vasiliev, Vitaly Dementiev and Nikita Andriyanova sequences a

Ulyanovsk State Technical University, Severny Venets, Ulyanovsk, 432027 Russia

Konstantin Vasiliev, Vitaly Dementiev and Nikita Andriyanova Abstract

a

Ulyanovsk State Technical University, Severny Venets, Ulyanovsk, 432027 Russia

The article considers a complex approach to image processing based on the application of mathematical models. At the same time, special attention is paid to the processing of images’ sequences. Moreover, the proposed algorithms are developed for Abstract processing satellite multispectral images. The presented filtering algorithm is of great applied importance as well as an algorithm for objects aand delineating boundaries. this paper,based description complex of images is achieved by using Thedetecting article considers complex approach to imageInprocessing on the of application mathematical models. At thedoubly same stochastic models of random the quasi-isotropic generated images is obtained by using time, special attention is paidfields to theand processing of images’properties sequences.ofMoreover, the proposed algorithms are autoregressive developed for models withsatellite multiple roots of the images. characteristic equations.filtering algorithm is of great applied importance as well as an algorithm processing multispectral The presented

for detecting objects Published and delineating boundaries. In this paper, description of complex images is achieved by using doubly © 2018 The Authors. Elsevier Ltd. filtering, Keywords:doubly stochastic models,by image processing, autoregression with multiple roots stochastic models of random fields and theBY-NC-ND quasi-isotropic properties of generated images is obtained by using autoregressive This is an open access article under the CC license (https://creativecommons.org/licenses/by-nc-nd/4.0/) models with multiple roots of the characteristic equations. Selection and peer-review under responsibility of KES International.

1. Introduction Keywords:doubly stochastic models, image processing, filtering, autoregression with multiple roots In recent years, a large number of works related to the processing of various kinds of images and sequences of 1. Introduction images have been published. This is due to a wide range of applications of image processing techniques. An important example of problems directly related to image processing is remote sensing of the Earth. In recentthe years, a largemade number of works related of to satellite the processing of various kinds of images and Despite advances in the development monitoring systems at present there is sequences a number of images have been published. due tothe a further wide range of applications of image An significant unresolved problemsThis thatisimpede development of computer visionprocessing techniques.techniques. Among these problems, example one can single out thedirectly problems of describing arrays of correlated data and their optimal important of problems related to image multidimensional processing is remote sensing of the Earth. andDespite suboptimal processing based on development formalized mathematical models. Analysis available literature on this the advances made in the of satellite monitoring systems atof present there is a number of subject1-20 unresolved showed thatproblems the complete of the problem of describing images has not been obtained. First, these most significant that solution impede the further development of computer vision techniques. Among problems, one can singleare out multispectral the problems images, of describing arrays of correlated data and of their optimal of the satellite imagery whichmultidimensional can be viewed as three-dimensional arrays brightness and suboptimal on formalized mathematical models. Analysis of the available on this values, composedprocessing of separatebased two-dimensional frames that correspond to the results of Earth's literature surface recording 1-20 showed thatrange. the complete solutionthe of time the problem of describing imagescorresponds has not beentoobtained. First, most subject in a separate spectral Accordingly, sequence of such images a four-dimensional of the satellite imagery multispectral which be viewed as three-dimensional of brightness random field (RF), one are of the dimensionsimages, of which is a can discrete time. Thus, the analysis andarrays processing of such values, composed data of separate two-dimensional frames that correspond to the results of the Earth's recording multidimensional naturally become more complicated than the usual two-dimensional imagessurface processing. The in a separate range. associated Accordingly, thethe time sequence of of remote such images corresponds to pronounced a four-dimensional second sourcespectral of problems with processing sensing data is the spatial random field (RF), one of satellite the dimensions is aindividual discrete time. Thus,ofthe analysis and processing of such heterogeneity of individual images. of Fig.which 1 shows fragments frames of multispectral images. multidimensional data naturally become more complicated than the usual two-dimensional images processing. The second source of problems associated with the processing of remote sensing data is the pronounced spatial heterogeneity ofThe individual satellitebyimages. 1877-0509  2018 Authors. Published Elsevier Fig. 1 shows individual fragments of frames of multispectral images. B.V. Peer-review under responsibility of KES International

1877-0509 © 2018 2018 The TheAuthors. Authors. Published by Elsevier Ltd. 1877-0509  Published by Elsevier This is an open access article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) B.V. Peer-review under responsibility of KES International Selection and peer-review under responsibility of KES International. 10.1016/j.procs.2018.07.208

50

Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58

2

Author name / Procedia Computer Science 00 (2018) 000–000

Fig. 1. Examples of satellite images Analysis of these fragments and other satellite images shows the presence of two important features. First, each image is composed of various objects: rivers, forests, fields, etc., in most cases separated by clearly observable boundaries. It is obvious that the properties of the images of these objects are not closely connected with each other. Secondly, the probabilistic characteristics of different points, even within the same object, are not constant. However, these characteristics within the object in a significant number of cases vary with a sufficiently low rate, which does not lead to visually observed boundaries. These properties, despite their apparent simplicity, remain outside the framework of most existing mathematical models. 3. Analysis of related work in image representation and processing A significant degree of novelty of the topic is confirmed by the fact that the methods we proposed were not previously studied in the modern works dealing with image processing. The analysis of works for 2016 shows that specialists are striving to use neural networks and large data in the field of image processing. However, this approach requires considerable computing resources. The authors carry out a new comparative study of algorithms for modeling multispectral images and image simulation algorithms with nonlocal properties in work6. It should be noted that this topic corresponds to the applied tasks of processing medical images and is of little use in the processing of satellite material, which does not reduce the importance of the study. Nevertheless, attempts to apply standard algorithms6 to satellite images do not lead to positive results. Authors of work9 explored a new problem of creating images from visual attributes. They propose to model the image as a foreground and background composition. These algorithms are applicable to solving image restoration problems. However, we have significant difficulties associated with the structure of characteristics set, which leads to a number of problems in theoretical constructions. Paper10 is devoted to the creation of an automatic system for monitoring building infrastructure process. The prototype model of the system uses several methods for character recognition to automatically measure actual progress in various parts of an infrastructure project using high-resolution remote sensing images. Unfortunately, the results obtained can only be applied to the technological process of their use in construction. We also should note the relevance of studies related to the processing of remote sensing data of the Earth. So the authors of study15 proposed an algorithm for calculating a three-dimensional model from several satellite images of the same terrain. An important feature of the method is its effective work, even if the images are observed at different times. But the model has some drawbacks. For example, it is difficult to forecast the algorithm results on complex structure images, including forests, rivers, fields and etc. The classification of satellite images is described in paper16. Authors made attempt to classify a number of objects. The use of matrix expressions makes the understanding of the article rather complicated. It should be assumed that the proposed algorithms based on matrix mixing will be very computationally expensive. The image modeling subject was continued in paper 17. The shortcomings of the work include the inability to compensate for various natural factors, e.g. vegetation indices, that affect the height of the simulated object. In paper20 the use of the hadoop system was proposed to accelerate the operation of the k-average segmentation algorithm for sufficiently large satellite images. However, the use of the kmeans algorithm is associated with significant errors when working with clusters with similar probabilistic characteristics, so the prospects for the practical application of the solution found are highly questionable.



Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

51 3

4. Doubly stochastic models based on autoregressions with multiple roots To overcome the difficulties mentioned earlier we will consider the processed satellite multispectral image and sequences of RF, given on a rectangular grid  of corresponding dimension so that its values xi  F ( x j , i , i ) , where i   , j  D ; D is a neighborhood of a point xi , determining its value; F () is some transformation;  i

are the model parameters, representing the implementation of an independent RF  i . Mathematical models having the parameters as the realization of auxiliary RF, are called doubly stochastic models (DSM)2,7,12. The important advantage of DSM is in the fact that the aggregate of auxiliary RFs { i } allows to form various, including nonuniform, realizations {xi } , which in their properties are close to real observations. As an important example of the DSM, let us consider the following autoregressive (AR) mathematical model of RF, defined on a rectangular N– dimensional grid   {i  (i1, i2 ,..., iN ) : (ik  1,2,..., M k ), k  1,2,..., N} : (1) x      x    ,i , j  , i

i

i,j

jDi

ij

i

i

where X  xi , i   is the simulated RF, defined on

 , {i , j , i , i : i  , j  Di } are model coefficients;

  { i , i  } is RF of white Gaussian noise; Di is the causal region of local states7 for the point i . Suppose that the coefficients

i , j  i 

i , j



l Di j

and  i ,

ri , j  i l , j   i , j i , j ;

 r 

l Di

 i of this model are RFs defined by the following relations:

l,j

ij

  i  i ;  i 

 r 

l Di

l,j

ij

   i  i ,

(2)

where {rl , j ,  i , j , rl , j , rl , j ,  i : i  , l  Dl } are constant coefficients; Di , j , Di , Di are regions of local states of RF {i , j },{ i } and { i } ;   { i , j , i , j  } ,

  {  i , i  } ,    {  i , i  }

are auxiliary

white RF. The random nature of the coefficients of model (1) makes it possible to use it to describe multivariate signals inhomogeneous in space and no stationary in time. The drawbacks of the model (1), determined by its AR nature, are the causality and anisotropy of the covariance functions (CF) generated by the RF. In addition, the presence of a large number of parameters determining the behavior of simulated RFs is specific for the model (1). Let us consider successively the possibilities to eliminate these shortcomings. The analysis showed that it is possible to single out a group from the whole variety of generators of multidimensional AR models, which makes it possible to simulate the RFs close to isotropic ones. Such a group is the AR model with multiple roots of the characteristic equations (ARMR). For a multidimensional grid  in the operator form, such models will have the following form: N

 (1   z k 1

where

k

k , 

dimension;

) xi  i ,

1 nk k

(3)

are model coefficients; n k are coefficients that determine the multiplicity of the model by k–s

z k1 is

l shift operator z k ( xi1,i 2 ,..., ik ,..., iN )  xi1,i 2 ,... i

N

separable, i.e. B(i , i ,...,i )   2  B (i ) , where 1 2 N x k k

k l

,... iN )

. The CF for the model under consideration is

Bk (ik ) is CF of the corresponding one-dimensional AR;

k 1

Bk (ik ) 

n k 1

 g ( nk , l , ik ) l 0

(nk  ik  1)!(2nk  l  2)!  k2 ( n  l 1) ; . g ( nk , l , ik )  2 2 k  l 1 l!(nk  1)!(nk  l  1)!(nk  ik  l  1)! (1   k ) k

For example, for an important case of a two-dimensional ARMR with roots of multiplicity 2, we get:

Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

52 4

B(i1 , i2 )   x2 (1 

1  12 1   22 | i1 |)(1  | i2 |) 1|i |  2|i | . 2 2 1  1 1  2 1

(4)

2

It can be shown 5 that when 1   i  1 such a CF can be described by equations close to elliptic equations. So, for the two-dimensional case the covariance function of such a RF is

 i2 i2  B(i1 , i2 )   x2 1  12  22  i12i22 (1  1 ) n (1   2 ) n A( 1 ) B(  2 ) ,  a b  1

where

2

a

1  12 ; (1  12 )(1  1 )

1   22 ; A(  )  1  1 ; B(  )  1   2 . b 2 1 2 (1   2 )(1   2 ) 1  12 1   22 Obviously, the sections B (i1 , i2 ) are ellipses close to B (i1 , i2 )  1 level asymptotically when 1   i  0 and when we have a large values of the multiplicity n1 and n 2 . Experiments show that significant proximity to isotropy can be clearly seen when n1  n2  2 . Also it should be noted that any ARMR is determined by the whole 3N

parameters, where N is the dimension of the described RF. When using the ARMR (3) as the basis for DSM (1) in the event that Di  Di , j  Di  Di for all i , j   , the number of DSM parameters is only 10  3N and

does not depend on the number of elements Di . As an important example, we write DSM based on ARMR of multiplicities (2,2):

xij  aij  FARMR2, 2 ( 1ij ,  2ij , bij ,  ij )  aij  2 1ij xi 1 j  2  2ij xij 1  4 1ij  2ij xi 1, j 1 

 12ij xi  2 j   22ij xij  2  12ij  2ij xi  2 j 1  1ij  22ij xi 1 j  2  12ij  22ij xi  2 j  2  bij ij

(5)

1ij  FARMR2, 2 (r11, r12 ,  1 , 1ij ) ,  2ij  FARMR2, 2 (r21 , r22 ,  2 ,  2ij ) ,  ij  FARMR2, 2 (r 1 , r 2 ,   , ij ) , bij  FARMR2, 2 (r 1 , r 2 ,   ,  ij ) are random values determined by the ARMR;  ij , ij , 1ij ,  2ij ,  ij are Gaussian

where

white RFs. Unfortunately, the CF of stochastic processes and RFs generated by DSM in the general case is described by very complicated expressions due to the presence of doubly correlations between the process itself and its basic parameters. Nevertheless, at slow (in comparison with the correlation interval of the RF being formed) changes in the base RFs based on the ARMR with multiple roots of multiplicity 2, the following formula can be obtained:

 i 2 j 2 j i 2 j  2 | i1 |   Ci   1 m 1 (2 j1  1)!! B(i1 , i2 )   x2 1 | i1 |  1  m21  (1  m 1 ) | i1 |  21  j  0 ,   i Ci2 j  2 2j mi 2 2 j (2 j2  1)!!  2 | i2 |   1 | i2 |  1  m2 2  (1  m 2 ) | i2 |  2 2  j  0  1

1

1

1

1

1

1

2

2

2

2

(6)

2

2

2

where



2 x

is variance of RF;

1ij and  2ij .

m  1 , m 2 ,  ,   2 2 1

2

are mathematical expectations and variances of auxiliary RFs

Fig. 2 shows the implementation of a RF on the basis of DSM, DSM on the basis of the ARMR and the corresponding CF.

(a) (b) (c) (d) Fig. 2 Realization of the doubly stochastic RF (a), RF generated by the ARMR (c) and their CF (b, d). Model parameters are following ( m  x  m  y  0.8 ,       0.001, r1x  r2 x  r1 y  r2 y  0.99 ). x y



Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

53 5

When describing real signals, it is also necessary to solve the problem of identifying the parameters of DSM. For a two-dimensional case, this means that the RF generated by the DSM must have characteristics of the real image {Z ij } . We can estimate DSM parameters r ,  , r ,  , r ,  : i  , l  D which determine the i i, j i i l , j l,j l , j





behavior of auxiliary RFs using an identification technique based on a combination of estimation in a sliding window and pseudo-gradient procedures8. For the two-dimensional case of ARMR-based DSMs, four parameters are evaluated at each point: mathematical expectation, variance, row-wise and column-wise correlation. The first two parameters are determined from the selection of pixels in the sliding window, the evaluation of the latter two is performed using a pseudo-gradient procedure that minimizes the functional M M 1  ( x  m )( x ( xi 1 j  mxij )( xij  mxij )  2 2 1 ij xij i 1 j  mxij ) ~ ~ ; J (r )  B1 ({xij })  B1 ({xij })  B2 ({xij })  B2 ({xij }) , where B1 ({xij })     M 1 (M 2  2) j 1 i  2   xij2  xij2 



B2 ({xij }) 

 



1

2

M 1 M  ( x  m )( x ( xij1  mxij )( xij  mxij )  1 xij ij 1  m xij )  ij  are statistics that determine the rate of change of   2    xij  xij2 M 2 ( M 1  2) j 2 i 1   1

2

xij } ; {~ 2 2 xij } , {mxij },{m~x ij },{ xij },{ ~x ij } are estimates of the mathematical expectations and variances for {xij } and {~

correlation

properties

horizontally

and

vertically

for

real

{xij }

and

simulated

images

obtained by a sliding window; M 1 and M 2 are width and height of the image. Fig. 3 shows an example of using the proposed method of identification and imitation of a real image.

(a) (b) (c) Fig. 3. Formation of an image with changing properties: fragment of an original image (a); implementation of a RF generated by a DSM based on the first-order ARMR (b); realization of a RF generated by a DSM based on the second-order ARMR (c). A simple visual analysis of the images in Fig. 3 shows the proximity of simulated images to real ones. This confirms the adequacy of DSM in the description of satellite images. The variances of errors also show the advantages of DSM model based on higher order ARMR. Variances of the fitting errors are following: variance for case (b) is 0.49; variance for case (c) is 0,37. 5. Filtering algorithm The use of DSM on the basis of ARMR enables to form effective algorithms for filtering real images. Indeed, suppose that a two-dimensional image can be described using model (5). In doing so, we will use the followng values of parameters: aij  0 , bij   x2 . Let the observation zij  xij  nij be the sum of information RF and White Gaussian RF{nij } having variance

 n2  M (nij2 ) . Let us solve the problem of restoring samples {xij } from observations {z ij } .



To do this we compose the following vector of elements xij  xxij ,  xij ,  yij



of length 6 M 1  6 , where

T xxij  xij xij 1 ...xi1 xi 1M ...xi 1,1 xi  2 M ...xi  2 j  2  and other parameters can be written using a similar equations. 1

1

2M 1  2 , which moves along the image (Fig. 4). Taking into account the regions in Fig. 4, model (5) can be written in compact form xij   ij ( xij )   ij , where  ij ( xij ) is matrix transformation, described in detail in work11,  ij  ( ij ,   1ij ,   2 ij ) . Let us note that any model

Then in sequential filtering we will use a window of size

(2) can be reduced to this form. Varying the complexity of the DSM, one can obtain the necessary accuracy of the estimation.

Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

54 6

Fig. 4. The region of local states and support elements for DSM based on ARMR of the multiplicity model (2,2) Using these relations and the method of recurrent nonlinear filtering, we can obtain the following twodimensional doubly stochastic filter (DSF): (7) xˆij  xˆ Eij  Bij ( zij  xˆ Eij ) , where

xˆ Eij

is the first element of the vector

xˆ Eij ; Bij  PEijC T Dij1 ; C  (1,0,...,0) ; Dij  CPEijC T   n2 ;

PEij  M {( xˆ Eij  xij )( xˆ Eij  xij )}  i', j 1 ( xˆij 1 ) Pij 1i', j 1 ( xˆij 1 )T  Vij . Filtering errors at each step are determined by the matrix Pij  ( E  Bij C ) PEij having size of (6M 1  6  6 M 1  6) . The obtained algorithm has important features. First, no large-scale matrix inversion is required, as in the case of recurrent Kalman estimation 3. Second, when forming an estimate at a point (i, j ) we use all the elements on the left and above from this point, and we make a new estimate for elements included into the vector

xˆ ij , which are previous for xˆ ij . Thus, this alleviates the

lack of causality of the AR construction used in DSM (2). Thirdly, the result of filtering is not only a set of estimates xˆ ij , but also the estimates of the correlation relations ˆ1ij and ˆ 2 ij . This feature allows the use of DSF not only to compensate for noise, but also as an element of algorithms for texture-correlation analysis, for example, in the segmentation of images. Fig. 5 shows an example of filtering a satellite image.

(a) (b) (c) Fig. 5. Satellite image filtering: (a) corresponds to original image, (b) corresponds to distorted image, (c) corresponds to filtered result). Fig. 6 shows the variance of the image filtering error versus the noise variance.

Fig. 6 Satellite image filtering efficiency. Curve A1 is vector Kalman filter without interpolation, curve A2 is discrete Wiener filter, curve A3 is Kalman filter with interpolation, curve A4 is DSF. A simple analysis of the curves in Fig. 6 shows the effectiveness of the proposed filtering method, the gain of which compared to other filters is up to 120% in terms of the variance of the estimation error. The presented filter can be generalized for the case of processing a multispectral image. Here we assume that all N of two-dimensional frames of this multispectral image are set by the RF {xijk }, k  1,2,..., N , placed on the same

grid  . We introduce matrix R which characterizes correlations between individual frames. To do this, we use the following statistics M

Rx 3 (c, d ) 

N1 N 2

 ( z m 1 i 1 j 1

c mij

d  mmc )( z mij  mmd )

M mc  md

,

(8)



Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

55 7

where M is number of multispectral images in the sample; z mij is scalar value of calculation of c-th multispectral c

image at the point (i, j ) on c–th frame; mmc and  mc are estimates of the mathematical expectation and the standard deviation on c–th frame of m–th multispectral images in the whole sample. We will also assume that the features of registering objects on these frames ensure the closeness of intra-frame correlation characteristics on all frames. Then the first frame of such a multispectral image can be described using the model (5), and for the description of the second and subsequent frames one can use the following relation: k M1 M 2 (9) x k  R(l , k ) x l  v k , ij

 i 1

ij



i11 j11

i1 j 1

i1 j 1

where vi1 j1 are elements of a triangular matrix V , such that example, first frame in accordance with (6);

 ik1 j1

VV T ;

B is covariance matrix of a separate, for k

2 2 are white noise samples with variance  k   1  R( p, k ) . p 1

Matrix V can be obtained using the well-known Cholesky transform. Relations (9) allow us to modify the filter (7) for the case of a multispectral image. To achieve this it is necessary to supplement the previously considered vector x xij by scalar samples xij2 ,..., xijN and re-write the non-linear equality

xij   ij ( xij )   ij taking into account the corresponding changes in the vector  ij . Then for filtering we can use relation (7), taking into account that at each point with coordinates (i, j ) we have not only one scalar observation z ij ,

but vector of such observations, whose values correspond to the values of the multispectral image at the point (i, j ) on the corresponding frame. Fig. 7 shows fragments of individual frames of three test multispectral satellite images obtained from the Landsat 8 spacecraft. In each processed multispectral image, eight spectral ranges were used. The level of interframe correlation was from 0.61 (ranges "SWIR-Blue") to 0.99 ("Blue-Green" bands).

(a) Image 1 (b) Image 2 (c) Image 3 Fig. 7. Fragments of multispectral images used to assess the effectiveness In order to analyze the efficiency of the algorithms found, the images were mixed with white RFs of different intensity q   x2  n2 , after which their filtering was consistently performed using a different number of spectral ranges. Table 1 shows the ratio of the variance of filtering errors versus the variance of the image (MSE/  x2 ). Table 1 Filtering Multispectral Images Number of ranges Image 1 used for filtering q=2 q=5 q=10 1 0.083 0.042 0.033 2 0.067 0.037 0.029 3 0.064 0.036 0.029 4 0.058 0.033 0.027 5 0.053 0.031 0.026 6 0.052 0.031 0.024 7 0.049 0.028 0.023 8 0.049 0.027 0.023

q=2 0.068 0.059 0.056 0.051 0.048 0.046 0.043 0.042

Image 2 q=5 0.036 0.032 0.029 0.028 0.027 0.026 0.024 0.021

q=10 0.028 0.026 0.024 0.022 0.021 0.021 0.020 0.019

q=2 0.081 0.067 0.063 0.056 0.054 0.054 0.051 0.050

Image 3 q=5 0.039 0.036 0.035 0.033 0.032 0.031 0.029 0.028

q=10 0.034 0.028 0.027 0.026 0.025 0.025 0.024 0.022

The analysis of the data presented in Table 1 shows that with the increase in the number of spectral bands used for processing, the effectiveness of this treatment is substantially increased. So the use of all available eight spectral

Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

56 8

ranges allows to reduce the variance of the filtering error by an average of 80% compared to the results of processing a single frame. Fig. 8 is a graphical illustration of the process of sequentially combining the estimates in the previous image and observations on the current multispectral image.

Fig. 8 Processing a sequence of multispectral images The processing algorithms presented in this paper can be combined in the framework of the following scheme for processing the time sequence of multispectral images: initial evaluation of the parameters of the DSM on an individual frame of the multispectral image; performing DSF of the entire multispectral image; identify the boundaries between objects and the objects themselves on the multispectral image; perform non-causal filtering of individual frames of the multispectral image; perform tensor filtering of the next multispectral image; refinement of local correlation and probabilistic parameters on the current image. Table 2. Processing results Algorithm

t=1

Image 1 t=2 t=4

t=8

t=1

Image 2 t=2 t=4

t=8

t=1

Image 3 t=2 t=4

t=8

Anisotropic multiresolution (MR) LPA Denoising (BLUE range)

0.028

0.026

0.029

0.028

0.02

0.021

0.021

0.022

0.027

0.031

0.028

0.027

Algorithm (6) (BLUE range)

0.033

0.035

0.033

0.34

0.028

0.026

0.032

0.031

0.034

0.035

0.034

0.031

Algorithm (6) for a single multispectral image

0.027

0.028

0.028

0.026

0.021

0.023

0.022

0.02

0.028

0.029

0.027

0.028

Using a non-causal forecast (8)

0.023

0.024

0.023

0.023

0.018

0.019

0.018

0.017

0.022

0.024

0.023

0.024

Algorithm based on the use of a tensor filter

0.023

0.020

0.018

0.017

0.018

0.015

0.014

0.014

0.022

0.018

0.017

0.017

Table 2 summarizes the results of the analysis of the efficiency of the satellite image processing procedures found in Fig. 7 for cases of processing a single frame, the entire multispectral image, additional non-causal processing using a variable-size sliding window, tensor filtering the time sequence of the multispectral images. For



Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

57 9

comparison, the results of using the algorithm Anisotropic multiresolution (MR) LPA Denoising are given. The signal-to-noise ratio for all cases is q=10. An analysis of the presented data shows that the DSF, when processing a single two-dimensional frame, is slightly inferior to the reference LPA algorithm. However, the correlation fields obtained with the help of the DSF allow joint processing of all frames of the multispectral image, generate a non-causal prediction (8), and process the time sequence of the multispectral images. Using all above-mentioned techniques one can increase the normalized MSE gain by 50% in comparison with the LPA algorithm. 6. Boundaries detection We can use filtering algorithm for detecting boundaries. The main task is to filter image from different image corners. Then we can use adaptive properties of DSF to distinguish boundaries on image after subtracting the result of filtration (the first corner of image) from the result of filtration (second corner of image). we can illustrate result of the proposed algorithm. Fig. 9a, 10a show images simulated by the two-dimensional doubly-stochastic model, as well as real satellite images. Fig. 9b, 10b show images composed of squares of the difference between the estimates of the forward and backward DSF. Fig. 9c, 10c show the detected borders.

a b c Fig. 9 Detecting the borders of the simulated image

a b c Fig. 10 Detecting the borders of a satellite image Thus, we present an algorithm for isolating borders between regions with slowly varying statistical and correlation properties. The applicability of the algorithm for processing two-dimensional spatially inhomogeneous images is also shown. Conclusion Thus, we have investigated doubly stochastic models on multiple roots. In this paper we introduce a new family of image filtration procedures and their sequences. The peculiarity of this procedure is the possibility of simultaneous estimation of brightness and correlation properties of the image. This makes it possible to process spatially inhomogeneous images without performing labor-intensive preliminary segmentation of them. The algorithms found can easily be generalized to the case of processing of multispectral satellite images and time sequences of such images. A comparative study of the proposed procedures and the well-known LPA algorithm is performed. As a result, it was found that the proposed approach allows to obtain a gain of up to 40% on the average dispersion of the estimation error. This allows us to recommend the found procedures for the preliminary processing of real satellite material when solving problems of image reconstruction and detecting anomalies of various kinds on them.

58 10

Konstantin Vasiliev et al. / Procedia Computer Science 126 (2018) 49–58 Author name / Procedia Computer Science 00 (2018) 000–000

Acknowledgements The work was supported by a grant from the RFBR and the Administration of the Ulyanovsk Region No. 16-41732027, and also by the RFBR grant No. 17-01-00179 A. References 1. Woods J.W. Two-dimensional Kalman filtering //Topics in Applied Physics, Berlin, 1981, v.42, pp.155-208. 2. Woods J.W., Dravida S., Mediavilla R. Image Estimation Using Doubly Stochastic Gaussian Random Field Models // Pattern Analysis and Machine Intelligence, issue №2, vol. 9 – February, 1987, pp. 245-253. 3. Paliy, D., A.Foi, R. Bilcu, V. Katkovnik, “Denoising and Interpolation of Noisy Bayer Data with Adaptive Cross-Color Filters”, SPIE-IS&T Electronic Imaging, Visual Communications and Image Processing 2008, vol. 6822, San Jose, CA, January 2008. 4. Charles A. Bouman Model Based Imaging Processing. Purdue University, 2013, 414 pp. 5. Vasil’ev K.K., Dementev V.E. Analysis of the correlation properties of autogressive random fields // Pattern Recognition and Image Analysis, 2005, Vol 15, No. 2. – p. 340-342 6. M. Lorenzi, I.J. Simpson, A.F. Mendelson, S.B. Vos, M.J. Cardoso, M. Modat, J.M. Schott, S. Ourselin Multimodal Image Anal ysis in Alzheimer’s Disease via Statistical Modelling of Non-local Intensity Correlations// Scientific Reports 6, Article number: 22161 (2016) doi:10.1038/srep22161 - 8 p. 7. Vasil'ev K.K., Dement'ev V.E., Andriyanov N.A. Doubly stochastic models of images // Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2015. Т. 25. № 1. P. 105-110. 8. Vasil'ev K.K., Dement'ev V.E., Andriyanov N.A. Application of mixed models for solving the problem on restoring and estimating image parameters // Pattern Recognition and Image Analysis (Advances in Mathematical Theory and Applications). 2016. V. 26. № 1. P. 240-247 9. X. Yan, J. Yang, K. Sohn, H. Lee Attribute2Image: Conditional Image Generation from Visual Attributes // 19 pages, accepted by ECCV 2016, The 14th European Conference on Computer Vision (2016) 10. Automated progress monitoring system for linear infrastructure projects using satellite remote sensing / Ali Behnam, Darshana Charitha Wickramasinghe, Moataz Ahmed Abdel Ghaffar, Tuong Thuy Vub, Yu Hoe Tang, Hazril Bin Md Isa // Automation in Construction. Volume 68, 2016. p. 114–127. 11. Andriyanov, N.A., Vasiliev, K.K., Dementiev, V.E. Anomalies detection on spatially inhomogeneous polyzonal images // CEUR Workshop Proceedings Volume 1901, 2017, P. 10-15 12. Vasiliev, K.K., Dementiev, V.E., Andriyanov, N.A. Filtration and restoration of satellite images using doubly stochastic random fields //CEUR Workshop Proceedings, Volume 1814, 2017, P. 10-20 13. https://landsat.gsfc.nasa.gov/ 14. Image Local Polynomial Approximation (LPA) and its Applications. (Draft, March 2011) Guennadi Levkine (email: [email protected]) Vancouver, Canada. 15. Gabriele Facciolo, Carlo De Franchis, Enric Meinhardt-Llopis Automatic 3D Reconstruction from Multi-date Satellite Images // Computer Vision and Pattern Recognition Workshops (CVPRW), 2017 IEEE Conference, - 8 p. 16. K. Radhika, S. Varadarajan Multi class classification of satellite images // Innovative Mechanisms for Industry Applicati ons (ICIMIA), 2017 International Conference, - 11 p. 17. K. Jacobsen Problems and limitations of satellite image orientation for determination of height models // The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-1/W1, 2017 ISPRS Hannover Workshop: HRIGI 17 – CMRT 17 – ISA 17 – EuroCOW 17, 6–9 June 2017, Hannover, Germany, p. 257-264 18. Favorskaya M.N., Jain L.C. Theoretical and practical solutions in remote sensing // Intelligent Systems Reference Library. 2018. V. 135. P. 19. 19. Vasiliev K. K., Andriyanov N. A. Synthesis and analysis of doubly stochastic models of images// 2nd International Worksho p on Radio Electronics and Information Technologies, REIT 2017; Yekaterinburg; Russian Federation; 15 November 2017. CE UR Workshop Proceedings, Volume 2005, 2017, P. 145-154 20. Mengzhao Yang, Haibin Mei and Dongmei Huang An effective detection of satellite images via k -means clustering on hadoop system // International Journal of Innovative Computing, Information and Control, Volume 13, Number 3, June 2017 p. 18-27