- Email: [email protected]
Piston sensing for sparse aperture systems with broadband extended objects via a single convolutional neural network
Optics and Lasers in Engineering 128 (2020) 106005
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Piston sensing for sparse aperture systems with broadband extended objects via a single convolutional neural network Xiafei Ma a,b,c, Zongliang Xie a,b,c,d,∗, Haotong Ma a,b,c,∗, Yangjie Xu a,b,c, Dong He a,b,c, Ge Ren a,b,c a
Key Laboratory of Optical Engineering, Chinese Academy of Sciences, Chengdu 610209, China The Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China c University of Chinese Academy of Sciences, Beijing 100039, China d State Key Laboratory of Pulsed Power Laser Technology, Hefei 230037, China b
a r t i c l e Keywords: Deep learning Piston sensing Sparse aperture systems
i n f o
a b s t r a c t It is crucial for sparse aperture systems to preserve imaging quality, which can be addressed when fast corrections of pistons within a fraction of a wavelength are available. In this paper, we demonstrate that only a single deep convolutional neural network is sufficient to extract pistons from wide-band extended images once being appropriately trained. To eliminate the object characters, the feature vector is calculated as the input by a pair of focused and defocused images. This method possesses the capability of fine phasing with high sensing accuracy, and a large-scale capture range without the use of combined wavelengths. Simple and fast, the proposed technique might find wide applications in phasing telescope arrays or segmented mirrors.
1. Introduction Optical sparse aperture systems, including segmented telescopes [1] and telescope arrays [2], are capable of providing high resolution of large monolithic apertures with reduced costs and weight. It is crucial to control the optical path lengths between sub-apertures, also known as pistons, within a fraction of a wavelength for the optimal imaging performance, or the imaging resolution will decrease sharply. Many methods proposed for piston sensing need unresolved sources so as to detect pistons from the interference fringes [3] or modulation transfer functions in Fourier domain [4,5]. Extracting pistons from extended objects is challenging since the convolutions between the targets and the point spread functions smooth the fringes. Phase diversity technique can realize piston sensing whatever the size of the object with focused and defocused images [6]. Though, it needs heavy computational burden and suffers 2𝜋 ambiguity. Alternatively, phasing approaches using optimization procedures can also handle extended targets [7,8], but a large capture range and instant correction can not be obtained with these approaches. Recently, with deep learning developing fast, researchers have demonstrated that the state-of-art convolutional neural network (CNN) can directly estimates wavefront represented by Zernike coefficients from the intensity images of not only point sources but also specified extended objects [9]. CNNs have also been applied for piston sensing. Guerra-Ramos et al. open a new door of deep learning-based piston sens∗
ing by means of simulation [10]. They trained 2 shallow CNNs, one for piston step values and the other for ambiguity range, and used 4 different wavelengths, achieving high sensing accuracy and an ample capture range. Then a single network for piston sensing is developed with broadband illumination [11]. However, these methods are still limited to the point sources. Li et al. have proposed a CNN-based piston sensing technique for extended objects with a pair of focused and defocused images [12]. In their study, 5 CNNs are trained with simulated dataset and 4 different wavelengths are also utilized for disambiguation. As a result, their technique succeeds in correcting the pistons ranging in [0, 10𝜆] to [0, 𝜆], and then traditional phase diversity is jointly used for fine phasing. In this paper, we demonstrate that using only a single deep convolutional neural network (DCNN) is sufficient to detect pistons from broadband extended images, and a large capture range up to the coherent length of the wide-band light can be achieved without combined wavelengths. In our implementation, focused and defocused broadband images are used to calculate the feature vectors, which are independent of the object characters. Then we train a DCNN by using the feature vectors as the inputs and the corresponding pistons as the outputs. As the training is done with advanced deep learning strategies, the DCNN is capable of serving as a piston sensor owning high sensing accuracy and an ample capture range. For centrosymmetric configurations, the feature vector might not sensitive enough to changes of pistons, thus affecting the piston sensing performance. This issue can be solved by using a diaphragm to break the symmetry.
Corresponding authors. E-mail addresses: [email protected] (Z. Xie), [email protected] (H. Ma).
https://doi.org/10.1016/j.optlaseng.2020.106005 Received 5 September 2019; Received in revised form 18 December 2019; Accepted 3 January 2020 0143-8166/© 2020 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license. (http://creativecommons.org/licenses/by-nc-nd/4.0/)
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
There are mainly three advantages associated with the reported method with regard to the previous work for extended objects. Firstly, compared with the usage of multiple CNNs, using only a single DCNN can’t only reduce the training complexity, but also cut down the recognition time. Secondly, the proposed technique discerns pistons directly from the broadband intensity summations rather than the intensities of separated wavelengths, thus removing the requirement of spectral optics and compacting the physical setup. Finally, it can work as an independent piston sensor without the joint use of other piston sensing approaches. This paper is structured as follows: we describe the optical imaging model of data generation and the implementation of the DCNN-based method in Section 2. The results of several simulations and detailed discussion are presented in Section 3. Concluding thoughts are provided in Section 4.
𝑁 ( ) ( ) ∑ ( ) 2𝜋𝑖 𝑃 (𝐮, 𝜆) = 𝑝 𝐮 − 𝐮1 + 𝑝 𝐮 − 𝐮𝑛 exp 𝑂𝑃 𝐷𝑛 , 𝜆 𝑛=2
2.1. Optical model of data generation As a data-driven technology, the training results of the CNN are supremely dependent on the training dataset. In the deep learningbased piston sensing methods for point sources, the deep models can directly establish the mapping relation between pistons and corresponding point spread functions. However, when the detecting targets are extended objects instead, the end-to-end detection mode that raw images corresponding to different pistons are directly used as the network inputs without preprocessing would markedly increase the scale of the database and lead to worse generalization, since different imaging objects own different imaging characteristics. It is difficult and makes no sense to construct training datasets for every object, so we turn to construct a feature vector which can free the method from the limitation of the imaging content. Thanks to the previous work of Kendrick [13], a feature vector model using a pair of focus diversity images is proposed, which can be expressed as 𝐺 ⋅ 𝐺𝑑∗ − 𝐺∗ ⋅ 𝐺𝑑
𝐺 ⋅ 𝐺∗ + 𝐺𝑑 ⋅ 𝐺𝑑∗
,
(1)
where G and Gd are the Fourier transforms of the focused and defocused images, respectively, while G∗ and 𝐺𝑑∗ are their complex conjugates. Such a feature vector is only related to the pistons of the sparse aperture system while being independent of the imaging target, which is exactly what the deep learning pursues. Thus, the feature vectors are used here as the training inputs. What is different is that the calculations of the character transform make use of polychromatic intensity summations rather than separated intensity distributions of multiple wavelengths for a large capture range, which reduces the computational budget and relaxes the physical setup. The broadband image captured on the focal plane can be modeled as 𝐼 (𝐱) = 𝑜(𝐱) ⊗ ℎ(𝐱),
(2)
where o is the ideal intensity of the imaging object, h is the focal broadband point spread function of the sparse aperture system, ⊗ is the convolution operator, and x is a 2D vector in the image plane. According to the imaging principle, the focal point spread function at a specific imaging wavelength 𝜆 can be represented as ℎ(𝐱, 𝜆) = |FT(𝑃 (𝐮, 𝜆))| , 2
(3)
where P(u, 𝜆) is the generalized pupil function corresponding to the wavelength 𝜆, FT(⋅) represents Fourier transform and u is a 2D vector in the pupil plane, respectively. The alignment errors of sparse aperture systems, involving tip-tilt and pistons seriously affect the imaging performance. Shack–Hartman wavefront sensors can handle the tip-tilt detection but can’t sense the
(4)
where the first sub-aperture is regarded as the reference, N is the total number of sub-apertures, p is the binary pupil function of an unit, un is the center position vector of the nth sub-aperture, and OPDn is the relative piston between the nth sub-aperture and the reference. Ranging from 𝜆1 to 𝜆m , the broadband point spread function can be modeled as an integral form [14] ℎ (𝐱 ) =
2. Method
𝑀𝑠ℎ𝑎𝑟𝑝𝑛𝑒𝑠𝑠 =
discontinuous pistons. The purpose of this paper is to detect the pistons, so here the assumption that the other phase distortions are eliminated is made, as commonly used in other researches on piston sensing [5,10]. With pistons taken into consideration, the corresponding generalized pupil function can be expressed as
𝜆𝑚
∫𝜆1
ℎ(𝐱, 𝜆)𝑠(𝜆)𝑑𝜆,
(5)
where s(𝜆) is the spectral coefficient at each wavelength. The defocus broadband image takes the similar physical model. The difference is involved in the generalized pupil function, which can be represented as 𝑃𝑑𝑒𝑓 𝑜𝑐𝑢𝑠 (𝐮, 𝜆) = 𝑃 (𝐮, 𝜆) ⋅ 𝐴(𝐮) exp(𝑗𝜑(𝐮, 𝜆)),
(6)
where A is the binary pupil function of the minimum virtual aperture surrounding all the sub-apertures, denotes the defocus aberration loaded. 2.2. DCNN implementation After training dataset preparation is completed, we construct the DCNN in TensorFlow and train it for piston sensing. As discussed above, the CNN used in the previous work for extended objects is too shallow to discern the pistons of all the sub-apertures from a frame of feature vector [10], resulting in the requirement of multiple CNNs. Here we establish a single DCNN with more network layers, in possession of powerful capability of characterizing a complicated analytic model. This DCNN learns the statistical transformation between the broadband feature vectors and the pistons of all the sub-apertures via a large amount of samples training. Once trained, this network uses the feature vectors computed by the broadband focused and defocused images as inputs, and quickly output all the pistons of the system, as shown in Fig. 1. The deep network in the proposed method has 29 layers in total, whose specific topology is also described in Fig. 1. The convolutional layers with ReLU activation functions are used for feature extraction. The max-pooling layers can diminish the parameters of the network and lighten the computation burden of the training. The training process operates through forward and back propagation until the loss function converges to a global optimum solution, which is defined as the root mean squared error (RMSE) between the output pistons and the ground truths. It’s noted that some advanced strategies are implemented here for satisfactory learning, such as Batch Normalization used to prevent gradient vanishing, dropout and data augmentation techniques aiming at avoiding over-fitting. 3. Results 3.1. Performance on the three-aperture system Similar to the previous work [9,10], we use simulated data to validate the efficiency of the proposed deep learning-based technique. We first model a three-aperture system as shown in Fig. 2 to validate the feasibility of the proposed method, where the diameter of sub-aperture is 10 mm. And in the focal plane with a distance of 500 mm, a camera with the pixel pitch of 1.67 𝜇m is settled to record the raw extended images.
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 1. Principle of the proposed DCNN-based piston sensing method and detailed schematic of the DCNN architecture, indicating the number of layers, the property of each layer, etc. The feature vector, calculated by a pair of focus diversity broadband images, is input into the DCNN and the pistons of all the sub-apertures are outputs.
Fig. 2. Configuration of the three-aperture system.
To start with, it is of the essence to generate the training database. We use a USAF resolution target as the object, illuminated by broadband light ranging from 500 nm to 600 nm. The capture range achieving the coherence length is within ±3 𝜇m. The wide-band imaging is simulated by accumulating 21 monochrome point spread functions with the sampling interval of 5 nm and equal weight contribution. To construct training data for the three-aperture system we model in this study, 10,000 sets of random pistons ranging from −3 𝜇m to 3 𝜇m are introduced into the sub-apertures and corresponding focused and defocused images are captured, of which a group of examples are shown as Fig. 3(a1) and (a2). Based on the raw images, 10,000 broadband feature vectors are calculated by using Eq. (1), of which 6 frames are presented as examples in Fig. 3(b1)–(b6). As described above, the feature vectors are ready for the utilization of the inputs while the corresponding pistons ready for the outputs. After training procedure is done, testing dataset, which is exclusive of the training images, is needed to test the performance of the DCNN trained for piston sensing. Besides, to demonstrate the independence of the DCNN on object characters, another image different from the one for training is used as the imaging target to generate testing datasets. We randomly load 1000 sets of pistons ranging from −3 𝜇m to 3 𝜇m on the system and acquire the focus diversity images, of which a pair of examples are given as Fig. 4(a1) and (a2). 1000 broadband feature vectors are then computed, of which 6 frames of samples are presented in Fig. 4(b1)–(b6). They are inputted into the DCNN and the predicted pistons are outputted quickly. The estimation for a single image takes
Fig. 3. Examples of the training database for the three-aperture system. (a1) and (a2) A pair of focused and defocused broadband image samples using the USAF target. (b1)–(b6) The feature vector samples corresponding to different pistons.
Fig. 4. Examples of the testing database for the three-aperture system. (a1) and (a2) A pair of focused and defocused broadband image samples using Lena picture. (b1)–(b6) The feature vector samples corresponding to different pistons.
only about 12.4 ms using the CPU of Intel(R) Core(Tm) i7-6800K and the graphics processing unit (GPU) of NVDIA GeForce GTX 1080 Ti. The evolution of the loss function over the course of the training for the three-aperture system is exhibited in Fig. 5, from which we can see that the DCNN converges after about 4000 iterations, indicating that the network has probably learnt the complex mapping relation between the broadband feature vectors and pistons. Here we take the average value of the RMSE over all testing samples as the assessment criteria, which is
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 5. Evolution of the loss function during training for the three-aperture system.
Fig. 6. 500 RMSEs randomly selected from the testing dataset for the three-aperture system.
about 12 nm for the three-aperture system. The RMSEs between the predicted pistons and the ground truths of 500 testing samples randomly selected are shown in Fig. 6, from which it is easy to be figured out that the sensing deviation can be exactly controlled within 40 nm. Specifically, with respect to the other criterion of 0.1𝜆 at the wavelength of 600 nm referred to the piston error tolerance in the analysis of Chung et al. [2], the sensing accuracy of the trained DCNN reaches 100%. To better illustrate the sensing capability, the distributions of the residual RMSEs on training samples and testing samples are depicted in Fig. 7, from which we can see that the results of these two datasets are highly consistent. The testing results demonstrate that our proposed technique can be used for fine phasing while the previous CNN-based method for extended objects can only perform coarse phasing due to the sensing error within 𝜆. 3.2. Performance on the six-aperture system The sub-aperture number is a key factor influencing the sensing accuracy of the proposed technique. There seems contradiction between the systematic complexity and the required accuracy due to two aspects. First and foremost, the complexity of the imaging model will greatly increase with the number of sub-aperture increasing, which means the non-linear mapping relation is more complex and it is more difficult for the network to fit that function. Second, since there are more unknown pistons to be estimated, the training data needed and the amount of computation will also greatly increase. It is necessary to evaluate the performance of this method on imaging systems with more sub-apertures. In this section, a sparse aperture system with 6 units is built, of which the configuration is shown in Fig. 8. A sub-aperture is located in the center as a benchmark and the other 5 sub-apertures are distributed
Fig. 7. Distributions of RMSEs over all the training and testing dataset for the three-aperture system.
along a circle with the radius of 145 mm. The diameter d of the subaperture is 100 mm and the focal length is 6 m. 100,000 samples for training and another 1000 samples for testing are collected in the case of the six-aperture imaging system. The estimation time for a single image is about 12.7 ms. As expected, the sensing accuracy for the six-aperture systems decreases slightly with respect to that for the three-aperture system. It can be recognized that most errors
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 8. Configuration of the six-aperture system.
Fig. 11. Wavefront aberrations introduced in the three-aperture system.
Fig. 9. 500 RMSEs randomly selected from the testing dataset for the sixaperture system.
The training process includes two parts: forward propagation of data and the back propagation of error. The parameters are updated through repeated iterations of these two steps. When the number of sub-apertures increases, the non-linear relation is more complex so that more training samples and iterations needed increase greatly. Because of the huge amounts of iterations, it takes several hours or days to accomplish the training process. However, when we utilize the trained network to detect the piston, only forward propagation performs once. The increase of the neurons in output layer has little effect on piston estimation time. In fact, our simulations show that the estimation time for the six-aperture system is very close to that for the three-aperture system. 3.3. Performance considering aberrations
Fig. 10. Distributions of RMSEs over all the training and testing dataset for the six-aperture system.
are located within the region of [0, 60] nm from the RMSE values of 500 random testing samples which are shown in Fig. 9. Also, as shown in Fig. 10, there is a certain difference between the sensing results of the training and testing dataset. Though, the accuracy decreasing is acceptable and the proposed technique still presents much higher detecting accuracy on the six-aperture system than the previous one for extended objects. Specifically, the sensing accuracy of the trained DCNN reaches 91.6% with respect to the criterion of 0.1𝜆, and the average RMSE over the test dataset can achieve 32 nm, which suffices for fine phasing.
Note that the simulations above are carried out in the assumption that the tilt-tip and other sub-aperture aberrations are eliminated entirely. However, these aberrations can be hardly eliminated entirely in experiments. There are some inevitable imperfections caused by manufacture errors in the system. Therefore, another simulation is also implemented, in which slight wavefront aberrations describing the system distortions are introduced in the three-aperture imaging system, to evaluate the influence of aberrations on sensing accuracy. The RMSE of each aberrations with 11 Zernike coefficients excluding the first one is set as 0.05𝜆 (𝜆 = 600 nm), and the introduced aberrations for training samples are shown in Fig. 11. The added system aberrations are the same along all the training. However, considering the randomness of the aberration in real piston detection scenario, we introduce dynamic aberrations in testing samples, which are different from those for training.The specific training and testing results are shown as the histogram in Fig. 12, which indicate that the sensing performance is similar to that without the presence of other aberrations. The average RMSE is 15 nm over all testing samples and the detecting accuracy also achieves 100% with regard to the criterion of 0.1𝜆. The aberrations affect the intensity distributions in an irregular way, and consequently disturb the mapping process between the feature vectors and pistons. However, the proposed method can still learn the distorted mapping functions via a large amount of samples and shows a good adaptability to these influence factors, which means that it is robust against aberrations and can be useful in practical systems. According to all the testing results above, it is demonstrated that the proposed deep learning-based technique is efficient for extracting pistons from extended objects with a broad capture range, high precision, and fast response time.
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 12. Distributions of RMSEs over all the training and testing dataset for the three-aperture system considering aberrations.
Fig. 13. Configurations of two Golay-6 arrays with an additional sub-aperture.
3.4. Discussion on impacts of redundancy The redundancy in the spatial frequencies has a considerable influence on the performance of some cophasing algorithms [15,16]. According to the previous researches, the estimation error of phase diversity technique increases with the level of redundancy, as there are two or
more pairs of sub-apertures contributing to the same spatial frequency peaks in redundant configuration, which can lead to an ambiguity. In this section, we discuss the impacts of array redundancy on the proposed piston sensing method. Two kinds of redundant arrays, centrosymmetric and noncentrosymmetric, are involved. First, we analyze the influence of redundancy of the noncentrosymmetric configuration by implementing additional simulations. Two Golay-6 arrays with an additional sub-aperture shown in Fig. 13 are modeled. Sub-apertures number 4 and 5 in configuration Golay-6-1 are more redundant than other ones for the baselines 4-6/5-3, 5-4/3-6/6-1, 4-1/5-6 are all redundant. For configuration Golay-6-2, the additional sub-aperture is placed at the center, and there is no redundancy. As the reference [16] concludes, for phase diversity technique, the estimation errors would be visibly greater on sub-apertures 4 and 5 in configuration Golay-6-1 due to the redundancy, while the accuracies on all sub-apertures in configuration Golay-6-2 would be at similar level. In our simulation, two training datasets composed of 120,000 samples are generated to train the sensing networks for Golay-6-1 and Golay-6-2, respectively. The testing results using other 1000 samples described in Fig. 14 show that there is no evident degradation caused by the redundancy on the estimation accuracies of sub-apertures 4 and 5 in configuration Golay-6-1. On the contrary, the sensing performances for Golay6-1 and Golay-6-2 are almost consistent, indicating few effects of redundancy. From the analysis, the performance of the proposed method using CNN seems not subject to the redundancy in a noncentrosymmetric array. Secondly, we check the effectiveness of the proposed approach on redundant centrosymmetric arrays, which are commonly used in segmented telescopes. Another simulation is carried out with a centrosymmetric segmented mirror shown in Fig. 15(a). 120,000 samples are used to train the network and the loss function converges indeed. However, the average RMSE over 1000 testing samples reaches up to 191 nm, which apparently can not meet the request of fine phasing. We infer that it is because the feature vector for the redundant centrosymmetric array might not be sensitive enough to the changes of pistons due to the full redundancies, thus making the mapping relations difficult to be precisely modeled by a DCNN and leading to decrease in sensing accuracy. It’s an interesting issue and needs our further study. Though, we also give a solution to solve this problem. We place a circular diaphragm in front of one mirror segment as shown in Fig. 15(b), which breaks the configuration symmetry. Then another simulation is implemented and 120,000 samples are used to train the network. The average RMSE of the trained network over 1000 testing samples is 47 nm, which is much better than the result on the centrosymmetric structure. Fig. 16 shows the distributions of RMSEs over all the testing dataset for the config-
Fig. 14. Influences of redundancy on the estimation error.
X. Ma, Z. Xie and H. Ma et al.
Fig. 15. Configurations of segmented mirror without (a) and with (b) a circular diaphragm.
Optics and Lasers in Engineering 128 (2020) 106005
plicability of the proposed technology in practical environments. The configuration redundancy has few effects on the sensing accuracy for a noncentrosymmetric array. For the centrosymmetric configuration, the method using a single CNN still works with a diaphragm used to break the symmetry. Based on the above advantages, it is believed that the proposed technique might find wide applications in phasing telescope arrays or segmented mirrors. For actual implementations, further work needs to consider the effect of atmospheric turbulence. Since the sparse distributed pupils decompose the monolithic turbulence distortion into dominant segmented piston and tip-tilt, further study on how to use CNN to simultaneously sense both the piston and tip-tilt is needed in future developments. It also requires a fast detection to correct atmospheric turbulence. There are several means to reduce the sensing time. First, we can try to reduce the image size on the premise of retaining useful information. Second, optimizing the configuration of the network to decrease the parameter size is an alternative approach. Besides, a better equipped computing device can also accelerate computation speed. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work is supported by Open Research Fund of State Key Laboratory of Pulsed Power Laser Technology (SKL2018KF05); Excellent Youth Foundation of Sichuan Scientific Committee (2019JDJQ0012); Youth Innovation Promotion Association, CAS (2018411); CAS “Light of West China” Program; Young Talents of Sichuan Thousand People Program. References
Fig. 16. Testing results for configurations (a) and (b).
urations (a) and (b). With this approach, the proposed method using a single CNN still can be applied to centrosymmetric configurations. Solutions to achieve higher sensing precision will be investigated and further study on the impact of pupil configuration is also needed in the future work. 4. Conclusions In this paper, we propose the use of a single DCNN to extract pistons from broadband extended images and prove the effectiveness by simulated data. The training is time-consuming for CNN, but once trained, the network-based techniques work fast without iterative process involved. Using only a single DCNN here can reduce the training complexity, as well as further improve the recognition speed. It is an image-based piston sensing method, thus requiring no additional optical equipment to be installed except for the usage of focus division common in phase diversity. It is available to realize a large capture range through the broadband spectrum rather than combined wavelengths, which would relax the physical setup by removing spectral optics for wavelength separation, such as prisms and gratings. Specifically, a capture range of 10𝜆, in units of the largest wavelength, is attained in the case of the broadband of 100 nm. The detection accuracy of the average RMSE of 12 nm for the three-aperture imaging system and 32 nm for the six-aperture imaging system are realized, which are sufficient for fine phasing without the joint use of other means. Besides, the trained DCNN can effectively recognize the pistons even when the system has certain aberrations, which illustrates the ap-
[1] Clampin M. Status of the James Webb space telescope observatory. Proc SPIE 2012;8442:84422A. [2] Chung SJ, Miller DW, de Weck OL. ARGOS testbed: study of multidisciplinary challenges of future spaceborne interferometric arrays. Opt Eng 2004;43(9):2156–67. [3] van Dam MA, McLeod BA, Bouchez AH. Dispersed fringe sensor for the Giant Magellan Telescope. Appl Opt 2016;55(3):539–47. [4] Mourard D, Dali Ali W, Meilland A, Tarmoul N, Patru F, Clausse JM, et al. Group and phase delay sensing for cophasing large optical arrays. Mon Not R Astron Soc 2014;445(2):2082–92. [5] Jiang J, Zhao W. Phasing piston error in segmented telescopes. Opt Express 2016;24(17):19123–37. [6] Paxman RG, Fienup JR. Optical misalignment sensing and image reconstruction using phase diversity. J Opt Soc Am A 1988;5:914–23. [7] Xie Z, Ma H, He X, Qi B, Ren G, Dong L, et al. Adaptive piston correction of sparse aperture systems with stochastic parallel gradient descent algorithm. Opt Express 2018;26(8):9541–51. [8] Paykin I, Yacobi L, Adler J, Ribak EN. Phasing a segmented telescope. Phys Rev E 2015;91(2):023302. [9] Yohe N, Matias V, Ryoichii H, Katsuhisai K, Mamoru S, Jun T, et al. Deep learning wavefront sensing. Opt Express 2019;27(1):240–51. [10] Guerra-Ramos D, Díaz-García L, Trujillo-Sevilla J, Rodríguez-Ramos JM. Piston alignment of segmented optical mirrors via convolutional neural networks. Opt Lett 2018;43(17):4264–7. [11] Ma X, Xie Z, Ma H, Xu Y, Ren G, Liu Y. Piston sensing of sparse aperture systems with a single broadband image via deep learning. Opt Express 2019;27(11):16058–70. [12] Li D, Xu H, Wang D, Yan D. Large-scale piston error detection technology for segmented optical mirrors via convolutional neural networks. Opt Lett 2019;44(5):1170–3. [13] Kendrick RL, Acton DS, Duncan AL. Phase-diversity wave-front sensor for imaging systems. Appl Opt 1994;33(27):6533–46. [14] Bolcar MR, Fienup JR. Sub-aperture piston phase diversity for segmented and multi-aperture systems. Appl Opt 2009;48(1):A5–A12. [15] Harvey JE, Rockwell RA. Performance characteristics of phased array and thinned aperture optical telescopes. Opt Eng 1988;27(9):762–8. [16] Baron F, Cassaing F, Blanc A, Laubie D. Cophasing a wide field multi-aperture array by phase diversity: influence of aperture redundancy and dilution. Opt Eng 2003;4852(2):663–73.
Contents lists available at ScienceDirect
Optics and Lasers in Engineering journal homepage: www.elsevier.com/locate/optlaseng
Piston sensing for sparse aperture systems with broadband extended objects via a single convolutional neural network Xiafei Ma a,b,c, Zongliang Xie a,b,c,d,∗, Haotong Ma a,b,c,∗, Yangjie Xu a,b,c, Dong He a,b,c, Ge Ren a,b,c a
Key Laboratory of Optical Engineering, Chinese Academy of Sciences, Chengdu 610209, China The Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu 610209, China c University of Chinese Academy of Sciences, Beijing 100039, China d State Key Laboratory of Pulsed Power Laser Technology, Hefei 230037, China b
a r t i c l e Keywords: Deep learning Piston sensing Sparse aperture systems
i n f o
a b s t r a c t It is crucial for sparse aperture systems to preserve imaging quality, which can be addressed when fast corrections of pistons within a fraction of a wavelength are available. In this paper, we demonstrate that only a single deep convolutional neural network is sufficient to extract pistons from wide-band extended images once being appropriately trained. To eliminate the object characters, the feature vector is calculated as the input by a pair of focused and defocused images. This method possesses the capability of fine phasing with high sensing accuracy, and a large-scale capture range without the use of combined wavelengths. Simple and fast, the proposed technique might find wide applications in phasing telescope arrays or segmented mirrors.
1. Introduction Optical sparse aperture systems, including segmented telescopes [1] and telescope arrays [2], are capable of providing high resolution of large monolithic apertures with reduced costs and weight. It is crucial to control the optical path lengths between sub-apertures, also known as pistons, within a fraction of a wavelength for the optimal imaging performance, or the imaging resolution will decrease sharply. Many methods proposed for piston sensing need unresolved sources so as to detect pistons from the interference fringes [3] or modulation transfer functions in Fourier domain [4,5]. Extracting pistons from extended objects is challenging since the convolutions between the targets and the point spread functions smooth the fringes. Phase diversity technique can realize piston sensing whatever the size of the object with focused and defocused images [6]. Though, it needs heavy computational burden and suffers 2𝜋 ambiguity. Alternatively, phasing approaches using optimization procedures can also handle extended targets [7,8], but a large capture range and instant correction can not be obtained with these approaches. Recently, with deep learning developing fast, researchers have demonstrated that the state-of-art convolutional neural network (CNN) can directly estimates wavefront represented by Zernike coefficients from the intensity images of not only point sources but also specified extended objects [9]. CNNs have also been applied for piston sensing. Guerra-Ramos et al. open a new door of deep learning-based piston sens∗
ing by means of simulation [10]. They trained 2 shallow CNNs, one for piston step values and the other for ambiguity range, and used 4 different wavelengths, achieving high sensing accuracy and an ample capture range. Then a single network for piston sensing is developed with broadband illumination [11]. However, these methods are still limited to the point sources. Li et al. have proposed a CNN-based piston sensing technique for extended objects with a pair of focused and defocused images [12]. In their study, 5 CNNs are trained with simulated dataset and 4 different wavelengths are also utilized for disambiguation. As a result, their technique succeeds in correcting the pistons ranging in [0, 10𝜆] to [0, 𝜆], and then traditional phase diversity is jointly used for fine phasing. In this paper, we demonstrate that using only a single deep convolutional neural network (DCNN) is sufficient to detect pistons from broadband extended images, and a large capture range up to the coherent length of the wide-band light can be achieved without combined wavelengths. In our implementation, focused and defocused broadband images are used to calculate the feature vectors, which are independent of the object characters. Then we train a DCNN by using the feature vectors as the inputs and the corresponding pistons as the outputs. As the training is done with advanced deep learning strategies, the DCNN is capable of serving as a piston sensor owning high sensing accuracy and an ample capture range. For centrosymmetric configurations, the feature vector might not sensitive enough to changes of pistons, thus affecting the piston sensing performance. This issue can be solved by using a diaphragm to break the symmetry.
Corresponding authors. E-mail addresses: [email protected] (Z. Xie), [email protected] (H. Ma).
https://doi.org/10.1016/j.optlaseng.2020.106005 Received 5 September 2019; Received in revised form 18 December 2019; Accepted 3 January 2020 0143-8166/© 2020 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license. (http://creativecommons.org/licenses/by-nc-nd/4.0/)
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
There are mainly three advantages associated with the reported method with regard to the previous work for extended objects. Firstly, compared with the usage of multiple CNNs, using only a single DCNN can’t only reduce the training complexity, but also cut down the recognition time. Secondly, the proposed technique discerns pistons directly from the broadband intensity summations rather than the intensities of separated wavelengths, thus removing the requirement of spectral optics and compacting the physical setup. Finally, it can work as an independent piston sensor without the joint use of other piston sensing approaches. This paper is structured as follows: we describe the optical imaging model of data generation and the implementation of the DCNN-based method in Section 2. The results of several simulations and detailed discussion are presented in Section 3. Concluding thoughts are provided in Section 4.
𝑁 ( ) ( ) ∑ ( ) 2𝜋𝑖 𝑃 (𝐮, 𝜆) = 𝑝 𝐮 − 𝐮1 + 𝑝 𝐮 − 𝐮𝑛 exp 𝑂𝑃 𝐷𝑛 , 𝜆 𝑛=2
2.1. Optical model of data generation As a data-driven technology, the training results of the CNN are supremely dependent on the training dataset. In the deep learningbased piston sensing methods for point sources, the deep models can directly establish the mapping relation between pistons and corresponding point spread functions. However, when the detecting targets are extended objects instead, the end-to-end detection mode that raw images corresponding to different pistons are directly used as the network inputs without preprocessing would markedly increase the scale of the database and lead to worse generalization, since different imaging objects own different imaging characteristics. It is difficult and makes no sense to construct training datasets for every object, so we turn to construct a feature vector which can free the method from the limitation of the imaging content. Thanks to the previous work of Kendrick [13], a feature vector model using a pair of focus diversity images is proposed, which can be expressed as 𝐺 ⋅ 𝐺𝑑∗ − 𝐺∗ ⋅ 𝐺𝑑
𝐺 ⋅ 𝐺∗ + 𝐺𝑑 ⋅ 𝐺𝑑∗
,
(1)
where G and Gd are the Fourier transforms of the focused and defocused images, respectively, while G∗ and 𝐺𝑑∗ are their complex conjugates. Such a feature vector is only related to the pistons of the sparse aperture system while being independent of the imaging target, which is exactly what the deep learning pursues. Thus, the feature vectors are used here as the training inputs. What is different is that the calculations of the character transform make use of polychromatic intensity summations rather than separated intensity distributions of multiple wavelengths for a large capture range, which reduces the computational budget and relaxes the physical setup. The broadband image captured on the focal plane can be modeled as 𝐼 (𝐱) = 𝑜(𝐱) ⊗ ℎ(𝐱),
(2)
where o is the ideal intensity of the imaging object, h is the focal broadband point spread function of the sparse aperture system, ⊗ is the convolution operator, and x is a 2D vector in the image plane. According to the imaging principle, the focal point spread function at a specific imaging wavelength 𝜆 can be represented as ℎ(𝐱, 𝜆) = |FT(𝑃 (𝐮, 𝜆))| , 2
(3)
where P(u, 𝜆) is the generalized pupil function corresponding to the wavelength 𝜆, FT(⋅) represents Fourier transform and u is a 2D vector in the pupil plane, respectively. The alignment errors of sparse aperture systems, involving tip-tilt and pistons seriously affect the imaging performance. Shack–Hartman wavefront sensors can handle the tip-tilt detection but can’t sense the
(4)
where the first sub-aperture is regarded as the reference, N is the total number of sub-apertures, p is the binary pupil function of an unit, un is the center position vector of the nth sub-aperture, and OPDn is the relative piston between the nth sub-aperture and the reference. Ranging from 𝜆1 to 𝜆m , the broadband point spread function can be modeled as an integral form [14] ℎ (𝐱 ) =
2. Method
𝑀𝑠ℎ𝑎𝑟𝑝𝑛𝑒𝑠𝑠 =
discontinuous pistons. The purpose of this paper is to detect the pistons, so here the assumption that the other phase distortions are eliminated is made, as commonly used in other researches on piston sensing [5,10]. With pistons taken into consideration, the corresponding generalized pupil function can be expressed as
𝜆𝑚
∫𝜆1
ℎ(𝐱, 𝜆)𝑠(𝜆)𝑑𝜆,
(5)
where s(𝜆) is the spectral coefficient at each wavelength. The defocus broadband image takes the similar physical model. The difference is involved in the generalized pupil function, which can be represented as 𝑃𝑑𝑒𝑓 𝑜𝑐𝑢𝑠 (𝐮, 𝜆) = 𝑃 (𝐮, 𝜆) ⋅ 𝐴(𝐮) exp(𝑗𝜑(𝐮, 𝜆)),
(6)
where A is the binary pupil function of the minimum virtual aperture surrounding all the sub-apertures, denotes the defocus aberration loaded. 2.2. DCNN implementation After training dataset preparation is completed, we construct the DCNN in TensorFlow and train it for piston sensing. As discussed above, the CNN used in the previous work for extended objects is too shallow to discern the pistons of all the sub-apertures from a frame of feature vector [10], resulting in the requirement of multiple CNNs. Here we establish a single DCNN with more network layers, in possession of powerful capability of characterizing a complicated analytic model. This DCNN learns the statistical transformation between the broadband feature vectors and the pistons of all the sub-apertures via a large amount of samples training. Once trained, this network uses the feature vectors computed by the broadband focused and defocused images as inputs, and quickly output all the pistons of the system, as shown in Fig. 1. The deep network in the proposed method has 29 layers in total, whose specific topology is also described in Fig. 1. The convolutional layers with ReLU activation functions are used for feature extraction. The max-pooling layers can diminish the parameters of the network and lighten the computation burden of the training. The training process operates through forward and back propagation until the loss function converges to a global optimum solution, which is defined as the root mean squared error (RMSE) between the output pistons and the ground truths. It’s noted that some advanced strategies are implemented here for satisfactory learning, such as Batch Normalization used to prevent gradient vanishing, dropout and data augmentation techniques aiming at avoiding over-fitting. 3. Results 3.1. Performance on the three-aperture system Similar to the previous work [9,10], we use simulated data to validate the efficiency of the proposed deep learning-based technique. We first model a three-aperture system as shown in Fig. 2 to validate the feasibility of the proposed method, where the diameter of sub-aperture is 10 mm. And in the focal plane with a distance of 500 mm, a camera with the pixel pitch of 1.67 𝜇m is settled to record the raw extended images.
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 1. Principle of the proposed DCNN-based piston sensing method and detailed schematic of the DCNN architecture, indicating the number of layers, the property of each layer, etc. The feature vector, calculated by a pair of focus diversity broadband images, is input into the DCNN and the pistons of all the sub-apertures are outputs.
Fig. 2. Configuration of the three-aperture system.
To start with, it is of the essence to generate the training database. We use a USAF resolution target as the object, illuminated by broadband light ranging from 500 nm to 600 nm. The capture range achieving the coherence length is within ±3 𝜇m. The wide-band imaging is simulated by accumulating 21 monochrome point spread functions with the sampling interval of 5 nm and equal weight contribution. To construct training data for the three-aperture system we model in this study, 10,000 sets of random pistons ranging from −3 𝜇m to 3 𝜇m are introduced into the sub-apertures and corresponding focused and defocused images are captured, of which a group of examples are shown as Fig. 3(a1) and (a2). Based on the raw images, 10,000 broadband feature vectors are calculated by using Eq. (1), of which 6 frames are presented as examples in Fig. 3(b1)–(b6). As described above, the feature vectors are ready for the utilization of the inputs while the corresponding pistons ready for the outputs. After training procedure is done, testing dataset, which is exclusive of the training images, is needed to test the performance of the DCNN trained for piston sensing. Besides, to demonstrate the independence of the DCNN on object characters, another image different from the one for training is used as the imaging target to generate testing datasets. We randomly load 1000 sets of pistons ranging from −3 𝜇m to 3 𝜇m on the system and acquire the focus diversity images, of which a pair of examples are given as Fig. 4(a1) and (a2). 1000 broadband feature vectors are then computed, of which 6 frames of samples are presented in Fig. 4(b1)–(b6). They are inputted into the DCNN and the predicted pistons are outputted quickly. The estimation for a single image takes
Fig. 3. Examples of the training database for the three-aperture system. (a1) and (a2) A pair of focused and defocused broadband image samples using the USAF target. (b1)–(b6) The feature vector samples corresponding to different pistons.
Fig. 4. Examples of the testing database for the three-aperture system. (a1) and (a2) A pair of focused and defocused broadband image samples using Lena picture. (b1)–(b6) The feature vector samples corresponding to different pistons.
only about 12.4 ms using the CPU of Intel(R) Core(Tm) i7-6800K and the graphics processing unit (GPU) of NVDIA GeForce GTX 1080 Ti. The evolution of the loss function over the course of the training for the three-aperture system is exhibited in Fig. 5, from which we can see that the DCNN converges after about 4000 iterations, indicating that the network has probably learnt the complex mapping relation between the broadband feature vectors and pistons. Here we take the average value of the RMSE over all testing samples as the assessment criteria, which is
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 5. Evolution of the loss function during training for the three-aperture system.
Fig. 6. 500 RMSEs randomly selected from the testing dataset for the three-aperture system.
about 12 nm for the three-aperture system. The RMSEs between the predicted pistons and the ground truths of 500 testing samples randomly selected are shown in Fig. 6, from which it is easy to be figured out that the sensing deviation can be exactly controlled within 40 nm. Specifically, with respect to the other criterion of 0.1𝜆 at the wavelength of 600 nm referred to the piston error tolerance in the analysis of Chung et al. [2], the sensing accuracy of the trained DCNN reaches 100%. To better illustrate the sensing capability, the distributions of the residual RMSEs on training samples and testing samples are depicted in Fig. 7, from which we can see that the results of these two datasets are highly consistent. The testing results demonstrate that our proposed technique can be used for fine phasing while the previous CNN-based method for extended objects can only perform coarse phasing due to the sensing error within 𝜆. 3.2. Performance on the six-aperture system The sub-aperture number is a key factor influencing the sensing accuracy of the proposed technique. There seems contradiction between the systematic complexity and the required accuracy due to two aspects. First and foremost, the complexity of the imaging model will greatly increase with the number of sub-aperture increasing, which means the non-linear mapping relation is more complex and it is more difficult for the network to fit that function. Second, since there are more unknown pistons to be estimated, the training data needed and the amount of computation will also greatly increase. It is necessary to evaluate the performance of this method on imaging systems with more sub-apertures. In this section, a sparse aperture system with 6 units is built, of which the configuration is shown in Fig. 8. A sub-aperture is located in the center as a benchmark and the other 5 sub-apertures are distributed
Fig. 7. Distributions of RMSEs over all the training and testing dataset for the three-aperture system.
along a circle with the radius of 145 mm. The diameter d of the subaperture is 100 mm and the focal length is 6 m. 100,000 samples for training and another 1000 samples for testing are collected in the case of the six-aperture imaging system. The estimation time for a single image is about 12.7 ms. As expected, the sensing accuracy for the six-aperture systems decreases slightly with respect to that for the three-aperture system. It can be recognized that most errors
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 8. Configuration of the six-aperture system.
Fig. 11. Wavefront aberrations introduced in the three-aperture system.
Fig. 9. 500 RMSEs randomly selected from the testing dataset for the sixaperture system.
The training process includes two parts: forward propagation of data and the back propagation of error. The parameters are updated through repeated iterations of these two steps. When the number of sub-apertures increases, the non-linear relation is more complex so that more training samples and iterations needed increase greatly. Because of the huge amounts of iterations, it takes several hours or days to accomplish the training process. However, when we utilize the trained network to detect the piston, only forward propagation performs once. The increase of the neurons in output layer has little effect on piston estimation time. In fact, our simulations show that the estimation time for the six-aperture system is very close to that for the three-aperture system. 3.3. Performance considering aberrations
Fig. 10. Distributions of RMSEs over all the training and testing dataset for the six-aperture system.
are located within the region of [0, 60] nm from the RMSE values of 500 random testing samples which are shown in Fig. 9. Also, as shown in Fig. 10, there is a certain difference between the sensing results of the training and testing dataset. Though, the accuracy decreasing is acceptable and the proposed technique still presents much higher detecting accuracy on the six-aperture system than the previous one for extended objects. Specifically, the sensing accuracy of the trained DCNN reaches 91.6% with respect to the criterion of 0.1𝜆, and the average RMSE over the test dataset can achieve 32 nm, which suffices for fine phasing.
Note that the simulations above are carried out in the assumption that the tilt-tip and other sub-aperture aberrations are eliminated entirely. However, these aberrations can be hardly eliminated entirely in experiments. There are some inevitable imperfections caused by manufacture errors in the system. Therefore, another simulation is also implemented, in which slight wavefront aberrations describing the system distortions are introduced in the three-aperture imaging system, to evaluate the influence of aberrations on sensing accuracy. The RMSE of each aberrations with 11 Zernike coefficients excluding the first one is set as 0.05𝜆 (𝜆 = 600 nm), and the introduced aberrations for training samples are shown in Fig. 11. The added system aberrations are the same along all the training. However, considering the randomness of the aberration in real piston detection scenario, we introduce dynamic aberrations in testing samples, which are different from those for training.The specific training and testing results are shown as the histogram in Fig. 12, which indicate that the sensing performance is similar to that without the presence of other aberrations. The average RMSE is 15 nm over all testing samples and the detecting accuracy also achieves 100% with regard to the criterion of 0.1𝜆. The aberrations affect the intensity distributions in an irregular way, and consequently disturb the mapping process between the feature vectors and pistons. However, the proposed method can still learn the distorted mapping functions via a large amount of samples and shows a good adaptability to these influence factors, which means that it is robust against aberrations and can be useful in practical systems. According to all the testing results above, it is demonstrated that the proposed deep learning-based technique is efficient for extracting pistons from extended objects with a broad capture range, high precision, and fast response time.
X. Ma, Z. Xie and H. Ma et al.
Optics and Lasers in Engineering 128 (2020) 106005
Fig. 12. Distributions of RMSEs over all the training and testing dataset for the three-aperture system considering aberrations.
Fig. 13. Configurations of two Golay-6 arrays with an additional sub-aperture.
3.4. Discussion on impacts of redundancy The redundancy in the spatial frequencies has a considerable influence on the performance of some cophasing algorithms [15,16]. According to the previous researches, the estimation error of phase diversity technique increases with the level of redundancy, as there are two or
more pairs of sub-apertures contributing to the same spatial frequency peaks in redundant configuration, which can lead to an ambiguity. In this section, we discuss the impacts of array redundancy on the proposed piston sensing method. Two kinds of redundant arrays, centrosymmetric and noncentrosymmetric, are involved. First, we analyze the influence of redundancy of the noncentrosymmetric configuration by implementing additional simulations. Two Golay-6 arrays with an additional sub-aperture shown in Fig. 13 are modeled. Sub-apertures number 4 and 5 in configuration Golay-6-1 are more redundant than other ones for the baselines 4-6/5-3, 5-4/3-6/6-1, 4-1/5-6 are all redundant. For configuration Golay-6-2, the additional sub-aperture is placed at the center, and there is no redundancy. As the reference [16] concludes, for phase diversity technique, the estimation errors would be visibly greater on sub-apertures 4 and 5 in configuration Golay-6-1 due to the redundancy, while the accuracies on all sub-apertures in configuration Golay-6-2 would be at similar level. In our simulation, two training datasets composed of 120,000 samples are generated to train the sensing networks for Golay-6-1 and Golay-6-2, respectively. The testing results using other 1000 samples described in Fig. 14 show that there is no evident degradation caused by the redundancy on the estimation accuracies of sub-apertures 4 and 5 in configuration Golay-6-1. On the contrary, the sensing performances for Golay6-1 and Golay-6-2 are almost consistent, indicating few effects of redundancy. From the analysis, the performance of the proposed method using CNN seems not subject to the redundancy in a noncentrosymmetric array. Secondly, we check the effectiveness of the proposed approach on redundant centrosymmetric arrays, which are commonly used in segmented telescopes. Another simulation is carried out with a centrosymmetric segmented mirror shown in Fig. 15(a). 120,000 samples are used to train the network and the loss function converges indeed. However, the average RMSE over 1000 testing samples reaches up to 191 nm, which apparently can not meet the request of fine phasing. We infer that it is because the feature vector for the redundant centrosymmetric array might not be sensitive enough to the changes of pistons due to the full redundancies, thus making the mapping relations difficult to be precisely modeled by a DCNN and leading to decrease in sensing accuracy. It’s an interesting issue and needs our further study. Though, we also give a solution to solve this problem. We place a circular diaphragm in front of one mirror segment as shown in Fig. 15(b), which breaks the configuration symmetry. Then another simulation is implemented and 120,000 samples are used to train the network. The average RMSE of the trained network over 1000 testing samples is 47 nm, which is much better than the result on the centrosymmetric structure. Fig. 16 shows the distributions of RMSEs over all the testing dataset for the config-
Fig. 14. Influences of redundancy on the estimation error.
X. Ma, Z. Xie and H. Ma et al.
Fig. 15. Configurations of segmented mirror without (a) and with (b) a circular diaphragm.
Optics and Lasers in Engineering 128 (2020) 106005
plicability of the proposed technology in practical environments. The configuration redundancy has few effects on the sensing accuracy for a noncentrosymmetric array. For the centrosymmetric configuration, the method using a single CNN still works with a diaphragm used to break the symmetry. Based on the above advantages, it is believed that the proposed technique might find wide applications in phasing telescope arrays or segmented mirrors. For actual implementations, further work needs to consider the effect of atmospheric turbulence. Since the sparse distributed pupils decompose the monolithic turbulence distortion into dominant segmented piston and tip-tilt, further study on how to use CNN to simultaneously sense both the piston and tip-tilt is needed in future developments. It also requires a fast detection to correct atmospheric turbulence. There are several means to reduce the sensing time. First, we can try to reduce the image size on the premise of retaining useful information. Second, optimizing the configuration of the network to decrease the parameter size is an alternative approach. Besides, a better equipped computing device can also accelerate computation speed. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work is supported by Open Research Fund of State Key Laboratory of Pulsed Power Laser Technology (SKL2018KF05); Excellent Youth Foundation of Sichuan Scientific Committee (2019JDJQ0012); Youth Innovation Promotion Association, CAS (2018411); CAS “Light of West China” Program; Young Talents of Sichuan Thousand People Program. References
Fig. 16. Testing results for configurations (a) and (b).
urations (a) and (b). With this approach, the proposed method using a single CNN still can be applied to centrosymmetric configurations. Solutions to achieve higher sensing precision will be investigated and further study on the impact of pupil configuration is also needed in the future work. 4. Conclusions In this paper, we propose the use of a single DCNN to extract pistons from broadband extended images and prove the effectiveness by simulated data. The training is time-consuming for CNN, but once trained, the network-based techniques work fast without iterative process involved. Using only a single DCNN here can reduce the training complexity, as well as further improve the recognition speed. It is an image-based piston sensing method, thus requiring no additional optical equipment to be installed except for the usage of focus division common in phase diversity. It is available to realize a large capture range through the broadband spectrum rather than combined wavelengths, which would relax the physical setup by removing spectral optics for wavelength separation, such as prisms and gratings. Specifically, a capture range of 10𝜆, in units of the largest wavelength, is attained in the case of the broadband of 100 nm. The detection accuracy of the average RMSE of 12 nm for the three-aperture imaging system and 32 nm for the six-aperture imaging system are realized, which are sufficient for fine phasing without the joint use of other means. Besides, the trained DCNN can effectively recognize the pistons even when the system has certain aberrations, which illustrates the ap-
[1] Clampin M. Status of the James Webb space telescope observatory. Proc SPIE 2012;8442:84422A. [2] Chung SJ, Miller DW, de Weck OL. ARGOS testbed: study of multidisciplinary challenges of future spaceborne interferometric arrays. Opt Eng 2004;43(9):2156–67. [3] van Dam MA, McLeod BA, Bouchez AH. Dispersed fringe sensor for the Giant Magellan Telescope. Appl Opt 2016;55(3):539–47. [4] Mourard D, Dali Ali W, Meilland A, Tarmoul N, Patru F, Clausse JM, et al. Group and phase delay sensing for cophasing large optical arrays. Mon Not R Astron Soc 2014;445(2):2082–92. [5] Jiang J, Zhao W. Phasing piston error in segmented telescopes. Opt Express 2016;24(17):19123–37. [6] Paxman RG, Fienup JR. Optical misalignment sensing and image reconstruction using phase diversity. J Opt Soc Am A 1988;5:914–23. [7] Xie Z, Ma H, He X, Qi B, Ren G, Dong L, et al. Adaptive piston correction of sparse aperture systems with stochastic parallel gradient descent algorithm. Opt Express 2018;26(8):9541–51. [8] Paykin I, Yacobi L, Adler J, Ribak EN. Phasing a segmented telescope. Phys Rev E 2015;91(2):023302. [9] Yohe N, Matias V, Ryoichii H, Katsuhisai K, Mamoru S, Jun T, et al. Deep learning wavefront sensing. Opt Express 2019;27(1):240–51. [10] Guerra-Ramos D, Díaz-García L, Trujillo-Sevilla J, Rodríguez-Ramos JM. Piston alignment of segmented optical mirrors via convolutional neural networks. Opt Lett 2018;43(17):4264–7. [11] Ma X, Xie Z, Ma H, Xu Y, Ren G, Liu Y. Piston sensing of sparse aperture systems with a single broadband image via deep learning. Opt Express 2019;27(11):16058–70. [12] Li D, Xu H, Wang D, Yan D. Large-scale piston error detection technology for segmented optical mirrors via convolutional neural networks. Opt Lett 2019;44(5):1170–3. [13] Kendrick RL, Acton DS, Duncan AL. Phase-diversity wave-front sensor for imaging systems. Appl Opt 1994;33(27):6533–46. [14] Bolcar MR, Fienup JR. Sub-aperture piston phase diversity for segmented and multi-aperture systems. Appl Opt 2009;48(1):A5–A12. [15] Harvey JE, Rockwell RA. Performance characteristics of phased array and thinned aperture optical telescopes. Opt Eng 1988;27(9):762–8. [16] Baron F, Cassaing F, Blanc A, Laubie D. Cophasing a wide field multi-aperture array by phase diversity: influence of aperture redundancy and dilution. Opt Eng 2003;4852(2):663–73.