- Email: [email protected]
Modeling and analyzing point cloud generation in missile-borne LiDAR
Defence Technology xxx (xxxx) xxx
Contents lists available at ScienceDirect
Defence Technology journal homepage: www.elsevier.com/locate/dt
Modeling and analyzing point cloud generation in missile-borne LiDAR Min-le Li a, b, Yi-hua Hu a, b, *, Nan-xiang Zhao a, b, Qi-shu Qian a, b a b
State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering, National University of Defense Technology, Hefei, 230037, China Anhui Province Key Laboratory of Electronic Restriction Technology, Hefei, 230037, China
a r t i c l e i n f o
a b s t r a c t
Article history: Received 31 July 2019 Received in revised form 16 September 2019 Accepted 15 October 2019 Available online xxx
The missile-borne LiDAR has an essential prospect in precise guidance. However, the instability of the missile has a significant impact on the precision of LIDAR point cloud, thus modeling and analyzing the influencing factors of point cloud generation is necessary. In this investigation, the authors modeled the point cloud calculation process of missile-borne linear array scanning LiDAR under ideal conditions and analyzed the effect of position change on the field of view (FOV) in the course of platform motion. Subsequently, the authors summarized that the stability concerning the field of view and the target tracking are dependent on three limiting conditions. Finally, the authors derived an error formula and analyzed several typical errors in missile platform flight, including scanner error, POS error, system integration error, and platform vibration error. Based on theoretical analysis and simulation experiments, the model proposed in this paper can provide a theoretical basis for the design of the missile-borne LiDAR system and the selection of related instruments in practice. © 2019 Production and hosting by Elsevier B.V. on behalf of China Ordnance Society. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Keywords: LiDAR Point cloud Precise guidance Vibration
1. Introduction Precision guided weapons have become an indispensable part of modern warfare because of their high hit rate, great combat effectiveness, and deterrence. Therefore, this type of weapon has attracted close attention worldwide [1e3]. Precision guidance weapons include radar guidance, infrared guidance, television guidance, and laser guidance. Laser imaging guidance technology has the advantages of high resolution, strong anti-jamming ability, and all-weather combat, and is attracting more and more research attention [4]. The United States is at the forefront of the world in the research of Light Detection and Ranging (LiDAR) guidance [5]. Since the 1980s, many military scientific research institutes in the United States have developed three-dimensional LiDAR seekers. At present, LiDAR seekers with specific algorithms have the ability of automatic target recognition. Multiple types of missiles are equipped with this type of LiDAR seeker. For example, the United
* Corresponding author. State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering, National University of Defense Technology, Hefei, 230037, China. E-mail address: [email protected] (Y.-h. Hu). Peer review under responsibility of China Ordnance Society
States Army Research Laboratory (ARL) and other units have developed and tested low-cost autonomous attack system (LOCAAS), which is a highly intelligent, unmanned aircraft launched outside the defense zone. The LOCAAS seeker has the ability of three-dimensional imaging of the target. It can scan accurately and autonomously at a rate of 100 Hz, and the scanning width is 750 m. Once the target is captured, the scanning width will be reduced to 100 m, the field of view angle is 4e6 , the operating distance is 1e1.25 km, and the resolution can be achieved to 15 cm. The target identification algorithm has an advanced feature extraction function. Since 1999, the United States has been developing a Flash LiDAR, the non-scanning three-dimensional imaging LiDAR, the imaging rate of which is at least five times higher than that of the scanning imaging LiDAR used in the previous LOCASS. NetFires (Network Fires, Network Fire Support) is a joint project proposed by DARPA and Army of the United States. It aims to provide the Army with the next generation of attack and patrol precision-guided missiles. The patrol attack missiles of NetFires weapon system are equipped with active LiDAR seekers that can automatically identify targets at 230 m altitude and 400 km/h speed. Besides, the general LiDAR seeker and the “vertical saw” imaging LiDAR system in the United States also use threedimensional imaging technology, which can automatically identify targets in various environments [5].
https://doi.org/10.1016/j.dt.2019.10.003 2214-9147/© 2019 Production and hosting by Elsevier B.V. on behalf of China Ordnance Society. This is an open access article under the CC BY-NC-ND license (http:// creativecommons.org/licenses/by-nc-nd/4.0/).
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
2
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
As for the researches on LiDAR modeling, there are many achievements in the research of LiDAR simulation all over the world [6]. O. Kechagias-Stamatis [7] evaluated 3D local descriptors and proposed an architecture [8] for future missile seeker with automatic target recognition (ATR) capabilities. Many research institutes, such as Oktal Synthetic Environment Company of France [9] and the Swedish National Defense Research Bureau [10], have developed and put into use the simulation platform of LiDAR. Huazhong University of Science and Technology [11] established an airborne dual-wedge LiDAR simulation platform, and obtained point cloud images in different scenarios, but the single-point scanning efficiency is lower than linear array scanning LiDAR. Scanning LiDAR is simulated and modeled by National Defense University [12]. Range and intensity images are obtained by analyzing the echo information in each beam, but the action distance is relatively close, and the scanning period is longer than linear array scanning LiDAR. At present, there is no missile-borne LiDAR simulation platform in China. As for the researches on platform motion error analysis, since the related literatures on missile-borne LiDAR point cloud motion error analysis are seldom published, this part reviews the published research works about image distortion correction of the missile-borne infrared imaging guidance system and related research on motion errors of satellite-borne and airborne LiDAR, so as to provide a reference for related research of missile-borne LiDAR. H Richard et al. [13] studied the image distortion caused by aero-optics of the supersonic aircraft, measured the wavefront parameters by the background oriented schlieren (BOS) technique, and obtained the offset of all targets on the imaging surface by difference method. CSANYI et al. [14] corrected the point cloud error of airborne LiDAR by setting markings on the ground. Dickman et al. [15] studied the influence of posture angle random measurement error on airborne LiDAR plane terrain imaging through numerical simulation but did not carry out more complex terrain simulation. Wang et al. [16] analyzed the causes of measurement errors affecting the accuracy of airborne laser scanning point clouds. By analyzing the working principle of conical rotating airborne laser scanning, the mapping relationship between error factors and point clouds accuracy was established. The equation between error factors and three-dimensional coordinate errors of point clouds was established by multiple linear regression method, and the causes of errors were quantitatively evaluated. The significant effect of the element on accuracy provides a theoretical basis for error allocation and suppression and compensation of essential factors in the airborne laser scanning system. Based on our literature review, we found that those exist studies cannot be applied to point cloud generation and motion error analysis of missile-borne linear array scanning LiDAR. Because of the high cost of research and development of the missile-borne platform, using an accurate system simulation model to guide practice is necessary. Besides, the movement of the platform often leads to the distortion of point cloud acquired by LiDAR in the process of scanning, and relevant research needs to be carried out. In this paper, firstly, the point cloud calculation process of missile-borne linear array scanning LiDAR under ideal conditions is modeled. On this basis, the effect of posture change on the field of view in the course of platform motion is analyzed, and three limiting conditions are put forward. Only when these conditions are satisfied, the stability of the field of view can be guaranteed, and the target tracking can be carried out. Finally, the authors derived an error formula and analyzed several typical errors in missile platform flight, including scanner error, POS error, system integration error, and platform vibration error.
2. Material and methods 2.1. The method of calculating point cloud coordinates of missileborne LiDAR Fig. 1 shows the scanning principle of linear scan LiDAR system for ground targets, and the beam vector of LiDAR is set as S. The direction of laser emission is Y axis, the direction of pitch angle increase is Z axis, and the X axis is determined according to the right hand rule. For the missile-borne LiDAR system, the starting point Vs ðxV ; yV ; zV ÞT is the platform of the optical system and can be measured by dynamic differential GPS. The mode of S represents the distance between the optical system and the ground laser foot and the direction of S is the platform posture parameter ða; b; gÞ as pitch - yaw - roll, which can be obtained by the high-precision posture measuring device. According to the above parameters, the three-dimensional coordinates of the ground point P in the geodetic coordinate system can be calculated. The system can acquire high-precision laser point cloud data by continuously transmitting laser pulses and receiving echo information through scanning. The difficulty of simulation technology of missile-borne Linear Array scanning LiDAR is that there are great differences in the motion of missiles, not only in flight speed but also in rotation speed. For each laser landing point, the collected information includes the range information from the landing point to the LiDAR (1D), the real-time position information (3D), the platform posture (3D), the position and posture relationship between the platform and the laser (6D) and the beam splitting angle (1D). More than 14 dimensions to calculate the position of each landing point. In order to facilitate analysis, Fig. 2 shows the coordinate system. The black coordinate system in Fig. 2 represents the local coordinate system: take the point projected to the ground from the original position of the missile as the origin, the vertical upward axis is the Z-axis, and the forward direction of the missile is the Yaxis. The X axis is determined according to the right-hand rule. The blue coordinate system in Fig. 2 represents the instantaneous coordinate system: the direction of laser emission is Y0 axis, the direction of elevation increase is Z0 axis, and the X0 axis is determined according to the right hand rule. The 64-line lasers are calibrated with serial numbers 1 to 64 in the positive direction of X0-axis, and the 32nd laser is aligned with the Y0-axis. Four steps accomplish the calculation of point cloud. First, the coordinates of the landing point in the instantaneous coordinate system are calculated according to the distance and beam splitting information. For the nth unit time, the coordinates of the laser point corresponding to the linear array laser beam are as follows:
2
3 2 3 xsl sn tanðan Þ 4 ysl 5 ¼ 4 5 0 zsl 1,sn
(1)
Fig. 1. Missile-borne linear array LiDAR system for target detection and tracking.
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
3 33 22 3 2 xLH xN xI 4 yLH 5 ¼ RN ,44 yI 5 4 yN 5 5 zLH zI zN
3
2
(4)
Here, the transformation matrix RN ¼ RfN ,RqN ,R4N representing the platform coordinate system to the local horizontal coordinate system is related to the posture angles in the three directions of rotation, pitch, and yaw, and the form is similar to Eq. (3). lN ¼ ðxN ; yN ; zN ÞT represents the eccentricity between the phase center of the GPS antenna and the inertial platform. Moreover, to convert the local horizontal coordinate system to the WGS-84 coordinate system.
2 3 3 xLH x84 4 y84 5 ¼ RW ,4 yLH 5 þ l84 z84 zLH 2
Fig. 2. Instantaneous scanning coordinate system and platform coordinate system.
Among them, sn represents the distance between 64 laser beams at the nth unit time and an ¼ 15:5,16 mrad is the row vector formed by the angle between the corresponding 64 laser beams and the Y ’ axis. Second, according to the position and posture relationship between the laser and the platform, the instantaneous coordinate system is converted to the platform coordinate system, as shown by Eq. (2).
where, the transformation matrix RW ¼ RfW ,RqW ,R4W representing the platform coordinate system to the local horizontal coordinate system is related to three posture angles in a form similar to Eq. (3). l84 ¼ ðx84 ; y84 ; z84 ÞT represents the coordinates of the phase center of the GPS antenna in the WGS-84 coordinate system. In summary, the coordinates of the laser point in WGS-84 coordinate system are as follows:
2 2 2 3 3 2 3 2 333 sn ,tanðan Þ x84 xP xN 4 y84 5 ¼ RW ,4RN ,4RP ,4 5 þ 4 yP 5 4 yN 5 5 5 0 z84 zP zN 1,sn 3 2 x84 þ 4 y84 5 z84 2
(6) Simplified to vector form,
P W ¼ RW ðRN ðRP r þ lP lN Þ Þ þ l84 2 3 3 xsl xI 4 yI 5 ¼ RP ,4 ysl 5 þ lP zI zsl 2
(2)
Among them, the matrix lP ¼ ðxP ; yP ; zP ÞT represents the influence of displacement in the process of transformation, and the matrix RP ¼ RfP ,RqP ,R4P is used to represent the error matrix consisting of three orientations angles: rotation, pitch, and yaw.
8 > > 3 2 > > > cos fV;n sin fV;n 0 > > > 7 6 f > > RP ¼ 4 sin fV;n cos fV;n 05 > > > > > 0 0 1 > > > 2 3 > > 1 0 0 > < 6 cos qV;n 7 RqP ¼ 4 0 sin qV;n 5 > > > 0 cos q q sin > V;n V;n > > 3 2 > > > cos 4V;n sin 4V;n 0 > > > 7 6 > > R4P ¼ 4 sin 4V;n cos 4V;n 05 > > > > > 0 0 1 > > :
(3)
Third, according to the real-time position and posture information of the platform itself in the local horizontal reference coordinate system, the coordinates are transformed into the local horizontal reference coordinate system.
(5)
(7)
where, r represents the position vector of the laser point in the instantaneous coordinate system and P W is the coordinate of the laser foot in the WGS-84 coordinate system. 2.2. The effect of motion on point cloud calculation The flight altitude of the missile can reach several kilometers. We utilized a 64-element linear array scanning laser radar, with the interval of each beam being 0.5 mrad and the instantaneous field of view range being 32 mrad. The laser frequency is 10 kHz, and the frame frequency is 20 Hz, and each frame has 500 scans. At this altitude, the influence of translation motion of missile on point cloud computing can be neglected, but the influence of posture change and the vibration is larger. Thus it must be considered. By analyzing the influence level of these two factors, this paper provides a reference for the selection of the gyro platform and shock absorber. 2.2.1. Three limiting conditions of posture change on the field of view In order to achieve precise guidance, it is necessary to ensure that the overlapping field of view of two adjacent frames has a larger overlap area, so as to ensure that the target is not lost. For the terminal guidance of the missile, the forward-looking method is used to detect the missile. The effect of the translation motion on the field of view is relatively small, while the change of posture will have a more significant impact. According to the different speed of posture change, we put forward three limiting conditions. Only
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
4
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
when these three conditions are satisfied, the stability of the field of view can be guaranteed. 1) The limiting condition of coincidence of transmitting and receiving footprints: the scanning time of a complete frame of the point cloud is 0.05 s, and each scanning time is t0 ¼ 104 s. The distance between transmitting and receiving is r and the time internal is Dt ¼ 2r=c. When the distance is within [500,7000] m, it can be calculated that 3:33 106 Dt 4:67 105 s. The field of view of a single receiving unit is U ¼ 5 104 rad, and when the 5104 rad ¼ 10:71 rad (1.7r/ posture changes at a speed of a_ > tU0 ¼ 4:6710 5 s =s s), the signal cannot be received. 2) Limiting condition of the field of view regularization: for two adjacent scans, if the angle and position deviation is too large, it will not be able to achieve parallel or approximate parallel relationship. According to the above analysis, when the posture change speed is less than 10.71 rad/s, it can ensure effective reception. At this time, the rotation angle is taken as an example to consider the parallel situation of adjacent scans. Because the time interval between two adjacent scans is t0 ¼ 104 s, the rotation angle difference between two adjacent scans is 1.07 mrad, namely 0.061, and within a frame (500 scans) the total deviation is 30.65 . The irregular change of the field of view makes it difficult to achieve continuous and practical coverage of the target. The setting of limiting conditions for target reconstruction is more of empirical value. 3) Limiting condition of overlap of the adjacent field of view: the system covers a limited field of view. When the posture changes too fast, the overlap of adjacent two frames is too small, resulting in the loss of the target. A complete frame point cloud can cover 32 mrad in the scanning direction by oscillating the pendulum mirror to and fro in the scanning time of tf ¼ 0:05s, thus the speed of the pendulum mirror is about Ur z0:032rad =0:05s ¼ 0:64ðrad=sÞ.The posture angle changes at different speeds a_ , which will also have a significant impact on the unfolding effect of the scanning. The new coverage is expressed as U0 ¼ ðUr a_ Þtf . It can be divided into the following three situations: (1) The change speed of posture angle is less than the swing speed of the swing mirror. If the direction is opposite, the coverage will be reduced (between 0 and one times), and if the direction is the same, the coverage will be increased (between 1 and two times). (2) The change speed of posture angle is equal to the swing speed of the swing mirror. If the direction is opposite, the coverage will be reduced to 0, and the scanning cannot be deployed effectively. If the direction is the same, the coverage will be increased by two times. (3) The change speed of posture angle is faster than the swing speed of the swing mirror. If the direction is opposite, the coverage will extend to the opposite direction; if the direction is the same, the coverage will increase. It is evident that when the change speed of posture angle is equal to or higher than the swing speed of the swing mirror, the coverage of the front and rear field of view will change significantly, even without overlap. This situation can easily lead to the loss of targets. Therefore, it should be less than half of the swing speed of the pendulum mirror, i.e. a_ < 0:32ðrad=sÞ. Under the above limiting conditions, it is necessary to select the appropriate servo device to solve the problem and eliminate the rapid posture change. At the same time, miniaturization design should be taken into account.
2.2.2. Effect of motion error on generating the point cloud The error of LiDAR can be roughly divided into the following categories:
(1) Scanner errors, including ranging errors and scanning angle errors; (2) POS error, namely GPS error and INS error; (3) System integration errors, including installation errors, time deviations. (4) Other errors, including coordinate transformation errors, errors caused by the irregular motion of the platform, elevation errors of type II, and some random errors. Considering various errors, Eq. (8) shows the geometric model of the missile-borne LiDAR
P *W ¼ RW ðDRN RN ðDRP RP ðr þ DrÞ þ lPN þ DlPN Þ Þ þ l84 þ Dl84 þ DlTB (8) where, P *W is the position vector of laser foot under the WGS-84 coordinate system affected by various errors. The term DRN represents the rotation matrix of the platform posture angle measurement error and DRP indicates the system placement error rotation matrix. DlPN means the measurement error of eccentricity component from laser launching point to GPS antenna phase center. Moreover, the Dl84 is the positioning error vector of GPS while DlTB represents the position error vector caused by time deviation. 2.3. Error effect of scanner (1) Ranging error. Laser scanning ranging errors are attributed to the following aspects [17]. The first point is the instrument error. It includes the non-parallel errors of laser signal transmitting and receiving, the measurement errors of laser pulse signal propagation time, the rotation and vibration errors of the reflector, and the zero-point errors of the pulse. The second point is the atmospheric refraction error. When the laser penetrates the atmosphere, it is also affected by the refraction error of the atmosphere (troposphere). The degree of influence is related to the wavelength of the laser pulse. The atmospheric refraction error of the same signal is related to temperature, pressure, and humidity. The wavelength of the laser pulse is about 1 mm, while that of the GPS carrier signal is about 20 cm. Therefore, compared with GPS signal, the laser pulse signal is much less affected by atmospheric refraction, and the absolute amount of tropospheric delay improvement in zenith direction is only a few millimeters, which is much smaller than the accuracy of LiDAR measurement at present. After simple model modification, the residual error can be neglected. The third point is errors related to the reflector. Different physical characteristics of the surface will produce different reflections. When diffuse reflection occurs, a large number of backscattering signals are received, resulting in larger receiving noise. On the contrary, if the signal is transmitted to a smooth surface, mirror reflection is formed, and backscattering signals are very few. Besides, if angular reflection occurs, backscattering may be the result of multiple reflections. Thus the measured time delay cannot represent real-time delays. The ranging accuracy is also related to the roughness of the ground, the slope of the terrain and the interference of the ground objects. Generally speaking, after system error correction, the ranging error of laser scanning can be controlled in a minimal range and can be neglected. (2) Scanning angle error. The scanner uses the scanning prism to turn the laser beam from different angles by driving the motor. Non-uniform speed of the scanning motor, the
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
5
Fig. 3. Point cloud data of typical 3D models.
Table 1 The simulation information of typical 3D models. Models
Number of surface elements
Running time/s
F15EL Missile T72tank
9415 12970 14562
4 5 7
vibration of the scanning mirror, temperature change, and beam divergence angle [18] also bring errors to the scanning angle. Scanning angle error will affect the coordinates of laser foot in the instantaneous coordinate system by converting to ranging error and posture angle error. There are three main types of scanning errors: the instantaneous scanning angle error, the pointing error, and the Scanning plane distortion. When a laser scanner is installed, the laser scanning plane cannot be completely perpendicular to the xaxis of the laser coordinate system, resulting in distortion of the scanning plane. These are small angle values, usually a few percent [19].
2.4. POS error effect GPS measurement data to determine the position of the aircraft platform and sensor. In the process of GPS dynamic positioning, it will be affected by tropospheric delay error, ionospheric delay error,
Fig. 4. Effect of motion on scanning field of view.
satellite clock difference, phase center instability, observation noise, multi-path effect, and so on. GPS positioning error is the main factor affecting the accuracy of laser foot, but because it will change with the change of the detection environment, it is not easy to eliminate and model. It is difficult to eliminate these errors in the later data processing. The main work to reduce GPS positioning error is in data acquisition, which can be reduced by establishing
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
6
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
Fig. 5. Effect of vibration on point cloud calculation accuracy.
multiple ground reference stations uniformly in the measuring area. In the actual operation process of the airborne platform, more than three ground reference stations should be placed, and the distance between ground references stations should not be more than 25 km. Better results can be obtained by GPS solution. At present, the best GPS positioning error is about 5 cm [20]. However, the missile-borne platform cannot set up a large number of ground reference stations, and can only compensate for the corresponding losses by choosing instruments with higher accuracy. Posture measurement error is one of the factors affecting the positioning accuracy of the LiDAR system. By connecting the rigid IMU with the laser scanner, the posture of the two is the same. IMU posture measurement accuracy will be affected by accelerometer proportional error, velocimeter constant error, random drift, gyroscope system drift, and other factors. Besides, posture angle error varies with time and measurement location [19]. At present, according to the international advanced GPS/INS combination accuracy level, the yaw angle can reach 0.005 , while the roll angle and the pitch angle can reach 0.0025 .
deviation is manifested in the following characteristic [19].
2.5. Effect of system integration error
3.1. Testing of point cloud calculation method
The first point is the installation error. The most critical error is the boresight error between scanner and IMU. Installation of LiDAR measurement system requires that the coordinate axes of the laser scanning coordinate system and inertial platform reference coordinate system are parallel to each other, but there will be errors in installation. The placement angle error should also include the translation error between the coordinate systems of each instrument, namely, the eccentricity error. Generally speaking, the placement angle error needs to be measured by platform calibration. The second point is the time deviation. Time deviation includes synchronization error and interpolation error. In order to realize the three-dimensional positioning of the target, the orientation, positioning, and ranging of the LiDAR system must be carried out at the same time. It cannot guarantee the complete synchronization of time, which cause variable positioning error. When the platform is in a stable flight state, the influence of time deviation between ranging and posture angle measurement can be ignored. On the contrary, when the platform is not stable, the time deviation will have a great impact on the accuracy of the three-dimensional measurement. The interpolation error is caused by the different sampling frequencies of each subsystem in the platform-borne LiDAR. The sampling frequency of GPS data is the lowest, generally 1e20 Hz, and the sampling frequency of INS is about 200 Hz while the highest of the scanner is 500 kHz. Therefore, interpolating INS posture data and GPS data with low sampling rate is necessary, and using GPS position, posture interpolation between adjacent scanning lines leads to interpolation error. The time
In order to verify the point cloud calculation method, we establish a simulation environment. We input the object or scene to be scanned into the simulation system and convert the input data into the instantaneous coordinate system of the missile. Because the model is usually composed of several triangular slices, we use ray-tracing method [21] to obtain the intersection point of laser beam and target and record the distance value, the position value and the posture information of platform. The point cloud computing method we proposed is precisely using the above information to complete the point cloud coordinate calculation. The simulation conditions are as follows: the laser radar sensor platform is a missile, and its distance from the origin of the target scene coordinates sets to 3000 m. The flight speed is 1 Mach, and the rotating speed is 1 r/s. The elevation angle of the laser beam center direction relative to the target scene coordinate system is 30 , and the azimuth angle is set to 0 . The parameters of the laser radar system simulated in this paper are as follows: repetition frequency is 10 KHz, the frame frequency is 20 Hz. Thus, each frame has the most point cloud images. 64 500 points can be obtained, and the divergence angle of the laser beam is 0.5 rad. Computer environment is CPU, Intel Core i5, 4-core 3.3 GHz; memory, 4 GB. The experimental results Fig. 3 and Table 1 show that the proposed model can calculate the point cloud of LiDAR satisfactorily, and the obtained point cloud data can depict the shape and surface structure of the object in a specific perspective more truthfully, thus providing the original data for precise guidance. It is worth noting that the distribution of point clouds will be sparser when the distance is longer. It is difficult to detect small targets. Although the
2.6. Influence of platform vibration error Different platforms have different vibration factors. The vibration affects the missile-borne LiDAR positioning equation by generating scanning angle error, placement angle error, and flight trajectory error, resulting in inaccurate point cloud calculation. Usually, the offset of the vibration to the three directions of x, y and z cannot cause large data deviation in practice. However, the impact of vibration on the collective positioning accuracy caused by missile-borne LiDAR cannot be neglected. If the corresponding measures are not taken to prevent the impact of vibration, the error caused by vibration will reduce the measurement accuracy of LiDAR. The traditional error correction method based on the datadriven polynomial model is difficult to eliminate its influence. Thus it is necessary to add appropriate shock absorber. 3. Results
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
7
Fig. 6. Point cloud deviations on the x, y and z axes.
density of point clouds can be improved by increasing resolution, it will also increase the power and cost of lasers. Therefore, the choice of equipment needs to weigh these two factors. 3.2. Simulation analysis of the influence of motion on the field of view In the influence of motion on the field of view, we simulate and analyze the limiting conditions. When the rotation speed is reduced to 1r/s, a_ ¼ 2pðrad =sÞ, the remaining conditions remain unchanged with section 3.1. In order to observe the effect of motion on the field of view, we scan the ground and summarized the
simulation results as follows: As can be seen from the Fig. 4, at the condition of a_ ¼ 2pðrad =sÞ, it can ensure that the front and back frames have an extensive range of overlap and the continuous coverage of the target. In practical application, because of the increase of follow-up devices, the actual posture changes will be less than the simulation conditions, and the actual detection field of view obtained can also achieve a relatively large coincidence effect. As far as possible to ensure that the target is always in the field of view in the course of platform movement can provide a prerequisite for the development of target tracking and precise guidance.
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
8
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
3.3. Simulation analysis of the effect of vibration on point cloud quality
Declaration of competing interest The authors declare that there is no conflict of interest.
Fig. 5 (a) and (c) show point clouds under ideal conditions and point clouds under vibration conditions. Fig. 5 (c) shows two contrast effects. According to Fig. 5, the vibration will affect the accuracy of point cloud calculation. Fig. 6 (a)-(c) are the point cloud deviations under ideal conditions and vibration conditions on the x, y and z axes, respectively. According to Figs. 6 (a)e(c), the difference among the three axes tend to impact the accuracy of the target position. However, when the vibration is controlled in a specific range, the influence of the vibration is not significant. Figs. 6 (d)e(f) show the maximum error on the x, y, and z axes at 0e500 unit times, respectively. According to the above analysis, the corresponding hardware or software measures need to be taken to reduce the impact of vibration. 4. Conclusions In this study, the factors influencing the calculation of point cloud for missile-borne linear array scanning LiDAR are modeled and analyzed. First, we model the point cloud calculation process of missile-borne linear array scanning LiDAR under ideal conditions. The experimental results show that the calculation method is effective. On this basis, in order to track the target, we analyze the effect of posture change on the field of view in the course of platform movement and propose three limiting conditions. Only when these conditions are satisfied, the stability of the field of view can be guaranteed, and the conditions for tracking can be provided. Finally, we establish error formulas and analyze several typical errors in missile platform flight, including scanner error, POS error, system integration error, and platform vibration error. Through theoretical analysis and simulation experiments, the model proposed in this paper can provide a theoretical basis and parameters for the design of the missile-borne LiDAR system and the selection of related instruments in practice. In the subsequent calculation, target detection and tracking algorithm should be studied in combination with guidance law. Funding This research was financially supported by the China National Natural Science Foundation (Grant No. 61871389), the Research Plan Project of the National University of Defense Technology (Grant No. ZK18-01-02) and the Director Foundation of State Key Laboratory of Pulsed Power Laser Technology (Grant No. SKL2018ZR09).
References [1] Snyder M, Qu Z, Hull R. Quad-segment polynomial trajectory guidance for impact-time control of precision-munition strike. IEEE Trans Aerosp Electron Syst 2016;52(6):3008e23. [2] Yu M, Gong L, Oh H, et al. Multiple model ballistic missile tracking with statedependent transitions and Gaussian PF. IEEE Trans Aerosp Electron Syst 2017;54(3):1066e81. [3] Kechagias-Stamatis O, Aouf N, Richardson MA. 3d automatic target recognition for future lidar missiles. IEEE Trans Aerosp Electron Syst 2016;52(6): 2662e75. R, Gollub MG, Sommer H, et al. Incremental-segment-based localization [4] Dube in 3-D point clouds. IEEE Robot Autom Lett 2018;3(3):1832e9. [5] Li J, Peng Q, Li Z, Li J, Wang R. Technology of Missile-borne laser active imaging guidance. Infrared Laser Eng 2014;43(4):1117e23. [6] Han Y, Sun H, Li Y, Guo H. Advances in foreign simulation softwares of imaging laser radar. Laser and Optoelectron Prog 2013;50(1):46e54. [7] Kechagias-Stamatis O, Aouf N. Evaluating 3d local descriptors for future lidar missiles with automatic target recognition capabilities. Imaging Sci J 2017: 1e10. [8] Kechagias-Stamatis O, Aouf N, Gray G, et al. Local feature based automatic target recognition for future 3D active homing seeker missiles. Aero Sci Technol 2017;73:309e17. [9] Douchin JA, Meynard T. Simulation of active EO imaging system based on SEWorkbench and OSMoS software tools. In: 5th international R Target and background modeling & simulation workshop; 2009. p. 1e18. [10] Chevalier TR, Steinvall OK. Laser radar modeling for simulation and performance evaluation. In: Proceedings of SPIE - the international society for optical engineering; 2009. 748206-748206-13. [11] Wu X. Study of moving target detection and recognition based on laser radar. Wuhan: Huazhong University of Science and Technology; 2015. [12] Chen X. Research on target feature extraction and recognition based on imaging ladar. Changsha: National University of Defense Technology; 2015. [13] Richard H, Raffel M. Principle and applications of the background oriented schlieren (BOS) method. Meas Sci Technol 2001;12:10. [14] Csanyi N, Toth CK. Improvement of lidar data accuracy using lidar specific ground targets. Photogramm Eng Remote Sens 2007;73(4):385e96. [15] Dickman J, Uijt De Haag M. Aircraft heading measurement potential from an airborne laser scanner using edge extraction. In: Proceedings of the 2007 IEEE aerospacz conference. Big Sky Montana; 2007. p. 1e16. [16] Wang J, Liu J. Analysis and sorting of impacts of measurement errors on positioning accuracy of laser point cloud obtained from airborne laser scanning. Chin J Lasers 2014;41(4):0414001. [17] Toch C. The state-of-the-art in airborne data collection systems-Focused on LiDAR and optical imagery. Columbus: Center for Mapping; 2009 [The Ohio State University, Columbus]. [18] Reinshaw. Data sheet: RESR angular encoder system. 2013 [Error! Hyperlink reference not valid.]. [19] Lemmens M. Airborne LiDAR sensors. GIM Int 2007;21(2):24e7. [20] Goulden T. Prediction of error due to terrain slope in LiDAR observation. New Brunswick: Department of Geodesy and Geomatics Engineering; 2009. Technical Report No. 265, University of New Brunswick. [21] Chen X, Shi Z, Yang W, Lu X. A fast simulation algorithms of scanning laser radar. Chin J Lasers 2014;41(08):281e6.
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
Contents lists available at ScienceDirect
Defence Technology journal homepage: www.elsevier.com/locate/dt
Modeling and analyzing point cloud generation in missile-borne LiDAR Min-le Li a, b, Yi-hua Hu a, b, *, Nan-xiang Zhao a, b, Qi-shu Qian a, b a b
State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering, National University of Defense Technology, Hefei, 230037, China Anhui Province Key Laboratory of Electronic Restriction Technology, Hefei, 230037, China
a r t i c l e i n f o
a b s t r a c t
Article history: Received 31 July 2019 Received in revised form 16 September 2019 Accepted 15 October 2019 Available online xxx
The missile-borne LiDAR has an essential prospect in precise guidance. However, the instability of the missile has a significant impact on the precision of LIDAR point cloud, thus modeling and analyzing the influencing factors of point cloud generation is necessary. In this investigation, the authors modeled the point cloud calculation process of missile-borne linear array scanning LiDAR under ideal conditions and analyzed the effect of position change on the field of view (FOV) in the course of platform motion. Subsequently, the authors summarized that the stability concerning the field of view and the target tracking are dependent on three limiting conditions. Finally, the authors derived an error formula and analyzed several typical errors in missile platform flight, including scanner error, POS error, system integration error, and platform vibration error. Based on theoretical analysis and simulation experiments, the model proposed in this paper can provide a theoretical basis for the design of the missile-borne LiDAR system and the selection of related instruments in practice. © 2019 Production and hosting by Elsevier B.V. on behalf of China Ordnance Society. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Keywords: LiDAR Point cloud Precise guidance Vibration
1. Introduction Precision guided weapons have become an indispensable part of modern warfare because of their high hit rate, great combat effectiveness, and deterrence. Therefore, this type of weapon has attracted close attention worldwide [1e3]. Precision guidance weapons include radar guidance, infrared guidance, television guidance, and laser guidance. Laser imaging guidance technology has the advantages of high resolution, strong anti-jamming ability, and all-weather combat, and is attracting more and more research attention [4]. The United States is at the forefront of the world in the research of Light Detection and Ranging (LiDAR) guidance [5]. Since the 1980s, many military scientific research institutes in the United States have developed three-dimensional LiDAR seekers. At present, LiDAR seekers with specific algorithms have the ability of automatic target recognition. Multiple types of missiles are equipped with this type of LiDAR seeker. For example, the United
* Corresponding author. State Key Laboratory of Pulsed Power Laser Technology, College of Electronic Engineering, National University of Defense Technology, Hefei, 230037, China. E-mail address: [email protected] (Y.-h. Hu). Peer review under responsibility of China Ordnance Society
States Army Research Laboratory (ARL) and other units have developed and tested low-cost autonomous attack system (LOCAAS), which is a highly intelligent, unmanned aircraft launched outside the defense zone. The LOCAAS seeker has the ability of three-dimensional imaging of the target. It can scan accurately and autonomously at a rate of 100 Hz, and the scanning width is 750 m. Once the target is captured, the scanning width will be reduced to 100 m, the field of view angle is 4e6 , the operating distance is 1e1.25 km, and the resolution can be achieved to 15 cm. The target identification algorithm has an advanced feature extraction function. Since 1999, the United States has been developing a Flash LiDAR, the non-scanning three-dimensional imaging LiDAR, the imaging rate of which is at least five times higher than that of the scanning imaging LiDAR used in the previous LOCASS. NetFires (Network Fires, Network Fire Support) is a joint project proposed by DARPA and Army of the United States. It aims to provide the Army with the next generation of attack and patrol precision-guided missiles. The patrol attack missiles of NetFires weapon system are equipped with active LiDAR seekers that can automatically identify targets at 230 m altitude and 400 km/h speed. Besides, the general LiDAR seeker and the “vertical saw” imaging LiDAR system in the United States also use threedimensional imaging technology, which can automatically identify targets in various environments [5].
https://doi.org/10.1016/j.dt.2019.10.003 2214-9147/© 2019 Production and hosting by Elsevier B.V. on behalf of China Ordnance Society. This is an open access article under the CC BY-NC-ND license (http:// creativecommons.org/licenses/by-nc-nd/4.0/).
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
2
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
As for the researches on LiDAR modeling, there are many achievements in the research of LiDAR simulation all over the world [6]. O. Kechagias-Stamatis [7] evaluated 3D local descriptors and proposed an architecture [8] for future missile seeker with automatic target recognition (ATR) capabilities. Many research institutes, such as Oktal Synthetic Environment Company of France [9] and the Swedish National Defense Research Bureau [10], have developed and put into use the simulation platform of LiDAR. Huazhong University of Science and Technology [11] established an airborne dual-wedge LiDAR simulation platform, and obtained point cloud images in different scenarios, but the single-point scanning efficiency is lower than linear array scanning LiDAR. Scanning LiDAR is simulated and modeled by National Defense University [12]. Range and intensity images are obtained by analyzing the echo information in each beam, but the action distance is relatively close, and the scanning period is longer than linear array scanning LiDAR. At present, there is no missile-borne LiDAR simulation platform in China. As for the researches on platform motion error analysis, since the related literatures on missile-borne LiDAR point cloud motion error analysis are seldom published, this part reviews the published research works about image distortion correction of the missile-borne infrared imaging guidance system and related research on motion errors of satellite-borne and airborne LiDAR, so as to provide a reference for related research of missile-borne LiDAR. H Richard et al. [13] studied the image distortion caused by aero-optics of the supersonic aircraft, measured the wavefront parameters by the background oriented schlieren (BOS) technique, and obtained the offset of all targets on the imaging surface by difference method. CSANYI et al. [14] corrected the point cloud error of airborne LiDAR by setting markings on the ground. Dickman et al. [15] studied the influence of posture angle random measurement error on airborne LiDAR plane terrain imaging through numerical simulation but did not carry out more complex terrain simulation. Wang et al. [16] analyzed the causes of measurement errors affecting the accuracy of airborne laser scanning point clouds. By analyzing the working principle of conical rotating airborne laser scanning, the mapping relationship between error factors and point clouds accuracy was established. The equation between error factors and three-dimensional coordinate errors of point clouds was established by multiple linear regression method, and the causes of errors were quantitatively evaluated. The significant effect of the element on accuracy provides a theoretical basis for error allocation and suppression and compensation of essential factors in the airborne laser scanning system. Based on our literature review, we found that those exist studies cannot be applied to point cloud generation and motion error analysis of missile-borne linear array scanning LiDAR. Because of the high cost of research and development of the missile-borne platform, using an accurate system simulation model to guide practice is necessary. Besides, the movement of the platform often leads to the distortion of point cloud acquired by LiDAR in the process of scanning, and relevant research needs to be carried out. In this paper, firstly, the point cloud calculation process of missile-borne linear array scanning LiDAR under ideal conditions is modeled. On this basis, the effect of posture change on the field of view in the course of platform motion is analyzed, and three limiting conditions are put forward. Only when these conditions are satisfied, the stability of the field of view can be guaranteed, and the target tracking can be carried out. Finally, the authors derived an error formula and analyzed several typical errors in missile platform flight, including scanner error, POS error, system integration error, and platform vibration error.
2. Material and methods 2.1. The method of calculating point cloud coordinates of missileborne LiDAR Fig. 1 shows the scanning principle of linear scan LiDAR system for ground targets, and the beam vector of LiDAR is set as S. The direction of laser emission is Y axis, the direction of pitch angle increase is Z axis, and the X axis is determined according to the right hand rule. For the missile-borne LiDAR system, the starting point Vs ðxV ; yV ; zV ÞT is the platform of the optical system and can be measured by dynamic differential GPS. The mode of S represents the distance between the optical system and the ground laser foot and the direction of S is the platform posture parameter ða; b; gÞ as pitch - yaw - roll, which can be obtained by the high-precision posture measuring device. According to the above parameters, the three-dimensional coordinates of the ground point P in the geodetic coordinate system can be calculated. The system can acquire high-precision laser point cloud data by continuously transmitting laser pulses and receiving echo information through scanning. The difficulty of simulation technology of missile-borne Linear Array scanning LiDAR is that there are great differences in the motion of missiles, not only in flight speed but also in rotation speed. For each laser landing point, the collected information includes the range information from the landing point to the LiDAR (1D), the real-time position information (3D), the platform posture (3D), the position and posture relationship between the platform and the laser (6D) and the beam splitting angle (1D). More than 14 dimensions to calculate the position of each landing point. In order to facilitate analysis, Fig. 2 shows the coordinate system. The black coordinate system in Fig. 2 represents the local coordinate system: take the point projected to the ground from the original position of the missile as the origin, the vertical upward axis is the Z-axis, and the forward direction of the missile is the Yaxis. The X axis is determined according to the right-hand rule. The blue coordinate system in Fig. 2 represents the instantaneous coordinate system: the direction of laser emission is Y0 axis, the direction of elevation increase is Z0 axis, and the X0 axis is determined according to the right hand rule. The 64-line lasers are calibrated with serial numbers 1 to 64 in the positive direction of X0-axis, and the 32nd laser is aligned with the Y0-axis. Four steps accomplish the calculation of point cloud. First, the coordinates of the landing point in the instantaneous coordinate system are calculated according to the distance and beam splitting information. For the nth unit time, the coordinates of the laser point corresponding to the linear array laser beam are as follows:
2
3 2 3 xsl sn tanðan Þ 4 ysl 5 ¼ 4 5 0 zsl 1,sn
(1)
Fig. 1. Missile-borne linear array LiDAR system for target detection and tracking.
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
3 33 22 3 2 xLH xN xI 4 yLH 5 ¼ RN ,44 yI 5 4 yN 5 5 zLH zI zN
3
2
(4)
Here, the transformation matrix RN ¼ RfN ,RqN ,R4N representing the platform coordinate system to the local horizontal coordinate system is related to the posture angles in the three directions of rotation, pitch, and yaw, and the form is similar to Eq. (3). lN ¼ ðxN ; yN ; zN ÞT represents the eccentricity between the phase center of the GPS antenna and the inertial platform. Moreover, to convert the local horizontal coordinate system to the WGS-84 coordinate system.
2 3 3 xLH x84 4 y84 5 ¼ RW ,4 yLH 5 þ l84 z84 zLH 2
Fig. 2. Instantaneous scanning coordinate system and platform coordinate system.
Among them, sn represents the distance between 64 laser beams at the nth unit time and an ¼ 15:5,16 mrad is the row vector formed by the angle between the corresponding 64 laser beams and the Y ’ axis. Second, according to the position and posture relationship between the laser and the platform, the instantaneous coordinate system is converted to the platform coordinate system, as shown by Eq. (2).
where, the transformation matrix RW ¼ RfW ,RqW ,R4W representing the platform coordinate system to the local horizontal coordinate system is related to three posture angles in a form similar to Eq. (3). l84 ¼ ðx84 ; y84 ; z84 ÞT represents the coordinates of the phase center of the GPS antenna in the WGS-84 coordinate system. In summary, the coordinates of the laser point in WGS-84 coordinate system are as follows:
2 2 2 3 3 2 3 2 333 sn ,tanðan Þ x84 xP xN 4 y84 5 ¼ RW ,4RN ,4RP ,4 5 þ 4 yP 5 4 yN 5 5 5 0 z84 zP zN 1,sn 3 2 x84 þ 4 y84 5 z84 2
(6) Simplified to vector form,
P W ¼ RW ðRN ðRP r þ lP lN Þ Þ þ l84 2 3 3 xsl xI 4 yI 5 ¼ RP ,4 ysl 5 þ lP zI zsl 2
(2)
Among them, the matrix lP ¼ ðxP ; yP ; zP ÞT represents the influence of displacement in the process of transformation, and the matrix RP ¼ RfP ,RqP ,R4P is used to represent the error matrix consisting of three orientations angles: rotation, pitch, and yaw.
8 > > 3 2 > > > cos fV;n sin fV;n 0 > > > 7 6 f > > RP ¼ 4 sin fV;n cos fV;n 05 > > > > > 0 0 1 > > > 2 3 > > 1 0 0 > < 6 cos qV;n 7 RqP ¼ 4 0 sin qV;n 5 > > > 0 cos q q sin > V;n V;n > > 3 2 > > > cos 4V;n sin 4V;n 0 > > > 7 6 > > R4P ¼ 4 sin 4V;n cos 4V;n 05 > > > > > 0 0 1 > > :
(3)
Third, according to the real-time position and posture information of the platform itself in the local horizontal reference coordinate system, the coordinates are transformed into the local horizontal reference coordinate system.
(5)
(7)
where, r represents the position vector of the laser point in the instantaneous coordinate system and P W is the coordinate of the laser foot in the WGS-84 coordinate system. 2.2. The effect of motion on point cloud calculation The flight altitude of the missile can reach several kilometers. We utilized a 64-element linear array scanning laser radar, with the interval of each beam being 0.5 mrad and the instantaneous field of view range being 32 mrad. The laser frequency is 10 kHz, and the frame frequency is 20 Hz, and each frame has 500 scans. At this altitude, the influence of translation motion of missile on point cloud computing can be neglected, but the influence of posture change and the vibration is larger. Thus it must be considered. By analyzing the influence level of these two factors, this paper provides a reference for the selection of the gyro platform and shock absorber. 2.2.1. Three limiting conditions of posture change on the field of view In order to achieve precise guidance, it is necessary to ensure that the overlapping field of view of two adjacent frames has a larger overlap area, so as to ensure that the target is not lost. For the terminal guidance of the missile, the forward-looking method is used to detect the missile. The effect of the translation motion on the field of view is relatively small, while the change of posture will have a more significant impact. According to the different speed of posture change, we put forward three limiting conditions. Only
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
4
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
when these three conditions are satisfied, the stability of the field of view can be guaranteed. 1) The limiting condition of coincidence of transmitting and receiving footprints: the scanning time of a complete frame of the point cloud is 0.05 s, and each scanning time is t0 ¼ 104 s. The distance between transmitting and receiving is r and the time internal is Dt ¼ 2r=c. When the distance is within [500,7000] m, it can be calculated that 3:33 106 Dt 4:67 105 s. The field of view of a single receiving unit is U ¼ 5 104 rad, and when the 5104 rad ¼ 10:71 rad (1.7r/ posture changes at a speed of a_ > tU0 ¼ 4:6710 5 s =s s), the signal cannot be received. 2) Limiting condition of the field of view regularization: for two adjacent scans, if the angle and position deviation is too large, it will not be able to achieve parallel or approximate parallel relationship. According to the above analysis, when the posture change speed is less than 10.71 rad/s, it can ensure effective reception. At this time, the rotation angle is taken as an example to consider the parallel situation of adjacent scans. Because the time interval between two adjacent scans is t0 ¼ 104 s, the rotation angle difference between two adjacent scans is 1.07 mrad, namely 0.061, and within a frame (500 scans) the total deviation is 30.65 . The irregular change of the field of view makes it difficult to achieve continuous and practical coverage of the target. The setting of limiting conditions for target reconstruction is more of empirical value. 3) Limiting condition of overlap of the adjacent field of view: the system covers a limited field of view. When the posture changes too fast, the overlap of adjacent two frames is too small, resulting in the loss of the target. A complete frame point cloud can cover 32 mrad in the scanning direction by oscillating the pendulum mirror to and fro in the scanning time of tf ¼ 0:05s, thus the speed of the pendulum mirror is about Ur z0:032rad =0:05s ¼ 0:64ðrad=sÞ.The posture angle changes at different speeds a_ , which will also have a significant impact on the unfolding effect of the scanning. The new coverage is expressed as U0 ¼ ðUr a_ Þtf . It can be divided into the following three situations: (1) The change speed of posture angle is less than the swing speed of the swing mirror. If the direction is opposite, the coverage will be reduced (between 0 and one times), and if the direction is the same, the coverage will be increased (between 1 and two times). (2) The change speed of posture angle is equal to the swing speed of the swing mirror. If the direction is opposite, the coverage will be reduced to 0, and the scanning cannot be deployed effectively. If the direction is the same, the coverage will be increased by two times. (3) The change speed of posture angle is faster than the swing speed of the swing mirror. If the direction is opposite, the coverage will extend to the opposite direction; if the direction is the same, the coverage will increase. It is evident that when the change speed of posture angle is equal to or higher than the swing speed of the swing mirror, the coverage of the front and rear field of view will change significantly, even without overlap. This situation can easily lead to the loss of targets. Therefore, it should be less than half of the swing speed of the pendulum mirror, i.e. a_ < 0:32ðrad=sÞ. Under the above limiting conditions, it is necessary to select the appropriate servo device to solve the problem and eliminate the rapid posture change. At the same time, miniaturization design should be taken into account.
2.2.2. Effect of motion error on generating the point cloud The error of LiDAR can be roughly divided into the following categories:
(1) Scanner errors, including ranging errors and scanning angle errors; (2) POS error, namely GPS error and INS error; (3) System integration errors, including installation errors, time deviations. (4) Other errors, including coordinate transformation errors, errors caused by the irregular motion of the platform, elevation errors of type II, and some random errors. Considering various errors, Eq. (8) shows the geometric model of the missile-borne LiDAR
P *W ¼ RW ðDRN RN ðDRP RP ðr þ DrÞ þ lPN þ DlPN Þ Þ þ l84 þ Dl84 þ DlTB (8) where, P *W is the position vector of laser foot under the WGS-84 coordinate system affected by various errors. The term DRN represents the rotation matrix of the platform posture angle measurement error and DRP indicates the system placement error rotation matrix. DlPN means the measurement error of eccentricity component from laser launching point to GPS antenna phase center. Moreover, the Dl84 is the positioning error vector of GPS while DlTB represents the position error vector caused by time deviation. 2.3. Error effect of scanner (1) Ranging error. Laser scanning ranging errors are attributed to the following aspects [17]. The first point is the instrument error. It includes the non-parallel errors of laser signal transmitting and receiving, the measurement errors of laser pulse signal propagation time, the rotation and vibration errors of the reflector, and the zero-point errors of the pulse. The second point is the atmospheric refraction error. When the laser penetrates the atmosphere, it is also affected by the refraction error of the atmosphere (troposphere). The degree of influence is related to the wavelength of the laser pulse. The atmospheric refraction error of the same signal is related to temperature, pressure, and humidity. The wavelength of the laser pulse is about 1 mm, while that of the GPS carrier signal is about 20 cm. Therefore, compared with GPS signal, the laser pulse signal is much less affected by atmospheric refraction, and the absolute amount of tropospheric delay improvement in zenith direction is only a few millimeters, which is much smaller than the accuracy of LiDAR measurement at present. After simple model modification, the residual error can be neglected. The third point is errors related to the reflector. Different physical characteristics of the surface will produce different reflections. When diffuse reflection occurs, a large number of backscattering signals are received, resulting in larger receiving noise. On the contrary, if the signal is transmitted to a smooth surface, mirror reflection is formed, and backscattering signals are very few. Besides, if angular reflection occurs, backscattering may be the result of multiple reflections. Thus the measured time delay cannot represent real-time delays. The ranging accuracy is also related to the roughness of the ground, the slope of the terrain and the interference of the ground objects. Generally speaking, after system error correction, the ranging error of laser scanning can be controlled in a minimal range and can be neglected. (2) Scanning angle error. The scanner uses the scanning prism to turn the laser beam from different angles by driving the motor. Non-uniform speed of the scanning motor, the
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
5
Fig. 3. Point cloud data of typical 3D models.
Table 1 The simulation information of typical 3D models. Models
Number of surface elements
Running time/s
F15EL Missile T72tank
9415 12970 14562
4 5 7
vibration of the scanning mirror, temperature change, and beam divergence angle [18] also bring errors to the scanning angle. Scanning angle error will affect the coordinates of laser foot in the instantaneous coordinate system by converting to ranging error and posture angle error. There are three main types of scanning errors: the instantaneous scanning angle error, the pointing error, and the Scanning plane distortion. When a laser scanner is installed, the laser scanning plane cannot be completely perpendicular to the xaxis of the laser coordinate system, resulting in distortion of the scanning plane. These are small angle values, usually a few percent [19].
2.4. POS error effect GPS measurement data to determine the position of the aircraft platform and sensor. In the process of GPS dynamic positioning, it will be affected by tropospheric delay error, ionospheric delay error,
Fig. 4. Effect of motion on scanning field of view.
satellite clock difference, phase center instability, observation noise, multi-path effect, and so on. GPS positioning error is the main factor affecting the accuracy of laser foot, but because it will change with the change of the detection environment, it is not easy to eliminate and model. It is difficult to eliminate these errors in the later data processing. The main work to reduce GPS positioning error is in data acquisition, which can be reduced by establishing
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
6
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
Fig. 5. Effect of vibration on point cloud calculation accuracy.
multiple ground reference stations uniformly in the measuring area. In the actual operation process of the airborne platform, more than three ground reference stations should be placed, and the distance between ground references stations should not be more than 25 km. Better results can be obtained by GPS solution. At present, the best GPS positioning error is about 5 cm [20]. However, the missile-borne platform cannot set up a large number of ground reference stations, and can only compensate for the corresponding losses by choosing instruments with higher accuracy. Posture measurement error is one of the factors affecting the positioning accuracy of the LiDAR system. By connecting the rigid IMU with the laser scanner, the posture of the two is the same. IMU posture measurement accuracy will be affected by accelerometer proportional error, velocimeter constant error, random drift, gyroscope system drift, and other factors. Besides, posture angle error varies with time and measurement location [19]. At present, according to the international advanced GPS/INS combination accuracy level, the yaw angle can reach 0.005 , while the roll angle and the pitch angle can reach 0.0025 .
deviation is manifested in the following characteristic [19].
2.5. Effect of system integration error
3.1. Testing of point cloud calculation method
The first point is the installation error. The most critical error is the boresight error between scanner and IMU. Installation of LiDAR measurement system requires that the coordinate axes of the laser scanning coordinate system and inertial platform reference coordinate system are parallel to each other, but there will be errors in installation. The placement angle error should also include the translation error between the coordinate systems of each instrument, namely, the eccentricity error. Generally speaking, the placement angle error needs to be measured by platform calibration. The second point is the time deviation. Time deviation includes synchronization error and interpolation error. In order to realize the three-dimensional positioning of the target, the orientation, positioning, and ranging of the LiDAR system must be carried out at the same time. It cannot guarantee the complete synchronization of time, which cause variable positioning error. When the platform is in a stable flight state, the influence of time deviation between ranging and posture angle measurement can be ignored. On the contrary, when the platform is not stable, the time deviation will have a great impact on the accuracy of the three-dimensional measurement. The interpolation error is caused by the different sampling frequencies of each subsystem in the platform-borne LiDAR. The sampling frequency of GPS data is the lowest, generally 1e20 Hz, and the sampling frequency of INS is about 200 Hz while the highest of the scanner is 500 kHz. Therefore, interpolating INS posture data and GPS data with low sampling rate is necessary, and using GPS position, posture interpolation between adjacent scanning lines leads to interpolation error. The time
In order to verify the point cloud calculation method, we establish a simulation environment. We input the object or scene to be scanned into the simulation system and convert the input data into the instantaneous coordinate system of the missile. Because the model is usually composed of several triangular slices, we use ray-tracing method [21] to obtain the intersection point of laser beam and target and record the distance value, the position value and the posture information of platform. The point cloud computing method we proposed is precisely using the above information to complete the point cloud coordinate calculation. The simulation conditions are as follows: the laser radar sensor platform is a missile, and its distance from the origin of the target scene coordinates sets to 3000 m. The flight speed is 1 Mach, and the rotating speed is 1 r/s. The elevation angle of the laser beam center direction relative to the target scene coordinate system is 30 , and the azimuth angle is set to 0 . The parameters of the laser radar system simulated in this paper are as follows: repetition frequency is 10 KHz, the frame frequency is 20 Hz. Thus, each frame has the most point cloud images. 64 500 points can be obtained, and the divergence angle of the laser beam is 0.5 rad. Computer environment is CPU, Intel Core i5, 4-core 3.3 GHz; memory, 4 GB. The experimental results Fig. 3 and Table 1 show that the proposed model can calculate the point cloud of LiDAR satisfactorily, and the obtained point cloud data can depict the shape and surface structure of the object in a specific perspective more truthfully, thus providing the original data for precise guidance. It is worth noting that the distribution of point clouds will be sparser when the distance is longer. It is difficult to detect small targets. Although the
2.6. Influence of platform vibration error Different platforms have different vibration factors. The vibration affects the missile-borne LiDAR positioning equation by generating scanning angle error, placement angle error, and flight trajectory error, resulting in inaccurate point cloud calculation. Usually, the offset of the vibration to the three directions of x, y and z cannot cause large data deviation in practice. However, the impact of vibration on the collective positioning accuracy caused by missile-borne LiDAR cannot be neglected. If the corresponding measures are not taken to prevent the impact of vibration, the error caused by vibration will reduce the measurement accuracy of LiDAR. The traditional error correction method based on the datadriven polynomial model is difficult to eliminate its influence. Thus it is necessary to add appropriate shock absorber. 3. Results
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
7
Fig. 6. Point cloud deviations on the x, y and z axes.
density of point clouds can be improved by increasing resolution, it will also increase the power and cost of lasers. Therefore, the choice of equipment needs to weigh these two factors. 3.2. Simulation analysis of the influence of motion on the field of view In the influence of motion on the field of view, we simulate and analyze the limiting conditions. When the rotation speed is reduced to 1r/s, a_ ¼ 2pðrad =sÞ, the remaining conditions remain unchanged with section 3.1. In order to observe the effect of motion on the field of view, we scan the ground and summarized the
simulation results as follows: As can be seen from the Fig. 4, at the condition of a_ ¼ 2pðrad =sÞ, it can ensure that the front and back frames have an extensive range of overlap and the continuous coverage of the target. In practical application, because of the increase of follow-up devices, the actual posture changes will be less than the simulation conditions, and the actual detection field of view obtained can also achieve a relatively large coincidence effect. As far as possible to ensure that the target is always in the field of view in the course of platform movement can provide a prerequisite for the development of target tracking and precise guidance.
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003
8
M.-l. Li et al. / Defence Technology xxx (xxxx) xxx
3.3. Simulation analysis of the effect of vibration on point cloud quality
Declaration of competing interest The authors declare that there is no conflict of interest.
Fig. 5 (a) and (c) show point clouds under ideal conditions and point clouds under vibration conditions. Fig. 5 (c) shows two contrast effects. According to Fig. 5, the vibration will affect the accuracy of point cloud calculation. Fig. 6 (a)-(c) are the point cloud deviations under ideal conditions and vibration conditions on the x, y and z axes, respectively. According to Figs. 6 (a)e(c), the difference among the three axes tend to impact the accuracy of the target position. However, when the vibration is controlled in a specific range, the influence of the vibration is not significant. Figs. 6 (d)e(f) show the maximum error on the x, y, and z axes at 0e500 unit times, respectively. According to the above analysis, the corresponding hardware or software measures need to be taken to reduce the impact of vibration. 4. Conclusions In this study, the factors influencing the calculation of point cloud for missile-borne linear array scanning LiDAR are modeled and analyzed. First, we model the point cloud calculation process of missile-borne linear array scanning LiDAR under ideal conditions. The experimental results show that the calculation method is effective. On this basis, in order to track the target, we analyze the effect of posture change on the field of view in the course of platform movement and propose three limiting conditions. Only when these conditions are satisfied, the stability of the field of view can be guaranteed, and the conditions for tracking can be provided. Finally, we establish error formulas and analyze several typical errors in missile platform flight, including scanner error, POS error, system integration error, and platform vibration error. Through theoretical analysis and simulation experiments, the model proposed in this paper can provide a theoretical basis and parameters for the design of the missile-borne LiDAR system and the selection of related instruments in practice. In the subsequent calculation, target detection and tracking algorithm should be studied in combination with guidance law. Funding This research was financially supported by the China National Natural Science Foundation (Grant No. 61871389), the Research Plan Project of the National University of Defense Technology (Grant No. ZK18-01-02) and the Director Foundation of State Key Laboratory of Pulsed Power Laser Technology (Grant No. SKL2018ZR09).
References [1] Snyder M, Qu Z, Hull R. Quad-segment polynomial trajectory guidance for impact-time control of precision-munition strike. IEEE Trans Aerosp Electron Syst 2016;52(6):3008e23. [2] Yu M, Gong L, Oh H, et al. Multiple model ballistic missile tracking with statedependent transitions and Gaussian PF. IEEE Trans Aerosp Electron Syst 2017;54(3):1066e81. [3] Kechagias-Stamatis O, Aouf N, Richardson MA. 3d automatic target recognition for future lidar missiles. IEEE Trans Aerosp Electron Syst 2016;52(6): 2662e75. R, Gollub MG, Sommer H, et al. Incremental-segment-based localization [4] Dube in 3-D point clouds. IEEE Robot Autom Lett 2018;3(3):1832e9. [5] Li J, Peng Q, Li Z, Li J, Wang R. Technology of Missile-borne laser active imaging guidance. Infrared Laser Eng 2014;43(4):1117e23. [6] Han Y, Sun H, Li Y, Guo H. Advances in foreign simulation softwares of imaging laser radar. Laser and Optoelectron Prog 2013;50(1):46e54. [7] Kechagias-Stamatis O, Aouf N. Evaluating 3d local descriptors for future lidar missiles with automatic target recognition capabilities. Imaging Sci J 2017: 1e10. [8] Kechagias-Stamatis O, Aouf N, Gray G, et al. Local feature based automatic target recognition for future 3D active homing seeker missiles. Aero Sci Technol 2017;73:309e17. [9] Douchin JA, Meynard T. Simulation of active EO imaging system based on SEWorkbench and OSMoS software tools. In: 5th international R Target and background modeling & simulation workshop; 2009. p. 1e18. [10] Chevalier TR, Steinvall OK. Laser radar modeling for simulation and performance evaluation. In: Proceedings of SPIE - the international society for optical engineering; 2009. 748206-748206-13. [11] Wu X. Study of moving target detection and recognition based on laser radar. Wuhan: Huazhong University of Science and Technology; 2015. [12] Chen X. Research on target feature extraction and recognition based on imaging ladar. Changsha: National University of Defense Technology; 2015. [13] Richard H, Raffel M. Principle and applications of the background oriented schlieren (BOS) method. Meas Sci Technol 2001;12:10. [14] Csanyi N, Toth CK. Improvement of lidar data accuracy using lidar specific ground targets. Photogramm Eng Remote Sens 2007;73(4):385e96. [15] Dickman J, Uijt De Haag M. Aircraft heading measurement potential from an airborne laser scanner using edge extraction. In: Proceedings of the 2007 IEEE aerospacz conference. Big Sky Montana; 2007. p. 1e16. [16] Wang J, Liu J. Analysis and sorting of impacts of measurement errors on positioning accuracy of laser point cloud obtained from airborne laser scanning. Chin J Lasers 2014;41(4):0414001. [17] Toch C. The state-of-the-art in airborne data collection systems-Focused on LiDAR and optical imagery. Columbus: Center for Mapping; 2009 [The Ohio State University, Columbus]. [18] Reinshaw. Data sheet: RESR angular encoder system. 2013 [Error! Hyperlink reference not valid.]. [19] Lemmens M. Airborne LiDAR sensors. GIM Int 2007;21(2):24e7. [20] Goulden T. Prediction of error due to terrain slope in LiDAR observation. New Brunswick: Department of Geodesy and Geomatics Engineering; 2009. Technical Report No. 265, University of New Brunswick. [21] Chen X, Shi Z, Yang W, Lu X. A fast simulation algorithms of scanning laser radar. Chin J Lasers 2014;41(08):281e6.
Please cite this article as: Li M-l et al., Modeling and analyzing point cloud generation in missile-borne LiDAR, Defence Technology, https:// doi.org/10.1016/j.dt.2019.10.003