- Email: [email protected]
Fast Angle Adaption of a MEMS-based LiDAR System
8th IFAC IFAC Symposium Symposium on on Mechatronic Mechatronic Systems Systems 8th Vienna, Sept. 4-6, 2019 8th IFACAustria, Symposium Systems online at www.sciencedirect.com Vienna, Austria, Sept. on 4-6,Mechatronic 2019 Available 8th IFACAustria, Symposium Systems Vienna, Sept. on 4-6,Mechatronic 2019 Vienna, Austria, Sept. 4-6, 2019
ScienceDirect
IFAC PapersOnLine 52-15 (2019) 55–60
Fast Fast Fast
Angle Angle Angle
Philipp Philipp Philipp Philipp
Adaption Adaption of of a a MEMS-based MEMS-based LiDAR System Adaption of a MEMS-based LiDAR System LiDAR LiDAR System System ∗ ∗ ∗ ∗ ∗ ∗
Stelzer Steger Stelzer ∗∗∗ Andreas Andreas∗∗Strasser Strasser ∗∗∗ Christian Christian Steger ∗∗∗ ∗∗ ∗∗ ∗∗ Alberto Garcia Norbert Druml Stelzer Andreas Strasser Christian ∗∗ ∗∗ ∗ Garcia ∗∗ Norbert ∗ Druml ∗∗ Steger ∗ Alberto Stelzer Strasser Christian ∗∗ Steger AlbertoAndreas Garcia ∗∗ ∗∗ Norbert Druml ∗∗ Alberto Garcia Norbert Druml ∗ ∗ University of Technology, Graz, Austria, ∗ Graz ∗ Graz University of Technology, Graz, Austria, ∗ Graz University Technology, Graz, Austria, (e-mail: {stelzer, of strasser, steger}@tugraz.at). {stelzer, strasser, steger}@tugraz.at). ∗(e-mail: Graz University of Technology, Graz, Austria, ∗∗ ∗∗ Technologies Austria AG, (e-mail: {stelzer, strasser, steger}@tugraz.at). ∗∗ Infineon Technologies Austria AG, Graz, Graz, Austria, Austria, ∗∗ Infineon (e-mail: {stelzer, strasser, steger}@tugraz.at). ∗∗ (e-mail: {alberto.garcia, norbert.druml}@infineon.com). Infineon Technologies Austria AG, Graz, Austria, ∗∗ (e-mail: {alberto.garcia, norbert.druml}@infineon.com). Infineon Technologiesnorbert.druml}@infineon.com). Austria AG, Graz, Austria, (e-mail: {alberto.garcia, (e-mail: {alberto.garcia, norbert.druml}@infineon.com). Abstract: The The amount amount of of Advanced Advanced Driver-Assistance Driver-Assistance Systems Systems (ADAS) (ADAS) in in middle-class middle-class cars cars Abstract: are increasing steadily. Next generation ADAS will take control in fully automated vehicles. Abstract: The amount of Advanced Driver-Assistance Systems (ADAS) in middle-class cars are increasing Next generation ADAS will takeSystems control (ADAS) in fully automated vehicles. Abstract: Thesteadily. amount of Advanced Driver-Assistance in highly middle-class cars are increasing steadily. Next generation ADAS will of take in fullyfor automated vehicles. Light Detection And Ranging Ranging (LiDAR) will be one one of thecontrol key enablers enablers for automated Light Detection And (LiDAR) will be the key highly automated are increasing steadily. Next generation ADAS will take control in fully automated vehicles. or even autonomous vehicles. The 1D Micro-Electro-Mechanical System (MEMS) MicroLight Detection And Ranging (LiDAR) will be one of the key enablers for highly automated The 1D will Micro-Electro-Mechanical System (MEMS) or even autonomous vehicles.(LiDAR) MicroLight Detection Ranging one of the Field-of-View key enablers for highly automated Scanning LiDARAnd is able able to detect detectThe obstacles inbe a predefined predefined (FoV). Future ADAS or even autonomous vehicles. 1D Micro-Electro-Mechanical System (MEMS) MicroScanning LiDAR is to obstacles in a Field-of-View (FoV). Future ADAS or even autonomous vehicles. The 1Dchange Micro-Electro-Mechanical System (MEMS) Microapplications will require dynamically FoVs as fast as possible (e.g. quickly switching Scanning LiDAR is able to detect obstacles in a predefined Field-of-View (FoV). Future ADAS to detect dynamically change as fastField-of-View as possible (e.g. quickly switching applications will require Scanning LiDAR is able to obstacles in a FoVs predefined (FoV). Future ADAS applications will require to dynamically change FoVs fast as possible (e.g. quickly switching between long-range long-range narrow FoV and and short-range wideas FoV). between narrow FoV short-range wide FoV). applications will require to dynamically change FoVs as fast as possible (e.g. quickly switching In this paper we introduce a novel system architecture that enables fast angle adaption of between long-range narrow FoV and short-range wide FoV). 1D In this paper we introduce aFoV novel system architecture that enables fast angle adaption of aa 1D between long-range narrow and short-range wide FoV). MEMS Micro-Scanning LiDAR System. The system architecture wasfast implemented in an anofFPGA FPGA In this paper we introduce a novel system architecture that enables angle adaption a 1D MEMS Micro-Scanning LiDAR System. The system architecture was implemented in In this paper we introduce a novel system architecture that enables angle adaption a 1D prototype to prove prove its feasibility feasibility and to The evaluate its architecture performance. MEMS Micro-Scanning LiDAR System. system wasfast implemented in anofFPGA prototype to its and to evaluate its performance. MEMS Micro-Scanning LiDAR System. system prototype to prove its feasibility and to The evaluate its architecture performance.was implemented in an FPGA prototype to prove its feasibility andoftoAutomatic evaluateControl) its performance. © 2019, IFAC (International Federation Hosting by Elsevier Ltd. All rights reserved. Keywords: LiDAR, LiDAR, 1D 1D MEMS MEMS mirror, mirror, perception, perception, automated automated driving, driving, ADAS ADAS Keywords: Keywords: LiDAR, 1D MEMS mirror, perception, automated driving, ADAS Keywords: LiDAR, 1D MEMS mirror, perception, automated driving, ADAS 1. INTRODUCTION INTRODUCTION 1940 1940 Fritz Fritz Oswald Oswald suggested suggested an an electromechanical electromechanical braking braking 1. regulator which got into testing stage [Reichel, 2003, S. 1. INTRODUCTION 1940 Fritz Oswald suggested an electromechanical braking regulator which got into testing stage [Reichel, 2003, S. 1. INTRODUCTION 1940 Fritzway Oswald suggested an electromechanical braking regulator which got intoinitial testing stage S. 49]. The The from the initial idea of [Reichel, the ABS ABS 2003, concept For several decades Advanced Driver-Assistance Systems 49]. way from the idea of the concept For several decades Advanced Driver-Assistance Systems regulator which got into testing stage [Reichel, 2003, S. 49]. The way from the initial idea of the ABS concept For several decades Advanced Driver-Assistance Systems to serial production was quite long. Daimler AG was the (ADAS) get deployed in of ADAS serial production was initial quite long. Daimler AG was the (ADAS) getdecades deployed in the the majority majority of vehicles. vehicles.Systems ADAS to 49]. The way from the idea of the ABS concept For several Advanced Driver-Assistance (ADAS) getAnti-lock deployed Braking in the majority of vehicles. ADAS world’s car which officially introduced serialfirst production was quite long. was the range System (ABS), which was range from from Anti-lock Braking System of (ABS), which was to world’s first car manufacturer, manufacturer, whichDaimler officiallyAG introduced to serial production was quite long. Daimler AG was the (ADAS) getAnti-lock deployed in the majority vehicles. ADAS (ABS), which the second-generation anti-lock braking system in world’s first car manufacturer, which officially introduced introduced in the past e.g. in the European research range from Braking System was the second-generation anti-lock which braking systemintroduced in August August introduced in the past e.g. in the European research world’s first car manufacturer, officially range from Anti-lock Braking System (ABS), which was the second-generation anti-lock braking system in August 1978. From From December December 1978, 1978, customers customers had had the the opportuniopportuniprogram for forinoptimizing optimizing the Road Transport System in 1978. introduced the past the e.g. Road in theTransport EuropeanSystem research program in the second-generation anti-lock braking system in August introduced in the past e.g. in the European research 1978. From December 1978, customers had the opportuniprogram for optimizing the Road Transport System in ty to order the ABS system as accessory [Daimler AG, Europe [Williams, 1988], Brake-Byty to From order December the ABS 1978, systemcustomers as accessory [Daimler AG, Europe (PROMETHEUS) (PROMETHEUS) [Williams, 1988], to to System Brake-By1978. had the opportuniprogram for optimizing the Road Transport in Europe (PROMETHEUS) [Williams, 1988], to Brake-By2008]. The history of the ABS shows that the time of ty to order the ABS system as accessory [Daimler AG, Wire [Fleming, 2001]. ADAS have a long history, the first 2008]. The history of the ABS shows that the time of Wire [Fleming, 2001]. ADAS have a long history, the first ty to order the invention ABSof system as accessory [Daimler AG, Europe (PROMETHEUS) [Williams, 1988], toinvented Brake-By2001]. ADAS have a long history, an useful idea, and implementation takes time 2008]. The history the ABS shows that the time of braking force regulator for automobiles was in Wire [Fleming, the first an useful idea, invention and implementation takes time braking force regulator for automobiles was invented in 2008]. Theidea, history theiteration. ABS shows that the time of Wire [Fleming, 2001]. Karl ADAS have a[Reichel, long was history, the first invention and implementation takes time invented in an anduseful can’t handle inofone one Nowadays an ABS is 1928 by the German Wessel 2003, S. 44]. braking force regulator for automobiles and can’t handle in iteration. Nowadays an ABS is 1928 by force the German Karl Wessel [Reichel, 2003, S. 44]. an useful idea, invention and implementation takes time braking regulator for automobiles was invented in and can’t handle in one iteration. Nowadays an ABS is 1928 by the German Karl Wessel [Reichel, 2003, S. 44]. mandatory in vehicles in Europe according to law and This first invention didn’t go beyond the design stage. In This first invention didn’t go beyond the design stage. In mandatory in vehicles in Europe according to law and can’t handle in oneiniteration. Nowadays anlaw ABS is 1928 by the German Karl go Wessel [Reichel, 2003,stage. S. 44]. invention didn’t beyond the design In and there are ongoing generations like size, performance or romandatory in vehicles Europe according to and This first are ongoing generations like size, performance orand romandatory in vehicles in Europe according to law This first invention didn’t go beyond the design stage. In there bustness. The ADAS the driver there are ongoing like size, or robustness. The first firstgenerations ADAS supported supported theperformance driver but but didn’t didn’t there are ongoing generations like size, or bustness. Theof ADAS supported the driver but didn’t take control control offirst the car. The The driver is performance responsible forrostake the car. driver is responsible for sbustness. Theoffirst ADAS supported the driverperception. but didn’t teering, braking, acceleration and environment take control the car. The driver is responsible for steering, braking, acceleration and environment perception. take control of the car. Theand driver is responsible for sThese responsibilities fall level zero teering, braking, acceleration environment perception. These responsibilities fall into into level zero of of automation, automation, teering, braking, acceleration and environment perception. as defined in Automotive J3016 These responsibilities fallof into level zeroEngineers of automation, as defined in the the Society Society Automotive Engineers J3016 These responsibilities fallof level zero ofNow, automation, as defined in standard the Society ofinto Automotive Engineers J3016 (SAE J3016) standard [SAE, January 2014]. decades (SAE J3016) [SAE, January 2014]. Now, decades as defined in the Society of Automotive Engineers J3016 after the first ADAS, the PRogrammable sYSTems for (SAE J3016) standard [SAE, January 2014]. Now, decades after the first standard ADAS, the PRogrammable sYSTems for ININ(SAE J3016) [SAE, January 2014]. Now, decades telligence in automobilEs (PRYSTINE) project envisions after the first ADAS, the PRogrammable sYSTems for INtelligence in automobilEs (PRYSTINE) project envisions after the first ADAS, the PRogrammable sYSTems forperINthe of Urban Surround telligence in automobilEs (PRYSTINE) project envisions the development development of Fail-operational Fail-operational Urban Surround pertelligence in automobilEs (PRYSTINE) project envisions the development of Fail-operational Urban Surround perceptION (FUSION). (FUSION). The The overall overall objective objective and and challenge challenge ceptION the development of realize Fail-operational Urban Surround perof this project is to highly automated driving. The ceptION (FUSION). The overall objective and challenge of this project is to realize highly automated driving. The ceptION (FUSION). The overall objective and challenge SAE Level-2 (Partial automation) should be increased to of this project is to realize highly automated driving. The SAE Level-2 (Partial automation) should be increased to of thisLevel-3 project(Conditional is to realize highly automated driving. The automation) andbeabove above [Druml SAE Level-2 (Partial automation) should increased to SAE Level-3 (Conditional automation) and [Druml SAE Level-2 (Partial automation) should be increased SAE automation) andconcept above [Druml et al., al.,Level-3 2018a]. (Conditional Figure 11 shows shows PRYSTINE’s concept view to of et 2018a]. Figure PRYSTINE’s view of SAE (Conditional automation) andconcept aboveadaption [Druml sensor FUSION. Our focuses on the angle et al.,Level-3 2018a]. Figure 1paper shows PRYSTINE’s view of sensor FUSION. Our paper focuses on the angle adaption et al., 2018a]. Figure 1paper shows PRYSTINE’s concept view of capability of Light Detection And Ranging sensor FUSION. focuses on the angle (LiDAR) adaption capability of the the Our Light Detection And Ranging (LiDAR) sensor FUSION. Our paper focuses on the angle adaption capability of the Light Detection And Ranging (LiDAR) part. The The possibility possibility to to switch switch between between two two angles angles is is part. capability ofpossibility the Light Detection And Ranging (LiDAR) part. The for to switch driving between two angles is necessary highly automated respectively aunecessary for highly automated driving respectively aupart. The driving. possibility to switch betweenrespectively two anglesThe is tonomous The paper is structured as follows. necessary for highly automated driving auFig. 1. 1. PRYSTINE’s PRYSTINE’s concept concept view view of of a a Fail-operational Fail-operational tonomous driving. Theautomated paper is structured as follows. The Fig. necessary for highly driving respectively auoverview on related work of the used MEMS-based driving. The paper structured follows. The Fig. Urban 1. PRYSTINE’s concept view of a Fail-operational Surround (FUSION) [Druml overview on related work of is the used 1D 1Das MEMS-based Surround perceptION perceptION (FUSION) [Druml et et al., al., tonomous tonomous driving. The paper is structured follows. Fig. Urban 1. PRYSTINE’s concept view of a Fail-operational overview on related work of the used 1DasMEMS-based automotive LiDAR system is given in Section 2. The The The sys2018a]. Urban Surround perceptION (FUSION) [Druml et al., automotive LiDAR system is given in Section 2. sys2018a]. overview onLiDAR relatedsystem work of the used 1D MEMS-based Urban Surround perceptION (FUSION) [Druml et al., automotive is given in Section 2. The sys2018a]. automotive LiDAR system is given in Section 2. The sys2018a].
2405-8963 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2019 194 Copyright © under 2019 IFAC IFAC 194 Control. Peer review responsibility of International Federation of Automatic Copyright © 2019 IFAC 194 10.1016/j.ifacol.2019.11.649 Copyright © 2019 IFAC 194
2019 IFAC MECHATRONICS 56 Vienna, Austria, Sept. 4-6, 2019
Emitter Path
Laser Illumination Emitted Signal
Trigger and Laser Power Setting
MEMS Mirror Actuation
Sensing
(
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
MEMS Driver ASIC
Receiver Path ! "#
Photo Diodes
Receiver Circuits
dt
Reflected Signal
System Safety Controller (AURIX)
Point Cloud Data
Config and Status
FPGA / Dedicated LiDAR Hardware Accelerators
$$" % &'
Raw Data
Trigger and Gain Setting
Fig. 4. Essential MEMS mirror signals of the MEMS Driver ASIC [Druml et al., 2018b].
Fig. 2. System concept of a 1D MEMS-based automotive LiDAR system by Druml [Druml et al., 2018b]. tem architecture, which is enabling angle adaption, will be described in detail in Section 3 and the results including a short discussion will be provided in Section 4. A summary of the findings will conclude this paper in Section 5. 2. RELATED WORK The LiDAR systems that are currently available on the market, are expensive and bulky like the Velodyne HDL64E LiDAR [Velodyne LiDAR, 2016]. Due to this both industry and academia strive to investigate LiDAR systems using MEMS to minimizing size, making them shock resistant and low-cost [Tsuchiya, 2017] for mass-production and easy integration in automobile system concepts. At the present time two different approaches are pursued, the 1D and 2D solution [Holmstrm et al., 2014, Schenk et al., 2000]. Druml et al. are applying a 1D MEMS mirror because they are more robust in contrast to the 2D MEMS mirror. Since the system should be deployed in automotive applications it is required that the system is robust against shocks and vibration [Druml et al., 2018b]. The mirror of Druml’s LiDAR system is based on the patent of Krastev et al. [Krastev et al., 2013] which is describing an innovative 1D MEMS mirror.
Fig. 3. 1D micro-scanning LiDAR functional principle [Druml et al., 2018b]. 195
2.1 1D MEMS Micro-Scanning LiDAR In this Section, the 1D MEMS Micro-Scanning LiDAR concept by Druml is presented. Figure 2 depicted the 1D MEMS-based automotive LiDAR system by Druml. This chipset is consisting of an oscillating 1D MEMS mirror, a MEMS Driver ASIC for the mirror, a System Safety Controller (AURIX) [Infineon Technologies AG, 2018a], a laser illumination part, a Receiver Circuit and an array of photo diodes [Druml et al., 2018b]. To adapt the angle of the system the MEMS Driver needs to perform control adaptations. The MEMS Driver is responsible for actuating, sensing and controlling the movement of the MEMS mirror [Druml et al., 2018b]. In Figure 3, a 1D microscanning LiDAR illumination of a scenery is depicted. In this concept, laser pulses are stretched into a vertical line by the 1D MEMS mirror, which scans horizontally over the Field-of-View (FoV). The reflected light from obstacles is captured by a stationary detector [Infineon Technologies AG, 2018b]. Crucial signals of the MEMS mirror are provided by the MEMS Driver to be able to monitor the current state of the mirror and check for safe operation. Signals such as POSITION L (if the signal is high the mirror is on the left side, else on the right side) and DIRECTION L (if the signal is high the mirror is moving towards left, else towards right), which delivers precise information about the mirror’s current position. After a zero crossing occurs the position signal changes its level and after the maximum deflection reaches the level of the direction signal changes. The PHASE CLK signal provides accurate and high-frequent phase information of the current mirror position by counting in equi-temporal steps from 0 to nmax during one mirror swing [Druml et al., 2018b] as illustrated in Figure 4. The deflection angle of the MEMS mirror follows the actuation frequency. In principle it is possible to operate the MEMS mirror in two different modes, either the open control-loop or the closed control-loop, cf. [Borovic et al., 2005]. In the open control-loop mode the mirror is actuated by a settled frequency without any control strategies considered. Otherwise the control-loop mode can be used in a phase-locked loop (PLL) or in an amplitude-control loop. When the PLL is activated the system follows the
2019 IFAC MECHATRONICS Vienna, Austria, Sept. 4-6, 2019
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
Analog
57
Digital Angle Signaling
Angle Detector
Angle Error
MEMS Mirror PI Control
HV-Off Value
HV On/Off
Fig. 5. Druml’s MEMS mirror response curve [Druml et al., 2018b]. movement (phase) of the oscillating MEMS mirror precisely. When the MEMS mirror is operated in amplitudecontrol loop it is ensured that the maximum deflection angle of the MEMS mirror stays constant [Druml et al., 2018b]. The MEMS Mirror behaves like a non-linear harmonic oscillator, as seen in Figure 5. By changing the frequency the MEMS mirror’s operation point moves on the resonance curve. The MEMS mirror’s operation point is initially located on the lower resonance curve. On the lower resonance curve, the MEMS mirror is operated in open-control loop mode. In open-control loop mode, the frequency will be varied until the jump frequency fjump is reached. At this point the MEMS mirror jumps onto the top resonance curve. The jump frequency is found by decreasing the start frequency. If fjump is reached a jump in the phase relation and mirror’s deviation angle is observable. After the jump to the upper resonance curve happened, the PLL mode will be activated. The mirror is able to be operated among the two limits. The limits are the points where the mirror jumps at fjump and where the mirror is falling back at the fall-back frequency ff b . The jump and fall-back frequencies vary from production sample to sample due to manufacturing variability. Wide FoV
Narrow FoV
Fig. 6. Icon of a car’s LiDAR system which is changing the FoV permanently. 196
Mirror Subtiming
Fig. 7. Simplified State-of-the-Art amplitude control architecture. 2.2 State-of-the-Art Angle Adaption Procedure To adapt the angle during runtime is a sensitive topic. While the angle is adapted, the LiDAR’s FoV of the point cloud also changes, which needs to be taken into account from a safety point of view. Only if the point stability criterion is fulfilled safety can be guaranteed for the whole system. The laser isn’t allowed to shoot during the adaption step. It is critical for safety that the maximum angle is always stable. Thus, changing the angle as fast as possible is crucial. The State-of-the-Art (SoA) solution is able to adapt the mirror’s angle during runtime though the speed of the adaption is not yet satisfying. A possibility to change the mirror’s peak deviation angle and consequently the adjustment of FoV during runtime is symbolically indicated in Figure 6. The applied architecture is depicted in Figure 7. This architecture basically consists of an Angle Detector part, a PI Control unit and also the Mirror Subtiming block where the phase counter is essential for the HV-Off value comparison. The change of the FoV is divided into two major steps: (1) Set Desired Angle In principle the State-of-the-Art amplitude-control operates like the State-of-the-Art branch indicates in Figure 10 below. At first the angle should be changed. In the SoA procedure there has to be an amplitude setpoint. The amplitude setpoint is a function to affect the change of the mirror’s frequency. Changing the setpoint affects the oscillating frequency of the MEMS mirror. (2) Angle Amplitude Control (a) Angle Detector After setting this point, the internal Angle Detection block is detecting and evaluating the mismatch of the desired amplitude and the actual amplitude of the MEMS mirror and propagates the angle error. This error is forwarded to the PI Control block. (b) PI Control In this part, the HV-Off value will be derived from the angle error by summing up the calculated proportional and integrator values from the
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
PLL Error [Clocks]
PLL Error [Clocks]
2019 IFAC MECHATRONICS 58 Vienna, Austria, Sept. 4-6, 2019
500
0
-500 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
500
0
-500 0
10000
1000
2000
3000
4000
6000 5000 4000 1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
HV Phase Counter [1]
HV Phase Counter [1]
7000
0
6000
7000
8000
9000
10000
7000
8000
9000
10000
7000
8000
9000
10000
7000 6000 5000 4000 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1] Frequency [Hz]
Mirror Half Periods [1] Frequency [Hz]
5000
Mirror Half Periods [1]
Mirror Half Periods [1]
4700 4600 4500
4700 4600 4500 4400
4400 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
Mirror Half Periods [1]
Fig. 8. Measurement results of State-of-the-Art Angle Adaption from small angle to large angle. PI Control block. The resulted HV-Off value is forwarded to the Mirror Subtiming unit. (c) Mirror Subtiming In the Mirror Subtiming unit, the HV-Off value is set. This value is used to disable the HV at a certain timestep. The specific timesteps for turning the HV on and off are directly influencing the operation angle. For this reason, changing the HV-Off value also implies a change of the MEMS mirror’s angle. The duration of FoV changing behaves differently. Switching from a wider FoV to a narrower FoV takes longer than the other way around. The measurement results of an angle increase via the SoA concept is depicted in Figure 8. The required time to achieve the desired maximum angle, respective the frequency, is approximately 692 ms. The other case, limiting the FoV from wide to narrow is slower. This behavior is caused by the different amount of energy that is necessary between the acceleration and the deceleration of the MEMS mirror. The results of the SoA angle decreasing measurement are shown in Figure 9. The time until the frequency has settled at the adjusted frequency amounts to approximately 714 ms. The difference between narrow to wide FoV and wide to narrow FoV is about 22 ms. If the control parameters will be optimized the needed time could be reduced. In Figures 8 and 9, the HV Phase Counter and PLL Error signals are included because the HV Phase Counter signal shows that the HV-Off value is permanently adjusted by the PI Control unit and slowly approaches the required HV-Off timestep. The PLL Error signal shows a similar behavior. It is depicted that PLL mode is also active in the amplitude-control mode. These two control modes work competitively.
Fig. 9. Measurement results of State-of-the-Art Angle Adaption from large angle to small angle. we appended an additional block to our architecture which enabled us to set the HV-Off value directly. The functional overview is seen in Figure 10 which the flow is described. The change of the FoV with the novel procedure is also divided into two major steps: (1) Set Desired Angle The novel procedure is able to operate in two different modes. The first possible mode is already described in Section 2.2. The second and more interesting operation mode is shown in the novel branch of Figure 10. In the beginning the angle should be changed. To Change MEMS Mirror Angle Novel Activated
Fixing Amplitude Control
State-of-the-Art Activated
Angle Error greater than Treshold
Set Amplitude Setpoint
Set HV-Off Value
PI Control
PI Control
Angle Error smaller than Threshold
Set HV On/Off
Accelerate or Decelerate MEMS Mirror State-of-the-Art Novel
3. FAST ANGLE ADAPTION OF MEMS-BASED LIDAR SYSTEMS
Unfixing Amplitude Control
In this Section, we introduce our novel approach to speedup the angle adaption of the MEMS mirror which is depicted in Figure 11. In contrast to the SoA approach 197
Fig. 10. Functional overview how an MEMS mirror’s maximum angle is able to be adapted.
2019 IFAC MECHATRONICS Vienna, Austria, Sept. 4-6, 2019
PLL Error [Clocks]
Digital Angle Signaling
Angle Detector
Angle Error
Fast Angle Adaption
0
-500 1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
7000
8000
9000
10000
7000
8000
9000
10000
Mirror Half Periods [1]
Pre-Defined HV-Off Value
MEMS Mirror
59
500
0
Fixing Angle Adaption Control
HV Phase Counter [1]
Analog
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
PI Control
6000
5000
4000 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
HV On/Off
Frequency [Hz]
HV-Off Value
Mirror Subtiming
4700 4600 4500 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
Fig. 11. Simplified Novel Angle Adaption architecture.
Fig. 12. Measurement results of novel Angle Adaption from small angle to large angle. 4. RESULTS
At the end of the novel procedure we are reactivating the angle amplitude-control to enable a settlement of the controller. Nevertheless, the system is the whole time still in a safe-state but the activation of the angle amplitudecontrol also guarantees point stability. 198
In this section we provide the measurement results of our novel fast angle adaption procedure which has been introduced in Section 3. Figure 12 shows the measurements of a fast angle increment. It is clearly visible where the HV-Off value is adapted. At the moment where the HV-Off counter is changed instantly, the PLL error also spikes. Subsequently the PLL started its operation and adjusted the actuation frequency. After some time the PLL error is approximately zero and the correct frequency is achieved. In case of changing the angle from small to large the required time is near 52 ms.
PLL Error [Clocks]
The second measurement exhibits a fast angle minimization in Figure 13. Similar to fast increasing, the HV-Off value is set immediately to a specific value. The PLL error is increasing. Other than the angle enlargement, where the MEMS mirror has to be sped up, the MEMS mirror has to slow down for angle reduction. The deceleration needs more time to settle and adjust the proper frequency until 500
0
-500 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
7000
8000
9000
10000
7000
8000
9000
10000
HV Phase Counter [1]
Mirror Half Periods [1] 8000 7000 6000 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1] Frequency [Hz]
enable the novel procedure the amplitude-control has to be fixed that it can’t intervene. (2) Angle Amplitude Control (a) Fast Angle Adaption The Fast Angle Adaption block is fixing the Angle Detector block that the angle error doesn’t jam the predefined HV-Off value of the Fast Angle Adaption. Then the HV-Off value is forwarded to the PI Control unit. (b) PI Control Due to the amplitude-control being fixed, the angle error can’t be forwarded to the PI Control block any longer. Thus, the integrator takes the HV-Off value from the Fast Angle Adaption unit. Furthermore the PI Control hasn’t to adapt the integrator according to an angle error. For this reason the integrator value is taken for the HV-Off Value which is forwarded to the Mirror Subtiming block. (c) Mirror Subtiming This part provides the same functionality as the SoA method. The HV-Off value is set in the Subtiming Mirror unit. The specific HV-Off value disables the HV at a certain timestep and thus influences directly the maximum operation angle. (d) Angle Detector Right at the start of the novel procedure the Angle Detector part is fixed. When the mirror is accelerated or decelerated the Angle Detector part is after a specific time period unfixed afresh. The internal Angle Detection block continues to detect and evaluate the mismatch of the desired amplitude and the actual amplitude of the MEMS mirror. The Angle Detector has to be unfixed to enable settling of the controller.
4600 4500 4400 4300 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
Fig. 13. Measurement results of novel Angle Adaption from large angle to small angle.
2019 IFAC MECHATRONICS 60 Vienna, Austria, Sept. 4-6, 2019
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
REFERENCES
Table 1. Measurement results comparison
State-of-the-Art Procedure narrow FoV → wide FoV wide FoV → narrow FoV Novel Procedure narrow FoV → wide FoV wide FoV → narrow FoV
Begin
End
Time in ms
1180 545
7819 7396
692 714
446 472
943 1168
52 72
the PLL error is nearly zero. The duration among HV-Off value adaption and frequency adjustment is about 72 ms. Finally the results of the novel procedure and the SoA procedure are compared with each other. Table 1 shows clear evidence that the novel procedure is considerably faster than the SoA procedure. 5. CONCLUSIONS In our paper, we have introduced a novel angle adaption procedure for 1D MEMS Micro-Scanning LiDAR systems. The SoA procedure requires too much time during the system switches between wide FoV and narrow FoV, because the LiDAR system doesn’t fulfill the point stability criterion, which is a safety requirement, during switching time and this is too risky for automated driving. In Section 2.1 we have described the principally used 1D MEMS Micro-Scanning LiDAR system. For our design we have considered this LiDAR system’s characteristics to conceive and implement our fast angle adaption procedure. The SoA architecture respectively the procedure of the angle adaption procedure during runtime was discussed in Section 2.2, the novel architecture and its procedure was presented in Section 3. Achieved measurement results of the novel procedure and a comparison with the SoA results were demonstrated in the Section 4. In our paper we offer an approach to change the maximum angle from a narrower FoV to a wider FoV in about 52 ms. The other way around, to change the maximum angle from a wider FoV to a narrower FoV, the novel procedure is able to adjust the desired angle in approximately 72 ms. Wide to narrow FoV change is approximately 10 times and narrow to wide FoV change is about 13 times faster with the novel procedure. For highly automated driving applications a fast switching between wide and narrow FoV will be a key enabler for reliable environment perception and corresponding intervention during operation. Thus, a safe driving state for passengers and road participants is ensured by the usage of the novel architecture. ACKNOWLEDGEMENTS The authors would like to thank all national funding authorities and the ECSEL Joint Undertaking, which funded the PRYSTINE project under the grant agreement number 783190. PRYSTINE is funded by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT) under the program “ICT of the Future” between May 2018 and April 2021 (grant number 865310). More information: https://iktderzukunft.at/en/. 199
Borovic, B., Liu, A.Q., Popa, D., Cai, H., and Lewis, F.L. (2005). Open-loop versus closed-loop control of MEMS devices: choices and issues. Journal of Micromechanics and Microengineering, 15(10), 1917. URL http:// stacks.iop.org/0960-1317/15/i=10/a=018. Daimler AG (2008). Mercedes-Benz und die Erfindung des Anti-Blockier-Systems: 1978 ist ABS serienreif. https://media.daimler.com/marsMediaSite/ de/instance/ko.xhtml?oid=9913502. Accessed: 27.11.2018. Druml, N., Macher, G., Stolz, M., Armengaud, E., Watzenig, D., Steger, C., Herndl, T., Eckel, A., Ryabokon, A., Hoess, A., Kumar, S., Dimitrakopoulos, G., and Roedig, H. (2018a). PRYSTINE - PRogrammable sYSTems for INtelligence in AutomobilEs. In 2018 21st Euromicro Conference on Digital System Design (DSD), 618–626. doi:10.1109/DSD.2018.00107. Druml, N., Maksymova, I., Thurner, T., Van Lierop, D., Hennecke, M., and Foroutan, A. (2018b). 1D MEMS Micro-Scanning LiDAR. In The Ninth International Conference on Sensor Device Technologies and Applications (SENSORDEVICES 2018). Fleming, W.J. (2001). Overview of Automotive Sensors. IEEE Sensors Journal, 1(4), 296–308. doi:10.1109/7361. 983469. Holmstrm, S.T.S., Baran, U., and Urey, H. (2014). MEMS Laser Scanners: A Review. Journal of Microelectromechanical Systems, 23(2), 259–275. doi:10.1109/JMEMS. 2013.2295470. Infineon Technologies AG (2018a). AURIX System Safety Controller. https://iot-automotive.news/ aurix-microcontroller-tc3xx-family-infineonfuels-automated-driving-electromobility. Infineon Technologies AG (2018b). Lidar Sensors. https://www.infineon.com/cms/en/product/ sensor/lidar-sensors/. Krastev, K.T., Van Lierop, H.W., Soemers, H.M., Sanders, R.H.M., and Nellissen, A.J.M. (2013). MEMS scanning micromirror. US Patent 8,526,089. Reichel, H. (2003). Elektronische Bremssysteme: vom ABS zum Brake-by-Wire. Reihe Technik. expert-Verlag. SAE (January 2014). SAE International Standard J3016 - Taxonomy and Definitions for Terms Related to OnRoad Motor Vehicle Automated Driving Systems. Standard, SAE International. Schenk, H., Duerr, P., Kunze, D., Lakner, H.K., and Kueck, H. (2000). Design and modeling of large deflection micromechanical 1D and 2D scanning mirrors. doi: 10.1117/12.396479. URL https://doi.org/10.1117/ 12.396479. Tsuchiya, T. (2017). MEMS mirrors for automotive applications. In 2017 IEEE International Meeting for Future of Electron Devices, Kansai (IMFEDK), 82–83. doi:10.1109/IMFEDK.2017.7998053. Velodyne LiDAR (2016). HDL-64E. Williams, M. (1988). PROMETHEUS-The European research programme for optimising the road transport system in Europe. In IEE Colloquium on Driver Information, 1/1–1/9.
ScienceDirect
IFAC PapersOnLine 52-15 (2019) 55–60
Fast Fast Fast
Angle Angle Angle
Philipp Philipp Philipp Philipp
Adaption Adaption of of a a MEMS-based MEMS-based LiDAR System Adaption of a MEMS-based LiDAR System LiDAR LiDAR System System ∗ ∗ ∗ ∗ ∗ ∗
Stelzer Steger Stelzer ∗∗∗ Andreas Andreas∗∗Strasser Strasser ∗∗∗ Christian Christian Steger ∗∗∗ ∗∗ ∗∗ ∗∗ Alberto Garcia Norbert Druml Stelzer Andreas Strasser Christian ∗∗ ∗∗ ∗ Garcia ∗∗ Norbert ∗ Druml ∗∗ Steger ∗ Alberto Stelzer Strasser Christian ∗∗ Steger AlbertoAndreas Garcia ∗∗ ∗∗ Norbert Druml ∗∗ Alberto Garcia Norbert Druml ∗ ∗ University of Technology, Graz, Austria, ∗ Graz ∗ Graz University of Technology, Graz, Austria, ∗ Graz University Technology, Graz, Austria, (e-mail: {stelzer, of strasser, steger}@tugraz.at). {stelzer, strasser, steger}@tugraz.at). ∗(e-mail: Graz University of Technology, Graz, Austria, ∗∗ ∗∗ Technologies Austria AG, (e-mail: {stelzer, strasser, steger}@tugraz.at). ∗∗ Infineon Technologies Austria AG, Graz, Graz, Austria, Austria, ∗∗ Infineon (e-mail: {stelzer, strasser, steger}@tugraz.at). ∗∗ (e-mail: {alberto.garcia, norbert.druml}@infineon.com). Infineon Technologies Austria AG, Graz, Austria, ∗∗ (e-mail: {alberto.garcia, norbert.druml}@infineon.com). Infineon Technologiesnorbert.druml}@infineon.com). Austria AG, Graz, Austria, (e-mail: {alberto.garcia, (e-mail: {alberto.garcia, norbert.druml}@infineon.com). Abstract: The The amount amount of of Advanced Advanced Driver-Assistance Driver-Assistance Systems Systems (ADAS) (ADAS) in in middle-class middle-class cars cars Abstract: are increasing steadily. Next generation ADAS will take control in fully automated vehicles. Abstract: The amount of Advanced Driver-Assistance Systems (ADAS) in middle-class cars are increasing Next generation ADAS will takeSystems control (ADAS) in fully automated vehicles. Abstract: Thesteadily. amount of Advanced Driver-Assistance in highly middle-class cars are increasing steadily. Next generation ADAS will of take in fullyfor automated vehicles. Light Detection And Ranging Ranging (LiDAR) will be one one of thecontrol key enablers enablers for automated Light Detection And (LiDAR) will be the key highly automated are increasing steadily. Next generation ADAS will take control in fully automated vehicles. or even autonomous vehicles. The 1D Micro-Electro-Mechanical System (MEMS) MicroLight Detection And Ranging (LiDAR) will be one of the key enablers for highly automated The 1D will Micro-Electro-Mechanical System (MEMS) or even autonomous vehicles.(LiDAR) MicroLight Detection Ranging one of the Field-of-View key enablers for highly automated Scanning LiDARAnd is able able to detect detectThe obstacles inbe a predefined predefined (FoV). Future ADAS or even autonomous vehicles. 1D Micro-Electro-Mechanical System (MEMS) MicroScanning LiDAR is to obstacles in a Field-of-View (FoV). Future ADAS or even autonomous vehicles. The 1Dchange Micro-Electro-Mechanical System (MEMS) Microapplications will require dynamically FoVs as fast as possible (e.g. quickly switching Scanning LiDAR is able to detect obstacles in a predefined Field-of-View (FoV). Future ADAS to detect dynamically change as fastField-of-View as possible (e.g. quickly switching applications will require Scanning LiDAR is able to obstacles in a FoVs predefined (FoV). Future ADAS applications will require to dynamically change FoVs fast as possible (e.g. quickly switching between long-range long-range narrow FoV and and short-range wideas FoV). between narrow FoV short-range wide FoV). applications will require to dynamically change FoVs as fast as possible (e.g. quickly switching In this paper we introduce a novel system architecture that enables fast angle adaption of between long-range narrow FoV and short-range wide FoV). 1D In this paper we introduce aFoV novel system architecture that enables fast angle adaption of aa 1D between long-range narrow and short-range wide FoV). MEMS Micro-Scanning LiDAR System. The system architecture wasfast implemented in an anofFPGA FPGA In this paper we introduce a novel system architecture that enables angle adaption a 1D MEMS Micro-Scanning LiDAR System. The system architecture was implemented in In this paper we introduce a novel system architecture that enables angle adaption a 1D prototype to prove prove its feasibility feasibility and to The evaluate its architecture performance. MEMS Micro-Scanning LiDAR System. system wasfast implemented in anofFPGA prototype to its and to evaluate its performance. MEMS Micro-Scanning LiDAR System. system prototype to prove its feasibility and to The evaluate its architecture performance.was implemented in an FPGA prototype to prove its feasibility andoftoAutomatic evaluateControl) its performance. © 2019, IFAC (International Federation Hosting by Elsevier Ltd. All rights reserved. Keywords: LiDAR, LiDAR, 1D 1D MEMS MEMS mirror, mirror, perception, perception, automated automated driving, driving, ADAS ADAS Keywords: Keywords: LiDAR, 1D MEMS mirror, perception, automated driving, ADAS Keywords: LiDAR, 1D MEMS mirror, perception, automated driving, ADAS 1. INTRODUCTION INTRODUCTION 1940 1940 Fritz Fritz Oswald Oswald suggested suggested an an electromechanical electromechanical braking braking 1. regulator which got into testing stage [Reichel, 2003, S. 1. INTRODUCTION 1940 Fritz Oswald suggested an electromechanical braking regulator which got into testing stage [Reichel, 2003, S. 1. INTRODUCTION 1940 Fritzway Oswald suggested an electromechanical braking regulator which got intoinitial testing stage S. 49]. The The from the initial idea of [Reichel, the ABS ABS 2003, concept For several decades Advanced Driver-Assistance Systems 49]. way from the idea of the concept For several decades Advanced Driver-Assistance Systems regulator which got into testing stage [Reichel, 2003, S. 49]. The way from the initial idea of the ABS concept For several decades Advanced Driver-Assistance Systems to serial production was quite long. Daimler AG was the (ADAS) get deployed in of ADAS serial production was initial quite long. Daimler AG was the (ADAS) getdecades deployed in the the majority majority of vehicles. vehicles.Systems ADAS to 49]. The way from the idea of the ABS concept For several Advanced Driver-Assistance (ADAS) getAnti-lock deployed Braking in the majority of vehicles. ADAS world’s car which officially introduced serialfirst production was quite long. was the range System (ABS), which was range from from Anti-lock Braking System of (ABS), which was to world’s first car manufacturer, manufacturer, whichDaimler officiallyAG introduced to serial production was quite long. Daimler AG was the (ADAS) getAnti-lock deployed in the majority vehicles. ADAS (ABS), which the second-generation anti-lock braking system in world’s first car manufacturer, which officially introduced introduced in the past e.g. in the European research range from Braking System was the second-generation anti-lock which braking systemintroduced in August August introduced in the past e.g. in the European research world’s first car manufacturer, officially range from Anti-lock Braking System (ABS), which was the second-generation anti-lock braking system in August 1978. From From December December 1978, 1978, customers customers had had the the opportuniopportuniprogram for forinoptimizing optimizing the Road Transport System in 1978. introduced the past the e.g. Road in theTransport EuropeanSystem research program in the second-generation anti-lock braking system in August introduced in the past e.g. in the European research 1978. From December 1978, customers had the opportuniprogram for optimizing the Road Transport System in ty to order the ABS system as accessory [Daimler AG, Europe [Williams, 1988], Brake-Byty to From order December the ABS 1978, systemcustomers as accessory [Daimler AG, Europe (PROMETHEUS) (PROMETHEUS) [Williams, 1988], to to System Brake-By1978. had the opportuniprogram for optimizing the Road Transport in Europe (PROMETHEUS) [Williams, 1988], to Brake-By2008]. The history of the ABS shows that the time of ty to order the ABS system as accessory [Daimler AG, Wire [Fleming, 2001]. ADAS have a long history, the first 2008]. The history of the ABS shows that the time of Wire [Fleming, 2001]. ADAS have a long history, the first ty to order the invention ABSof system as accessory [Daimler AG, Europe (PROMETHEUS) [Williams, 1988], toinvented Brake-By2001]. ADAS have a long history, an useful idea, and implementation takes time 2008]. The history the ABS shows that the time of braking force regulator for automobiles was in Wire [Fleming, the first an useful idea, invention and implementation takes time braking force regulator for automobiles was invented in 2008]. Theidea, history theiteration. ABS shows that the time of Wire [Fleming, 2001]. Karl ADAS have a[Reichel, long was history, the first invention and implementation takes time invented in an anduseful can’t handle inofone one Nowadays an ABS is 1928 by the German Wessel 2003, S. 44]. braking force regulator for automobiles and can’t handle in iteration. Nowadays an ABS is 1928 by force the German Karl Wessel [Reichel, 2003, S. 44]. an useful idea, invention and implementation takes time braking regulator for automobiles was invented in and can’t handle in one iteration. Nowadays an ABS is 1928 by the German Karl Wessel [Reichel, 2003, S. 44]. mandatory in vehicles in Europe according to law and This first invention didn’t go beyond the design stage. In This first invention didn’t go beyond the design stage. In mandatory in vehicles in Europe according to law and can’t handle in oneiniteration. Nowadays anlaw ABS is 1928 by the German Karl go Wessel [Reichel, 2003,stage. S. 44]. invention didn’t beyond the design In and there are ongoing generations like size, performance or romandatory in vehicles Europe according to and This first are ongoing generations like size, performance orand romandatory in vehicles in Europe according to law This first invention didn’t go beyond the design stage. In there bustness. The ADAS the driver there are ongoing like size, or robustness. The first firstgenerations ADAS supported supported theperformance driver but but didn’t didn’t there are ongoing generations like size, or bustness. Theof ADAS supported the driver but didn’t take control control offirst the car. The The driver is performance responsible forrostake the car. driver is responsible for sbustness. Theoffirst ADAS supported the driverperception. but didn’t teering, braking, acceleration and environment take control the car. The driver is responsible for steering, braking, acceleration and environment perception. take control of the car. Theand driver is responsible for sThese responsibilities fall level zero teering, braking, acceleration environment perception. These responsibilities fall into into level zero of of automation, automation, teering, braking, acceleration and environment perception. as defined in Automotive J3016 These responsibilities fallof into level zeroEngineers of automation, as defined in the the Society Society Automotive Engineers J3016 These responsibilities fallof level zero ofNow, automation, as defined in standard the Society ofinto Automotive Engineers J3016 (SAE J3016) standard [SAE, January 2014]. decades (SAE J3016) [SAE, January 2014]. Now, decades as defined in the Society of Automotive Engineers J3016 after the first ADAS, the PRogrammable sYSTems for (SAE J3016) standard [SAE, January 2014]. Now, decades after the first standard ADAS, the PRogrammable sYSTems for ININ(SAE J3016) [SAE, January 2014]. Now, decades telligence in automobilEs (PRYSTINE) project envisions after the first ADAS, the PRogrammable sYSTems for INtelligence in automobilEs (PRYSTINE) project envisions after the first ADAS, the PRogrammable sYSTems forperINthe of Urban Surround telligence in automobilEs (PRYSTINE) project envisions the development development of Fail-operational Fail-operational Urban Surround pertelligence in automobilEs (PRYSTINE) project envisions the development of Fail-operational Urban Surround perceptION (FUSION). (FUSION). The The overall overall objective objective and and challenge challenge ceptION the development of realize Fail-operational Urban Surround perof this project is to highly automated driving. The ceptION (FUSION). The overall objective and challenge of this project is to realize highly automated driving. The ceptION (FUSION). The overall objective and challenge SAE Level-2 (Partial automation) should be increased to of this project is to realize highly automated driving. The SAE Level-2 (Partial automation) should be increased to of thisLevel-3 project(Conditional is to realize highly automated driving. The automation) andbeabove above [Druml SAE Level-2 (Partial automation) should increased to SAE Level-3 (Conditional automation) and [Druml SAE Level-2 (Partial automation) should be increased SAE automation) andconcept above [Druml et al., al.,Level-3 2018a]. (Conditional Figure 11 shows shows PRYSTINE’s concept view to of et 2018a]. Figure PRYSTINE’s view of SAE (Conditional automation) andconcept aboveadaption [Druml sensor FUSION. Our focuses on the angle et al.,Level-3 2018a]. Figure 1paper shows PRYSTINE’s view of sensor FUSION. Our paper focuses on the angle adaption et al., 2018a]. Figure 1paper shows PRYSTINE’s concept view of capability of Light Detection And Ranging sensor FUSION. focuses on the angle (LiDAR) adaption capability of the the Our Light Detection And Ranging (LiDAR) sensor FUSION. Our paper focuses on the angle adaption capability of the Light Detection And Ranging (LiDAR) part. The The possibility possibility to to switch switch between between two two angles angles is is part. capability ofpossibility the Light Detection And Ranging (LiDAR) part. The for to switch driving between two angles is necessary highly automated respectively aunecessary for highly automated driving respectively aupart. The driving. possibility to switch betweenrespectively two anglesThe is tonomous The paper is structured as follows. necessary for highly automated driving auFig. 1. 1. PRYSTINE’s PRYSTINE’s concept concept view view of of a a Fail-operational Fail-operational tonomous driving. Theautomated paper is structured as follows. The Fig. necessary for highly driving respectively auoverview on related work of the used MEMS-based driving. The paper structured follows. The Fig. Urban 1. PRYSTINE’s concept view of a Fail-operational Surround (FUSION) [Druml overview on related work of is the used 1D 1Das MEMS-based Surround perceptION perceptION (FUSION) [Druml et et al., al., tonomous tonomous driving. The paper is structured follows. Fig. Urban 1. PRYSTINE’s concept view of a Fail-operational overview on related work of the used 1DasMEMS-based automotive LiDAR system is given in Section 2. The The The sys2018a]. Urban Surround perceptION (FUSION) [Druml et al., automotive LiDAR system is given in Section 2. sys2018a]. overview onLiDAR relatedsystem work of the used 1D MEMS-based Urban Surround perceptION (FUSION) [Druml et al., automotive is given in Section 2. The sys2018a]. automotive LiDAR system is given in Section 2. The sys2018a].
2405-8963 © 2019, IFAC (International Federation of Automatic Control) Hosting by Elsevier Ltd. All rights reserved. Copyright © 2019 194 Copyright © under 2019 IFAC IFAC 194 Control. Peer review responsibility of International Federation of Automatic Copyright © 2019 IFAC 194 10.1016/j.ifacol.2019.11.649 Copyright © 2019 IFAC 194
2019 IFAC MECHATRONICS 56 Vienna, Austria, Sept. 4-6, 2019
Emitter Path
Laser Illumination Emitted Signal
Trigger and Laser Power Setting
MEMS Mirror Actuation
Sensing
(
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
MEMS Driver ASIC
Receiver Path ! "#
Photo Diodes
Receiver Circuits
dt
Reflected Signal
System Safety Controller (AURIX)
Point Cloud Data
Config and Status
FPGA / Dedicated LiDAR Hardware Accelerators
$$" % &'
Raw Data
Trigger and Gain Setting
Fig. 4. Essential MEMS mirror signals of the MEMS Driver ASIC [Druml et al., 2018b].
Fig. 2. System concept of a 1D MEMS-based automotive LiDAR system by Druml [Druml et al., 2018b]. tem architecture, which is enabling angle adaption, will be described in detail in Section 3 and the results including a short discussion will be provided in Section 4. A summary of the findings will conclude this paper in Section 5. 2. RELATED WORK The LiDAR systems that are currently available on the market, are expensive and bulky like the Velodyne HDL64E LiDAR [Velodyne LiDAR, 2016]. Due to this both industry and academia strive to investigate LiDAR systems using MEMS to minimizing size, making them shock resistant and low-cost [Tsuchiya, 2017] for mass-production and easy integration in automobile system concepts. At the present time two different approaches are pursued, the 1D and 2D solution [Holmstrm et al., 2014, Schenk et al., 2000]. Druml et al. are applying a 1D MEMS mirror because they are more robust in contrast to the 2D MEMS mirror. Since the system should be deployed in automotive applications it is required that the system is robust against shocks and vibration [Druml et al., 2018b]. The mirror of Druml’s LiDAR system is based on the patent of Krastev et al. [Krastev et al., 2013] which is describing an innovative 1D MEMS mirror.
Fig. 3. 1D micro-scanning LiDAR functional principle [Druml et al., 2018b]. 195
2.1 1D MEMS Micro-Scanning LiDAR In this Section, the 1D MEMS Micro-Scanning LiDAR concept by Druml is presented. Figure 2 depicted the 1D MEMS-based automotive LiDAR system by Druml. This chipset is consisting of an oscillating 1D MEMS mirror, a MEMS Driver ASIC for the mirror, a System Safety Controller (AURIX) [Infineon Technologies AG, 2018a], a laser illumination part, a Receiver Circuit and an array of photo diodes [Druml et al., 2018b]. To adapt the angle of the system the MEMS Driver needs to perform control adaptations. The MEMS Driver is responsible for actuating, sensing and controlling the movement of the MEMS mirror [Druml et al., 2018b]. In Figure 3, a 1D microscanning LiDAR illumination of a scenery is depicted. In this concept, laser pulses are stretched into a vertical line by the 1D MEMS mirror, which scans horizontally over the Field-of-View (FoV). The reflected light from obstacles is captured by a stationary detector [Infineon Technologies AG, 2018b]. Crucial signals of the MEMS mirror are provided by the MEMS Driver to be able to monitor the current state of the mirror and check for safe operation. Signals such as POSITION L (if the signal is high the mirror is on the left side, else on the right side) and DIRECTION L (if the signal is high the mirror is moving towards left, else towards right), which delivers precise information about the mirror’s current position. After a zero crossing occurs the position signal changes its level and after the maximum deflection reaches the level of the direction signal changes. The PHASE CLK signal provides accurate and high-frequent phase information of the current mirror position by counting in equi-temporal steps from 0 to nmax during one mirror swing [Druml et al., 2018b] as illustrated in Figure 4. The deflection angle of the MEMS mirror follows the actuation frequency. In principle it is possible to operate the MEMS mirror in two different modes, either the open control-loop or the closed control-loop, cf. [Borovic et al., 2005]. In the open control-loop mode the mirror is actuated by a settled frequency without any control strategies considered. Otherwise the control-loop mode can be used in a phase-locked loop (PLL) or in an amplitude-control loop. When the PLL is activated the system follows the
2019 IFAC MECHATRONICS Vienna, Austria, Sept. 4-6, 2019
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
Analog
57
Digital Angle Signaling
Angle Detector
Angle Error
MEMS Mirror PI Control
HV-Off Value
HV On/Off
Fig. 5. Druml’s MEMS mirror response curve [Druml et al., 2018b]. movement (phase) of the oscillating MEMS mirror precisely. When the MEMS mirror is operated in amplitudecontrol loop it is ensured that the maximum deflection angle of the MEMS mirror stays constant [Druml et al., 2018b]. The MEMS Mirror behaves like a non-linear harmonic oscillator, as seen in Figure 5. By changing the frequency the MEMS mirror’s operation point moves on the resonance curve. The MEMS mirror’s operation point is initially located on the lower resonance curve. On the lower resonance curve, the MEMS mirror is operated in open-control loop mode. In open-control loop mode, the frequency will be varied until the jump frequency fjump is reached. At this point the MEMS mirror jumps onto the top resonance curve. The jump frequency is found by decreasing the start frequency. If fjump is reached a jump in the phase relation and mirror’s deviation angle is observable. After the jump to the upper resonance curve happened, the PLL mode will be activated. The mirror is able to be operated among the two limits. The limits are the points where the mirror jumps at fjump and where the mirror is falling back at the fall-back frequency ff b . The jump and fall-back frequencies vary from production sample to sample due to manufacturing variability. Wide FoV
Narrow FoV
Fig. 6. Icon of a car’s LiDAR system which is changing the FoV permanently. 196
Mirror Subtiming
Fig. 7. Simplified State-of-the-Art amplitude control architecture. 2.2 State-of-the-Art Angle Adaption Procedure To adapt the angle during runtime is a sensitive topic. While the angle is adapted, the LiDAR’s FoV of the point cloud also changes, which needs to be taken into account from a safety point of view. Only if the point stability criterion is fulfilled safety can be guaranteed for the whole system. The laser isn’t allowed to shoot during the adaption step. It is critical for safety that the maximum angle is always stable. Thus, changing the angle as fast as possible is crucial. The State-of-the-Art (SoA) solution is able to adapt the mirror’s angle during runtime though the speed of the adaption is not yet satisfying. A possibility to change the mirror’s peak deviation angle and consequently the adjustment of FoV during runtime is symbolically indicated in Figure 6. The applied architecture is depicted in Figure 7. This architecture basically consists of an Angle Detector part, a PI Control unit and also the Mirror Subtiming block where the phase counter is essential for the HV-Off value comparison. The change of the FoV is divided into two major steps: (1) Set Desired Angle In principle the State-of-the-Art amplitude-control operates like the State-of-the-Art branch indicates in Figure 10 below. At first the angle should be changed. In the SoA procedure there has to be an amplitude setpoint. The amplitude setpoint is a function to affect the change of the mirror’s frequency. Changing the setpoint affects the oscillating frequency of the MEMS mirror. (2) Angle Amplitude Control (a) Angle Detector After setting this point, the internal Angle Detection block is detecting and evaluating the mismatch of the desired amplitude and the actual amplitude of the MEMS mirror and propagates the angle error. This error is forwarded to the PI Control block. (b) PI Control In this part, the HV-Off value will be derived from the angle error by summing up the calculated proportional and integrator values from the
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
PLL Error [Clocks]
PLL Error [Clocks]
2019 IFAC MECHATRONICS 58 Vienna, Austria, Sept. 4-6, 2019
500
0
-500 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
500
0
-500 0
10000
1000
2000
3000
4000
6000 5000 4000 1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
HV Phase Counter [1]
HV Phase Counter [1]
7000
0
6000
7000
8000
9000
10000
7000
8000
9000
10000
7000
8000
9000
10000
7000 6000 5000 4000 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1] Frequency [Hz]
Mirror Half Periods [1] Frequency [Hz]
5000
Mirror Half Periods [1]
Mirror Half Periods [1]
4700 4600 4500
4700 4600 4500 4400
4400 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
Mirror Half Periods [1]
Fig. 8. Measurement results of State-of-the-Art Angle Adaption from small angle to large angle. PI Control block. The resulted HV-Off value is forwarded to the Mirror Subtiming unit. (c) Mirror Subtiming In the Mirror Subtiming unit, the HV-Off value is set. This value is used to disable the HV at a certain timestep. The specific timesteps for turning the HV on and off are directly influencing the operation angle. For this reason, changing the HV-Off value also implies a change of the MEMS mirror’s angle. The duration of FoV changing behaves differently. Switching from a wider FoV to a narrower FoV takes longer than the other way around. The measurement results of an angle increase via the SoA concept is depicted in Figure 8. The required time to achieve the desired maximum angle, respective the frequency, is approximately 692 ms. The other case, limiting the FoV from wide to narrow is slower. This behavior is caused by the different amount of energy that is necessary between the acceleration and the deceleration of the MEMS mirror. The results of the SoA angle decreasing measurement are shown in Figure 9. The time until the frequency has settled at the adjusted frequency amounts to approximately 714 ms. The difference between narrow to wide FoV and wide to narrow FoV is about 22 ms. If the control parameters will be optimized the needed time could be reduced. In Figures 8 and 9, the HV Phase Counter and PLL Error signals are included because the HV Phase Counter signal shows that the HV-Off value is permanently adjusted by the PI Control unit and slowly approaches the required HV-Off timestep. The PLL Error signal shows a similar behavior. It is depicted that PLL mode is also active in the amplitude-control mode. These two control modes work competitively.
Fig. 9. Measurement results of State-of-the-Art Angle Adaption from large angle to small angle. we appended an additional block to our architecture which enabled us to set the HV-Off value directly. The functional overview is seen in Figure 10 which the flow is described. The change of the FoV with the novel procedure is also divided into two major steps: (1) Set Desired Angle The novel procedure is able to operate in two different modes. The first possible mode is already described in Section 2.2. The second and more interesting operation mode is shown in the novel branch of Figure 10. In the beginning the angle should be changed. To Change MEMS Mirror Angle Novel Activated
Fixing Amplitude Control
State-of-the-Art Activated
Angle Error greater than Treshold
Set Amplitude Setpoint
Set HV-Off Value
PI Control
PI Control
Angle Error smaller than Threshold
Set HV On/Off
Accelerate or Decelerate MEMS Mirror State-of-the-Art Novel
3. FAST ANGLE ADAPTION OF MEMS-BASED LIDAR SYSTEMS
Unfixing Amplitude Control
In this Section, we introduce our novel approach to speedup the angle adaption of the MEMS mirror which is depicted in Figure 11. In contrast to the SoA approach 197
Fig. 10. Functional overview how an MEMS mirror’s maximum angle is able to be adapted.
2019 IFAC MECHATRONICS Vienna, Austria, Sept. 4-6, 2019
PLL Error [Clocks]
Digital Angle Signaling
Angle Detector
Angle Error
Fast Angle Adaption
0
-500 1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
7000
8000
9000
10000
7000
8000
9000
10000
Mirror Half Periods [1]
Pre-Defined HV-Off Value
MEMS Mirror
59
500
0
Fixing Angle Adaption Control
HV Phase Counter [1]
Analog
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
PI Control
6000
5000
4000 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
HV On/Off
Frequency [Hz]
HV-Off Value
Mirror Subtiming
4700 4600 4500 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
Fig. 11. Simplified Novel Angle Adaption architecture.
Fig. 12. Measurement results of novel Angle Adaption from small angle to large angle. 4. RESULTS
At the end of the novel procedure we are reactivating the angle amplitude-control to enable a settlement of the controller. Nevertheless, the system is the whole time still in a safe-state but the activation of the angle amplitudecontrol also guarantees point stability. 198
In this section we provide the measurement results of our novel fast angle adaption procedure which has been introduced in Section 3. Figure 12 shows the measurements of a fast angle increment. It is clearly visible where the HV-Off value is adapted. At the moment where the HV-Off counter is changed instantly, the PLL error also spikes. Subsequently the PLL started its operation and adjusted the actuation frequency. After some time the PLL error is approximately zero and the correct frequency is achieved. In case of changing the angle from small to large the required time is near 52 ms.
PLL Error [Clocks]
The second measurement exhibits a fast angle minimization in Figure 13. Similar to fast increasing, the HV-Off value is set immediately to a specific value. The PLL error is increasing. Other than the angle enlargement, where the MEMS mirror has to be sped up, the MEMS mirror has to slow down for angle reduction. The deceleration needs more time to settle and adjust the proper frequency until 500
0
-500 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
7000
8000
9000
10000
7000
8000
9000
10000
HV Phase Counter [1]
Mirror Half Periods [1] 8000 7000 6000 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1] Frequency [Hz]
enable the novel procedure the amplitude-control has to be fixed that it can’t intervene. (2) Angle Amplitude Control (a) Fast Angle Adaption The Fast Angle Adaption block is fixing the Angle Detector block that the angle error doesn’t jam the predefined HV-Off value of the Fast Angle Adaption. Then the HV-Off value is forwarded to the PI Control unit. (b) PI Control Due to the amplitude-control being fixed, the angle error can’t be forwarded to the PI Control block any longer. Thus, the integrator takes the HV-Off value from the Fast Angle Adaption unit. Furthermore the PI Control hasn’t to adapt the integrator according to an angle error. For this reason the integrator value is taken for the HV-Off Value which is forwarded to the Mirror Subtiming block. (c) Mirror Subtiming This part provides the same functionality as the SoA method. The HV-Off value is set in the Subtiming Mirror unit. The specific HV-Off value disables the HV at a certain timestep and thus influences directly the maximum operation angle. (d) Angle Detector Right at the start of the novel procedure the Angle Detector part is fixed. When the mirror is accelerated or decelerated the Angle Detector part is after a specific time period unfixed afresh. The internal Angle Detection block continues to detect and evaluate the mismatch of the desired amplitude and the actual amplitude of the MEMS mirror. The Angle Detector has to be unfixed to enable settling of the controller.
4600 4500 4400 4300 0
1000
2000
3000
4000
5000
6000
Mirror Half Periods [1]
Fig. 13. Measurement results of novel Angle Adaption from large angle to small angle.
2019 IFAC MECHATRONICS 60 Vienna, Austria, Sept. 4-6, 2019
Philipp Stelzer et al. / IFAC PapersOnLine 52-15 (2019) 55–60
REFERENCES
Table 1. Measurement results comparison
State-of-the-Art Procedure narrow FoV → wide FoV wide FoV → narrow FoV Novel Procedure narrow FoV → wide FoV wide FoV → narrow FoV
Begin
End
Time in ms
1180 545
7819 7396
692 714
446 472
943 1168
52 72
the PLL error is nearly zero. The duration among HV-Off value adaption and frequency adjustment is about 72 ms. Finally the results of the novel procedure and the SoA procedure are compared with each other. Table 1 shows clear evidence that the novel procedure is considerably faster than the SoA procedure. 5. CONCLUSIONS In our paper, we have introduced a novel angle adaption procedure for 1D MEMS Micro-Scanning LiDAR systems. The SoA procedure requires too much time during the system switches between wide FoV and narrow FoV, because the LiDAR system doesn’t fulfill the point stability criterion, which is a safety requirement, during switching time and this is too risky for automated driving. In Section 2.1 we have described the principally used 1D MEMS Micro-Scanning LiDAR system. For our design we have considered this LiDAR system’s characteristics to conceive and implement our fast angle adaption procedure. The SoA architecture respectively the procedure of the angle adaption procedure during runtime was discussed in Section 2.2, the novel architecture and its procedure was presented in Section 3. Achieved measurement results of the novel procedure and a comparison with the SoA results were demonstrated in the Section 4. In our paper we offer an approach to change the maximum angle from a narrower FoV to a wider FoV in about 52 ms. The other way around, to change the maximum angle from a wider FoV to a narrower FoV, the novel procedure is able to adjust the desired angle in approximately 72 ms. Wide to narrow FoV change is approximately 10 times and narrow to wide FoV change is about 13 times faster with the novel procedure. For highly automated driving applications a fast switching between wide and narrow FoV will be a key enabler for reliable environment perception and corresponding intervention during operation. Thus, a safe driving state for passengers and road participants is ensured by the usage of the novel architecture. ACKNOWLEDGEMENTS The authors would like to thank all national funding authorities and the ECSEL Joint Undertaking, which funded the PRYSTINE project under the grant agreement number 783190. PRYSTINE is funded by the Austrian Federal Ministry of Transport, Innovation and Technology (BMVIT) under the program “ICT of the Future” between May 2018 and April 2021 (grant number 865310). More information: https://iktderzukunft.at/en/. 199
Borovic, B., Liu, A.Q., Popa, D., Cai, H., and Lewis, F.L. (2005). Open-loop versus closed-loop control of MEMS devices: choices and issues. Journal of Micromechanics and Microengineering, 15(10), 1917. URL http:// stacks.iop.org/0960-1317/15/i=10/a=018. Daimler AG (2008). Mercedes-Benz und die Erfindung des Anti-Blockier-Systems: 1978 ist ABS serienreif. https://media.daimler.com/marsMediaSite/ de/instance/ko.xhtml?oid=9913502. Accessed: 27.11.2018. Druml, N., Macher, G., Stolz, M., Armengaud, E., Watzenig, D., Steger, C., Herndl, T., Eckel, A., Ryabokon, A., Hoess, A., Kumar, S., Dimitrakopoulos, G., and Roedig, H. (2018a). PRYSTINE - PRogrammable sYSTems for INtelligence in AutomobilEs. In 2018 21st Euromicro Conference on Digital System Design (DSD), 618–626. doi:10.1109/DSD.2018.00107. Druml, N., Maksymova, I., Thurner, T., Van Lierop, D., Hennecke, M., and Foroutan, A. (2018b). 1D MEMS Micro-Scanning LiDAR. In The Ninth International Conference on Sensor Device Technologies and Applications (SENSORDEVICES 2018). Fleming, W.J. (2001). Overview of Automotive Sensors. IEEE Sensors Journal, 1(4), 296–308. doi:10.1109/7361. 983469. Holmstrm, S.T.S., Baran, U., and Urey, H. (2014). MEMS Laser Scanners: A Review. Journal of Microelectromechanical Systems, 23(2), 259–275. doi:10.1109/JMEMS. 2013.2295470. Infineon Technologies AG (2018a). AURIX System Safety Controller. https://iot-automotive.news/ aurix-microcontroller-tc3xx-family-infineonfuels-automated-driving-electromobility. Infineon Technologies AG (2018b). Lidar Sensors. https://www.infineon.com/cms/en/product/ sensor/lidar-sensors/. Krastev, K.T., Van Lierop, H.W., Soemers, H.M., Sanders, R.H.M., and Nellissen, A.J.M. (2013). MEMS scanning micromirror. US Patent 8,526,089. Reichel, H. (2003). Elektronische Bremssysteme: vom ABS zum Brake-by-Wire. Reihe Technik. expert-Verlag. SAE (January 2014). SAE International Standard J3016 - Taxonomy and Definitions for Terms Related to OnRoad Motor Vehicle Automated Driving Systems. Standard, SAE International. Schenk, H., Duerr, P., Kunze, D., Lakner, H.K., and Kueck, H. (2000). Design and modeling of large deflection micromechanical 1D and 2D scanning mirrors. doi: 10.1117/12.396479. URL https://doi.org/10.1117/ 12.396479. Tsuchiya, T. (2017). MEMS mirrors for automotive applications. In 2017 IEEE International Meeting for Future of Electron Devices, Kansai (IMFEDK), 82–83. doi:10.1109/IMFEDK.2017.7998053. Velodyne LiDAR (2016). HDL-64E. Williams, M. (1988). PROMETHEUS-The European research programme for optimising the road transport system in Europe. In IEE Colloquium on Driver Information, 1/1–1/9.