Development of a Fully Autonomous Micro Aerial Vehicle (MAV) for Ground Traffic Surveillance

ELSEVIER

Copyright © IFAC Control in Transportation Systems, Tokyo, Japan, 2003

IFAC PUBLICATIONS www.elsevier.comllocalelifac

DEVELOPMENT OF A FULLY AUTONOMOUS MICRO AERIAL VEHICLE (MAV) FOR GROUND TRAFFIC SURVEILLANCE Marco Buschmann, Stefan Winkler, Thomas Kordes, Hanns-Waiter Schulz, Peter Vorsmann

Institute of Aerospace Systems, Technical University of Bmunschweig, Germany

Abstract: This paper describes the development of an autonomously operating Micro Aerial Vehicle (MAV) at the Institute of Aerospace Systems, Technical University of Braunschweig, Germany. After presenting the theoretical work on autonomous MAV flight, the currect prototype "Carolo" is introduced, a MAV with a wingspan of 40 cm and a mass of 390 g. One of the main applications for the MAV is ground traffic surveillance, allowing quick and flexible visual information retrieval in real-time. The operational concept is presented as well as the implemented ground control software. The first autonomous flight of "Carolo" is scheduled for autumn 2003. Copyright © 2003 [FAC Keywords: autonomous vehicles, data acquisition, digital images, inertial measurement units, microsystems, road traffic, telemetry, traffic control

1. INTRODUCTION

Micro Aerial Vehicles (MAV) are small, bird-sized aeroplanes which can be equipped with a versatility of sensors to fulfil different missions. Recent developments in this field resulted in MAVs with a wingspan of 15 cm and less, as described exemplarily in (Grasmeyer and Keennon 2001). However, currently available MAVs are still remotecontrolled and have a range of only a few kilometres. So besides minimizing the physical size, research worldwide focuses on the increase of MAV autonomy, e.g. (Ettinger e.t al. 2002). The main research at the Institute of Aerospace Systems leads towards the first autonomously operating J\,IAV, providing the same grade of autonomy as an Unmanned Aerial Vehicle (UAV) while keeping the physical size of a MAV. The current prototype called "Carolo" as seen in Figure 1 is

Fig. 1. The Micro Aerial Vehicle "Carolo" a canard type of aeroplane with a propeller in pusher configuration. It has a wingspan of 40 cm, a mass of 390 g and an endurance of 45 minutes.

73

As payload, a colour video camera was integrated. The MAV is still remote-controlled, but already uses an on-board computer and sensors to implement flight stabilization. Subsystem development and testing is done with an upscale version, a MAV with a wingspan of 100 cm and a mass of 960 g. This MAV configuration can also be used for achieving longer endurance or integrating heavier payloads. Depending on the installed payload, different missions are possible. Carolo could be used to measure basic meteorological parameters like humidity, temperature, air pressure and wind velocity vector. This could be a replacement or a valuable supplement for conventional nonreturnable radiosondes. By implementing a digital image sensor, Carolo can be used to observe wide areas in case of catastrophes like flooding to coordinate rescue teams.

Fig. 2. The anatomy of Carolo

Telemetry

Beside these applications, the current research activities are focused on ground traffic surveillance for optimizing ground traffic flow. Due to it's small size and autonomous operation, Carolo can be utilized quickly at any place by personnel without special training. Propulsion

Flight Data Processing Unit

2. DEVELOPMENT OF A FULLY AUTONOMOUS MAV

Fig. 3. Data flow between the MAV's subsystems the MAV "Carolo". The model is based on nonlinear flight mechanics and uses an experimentally derived fourdimensional data set 1 , describing the aerodynamic properties (Kordes et al. 2002).

The basis for computer-controlled autonomous flight is the determination of the aircraft's state vector, consisting of the aeroplane's position, attitude and velocity in all three coordinates. Modern position and attitude determination systems are well capable of delivering this information with sufficient temporal resolution and spatial precision. Based on these data, the necessary control commands for the aeroplane's control surfaces and the propulsion system can be calculated in realtime in order to keep the aeroplane on a given flight path. This was already implemented by several Unmanned Aerial Vehicles (UAV) designs. However, due to the demand for sensor precision and high computational power, these UAV are comparably bulky and not easy to operate.

2.2 Overview of the MA V hardware The different subsystems of the Micro Aerial Vehicle "Carolo" and their arrangement is shown in Figure 2. The design is highly modular, simplifying development and testing as well as allowing quick and simple replacement of subsystems for adjusting the MAV to it's mission. Figure 3 shows the data flow between the subsystems. The following sections describe these subsystems in detail.

2.3 Propulsion and actuators 2.1 Simulation of the MA V's dynamic behaviour The Micro Aerial Vehicle is controlled with altogether three control surfaces: One elevator at the front and two ailerons in the main wing, one each side. By exciting the ailerons codirectionally, the ailerons can act as flaps. As actuators, conventional model plane servo motors are used. The

The first step towards the development of an autonomous MAV was the creation of a simulation environment. This environment is used to study the highly dynamic behaviour of the aeroplane and to develop and test the necessary flight control algorithms. The software tool "MATLAB" from MathWorks Inc. was used to develop a mathematical model of

1 Measurements at the wind tunnel of the Institute of Fluid Dynamics, Technical University of Braunschweig

74

The currently available MEMS devices fulfill all requirements regarding weight, dimensions and power consumption, but the accuracy is rather poor. However, it was shown that the accuracy of a MEMS-based IMU can be significantly improved by taking the unit's temperature into account and using proper calibration methods (Winkler et al. 2003). For evaluating the quality of the IMU data, a test flight with the research aircraft of the Institute of Flight Guidance and Control, Technical University of Braunschweig, was performed. The research aircraft of type DO-128 is equipped with a high precision differential GPS receiver as well as a LaserNav IMU from Honeywell, which were used as reference.

Fig. 4. IMU in test mounting dynamic behaviour of the actuators was investigated experimentally and a mathematical model was derived. This model was integrated into the simulating environment described earlier to allow realistic propagation of the MAVs behaviour during flight. It was shown that the response time of commercially available micro servos is fast enough to control a highly dynamic MAV like Carolo (Schulz et al. 2002).

2.5 The on-board computer

Central element of the autonomous MAV electronics is the on-board computer, processing the sensor data and controlling the aeroplane's actuators. Via a telemetry link, the computer is connected with ground control to report the MAV's status and receive flight path updates. In addition, the payload, a digital camera in the case of ground traffic observation, has to be interfaced.

As propulsion, a 2-bladed propeller powered by a brushless 3-phase DC motor with external rotor is used. The power consumption is approximately 50 W during launch and 20 W to 25 W at cruise speed. The pusher configuration allows better aerodynamic performance.

The tasks for the on-board computer can be split into two groups: Interfacing to other subsystems and data processing. These two tasks have very different requirements on the used hardware. The subsystem interfacing demands a versatility of different control and data lines while the data processing demands high computational power. As shown earlier in figure 3, the computer consists of two parts: The so-called Main System Controller (MSC) and the Flight Data Processing Unit (FDPU). This division was made in order to meet the different requirements of the above mentioned tasks as good as possible.

2.4 Sensors The basis for autonomous flight is the real-time determination of the state vector. For this, two sensor modules were integrated. One module is a commercially available GPS receiver for determining position and velocity in ECEF 2 system with an update rate of 1 Hz. However, this information is not sufficient for controlling highly dynamic aeroplanes. For this reason, the second module consists of a highly miniaturized Inertial Measurement Unit (IMU) with dimensions of 40 x 40 x 20 mm, as can be seen in figure 4.

The MSC consists of an ultra-Iow-power microcontroller with a multitude of integrated periphery. This simplifies the interconnections to the MAV's subsystems while allowing a comparably low computational power. The MSC also allows the connection of a telemetry module or payload module with low data rates.

The module incorporates commercially available microelectromechanical systems (MEMS) for measuring linear acceleration and angular rate of all three axes. It has integrated A/D-converters which sample the data at a rate of 100 Hz. The module was developed in two different variations, employing different brands of sensors. The main purpose of evaluating two different concepts is to gain valuable experience with MEMS-based sensors and to determine the most suitable IMU concept.

2

The FDPU consists of a 32 bit RISC 3 processor with a computational power of approximately 200 MIPS. Because all time-consuming I/O interfacing is done by the MSC, the FDPU can use it's computational power solely for hosting the flight control and navigation algorithms. However, payloads or telemetry links which demand high data rates or complex communication protocols have to be interfaced to the FDPU, due to the limited capacities of the MSC.

ECEF: Earth Centered, Earth Fixed

3

75

RISC: Reduced Instruction Set Computer

In addition to the MSC and the FDPU, a Flight Data Recorder (FDR) was implemented. The FDR records the data flow between MSC and FDPU and performs the same task for the MAY as a "black box" for conventional aircraft. This is not only useful for malfunction analysis and flight performance evaluation during developing and testing, but also allows the recording of sensor data, e.g. high-resolution images made by an on-board camera. As mass storage media, Multi Media Cards with 128 MBytes are used.

2.7.1. General Packet Radio Service (GPRS). The implementation of a GPRS 5 modem is in progress now. The module will allow bidirectional communication with low latency time and a guaranteed minimum data rate of 9.6 kbps and an average data rate of 28.8 kbps, depending on the load of the mobile phone network. This data rate is fully sufficient for mission control and in addition allows the transmission of sensor data like low-resolution images with moderate update rates. Depending on the mission, a tradeoff between image resolution and update rate has to be made. The use of a cellular mobile phone network instead of a dedicated, direct radio links has several advantages: Since the telecommunication infrastructure for GPRS (or similar systems) is already available in most countries, no initial costs arise and the MAY can be used in a wide geographic area with no or only minor changes in telemetry configuration.

2.6 Payload For ground traffic surveillance, the payload will consist of an appropriate camera system. The current prototype is equipped with a commercially available color camera module with an analog video output. The signal is transmitted with a dedicated miniaturized HF transmitter which allows a range of approximately 150 m. The complete system of camera and transmitter weights under 30 g. In order to increase both range and resolution, this camera will be replaced with a digital imaging system, allowing the use of the Flight Data Recorder for image storage and the MAY's telemetry link for real-time image transmission.

2.7.2. Future migration path: UMTS. The introduction of Universal Mobile Telecommunications System (UMTS) which is based on the Wideband Code Division Multiple Access (W-CDMA) system promises very high data rates. As an example, data rates of 384 kbit are announced for the introductory phase of UMTS in Germany. However, since the actual data rate depends on many factors like the number of users within a cell or the mobility of the user, the practically realizable data rate is difficult to predict.

For first experiences with ground traffic surveillance, the MAY will be equipped with a digital image sensor with a resolution of 1.3 megapixel. The internal memory of the camera module allows the storage of up to 50 full resolution images in JPEG format. The transmission of images to the ground station depends on the used telemetry link, as discussed in the next section.

Table 1 shows a comparison of three different telemetry concepts regarding their data rate in kilobit per second (kbps) and the time in seconds needed to transfer a digital image, assuming a hardware and software protocol overhead of 25 %. The listed data rate for GPRS is an average value since the actual data rate depends on the load of the cell. Hence GSM is listed since it reflects the guaranteed minimum data rate for GPRS. Because unpacked image data size is not feasible, a JPEG2000 algorithm will be used for in-flight data size reduction. For this comparison, the compression rate was set to 1:40, causing acceptable image quality losses. High resolution images have a size of 1024 x 768 pixels resulting in a data size of approximately 60 kbytes per image. For low resolution images, a size of 320 x 240 pixel is assumed, resulting in a size of 5.5 kbytes per image.

2.7 Telemetry The ~[AY prototype is equipped with a unidirectional data link based on a conventional remote control (R/C) for model planes. During the current test and development stage, the R/C receiver interfaces to the MSC, allowing the in-flight adjustment of parameters for the control algorithms and serving as backup for manual flight control in case of flight controller malfunction. In addition to this, a quasi-bidirectional data link was established with a module based on the Digital Enhanced Cordless Telecommunications (DECT) standard, allowing a data rate of approximately 100 kbps 4 . However, the connection proved to be reliable only for distances up to 150 m. While this is acceptable for test purposes, it is nor suitable for scientific or commercial MAY applications. 4

s based on the Global System for Mobile Communications (GSM)

bps: kilobits per second

76

resolution are not as high as for basic research purposes. reducing image data size. The conceptual design of such a mission is done together with local and state authorities as well as the local branch of the Gerneral German Automobile Club "ADAC Niedersachsen/Sachsen-Anhalt" .

Table 1. Data rate comparison of different telemetry concepts. System

Data Rate

GSM GPRS UMTS

9.6 kbps 28.8 kbps 384.0 kbps

Time per High Res. Image 62.50 s 20.83 s 1.56 s

Time per Low Res. Image 5.73 s 1.90 s 0.15 s

The MAV takes images from the current traffic conditions on the observed area and feeds the collected data via ePRS and the internet to the ~IAV server software. On ground, the image data is processed, distributed to an end user needing instant information, e.g. police or fire department. By using portable computers like Notebooks or Personal Digital Assistants, this information is also available to mobile users like rescue teams. The whole mission is initially planned and supervised by an operator at ground control, ensuring proper mission fulfillment. The following section describes ground control operation in more detail.

3. GROUND TRAFFIC SURVEILLANCE WITH THE AUTONOMOUS MAV "CAROLO"

This section describes the planned operation of the r"IAV "Carolo" for ground traffic surveillance and the actual state of development at the Institute of Aerospace Systems. The application of ground traffic surveillance with a l'vIAV can be split into two main groups: The first group is the systematic collection of ground traffic data. The second group is the retrieval of instantaneous traffic information, useful in case of traffic jams or accidents.

3.3 Ground control 3.1 Using the MA V for basic research

For the practical operation, ground control plays an important role. During the last months, a modular ground control concept was developed which allows the easy adjustment of the software's functionality according to the end user's demands. Ground control consists of a generic Personal Computer or Notebook, hosting the ground control software. Depending on the chosen telemetry concept, additional hardware is needed to exchange data with the MAV.

For basic research projects, cruising speed, flight altitude, image quality and camera optics have to be adjusted carefully in order to retrieve the desired visual information reliably. Real-time transmission of images is not necessary, allowing onboard storage of the image data. This greatly simplifies the requirements for the telemetry link. Together with the Institute of Transport, Road Engineering and Planning (Hannover University, Germany), the investigation of the demands on hardware and flight guidance for such a basic research mission are in progress. The goal of this project is to measure traffic density and vehicle velocities continiously using high-resolution aerial photograph sequences made by the MAV.

The software has a modular design based on a server-client structure. Each software module connects via UDP lIP 6 to the server module which coordinates the data flow. By using the Internet Protocol for communication between the software modules, the configuration of ground control is very versatile. This allows the server and the different software modules to run on PCs in different locations, assuming a network connection exists. Alternatively, the complete ground control software could run on a single Notebook. In case of GPRS, the MAV connects via the internet to the server and appears as an additional software module of ground control. To integrate telemetry links which are not directly lP-based, a serial communication client was implemented, allowing the transmission of ground control data packets via the PC's serial port. At present, the server software is implemented in two different versions in order to run under the operating systems Linux or Microsoft Windows.

3.2 Retrieval of instantaneous traffic information The main focus of the Institute of Aerospace Systems is the development of the necessary payload and telemetry systems for using the MAV "Car010" as a source of instantaneous traffic information. In this case, the goal is to quickly deliver visual information of the ground traffic situation to allow fast response of ground personnel like police or rescue teams and to aid in their coordination. The main advantage of a MAV over permanent, stationary installed systems is it's high flexibility which makes it an ideal tool for observing temporary obstacles like construction sites or guiding ground traffic on sporadically overloaded roads, e.g. in case of big events. In this case, real-time image transmission to ground control is necessary which results in high data load for the telemetry link. On the other hand, the demands of image

As a first client module, the ~;lission Planning and Supervision Module was implemented. This module provides the user with a digital map of the op6

77

UDP/IP: User Datagram Protocol/Internet Protocol

100cmcmis isused. used.Demonstration Demon stration autonom ous of of autonomous ofof100 flight is schedul autumn 2003. forfor autumn 2003. flight is scheduleded softwarpackage e package generic PCs develop ed, AAsoftware forfor generic PCs waswas developed, allowin g missionplanning plannin gandandmission mission control allowing mission control digitalmap. map.For Forcommunication commu nicatiobetween n between onona adigital groundcontrol control andthethe MAV, implem entatio n and MAV, thethe implementation ground of a GPRSbased radio modem progres s of a GPRS-based radio modem is isin inprogress nowThe . Thechosen chosentelemetry telemet ry concepwill t will allow now. concept allow thethe transmi ssionofoflow lowresolution resoluti onimages imagesfrom from thethe transmission MAVwith witha adata datarate rate approx imately MAV ofof approximately 30 30 images images per minute. prospec tivemigration migrati on per minute. AAprospective path forfor thethe path telemet rylink linkis isUMTS UMTSand andwill will bebe implemented telemetry implem ented soonasasend enduser userequipment equipm ent asassoon becomes becomeavailable. s available. Test mission s groundtraffic trafficsurveillance Test missions forforground surveill ancein in the area area ofofLower LowerSaxony, Saxony,Germany, the German y,will willtake take place inin close close cooperation coopera tionwith place withthethelocal localand and state authorities authori tiesand andthe state theGerneral Gerner alGerman Germa nAuAutomobil e Club Club "ADAC "ADAC NiedersachsenjSachsentomobile Nieders achsen/ Sachsen Anhalt" .The Thefirst firstfield Anhalt". fieldtest testis isplanned plannedforforspring spring 2004. 2004.

Fig. 5. Screens hot of the Mission Plannin g and Fig. 5. Screenshot of the Mission Planning and Superv ision Module Supervision Module eration al area. This map can consist of a digitize d erational area. This map can consist of a digitized topogra phic or city map, combin ed with elevatio n topographic or city map, combined with elevation informa tion, or digital landsca pe models. Figure 5 information, or digital landscape models. Figure 5 shows a screens hot of the current softwar e version. shows a screenshot of the current software version. InInthis d air raph combined combin ed thiscase, case, aa digitize digitized air photog photograph with n model used. The The user user withaadigital digital elevatio elevation model was was used. can nts on determ ining the map, map, thus thus determining canset set waypoi waypoints on the the special MAV's flight flight path. path. With With each each point, special the MAV's actions can be associa ted, conactions can be associated, e.g. e.g. "circlin "circlingg in constant height for 60 seconds ". During mission , the stant height for 60 seconds". During mission, the actual V isis display ed within the the MA MAV displayed actualposition positionof ofthe digital g supervi sion of digital map, map, allowin allowing supervision of the MAV's route. The ly flight path path can can be be adapted adapted manual manually route. Theflight atatany anytime timeby byediting editing the the flight flight path path on on the map and nt update andsending sendinga awaypoi waypoint update to to the the MAV. MAV. The Mission Plannin g and ision Module The Mission Planning and Superv Supervision Module display s only V informa displays only very very basic basic MA MAV information tion like position , course . An and height height. An Instrum Instrumentation position, courseand entatio n Module ment now, Moduleisisunder underdevelop development now, which which allows allows the ation ofofthe V's most thevisualiz visualization the MA MAV's most import important ant states , speed, stateslike likeattitude attitude, speed, fuel fuel and and basic basic health health monito ring inina aconven ient way monitoring convenient way similar similar to to aa conconvention al aeropla ne's instrum ents. Other Other tasks tasks are are ventional aeroplane's instruments. the entatio n ofofa avisualiz theimplem implementation visualization module for for ation module display ing and displaying and archivi archiving images from from the the ononng images board . boardcamera camera.

REFERENCES REFER ENCES

Ettinge r, S.S. M., Ettinger, M., M. M. C.C.Nechyba, Nechyb a,P.P.G.G lfju . Ifjuand and M. Waszak M. Flight Waszak(2002). (2002) .Vision-Guided Vision- Guided FlightStaStablity blity and and Control Contro lfor forMicro MicroAir AirVehicles. Vehicles.In:In: Proceedings Proceedings ofofthe the IEEE IEEEInternational Interna tionalConConference ference on on Intelligent Intelligent Robots Robots and andSystems. Systems. Vo!. Vol. 3.3. pp. pp. 2134-40. 2131-40 . Grasmeyer, Grasme yer, J.J. M. M. and and M. M. T.T . Keennon Keenno n(2001). (2001). Development Black Widow Develo pment ofof the the Black Widowmicro microairair vehicle. vehicle. In: In: AIAA, AIAA, Aerospace AerospaceSciences SciencesMeetMeeting ing and and Exhibit, Exhibit , 39th, 39th, Reno, Reno, NV, NV, Jan. Jan.8-11, 8-11, 2001. 2001. Kordes, Kordes , T., T., M. M. Buschmann Buschm ann and and P.P. Vorsmann Vorsma nn (2002). Modeling of the Nonlinear (2002). Modeli ng of the Nonline arDynamic Dynam ic Behavior Behavi or of of aa Micro-Aerial-Vehicle Micro-Aerial-Vehicle (MAV) (MAV) in an in an Environment Environ ment of of aa Turbulent Turbul ent AtmoAtmosphere. In: sphere. In: Proceedings Proceedings of of the the ICAS ICAS 2002 2002 (Toronto, 2002),. (Toront o, Canada, Canada, September September 8-13, 8-13, 2002), . International Interna tional Council Counci l of ofthe theAeronautical Aerona uticalSciSciences. pp. 552.1-552.10. ences. pp. 552.1-5 52.10. Kordes and Schulz, H.-W., H.-W., M. Schulz, M. Buschmann, Buschm ann, T. T. Kordes and P. Vorsma Vorsmann (2002). Simulation dyP. nn (2002). Simula tion des des dynamischen Mikroflugzeuges namisc hen Verhaltens Verhalt ens eines eines Mikroflugzeuges Wind und Eigenunter Bercksi Bercksichtigung unter chtigun g von von Wind und Eigenschaften realer Subsysteme. Jahrbuch der schafte n realer Subsys teme. In: In: Jahrbuch der Deutschen Gesellschaft fr LuftRaumDeutsch en Gesellschajt fr Lujt- und und Raumund fahrt 2002, Deutscher Luftfahrt 2002, Deutscher Lujt- und RaumRaumfahrtkongress 2002 (Stuttgart, Germany, fahrtko ngress 2002 (Stuttga rt , Germany, September 23-26, 2002). number DGLRSeptem ber 23-26, 2002). number DGLRJ T2002-168. JT2002 -168. Winkler, S., M. Buschmann, T. Kordes, H.-W. Winkle r, S. , M. Buschm ann, T. Kordes , H.-W. Schulz and P. Vorsmann (2003). MEMSSchulz and P. Vorsma nn (2003). MEtvlSbased IMU Development, Calibration and based IMU Develo pment, Calibra tion and Testing for Autonomous MAV Navigation. In: Testing for Autono mous MAV NavigaMeeting tion. In: Proceedings of the ION 59 th Annual Proceedings of the ION 59 th Annual Meeting (Albuquerque, NM, June 23-25, 2003),. The (Albuquerque, NM, June 23-25, 2003),. The Institute of Navigation. Institut e of Naviga tion.

4.4.CONCL CONCLUSION USION

In In this thedevelop development statusofofthe thefully fully thispaper, paper,the ment status autonom MAV"Carolo "Carolo" of the Institute of autonomous ous MAV " of the Institut e of Aerospa Aerospace Systems wasshown, shown,includin including a dece System s was g a descriptio MAV'ssubsyst subsystems. prototype scription n of of thetheMAV's ems. AAprototy pe with anda amass massofof390 390gg with a wingspan cmand a wingsp an ofof4040cm ThisMA MAV stilloperate operates alreadyassemb assembled. waswas already led. This V still s butalready alreadyinclude includes video remotely controlled remotel y control led but s a avideo and transmitter supplythe theuser userwith with camera camera and transm itter totosupply shortdistanc distances. The development video over livelive video over short es. The develop ment autonomous flight of all subsystems necessary of all subsyst ems necessa ry forforautonom ous flight and intensive systemintegra integration and finished waswas finished and intensiv e system tion and testing currently takesplace. place.AsAstest testplatform platform, testing current ly takes , upscale versionof of"Carolo "Carolo" witha awingsp wingspan an an upscale version " with an

78 78