- Email: [email protected]
Emergence of SFAP
C H A P T E R
21 Emergence of SFAP Natural scientists have a preference for visual over verbal descriptions of real-world phenomena. Wainer and Friendly (2019, p. 210)
21-1 OVERVIEW Small-format aerial photography (SFAP) first appeared in the late 19th century with balloons and kites and continued to develop throughout the 20th century with all manner of aircraft (see Chap. 1). By the 1990s, SFAP was recognized as a specialization within the greater realm of remote sensing (Warner et al. 1996; Bauer et al. 1997). Nonetheless, SFAP remained a tiny niche practiced by just a handful of hobbyists and even fewer professionals. At the turn of this century, for example, there were probably no more than a couple hundred people doing kite aerial photography worldwide, civilian unmanned aerial systems (UAS) hardly existed, and nearly all SFAP was accomplished via custom-built equipment or by simply pointing a camera out the window of an airplane or helicopter. Within the span of just two decades, SFAP has moved from niche status to mainstream activity involving many thousands of people with sizable commercial markets for readyto-use hardware and software, and diverse technical, scientific, and artistic applications.
21-2 SFAP TECHNOLOGIES The rapid rise of SFAP parallels development of digital cameras. At the beginning of this century, a typical compact digital camera had a sensor of just 2 megapixels (1600 by 1200 pixels), relatively slow response time, and limited functions and memory. An equivalent modern digital camera is likely to have a 24-megapixel sensor (6000 by 4000 pixels), much improved response time, many functions including WiFi, and a huge memory
Small-Format Aerial Photography and UAS Imagery https://doi.org/10.1016/B978-0-12-812942-5.00021-5
c apacity. Mirrorless interchangeable-lens cameras (MILC) have become quite effective for SFAP, and small digital cameras for color-infrared and multispectral imagery are also readily available now. The high quality of modern digital cameras is, perhaps, the single most-important factor for emergence of SFAP in the early 21st century. Several other technologies and societal developments have contributed to SFAP, namely GNSS, batteries, airborne platforms, 3D photogrammetry, Internet and mobile phones, and legal frameworks. Global navigation satellite system (GNSS), such as the US Global Positioning System (GPS), European Galileo, and Chinese BeiDou, is another technology that has moved into the mainstream. Prior to 2000, the United States GPS signals were intentionally degraded for security reasons. Known as selective availability, this procedure was discontinued in 2000 at the direction of President Clinton (GPS.gov 2018). The end of selective availability led to a revolution in GPS applications for civilian use. Every conceivable GPS device is now available to the public for automobiles, boats, bicycles, and personal wear, and GPS is built into many digital cameras. GPS-guided aircraft enable high-precision survey options for both manned and unmanned small-format aerial photography. Real-time kinematic (RTK) and post-processing kinematic (PPK) GNSS have achieved cm-dm positional accuracy for professional-grade UAS imagery. With Europe’s own GNSS, Galileo, at the brink of full operational capability, availability and accuracy of satellite positioning will further increase. Rechargeable batteries power nearly all devices necessary for SFAP both in cameras and unmanned platforms. In the late 1990s, the usual choices were alkaline batteries based on nickel-cadmium (NiCd) or nickel-metal-hydrate (NiMH). NiCd batteries have excellent performance under a variety of conditions, but they are highly toxic. NiMH batteries are less toxic and have higher specific energy; they have replaced most NiCd batteries (Battery University 2017).
349
© 2019 Elsevier B.V. All rights reserved.
350
21. Emergence of SFAP
Lithium-ion-polymer (LiPo) batteries have proliferated in the early 21st century for all types of devices employed for SFAP. Several varieties incorporate cobalt, manganese, or molybdenum (Schumm 2018), and a protection circuit is necessary for safely charging LiPo cells (Battery University 2017). Initial cost is high, but LiPo batteries may be recharged many more times than nickel-based batteries. Improved battery performance has made SFAP more reliable than in the past, although limitations still exist, and battery issues are among the most common operational and logistical problems. Unmanned SFAP platforms have evolved swiftly in the early 21st century, especially for free-flying UAS. Multicopters with 4 (quad), 6 (hexa), or 8 (octo) rotors have emerged as stable designs along with flying wings and other more-conventional fixed-wing types of model aircraft. Photographs may be taken by manual radio control, autonomous camera operation, or following pre-programmed routines guided by GNSS. UAS are autopiloted by an inertial measurement unit (IMU) and inertial navigation system (INS) along with additional sensors for measuring altitude and obstacle detection. Electronic, 3-axis gimbals stabilize and allow control of camera position, and First Person View (FPV) live-video downlinking allows the pilot to take the camera’s view in real time. The advantages of fixed-wing and multirotor UAS are combined in hybrid aircraft that have the abilities of vertical takeoff and landing (VTOL), hovering, and highspeed flight like a conventional model airplane. These platforms are currently much in the focus of UAS developers. At the leading edge of UAS design are robotic birds, known as ornithopters, such as the 2-m-wingspan SmartBird (Borrell 2018). Manned platforms, in contrast, have not changed much in recent years as far as SFAP is concerned. Small airplanes, helicopters, autogyros, and various kinds of ultra-light aircraft may be utilized with hand-held cameras or mounted cameras that are remotely controlled by the pilot or copilot-photographer.
Photogrammetry techniques have likewise moved forward rapidly. In truth, the Structure from Motion– Multi-View Stereo (SfM-MVS) approach, originally developed for computer vision, has revolutionized the potential for highly detailed 3D modeling and analysis of earth-surface phenomena by combining vertical and low-oblique SFAP imagery in highly redundant image networks. This capability has been utilized already for many geoscientific applications (e.g. Cook 2017; Feurer et al. 2018). The growth of Internet communication has been phenomenal since it was privatized in 1995. As of 2000, less than 20,000 websites existed worldwide. The milestone of one billion websites was achieved in 2014, and current websites total ~1.9 billion, although many of these are inactive (InternetLiveStats.com 2018). Online sharing of techniques and experiences has greatly benefitted small-format aerial photographers. Early adopters inspired many others to follow in their footsteps. Similar explosive growth took place for mobile (cellular) phones. The number of mobile-phone subscriptions worldwide was approximately 740,000 in 2000. By 2015, global subscriptions exceeded 7 billion (Statista.com 2018), approximately the world’s human population. This expansion was driven by increasingly sophisticated phones as well as enhanced network capacity and functionality (Table 21-1). Smartphone apps have been adapted for control of airborne platforms and camera functions. Legal aspects have moved forward along with SFAP technology. Laws and regulations vary considerably from country to country, but some common trends are noted during the past decade. Regarding manned aircraft and more-traditional unmanned tethered platforms, rules and regulations have been relatively stable for SFAP. Small UAS, on the other hand, have prompted many countries to revise and tighten regulations and restrictions for pilot qualifications and UAS operations. In the United States, for instance, the FAA is funding research and development that may lead to new rules
Table 21-1 Summary of mobile (cellular) phone network capacity and functionality. Based on SmartIPX (2015) and Johansen (2016). Network
1G
2G
3G
4G
5G
Period
1980s
1990s
2000s
2010s
2020s
Speed
2.4 kbps
64 kbps
2000 kbps
100 Mbps
1 Gbps
Functions
Basic voice Analog protocols
Designed for voice First digital standard
Voice with some data Mobile broadband
Designed for data IP-based protocols
[Still under development]
Note: pilot 5G networks are already available in Estonia to the public in limited areas (Hankewitz 2018).
351
21-3 Unmanned Aircraft for SFAP
that would support more complex, low-altitude UAS operations (FAA 2018), and the same may be expected in other countries as well. This provides a legal framework within which SFAP may be conducted safely and properly. Furthermore, ethical and privacy considerations have grown for appropriate use of SFAP platforms in different situations.
21-3 UNMANNED AIRCRAFT FOR SFAP The emphasis throughout this book has been for unmanned platforms employed at relatively low height for small-format aerial photography. From early kites to the newest experimental ornithopters, many unmanned aircraft have been designed, tested, and further refined as potential platforms for SFAP. Some have fallen by the wayside for various reasons of cost, logistics, flight operation, and legal restrictions. Other types of unmanned aircraft have emerged as viable platforms for SFAP. Aber et al. (2018b) compared the strengths and shortcomings of four types of unmanned SFAP aircraft, namely fixed-wing UAS, multi-
rotor UAS, kite, and helium blimp. They concluded that no single platform is ideal for SFAP in all circumstances. Cost of aircraft, logistical and local site issues, availability of fuel or electric power, and legal restrictions all may come into play for SFAP in different parts of the world. For example, tethered aircraft could be the only platforms legally allowed to fly in some situations. The requirement to lift heavier or multiple cameras or sensors adds further restrictions on feasible platforms. Helium for b alloons or blimps is readily available in the United States, but is costly or simply not an option in many places. Transporting larger LiPo batteries is restricted in some countries. In light of these many issues, seven unmanned platforms for SFAP are summarized in Table 21-2. On this basis, it should be clear that no single type of unmanned aircraft is ideal for SFAP in all situations. Indeed, each mission and location presents certain requirements and special conditions that favor one type of platform over another. For maximum flexibility, both a free-flying UAS and a tethered platform should be available to the small-format aerial photographer in order to cope with different circumstances.
TABLE 21-2 Comparison of four free-flying UAS and three tethered unmanned platforms for SFAP. Relative qualitative ratings for typical equipment and missions are based on many sources and the authors’ experiences. Based on Aber et al. (2018a & 2018b). SFAP comparison
Fixed-wing UAS
Multirotor UAS Multirotor UAS Hybrid VTOL (quadcopter) (hexacopter) UAS
Kite (tethered)
Platform cost
High
Moderate
High
High
Low to moderate Moderate to high High
Operating energy
Rechargeable battery
Rechargeable battery
Rechargeable battery
Rechargeable battery
Wind
Helium
Propane, heating gas
Low
Low
Low
Low
High
Low
Operating cost Low
Helium blimp (tethered)
Hot-air blimp (tethered)
Wind and temperature
Calm to moderate Calm to light Above freezing Above freezing
Calm to light Above freezing
Calm to light Light to strong Calm to light Calm to light Above freezing Any temperature Any temperature Any temperature
Portability
Moderate
Easy
Moderate
Moderate
Easy
Difficult
Difficult
Personnel
1 to 2
1
1 to 2
1 to 2
1 to 2
2 to 3
4 to 6
Flight time
One hour
15–25 min
15–25 min
One hour
Several hours
Several hours
45 min
Flying height*
400 feet
400 feet
400 feet
400 feet
500 feet
500 feet
500 feet
Stationary position
Not possible
Excellent
Excellent
Excellent
Good
Excellent
Excellent
Payload
½ to 6 kg
½ to 1 kg
1 to 6 kg
1 to 6 kg
½ to 3 kg
5 to 10 kg
5 to 25 kg
Technical level High
Moderate
High
High
Low
Low
Moderate
Vertical areal coverage
Large
Small
Moderate
Large
Small
Moderate
Moderate
Legal status*
Many restrictions
Many restrictions
Many restrictions
Many restrictions
Few restrictions Few restrictions Few restrictions
*
Flying height and legal status are given for the United States; regulations vary considerably in other countries from outright prohibition to few restrictions. Logistical factors and costs likewise vary greatly around the world.
352
21. Emergence of SFAP
21-4 SFAP CONVERGENCE Small-format aerial photography, thus, represents a convergence of multiple technologies, many of which had no common origin or initial relationships, along with societal recognition and development of legal frameworks. The emergence of SFAP is not due to any single person or institution. Rather, it represents the results of many inventors and self-builders using diverse materials, components, airborne platforms, and cameras. Many of the people involved freely shared their techniques and methods from many parts of the world. This convergence of innovation, manufacturing,
and public policy is an example of bricolage, a French term that refers to resourcefulness in the creation of a new technology from diverse sources and available resources (Nielsen 2009). Given the quick pace of recent technological and societal developments, the authors anticipate continued rapid improvements and growth of small-format aerial photography and UAS imagery in the near future. SFAP may be applied for subjects not yet imagined using cameras and airborne platforms that are yet to be designed and built. We look forward to the next decade of innovations for small-format aerial photography particularly for geoscience applications.
21 Emergence of SFAP Natural scientists have a preference for visual over verbal descriptions of real-world phenomena. Wainer and Friendly (2019, p. 210)
21-1 OVERVIEW Small-format aerial photography (SFAP) first appeared in the late 19th century with balloons and kites and continued to develop throughout the 20th century with all manner of aircraft (see Chap. 1). By the 1990s, SFAP was recognized as a specialization within the greater realm of remote sensing (Warner et al. 1996; Bauer et al. 1997). Nonetheless, SFAP remained a tiny niche practiced by just a handful of hobbyists and even fewer professionals. At the turn of this century, for example, there were probably no more than a couple hundred people doing kite aerial photography worldwide, civilian unmanned aerial systems (UAS) hardly existed, and nearly all SFAP was accomplished via custom-built equipment or by simply pointing a camera out the window of an airplane or helicopter. Within the span of just two decades, SFAP has moved from niche status to mainstream activity involving many thousands of people with sizable commercial markets for readyto-use hardware and software, and diverse technical, scientific, and artistic applications.
21-2 SFAP TECHNOLOGIES The rapid rise of SFAP parallels development of digital cameras. At the beginning of this century, a typical compact digital camera had a sensor of just 2 megapixels (1600 by 1200 pixels), relatively slow response time, and limited functions and memory. An equivalent modern digital camera is likely to have a 24-megapixel sensor (6000 by 4000 pixels), much improved response time, many functions including WiFi, and a huge memory
Small-Format Aerial Photography and UAS Imagery https://doi.org/10.1016/B978-0-12-812942-5.00021-5
c apacity. Mirrorless interchangeable-lens cameras (MILC) have become quite effective for SFAP, and small digital cameras for color-infrared and multispectral imagery are also readily available now. The high quality of modern digital cameras is, perhaps, the single most-important factor for emergence of SFAP in the early 21st century. Several other technologies and societal developments have contributed to SFAP, namely GNSS, batteries, airborne platforms, 3D photogrammetry, Internet and mobile phones, and legal frameworks. Global navigation satellite system (GNSS), such as the US Global Positioning System (GPS), European Galileo, and Chinese BeiDou, is another technology that has moved into the mainstream. Prior to 2000, the United States GPS signals were intentionally degraded for security reasons. Known as selective availability, this procedure was discontinued in 2000 at the direction of President Clinton (GPS.gov 2018). The end of selective availability led to a revolution in GPS applications for civilian use. Every conceivable GPS device is now available to the public for automobiles, boats, bicycles, and personal wear, and GPS is built into many digital cameras. GPS-guided aircraft enable high-precision survey options for both manned and unmanned small-format aerial photography. Real-time kinematic (RTK) and post-processing kinematic (PPK) GNSS have achieved cm-dm positional accuracy for professional-grade UAS imagery. With Europe’s own GNSS, Galileo, at the brink of full operational capability, availability and accuracy of satellite positioning will further increase. Rechargeable batteries power nearly all devices necessary for SFAP both in cameras and unmanned platforms. In the late 1990s, the usual choices were alkaline batteries based on nickel-cadmium (NiCd) or nickel-metal-hydrate (NiMH). NiCd batteries have excellent performance under a variety of conditions, but they are highly toxic. NiMH batteries are less toxic and have higher specific energy; they have replaced most NiCd batteries (Battery University 2017).
349
© 2019 Elsevier B.V. All rights reserved.
350
21. Emergence of SFAP
Lithium-ion-polymer (LiPo) batteries have proliferated in the early 21st century for all types of devices employed for SFAP. Several varieties incorporate cobalt, manganese, or molybdenum (Schumm 2018), and a protection circuit is necessary for safely charging LiPo cells (Battery University 2017). Initial cost is high, but LiPo batteries may be recharged many more times than nickel-based batteries. Improved battery performance has made SFAP more reliable than in the past, although limitations still exist, and battery issues are among the most common operational and logistical problems. Unmanned SFAP platforms have evolved swiftly in the early 21st century, especially for free-flying UAS. Multicopters with 4 (quad), 6 (hexa), or 8 (octo) rotors have emerged as stable designs along with flying wings and other more-conventional fixed-wing types of model aircraft. Photographs may be taken by manual radio control, autonomous camera operation, or following pre-programmed routines guided by GNSS. UAS are autopiloted by an inertial measurement unit (IMU) and inertial navigation system (INS) along with additional sensors for measuring altitude and obstacle detection. Electronic, 3-axis gimbals stabilize and allow control of camera position, and First Person View (FPV) live-video downlinking allows the pilot to take the camera’s view in real time. The advantages of fixed-wing and multirotor UAS are combined in hybrid aircraft that have the abilities of vertical takeoff and landing (VTOL), hovering, and highspeed flight like a conventional model airplane. These platforms are currently much in the focus of UAS developers. At the leading edge of UAS design are robotic birds, known as ornithopters, such as the 2-m-wingspan SmartBird (Borrell 2018). Manned platforms, in contrast, have not changed much in recent years as far as SFAP is concerned. Small airplanes, helicopters, autogyros, and various kinds of ultra-light aircraft may be utilized with hand-held cameras or mounted cameras that are remotely controlled by the pilot or copilot-photographer.
Photogrammetry techniques have likewise moved forward rapidly. In truth, the Structure from Motion– Multi-View Stereo (SfM-MVS) approach, originally developed for computer vision, has revolutionized the potential for highly detailed 3D modeling and analysis of earth-surface phenomena by combining vertical and low-oblique SFAP imagery in highly redundant image networks. This capability has been utilized already for many geoscientific applications (e.g. Cook 2017; Feurer et al. 2018). The growth of Internet communication has been phenomenal since it was privatized in 1995. As of 2000, less than 20,000 websites existed worldwide. The milestone of one billion websites was achieved in 2014, and current websites total ~1.9 billion, although many of these are inactive (InternetLiveStats.com 2018). Online sharing of techniques and experiences has greatly benefitted small-format aerial photographers. Early adopters inspired many others to follow in their footsteps. Similar explosive growth took place for mobile (cellular) phones. The number of mobile-phone subscriptions worldwide was approximately 740,000 in 2000. By 2015, global subscriptions exceeded 7 billion (Statista.com 2018), approximately the world’s human population. This expansion was driven by increasingly sophisticated phones as well as enhanced network capacity and functionality (Table 21-1). Smartphone apps have been adapted for control of airborne platforms and camera functions. Legal aspects have moved forward along with SFAP technology. Laws and regulations vary considerably from country to country, but some common trends are noted during the past decade. Regarding manned aircraft and more-traditional unmanned tethered platforms, rules and regulations have been relatively stable for SFAP. Small UAS, on the other hand, have prompted many countries to revise and tighten regulations and restrictions for pilot qualifications and UAS operations. In the United States, for instance, the FAA is funding research and development that may lead to new rules
Table 21-1 Summary of mobile (cellular) phone network capacity and functionality. Based on SmartIPX (2015) and Johansen (2016). Network
1G
2G
3G
4G
5G
Period
1980s
1990s
2000s
2010s
2020s
Speed
2.4 kbps
64 kbps
2000 kbps
100 Mbps
1 Gbps
Functions
Basic voice Analog protocols
Designed for voice First digital standard
Voice with some data Mobile broadband
Designed for data IP-based protocols
[Still under development]
Note: pilot 5G networks are already available in Estonia to the public in limited areas (Hankewitz 2018).
351
21-3 Unmanned Aircraft for SFAP
that would support more complex, low-altitude UAS operations (FAA 2018), and the same may be expected in other countries as well. This provides a legal framework within which SFAP may be conducted safely and properly. Furthermore, ethical and privacy considerations have grown for appropriate use of SFAP platforms in different situations.
21-3 UNMANNED AIRCRAFT FOR SFAP The emphasis throughout this book has been for unmanned platforms employed at relatively low height for small-format aerial photography. From early kites to the newest experimental ornithopters, many unmanned aircraft have been designed, tested, and further refined as potential platforms for SFAP. Some have fallen by the wayside for various reasons of cost, logistics, flight operation, and legal restrictions. Other types of unmanned aircraft have emerged as viable platforms for SFAP. Aber et al. (2018b) compared the strengths and shortcomings of four types of unmanned SFAP aircraft, namely fixed-wing UAS, multi-
rotor UAS, kite, and helium blimp. They concluded that no single platform is ideal for SFAP in all circumstances. Cost of aircraft, logistical and local site issues, availability of fuel or electric power, and legal restrictions all may come into play for SFAP in different parts of the world. For example, tethered aircraft could be the only platforms legally allowed to fly in some situations. The requirement to lift heavier or multiple cameras or sensors adds further restrictions on feasible platforms. Helium for b alloons or blimps is readily available in the United States, but is costly or simply not an option in many places. Transporting larger LiPo batteries is restricted in some countries. In light of these many issues, seven unmanned platforms for SFAP are summarized in Table 21-2. On this basis, it should be clear that no single type of unmanned aircraft is ideal for SFAP in all situations. Indeed, each mission and location presents certain requirements and special conditions that favor one type of platform over another. For maximum flexibility, both a free-flying UAS and a tethered platform should be available to the small-format aerial photographer in order to cope with different circumstances.
TABLE 21-2 Comparison of four free-flying UAS and three tethered unmanned platforms for SFAP. Relative qualitative ratings for typical equipment and missions are based on many sources and the authors’ experiences. Based on Aber et al. (2018a & 2018b). SFAP comparison
Fixed-wing UAS
Multirotor UAS Multirotor UAS Hybrid VTOL (quadcopter) (hexacopter) UAS
Kite (tethered)
Platform cost
High
Moderate
High
High
Low to moderate Moderate to high High
Operating energy
Rechargeable battery
Rechargeable battery
Rechargeable battery
Rechargeable battery
Wind
Helium
Propane, heating gas
Low
Low
Low
Low
High
Low
Operating cost Low
Helium blimp (tethered)
Hot-air blimp (tethered)
Wind and temperature
Calm to moderate Calm to light Above freezing Above freezing
Calm to light Above freezing
Calm to light Light to strong Calm to light Calm to light Above freezing Any temperature Any temperature Any temperature
Portability
Moderate
Easy
Moderate
Moderate
Easy
Difficult
Difficult
Personnel
1 to 2
1
1 to 2
1 to 2
1 to 2
2 to 3
4 to 6
Flight time
One hour
15–25 min
15–25 min
One hour
Several hours
Several hours
45 min
Flying height*
400 feet
400 feet
400 feet
400 feet
500 feet
500 feet
500 feet
Stationary position
Not possible
Excellent
Excellent
Excellent
Good
Excellent
Excellent
Payload
½ to 6 kg
½ to 1 kg
1 to 6 kg
1 to 6 kg
½ to 3 kg
5 to 10 kg
5 to 25 kg
Technical level High
Moderate
High
High
Low
Low
Moderate
Vertical areal coverage
Large
Small
Moderate
Large
Small
Moderate
Moderate
Legal status*
Many restrictions
Many restrictions
Many restrictions
Many restrictions
Few restrictions Few restrictions Few restrictions
*
Flying height and legal status are given for the United States; regulations vary considerably in other countries from outright prohibition to few restrictions. Logistical factors and costs likewise vary greatly around the world.
352
21. Emergence of SFAP
21-4 SFAP CONVERGENCE Small-format aerial photography, thus, represents a convergence of multiple technologies, many of which had no common origin or initial relationships, along with societal recognition and development of legal frameworks. The emergence of SFAP is not due to any single person or institution. Rather, it represents the results of many inventors and self-builders using diverse materials, components, airborne platforms, and cameras. Many of the people involved freely shared their techniques and methods from many parts of the world. This convergence of innovation, manufacturing,
and public policy is an example of bricolage, a French term that refers to resourcefulness in the creation of a new technology from diverse sources and available resources (Nielsen 2009). Given the quick pace of recent technological and societal developments, the authors anticipate continued rapid improvements and growth of small-format aerial photography and UAS imagery in the near future. SFAP may be applied for subjects not yet imagined using cameras and airborne platforms that are yet to be designed and built. We look forward to the next decade of innovations for small-format aerial photography particularly for geoscience applications.