- Email: [email protected]
Basic Principles of SFAP
C H A P T E R
2 Basic Principles of SFAP The sky is where I work, and the land and people are what I see … my aerial vantage point is a privileged one. A. Heisey (Heisey and Kawano 2001).
2-1 REMOTE SENSING Small-format aerial photography (SFAP) is a type of remote sensing, which is the science and art of gathering information about an object from a distance. In other words, measurements or observations are taken without making direct physical contact with the object in question. The electromagnetic spectrum is the energy that carries information through the atmosphere from the Earth’s surface to the small-format camera. Most digital cameras—as well as analog film cameras— are capable of operating in the spectral range that includes near-ultraviolet, visible, and near-infrared radiation, which is the spectrum emphasized in this book. For the most part, this electromagnetic radiation represents reflected solar energy—in other words, natural sunlight that illuminates the scene, reflects from surface objects, and strikes the electronic detector or film of the camera. There are several aspects in which actual SFAP deviates from ideal remote sensing (adapted from Lillesand et al. 2015).
2-1.1 Ideal Remote Sensing • Sunlight—constant solar energy over all wavelengths at known output, irrespective of time and place. • Neutral atmosphere—totally transparent atmosphere that would neither absorb nor scatter solar radiation. • Unique spectral signatures—each object would have a distinctive and known spectral response everywhere and at all times. • Super sensor—camera that would be highly sensitive through all wavelengths of interest and would be economical and practical to operate. • Real-time data handling—system that would allow instant downloading, processing, and presentation of images.
Small-Format Aerial Photography and UAS Imagery https://doi.org/10.1016/B978-0-12-812942-5.00002-1
• Multiple data users—images would be useful to scientists, engineers, managers, and others from all disciplines and applications.
2-1.2 Actual SFAP • Sunlight—varies with time and place in ways that cannot be fully predicted. Calibration is sometimes possible, but the exact nature of available solar energy is usually not known. • Atmosphere—varies according to latitude, season, time of day, local weather, etc. Selective absorption and scattering are the rule at most times and places. • Spectral signatures—all objects have theoretically unique signatures, but in practice these may change and cannot always be distinguished; many objects appear the same. • Real cameras—no existing small-format camera system may operate practically in all wavelengths of interest. Each camera is limited by its optics, electronic or film characteristics to certain wavelengths. Likewise certain cameras are limited by their high cost. • Data handling—digital cameras now generate imagery that may be handled quickly by either visual inspection or computer analysis. • Multiple users—no single combination of imagery and analysis satisfies all users. Many users are not familiar with subjects outside their immediate disciplines and thus cannot appreciate the full potential or limitations of SFAP. SFAP, like other types of remote sensing, is a compromise between the ideal and what is logistically feasible and financially affordable for a given project. In this regard, the relatively low cost, high spatial resolution, and field portability of SFAP offer some advantages not possible with other means of aerial remote sensing. SFAP normally exploits the so-called visible atmospheric window consisting of wavelengths from ~0.3 to 1.5 μm long (Fig. 2-1). On a cloud-free day, this range of wavelengths passes through the atmosphere with little scattering or absorption by gas molecules, aerosols, or
11
© 2019 Elsevier B.V. All rights reserved.
12
2. Basic Principles of SFAP
Fig. 2-1 Spectrum of visible light in micrometers (μm) wavelength. All visible colors are made up of three primary colors: blue (0.4–0.5 μm), green (0.5–0.6 μm), and red (0.6–0.7 μm). Near ultraviolet is ~0.3–0.4 μm, and near infrared is ~0.7–1.5 μm wavelengths.
fine dust. Given the low-height operation for most SFAP below 300 m, images are acquired in which the reflected radiation has suffered minimal degradation from atmospheric scattering or absorption. This is an important consideration in terms of clarity and spectral signatures of objects depicted in SFAP images.
2-2 COMMON ASPECTS OF SFAP Among different types of remote sensing, SFAP undoubtedly has the greatest variety in terms of aerial platforms and camera systems. Some basic aspects are common to all approaches, nonetheless, regardless of the type of platform, camera, or purpose for SFAP. These common aspects are introduced here and elaborated in more detail in subsequent chapters. Fig. 2-2 Three views of the Science Garden at Frankfurt
2-2.1 Image Vantage Aerial photographs may be taken in three vantages relative to the Earth’s surface, as determined by the tilt of the camera lens relative to the horizon (Fig. 2-2). The amount of tilt is called depression angle.
University’s Campus Riedberg, Germany. (A) High-oblique view showing the horizon with the Taunus mountain range in the background. (B) Low-oblique view in which the horizon is not visible. (C) Vertical view of the central part of the garden. The flowerbeds, used as nursery and for teaching purposes, are arranged according to plant taxonomy. Taken with on-board camera of quadcopter UAV.
2-2 Common Aspects of SFAP
13
• High-oblique vantage—side view, horizon and sky are visible, depression angle typically <20°. • Low-oblique vantage—side view, horizon is not visible, depression angle typically 20° to 87°. • Vertical vantage—view straight down, also called nadir, depression angle >87° to 90°. Vertical images are generally preferred for mapping and measurement purposes, as explained below, because the geometry of vertical images may be calculated. However, such vertical views often are difficult for many people to interpret unless they are quite familiar with the site and objects shown in the image. Oblique shots, on the other hand, provide overviews of sites and their surroundings that are easier for most people to recognize visually and understand readily (e.g. Ham and Curtis 1960). Yet, oblique photographs have substantial distortions in scene geometry that render accurate measurements difficult or impossible.
2-2.2 Photographic Scale and Resolution The scale of a vertical aerial photograph over flat terrain may be calculated simply in two ways (see Chap. 3). The scale (S) depends on the average height above the ground (Hg) and the lens focal length (f) of the camera. In either case, the units of measurement must be the same. S = f / Hg
(Eq. 2-1)
or S = photo distance ( d ) / ground distance ( D )
(Eq. 2-2)
In cases where objects of known size appear in the vertical photograph, the second method may be utilized for scale calculation (Fig. 2-3). If no objects of known size are visible in the photograph and the flying height above ground is known, the first method is employed. Scale is usually expressed as a fraction or ratio, such as 1/1000 or 1:1000, meaning one linear unit of measurement on the photograph equals 1000 units on the ground. In rugged terrain, however, photo scale varies because of large height differences within the photograph. Likewise oblique photos also display large scale variations. Scale is a fundamental property of routine aerial photographs, and is especially important for vertical airphotos used for measurements and photogrammetric purposes. Interpretability of aerial photographs is often determined by photo scale. For analog aerial photographs, the original film scale is the most commonly used characteristic for describing the amount of detail identifiable in the image. The actual photographic resolution is determined by the size of the smallest identifiable feature within an image.
Fig. 2-3 Biological study site near Pueblo, Colorado, United States. The north arrow is 4 m long by 1 m wide; it provides both a scale bar and a directional indicator for this vertical kite aerial photograph. Note two people standing next to the survey arrow. Photo taken with a compact digital camera.
For digital images, the original scale on the image sensor is not of much interest, as the scale of a digital image is easily changed when viewing it on a display device and becomes a property of that device. However, the original image resolution does not change with varying display scale, and the size of the smallest visible object depends directly on the size of the sensor cells or pixels in the electronic detector. In the case of digital imagery, ground sample distance (GSD) is, therefore, more appropriate as a measure for image scale (Comer et al. 1998). Consider a digital camera with a charge-coupled device (CCD); collection GSD is related to the size of each pixel element within the detector array. Using the scale calculations noted above, GSD may be determined as follows: GSD = ( pixel element size ) × H g / f
(Eq. 2-3)
However, a single pixel usually cannot be identified as a unique object by itself. For visual identification of distinct objects, generally a group of 4–9 pixels is the minimum necessary (Comer et al. 1998). This leads to a general rule of thumb (Hall 1997): • Positive recognition of objects in aerial photographs requires a GSD three to five times smaller than the object size. Digital images as well as small-format analog photographs are rarely, if ever, displayed at the original camera scale, which would be much too small for normal visual examination. Most usually, digital images are enlarged substantially for display on a computer monitor, in which the dot pitch (size) controls the image size and
14
2. Basic Principles of SFAP
scale, assuming one image pixel is displayed for each monitor dot. In this case, the display scale is a ratio of the collection GSD to the monitor dot pitch. Display scale = ( monitor dot pitch ) / GSD (Eq. 2-4) As an example, take a digital vertical photograph acquired at a height of 100 m using a camera with a 35-mm lens focal length and CCD pixel element size of 0.009 mm. Converting all units into meters, collection GSD would be (0.000009 × 100)/0.035 = 0.026 m (~2½ cm or 1 in.). Now, displaying this image at full size on a monitor with dot pitch of 0.26 mm, the display scale would be 0.00026/0.026 = 0.01 (or 1:100). A similar calculation could be done for printed digital images. The nominal pixel size for standard printing at 300 dpi (dots per inch) is about 0.085 mm. In this case, printed scale would be 0.000085/0.026 = 0.00327 (or about 1:300). Displaying or printing the image at smaller scales would mean losing some of its information content when viewing it on the screen or printout. This example demonstrates that display and printed scales are usually many times greater than is the original digital image scale, because the display/print pixels are many times larger than are the electronic detector pixel elements. The larger scales employed for display and printing of digital images do not imply more information or better interpretability, however, compared to the raw image data (Fig. 2-4). A digital number is simply
Fig. 2-4 Enlargement of the arrow and people in the previous figure. This image contains no more spatial or spectral information than the previous image; each pixel represents exactly the same ground area and color as before. All the details visible in the enlarged image are present in the original image.
a color value for a single pixel, regardless of the size at which the pixel is displayed. So far, we have used the term resolution, which is employed in different ways in remote sensing, for describing the spatial dimensions of SFAP. Other aspects of resolution include spectral, temporal, and radiometric properties (Jensen 2007). Suitable spectral resolution of the image may be equally or even more important than is spatial resolution for identification of certain objects. For example, color-infrared photography was developed originally for camouflage detection and is widely employed now for vegetation, soil, and water studies (Finney 2007). The combination of visible and near- infrared radiation reveals objects that may appear similar in visible light only (Fig. 2-5). Temporal resolution refers to how often a remote-sensing system may record an image of the same area. For satellite systems, this is dependent on their fixed orbital periods, image swath width, and off-nadir tilting capacity. SFAP, in contrast, offers much greater flexibility in adapting acquisition
Fig. 2-5 Color-visible (A) and color-infrared (B) kite aerial photographs of the campus of Emporia State University, Kansas, United States. A portion of the football field, dormitory buildings, parking lots, automobiles, grass, and deciduous trees. Photosynthetically active vegetation is depicted in red and pink colors in the infrared image. Note variations in tree appearance in the color-infrared version. Photos taken with a pair of analog SLR cameras.
2-2 Common Aspects of SFAP
15
Fig. 2-6 Time series of three vertical views of a small tributary to Oued Ouaar at La Glalcha near Taroudant (Morocco), taken before and after a heavy rainfall event in autumn 2010. Flow direction is bottom to top in the image; field of view ~50 m across. Between September 29 (A) and December 2 (B), stream channel erosion has resulted in substantial changes at and below the cut bank (CB) at the left side of the channel and at the point bar (PB) at the entry into the main wadi. As the ephemeral stream channel was still partly filled with water during the post-event survey, the survey was repeated 1 day later (December 3; C) to reveal the erosion and deposition changes in the channel bed. All images were taken between 4 and 5 pm; note the similarity of shadows in B and C and the discrepancy with image A, taken 9 weeks earlier, due to differences in seasonal sun position. Fixed-wing UAV photographs taken with digital MILC by IM, S. d’Oleire-Oltmanns and D. Peter.
time and repeat rate to the objectives of a survey. Images may be taken in intervals of minutes, days, or years depending on the requirements of a project. This allows the SFAP photographer to react timely when monitoring changes in a landscape (Fig. 2-6; see also Fig. 16-2). Even if time series are not required, the high temporal flexibility of SFAP in choosing a precise acquisition time is a great advantage over other remote-sensing systems. Deciduous vegetation, as an example, is strongly seasonal in character, and this situation may be exploited for identification of plant types (Fig. 2-7). Finally, the term radiometric resolution refers to the number of digital levels, also called precision or image depth, that the sensor uses for recording different intensities of radiation. For display devices and most standard image file format, 0–255 or 28 (8-bit) digital levels per image band are usual. Most SFAP camera sensors, however, record 12–16 bits for the raw image (see Chap. 6).
of tall objects near the edge of a vertical photograph, particularly for wide-angle fields of view (Fig. 2-8). The height of a tall vertical object may be calculated from its relief displacement, and the height of tall objects also may be determined from measurements of shadows.
2-2.3 Relief Displacement The camera lens operates much like the human eye, both of which produce single-point perspective views of the scene. This perspective causes increasing relief displacement of objects nearing the edge of view (see Chap. 3) and is most noticeable in vertical airphotos, because tall objects appear to lean away from the photo center. Conversely low objects are displaced toward the center. Relief displacement is minimal near the photo center and becomes extreme at the edge. This allows for a side view
Fig. 2-7 Vertical kite photograph in visible light showing water pools and vegetated hummocks in the central portion of Männikjärve Bog, Estonia. Moss species display distinctive green, gold, and red early autumn colors along with pale green dwarf pine trees on hummocks. These dramatic colors are not so distinct at other times of the year; precise timing of image acquisition is therefore highly valuable. Field of view ~60 m across. Taken with a compact digital camera; based on Aber et al. (2002).
16
2. Basic Principles of SFAP
Fig. 2-8 Wide-angle, vertical view over conifer forest at Kojšovská hol’a in southeastern Slovakia. Note how trees appear to lean away from photo center toward outer edges of the scene. Kite photo taken with a compact digital camera.
2-2.4 Stereoscopic Images Humans see in three dimensions, in other words depth perception, because our eyes provide overlapping fields of view from slightly different vantage points. The amount of depth perception in humans is limited to about 400 m distance, however, because of the relatively close spacing of our eyes only 6–7 cm apart (Drury 2001). Stereoscopic photography has been practiced since the middle nineteenth century to provide 3D imagery (Osterman 2007). Aerial
stereo photographs may be taken from widely separated positions (Fig. 2-9); the greater distance in image spacing produces exaggerated depth perception. Such overlapping pictures typically are viewed through a stereoscope (Fig. 2-10) or on-screen with anaglyph glasses or special stereoviewing hardware. Vertical stereophotos are important for visual photointerpretation and are the basis for many photogrammetric techniques (Ogleby 2007; see also Chaps. 3-3 and 11-4).
Fig. 2-9 Pair of oblique stereophotos showing a residential scene in Emporia, Kansas, United States. The pictures were taken simultaneously with two cameras spaced ~1 m apart. Note slight left-right offset in views; compare vehicle in lower left corner and house in lower right corner of each photograph. Kite photos taken with a pair of compact analog cameras.
2-3 Photographic Storage
17
Fig. 2-10 Sokkia mirror stereoscope (MS16). This model is the ideal size for viewing 4 × 6-in. (10 × 15-cm) prints made from 35-mm film or digital images.
2-3 PHOTOGRAPHIC STORAGE SFAP is more than just snapshots; usually the images are intended for long-term storage and reproduction years and even decades after they were acquired. However, neither film nor digital photography is everlasting; all photographic media are subject to long-term decay (Rosenthaler 2007). Thus, proper storage of the images becomes a significant issue for most SFAP projects. Geographic information typically consists of two kinds of data. First is the primary dataset comprised of location information and attribute data about individual features. Second is so-called metadata, which includes such information about the geographic dataset as its grid system, map projection, units of measurement, date of creation, camera model, lens focal length, and history of processing. Aerial photographs are one type of geographic information. The original image itself is the primary dataset. Metadata should contain information about location, date of image acquisition, type of camera and lens, exposure settings, altitude, and other relevant facts. For analog (film, print) photographs, such information could be written directly on the image, so there is no chance the image could be separated from its metadata (see Figs. 1-4 and 1-5). A more common approach is to place metadata on the margins, back, or frame of the photographic medium (Fig. 2-11). Still this approach is often omitted or incomplete for SFAP, and years later nobody would remember the where, when, or what aspects for a photograph. Some camera-related metadata (EXIF header) is built into image files collected with most modern digital cameras. The image file contains metadata, under file properties, such as image dimensions, file size in bytes, date and time the image was taken, type of data compression, and camera model as well as shutter speed, f-stop, and ISO setting. Some camera models also may include GPS data and look direction (Fig. 2-12).
Fig. 2-11 Example of 35-mm color film mounted in a plastic frame (slide). Metadata written on the frame include location, date of acquisition, and picture number. Kite photo taken with a compact analog camera, Estonia.
Fig. 2-12 Close-up view of radio-controlled rig for Nikon Coolpix 16-megapixel digital camera. The AW110 model is waterproof, dustproof, and shockproof. It has built-in GPS, world map, and compass functions, which are recorded in metadata for each image. This setup is designed for use with a tethered and relatively stationary platform, such as kite, blimp, or balloon.
Image file sizes have increased substantially in the first decades of digital photography, resulting in increased storage requirements. Consider a typical 15-min UAS survey with a rather modest 12-megapixel camera. An exposure interval of 5 s, excluding starting and landing phase, would result in about 150 images of ~5½ MB
18
2. Basic Principles of SFAP
jpg file-format size—a total storage space exceeding that of a 700 MB CD, which was a substantial format only 10 years ago. Although the capacity of contemporary storage media continues to increase as well as their miniaturization, their perpetuity remains an important issue. Both digital images and analog photographs are stored in some type of physical medium, the properties of which determine how long the image is likely to survive. The potential longevity of panchromatic (b/w) film and prints is on the order of one century or more; color film and prints have only about half these lifetimes or less (Jensen 2015; Rosenthaler 2007). In contrast, magnetic media, such as disks, tapes, and solid-state devices, last only one or two decades at most, before they degrade under the Earth’s magnetic field. Optical disks (CD, DVD) are thought to survive for a century or more, although they have not been in service long enough to really know their lifespan in practice. Another, perhaps more serious, long-term issue for digital preservation concerns the computer hardware and software necessary to read, transfer, and display digital files stored in a particular medium. Medium types, file storage formats, and computer operating systems have changed rapidly since digital data became commonplace. So, although the magnetic tape, optical disk, external hard drive, or hardware behind the cloud storage service may survive intact, an operational reading device and software to interpret the file format may no longer exist in many cases. High-capacity external hard drives and removable solid-state memory have become commonplace; however, like all electronic media they are subject to failure. Serial failures of computers and memory devices, in fact, have cost more than one researcher substantial datasets representing months or years of work. The option to store data “in the cloud” is also subject to the same potential failures as well as security risks. Large technology companies encourage users to link their devices and sync their personal data via wireless connections. While this may be quite convenient, it means that commercial enterprise now controls much of a person’s private information, which is a scary and dangerous development (Pogue 2014). Burge (2007) emphasized the importance of data migration periodically as new digital media are established and old media become obsolete; such migration is necessary about every 5 years (Rosenthaler 2007). Large companies and governmental agencies may have the capability to transfer and reformat digital files from one medium to another periodically. In one geospatial analysis laboratory, for example, digital image files were updated from original 9-track tapes, to compact cartridge tapes, to zip disks, and finally to optical disks as well as external hard drives. Nearly all people who work with digital imagery and data have experienced at least one hard-drive crash or data loss due to mechanical failure, viruses, or simple human error. Multiple backup files stored on different devices in separate locations provide
some insurance that critical datasets would not be lost. The investment in time and labor to accomplish this is certainly significant, yet necessary. Lawrence’s photograph of San Francisco in Ruins (see Fig. 1-4) has survived for more than a century. One could ask what the chances are for modern digital photographs to survive with their metadata intact for so long (Burge 2007). Recent history suggests that technical innovation and obsolescence will continue to happen rapidly for digital storage devices and file formats. Means of long-term storage for digital imagery is an issue yet to be fully resolved.
2-4 SUMMARY SFAP is based primarily on solar radiation reflected from the Earth’s surface in the visible and near-infrared portions of the spectrum. SFAP is a compromise between ideal photographic conditions and the reality of what is possible to accomplish under natural conditions within logistical constraints and financial limitations. SFAP may be taken in a range of viewing angles from high-oblique to vertical vantages. This facilitates depicting a study area in its broader landscape context as well as in map-like views best suited for accurate measurements. Photographic scale (S) and GSD are closely related, but distinct, concepts dealing with spatial aspects of photographs. Resolution relates to spatial, spectral, temporal, and radiometric aspects of images. Compared to other remote-sensing systems, SFAP offers particularly great advantages with regard to the degree and flexibility of spatial and temporal resolutions. A single photograph has geometric characteristics similar to the image sensed through a human eye, which creates a single-point perspective view. Two overlapping views of the same area result in stereoscopic imagery in which depth perception is apparent. This is the basis for much photointerpretation as well as photogrammetry. Photographic information consists of both the primary image data and metadata about the image. Metadata should include date of image acquisition, location, type of camera and lens, and other relevant information. These metadata should be saved in a manner that is physically difficult to separate from the image data. Various media and file formats have been used over the years for storing photographic images and digital datasets. Among the most long-lived are panchromatic film negatives and optical disks; the least stable are magnetic media. Storage of digital datasets is subject to rapid changes in media types and related computer hardware and software necessary for reading the digital files. Continued technical changes and obsolescence are likely to happen in the near future, which raises questions for long-term archival storage of photographs. Periodic data migration and multiple backup files stored on different devices in separate locations may provide some insurance that critical datasets would not be lost.
2 Basic Principles of SFAP The sky is where I work, and the land and people are what I see … my aerial vantage point is a privileged one. A. Heisey (Heisey and Kawano 2001).
2-1 REMOTE SENSING Small-format aerial photography (SFAP) is a type of remote sensing, which is the science and art of gathering information about an object from a distance. In other words, measurements or observations are taken without making direct physical contact with the object in question. The electromagnetic spectrum is the energy that carries information through the atmosphere from the Earth’s surface to the small-format camera. Most digital cameras—as well as analog film cameras— are capable of operating in the spectral range that includes near-ultraviolet, visible, and near-infrared radiation, which is the spectrum emphasized in this book. For the most part, this electromagnetic radiation represents reflected solar energy—in other words, natural sunlight that illuminates the scene, reflects from surface objects, and strikes the electronic detector or film of the camera. There are several aspects in which actual SFAP deviates from ideal remote sensing (adapted from Lillesand et al. 2015).
2-1.1 Ideal Remote Sensing • Sunlight—constant solar energy over all wavelengths at known output, irrespective of time and place. • Neutral atmosphere—totally transparent atmosphere that would neither absorb nor scatter solar radiation. • Unique spectral signatures—each object would have a distinctive and known spectral response everywhere and at all times. • Super sensor—camera that would be highly sensitive through all wavelengths of interest and would be economical and practical to operate. • Real-time data handling—system that would allow instant downloading, processing, and presentation of images.
Small-Format Aerial Photography and UAS Imagery https://doi.org/10.1016/B978-0-12-812942-5.00002-1
• Multiple data users—images would be useful to scientists, engineers, managers, and others from all disciplines and applications.
2-1.2 Actual SFAP • Sunlight—varies with time and place in ways that cannot be fully predicted. Calibration is sometimes possible, but the exact nature of available solar energy is usually not known. • Atmosphere—varies according to latitude, season, time of day, local weather, etc. Selective absorption and scattering are the rule at most times and places. • Spectral signatures—all objects have theoretically unique signatures, but in practice these may change and cannot always be distinguished; many objects appear the same. • Real cameras—no existing small-format camera system may operate practically in all wavelengths of interest. Each camera is limited by its optics, electronic or film characteristics to certain wavelengths. Likewise certain cameras are limited by their high cost. • Data handling—digital cameras now generate imagery that may be handled quickly by either visual inspection or computer analysis. • Multiple users—no single combination of imagery and analysis satisfies all users. Many users are not familiar with subjects outside their immediate disciplines and thus cannot appreciate the full potential or limitations of SFAP. SFAP, like other types of remote sensing, is a compromise between the ideal and what is logistically feasible and financially affordable for a given project. In this regard, the relatively low cost, high spatial resolution, and field portability of SFAP offer some advantages not possible with other means of aerial remote sensing. SFAP normally exploits the so-called visible atmospheric window consisting of wavelengths from ~0.3 to 1.5 μm long (Fig. 2-1). On a cloud-free day, this range of wavelengths passes through the atmosphere with little scattering or absorption by gas molecules, aerosols, or
11
© 2019 Elsevier B.V. All rights reserved.
12
2. Basic Principles of SFAP
Fig. 2-1 Spectrum of visible light in micrometers (μm) wavelength. All visible colors are made up of three primary colors: blue (0.4–0.5 μm), green (0.5–0.6 μm), and red (0.6–0.7 μm). Near ultraviolet is ~0.3–0.4 μm, and near infrared is ~0.7–1.5 μm wavelengths.
fine dust. Given the low-height operation for most SFAP below 300 m, images are acquired in which the reflected radiation has suffered minimal degradation from atmospheric scattering or absorption. This is an important consideration in terms of clarity and spectral signatures of objects depicted in SFAP images.
2-2 COMMON ASPECTS OF SFAP Among different types of remote sensing, SFAP undoubtedly has the greatest variety in terms of aerial platforms and camera systems. Some basic aspects are common to all approaches, nonetheless, regardless of the type of platform, camera, or purpose for SFAP. These common aspects are introduced here and elaborated in more detail in subsequent chapters. Fig. 2-2 Three views of the Science Garden at Frankfurt
2-2.1 Image Vantage Aerial photographs may be taken in three vantages relative to the Earth’s surface, as determined by the tilt of the camera lens relative to the horizon (Fig. 2-2). The amount of tilt is called depression angle.
University’s Campus Riedberg, Germany. (A) High-oblique view showing the horizon with the Taunus mountain range in the background. (B) Low-oblique view in which the horizon is not visible. (C) Vertical view of the central part of the garden. The flowerbeds, used as nursery and for teaching purposes, are arranged according to plant taxonomy. Taken with on-board camera of quadcopter UAV.
2-2 Common Aspects of SFAP
13
• High-oblique vantage—side view, horizon and sky are visible, depression angle typically <20°. • Low-oblique vantage—side view, horizon is not visible, depression angle typically 20° to 87°. • Vertical vantage—view straight down, also called nadir, depression angle >87° to 90°. Vertical images are generally preferred for mapping and measurement purposes, as explained below, because the geometry of vertical images may be calculated. However, such vertical views often are difficult for many people to interpret unless they are quite familiar with the site and objects shown in the image. Oblique shots, on the other hand, provide overviews of sites and their surroundings that are easier for most people to recognize visually and understand readily (e.g. Ham and Curtis 1960). Yet, oblique photographs have substantial distortions in scene geometry that render accurate measurements difficult or impossible.
2-2.2 Photographic Scale and Resolution The scale of a vertical aerial photograph over flat terrain may be calculated simply in two ways (see Chap. 3). The scale (S) depends on the average height above the ground (Hg) and the lens focal length (f) of the camera. In either case, the units of measurement must be the same. S = f / Hg
(Eq. 2-1)
or S = photo distance ( d ) / ground distance ( D )
(Eq. 2-2)
In cases where objects of known size appear in the vertical photograph, the second method may be utilized for scale calculation (Fig. 2-3). If no objects of known size are visible in the photograph and the flying height above ground is known, the first method is employed. Scale is usually expressed as a fraction or ratio, such as 1/1000 or 1:1000, meaning one linear unit of measurement on the photograph equals 1000 units on the ground. In rugged terrain, however, photo scale varies because of large height differences within the photograph. Likewise oblique photos also display large scale variations. Scale is a fundamental property of routine aerial photographs, and is especially important for vertical airphotos used for measurements and photogrammetric purposes. Interpretability of aerial photographs is often determined by photo scale. For analog aerial photographs, the original film scale is the most commonly used characteristic for describing the amount of detail identifiable in the image. The actual photographic resolution is determined by the size of the smallest identifiable feature within an image.
Fig. 2-3 Biological study site near Pueblo, Colorado, United States. The north arrow is 4 m long by 1 m wide; it provides both a scale bar and a directional indicator for this vertical kite aerial photograph. Note two people standing next to the survey arrow. Photo taken with a compact digital camera.
For digital images, the original scale on the image sensor is not of much interest, as the scale of a digital image is easily changed when viewing it on a display device and becomes a property of that device. However, the original image resolution does not change with varying display scale, and the size of the smallest visible object depends directly on the size of the sensor cells or pixels in the electronic detector. In the case of digital imagery, ground sample distance (GSD) is, therefore, more appropriate as a measure for image scale (Comer et al. 1998). Consider a digital camera with a charge-coupled device (CCD); collection GSD is related to the size of each pixel element within the detector array. Using the scale calculations noted above, GSD may be determined as follows: GSD = ( pixel element size ) × H g / f
(Eq. 2-3)
However, a single pixel usually cannot be identified as a unique object by itself. For visual identification of distinct objects, generally a group of 4–9 pixels is the minimum necessary (Comer et al. 1998). This leads to a general rule of thumb (Hall 1997): • Positive recognition of objects in aerial photographs requires a GSD three to five times smaller than the object size. Digital images as well as small-format analog photographs are rarely, if ever, displayed at the original camera scale, which would be much too small for normal visual examination. Most usually, digital images are enlarged substantially for display on a computer monitor, in which the dot pitch (size) controls the image size and
14
2. Basic Principles of SFAP
scale, assuming one image pixel is displayed for each monitor dot. In this case, the display scale is a ratio of the collection GSD to the monitor dot pitch. Display scale = ( monitor dot pitch ) / GSD (Eq. 2-4) As an example, take a digital vertical photograph acquired at a height of 100 m using a camera with a 35-mm lens focal length and CCD pixel element size of 0.009 mm. Converting all units into meters, collection GSD would be (0.000009 × 100)/0.035 = 0.026 m (~2½ cm or 1 in.). Now, displaying this image at full size on a monitor with dot pitch of 0.26 mm, the display scale would be 0.00026/0.026 = 0.01 (or 1:100). A similar calculation could be done for printed digital images. The nominal pixel size for standard printing at 300 dpi (dots per inch) is about 0.085 mm. In this case, printed scale would be 0.000085/0.026 = 0.00327 (or about 1:300). Displaying or printing the image at smaller scales would mean losing some of its information content when viewing it on the screen or printout. This example demonstrates that display and printed scales are usually many times greater than is the original digital image scale, because the display/print pixels are many times larger than are the electronic detector pixel elements. The larger scales employed for display and printing of digital images do not imply more information or better interpretability, however, compared to the raw image data (Fig. 2-4). A digital number is simply
Fig. 2-4 Enlargement of the arrow and people in the previous figure. This image contains no more spatial or spectral information than the previous image; each pixel represents exactly the same ground area and color as before. All the details visible in the enlarged image are present in the original image.
a color value for a single pixel, regardless of the size at which the pixel is displayed. So far, we have used the term resolution, which is employed in different ways in remote sensing, for describing the spatial dimensions of SFAP. Other aspects of resolution include spectral, temporal, and radiometric properties (Jensen 2007). Suitable spectral resolution of the image may be equally or even more important than is spatial resolution for identification of certain objects. For example, color-infrared photography was developed originally for camouflage detection and is widely employed now for vegetation, soil, and water studies (Finney 2007). The combination of visible and near- infrared radiation reveals objects that may appear similar in visible light only (Fig. 2-5). Temporal resolution refers to how often a remote-sensing system may record an image of the same area. For satellite systems, this is dependent on their fixed orbital periods, image swath width, and off-nadir tilting capacity. SFAP, in contrast, offers much greater flexibility in adapting acquisition
Fig. 2-5 Color-visible (A) and color-infrared (B) kite aerial photographs of the campus of Emporia State University, Kansas, United States. A portion of the football field, dormitory buildings, parking lots, automobiles, grass, and deciduous trees. Photosynthetically active vegetation is depicted in red and pink colors in the infrared image. Note variations in tree appearance in the color-infrared version. Photos taken with a pair of analog SLR cameras.
2-2 Common Aspects of SFAP
15
Fig. 2-6 Time series of three vertical views of a small tributary to Oued Ouaar at La Glalcha near Taroudant (Morocco), taken before and after a heavy rainfall event in autumn 2010. Flow direction is bottom to top in the image; field of view ~50 m across. Between September 29 (A) and December 2 (B), stream channel erosion has resulted in substantial changes at and below the cut bank (CB) at the left side of the channel and at the point bar (PB) at the entry into the main wadi. As the ephemeral stream channel was still partly filled with water during the post-event survey, the survey was repeated 1 day later (December 3; C) to reveal the erosion and deposition changes in the channel bed. All images were taken between 4 and 5 pm; note the similarity of shadows in B and C and the discrepancy with image A, taken 9 weeks earlier, due to differences in seasonal sun position. Fixed-wing UAV photographs taken with digital MILC by IM, S. d’Oleire-Oltmanns and D. Peter.
time and repeat rate to the objectives of a survey. Images may be taken in intervals of minutes, days, or years depending on the requirements of a project. This allows the SFAP photographer to react timely when monitoring changes in a landscape (Fig. 2-6; see also Fig. 16-2). Even if time series are not required, the high temporal flexibility of SFAP in choosing a precise acquisition time is a great advantage over other remote-sensing systems. Deciduous vegetation, as an example, is strongly seasonal in character, and this situation may be exploited for identification of plant types (Fig. 2-7). Finally, the term radiometric resolution refers to the number of digital levels, also called precision or image depth, that the sensor uses for recording different intensities of radiation. For display devices and most standard image file format, 0–255 or 28 (8-bit) digital levels per image band are usual. Most SFAP camera sensors, however, record 12–16 bits for the raw image (see Chap. 6).
of tall objects near the edge of a vertical photograph, particularly for wide-angle fields of view (Fig. 2-8). The height of a tall vertical object may be calculated from its relief displacement, and the height of tall objects also may be determined from measurements of shadows.
2-2.3 Relief Displacement The camera lens operates much like the human eye, both of which produce single-point perspective views of the scene. This perspective causes increasing relief displacement of objects nearing the edge of view (see Chap. 3) and is most noticeable in vertical airphotos, because tall objects appear to lean away from the photo center. Conversely low objects are displaced toward the center. Relief displacement is minimal near the photo center and becomes extreme at the edge. This allows for a side view
Fig. 2-7 Vertical kite photograph in visible light showing water pools and vegetated hummocks in the central portion of Männikjärve Bog, Estonia. Moss species display distinctive green, gold, and red early autumn colors along with pale green dwarf pine trees on hummocks. These dramatic colors are not so distinct at other times of the year; precise timing of image acquisition is therefore highly valuable. Field of view ~60 m across. Taken with a compact digital camera; based on Aber et al. (2002).
16
2. Basic Principles of SFAP
Fig. 2-8 Wide-angle, vertical view over conifer forest at Kojšovská hol’a in southeastern Slovakia. Note how trees appear to lean away from photo center toward outer edges of the scene. Kite photo taken with a compact digital camera.
2-2.4 Stereoscopic Images Humans see in three dimensions, in other words depth perception, because our eyes provide overlapping fields of view from slightly different vantage points. The amount of depth perception in humans is limited to about 400 m distance, however, because of the relatively close spacing of our eyes only 6–7 cm apart (Drury 2001). Stereoscopic photography has been practiced since the middle nineteenth century to provide 3D imagery (Osterman 2007). Aerial
stereo photographs may be taken from widely separated positions (Fig. 2-9); the greater distance in image spacing produces exaggerated depth perception. Such overlapping pictures typically are viewed through a stereoscope (Fig. 2-10) or on-screen with anaglyph glasses or special stereoviewing hardware. Vertical stereophotos are important for visual photointerpretation and are the basis for many photogrammetric techniques (Ogleby 2007; see also Chaps. 3-3 and 11-4).
Fig. 2-9 Pair of oblique stereophotos showing a residential scene in Emporia, Kansas, United States. The pictures were taken simultaneously with two cameras spaced ~1 m apart. Note slight left-right offset in views; compare vehicle in lower left corner and house in lower right corner of each photograph. Kite photos taken with a pair of compact analog cameras.
2-3 Photographic Storage
17
Fig. 2-10 Sokkia mirror stereoscope (MS16). This model is the ideal size for viewing 4 × 6-in. (10 × 15-cm) prints made from 35-mm film or digital images.
2-3 PHOTOGRAPHIC STORAGE SFAP is more than just snapshots; usually the images are intended for long-term storage and reproduction years and even decades after they were acquired. However, neither film nor digital photography is everlasting; all photographic media are subject to long-term decay (Rosenthaler 2007). Thus, proper storage of the images becomes a significant issue for most SFAP projects. Geographic information typically consists of two kinds of data. First is the primary dataset comprised of location information and attribute data about individual features. Second is so-called metadata, which includes such information about the geographic dataset as its grid system, map projection, units of measurement, date of creation, camera model, lens focal length, and history of processing. Aerial photographs are one type of geographic information. The original image itself is the primary dataset. Metadata should contain information about location, date of image acquisition, type of camera and lens, exposure settings, altitude, and other relevant facts. For analog (film, print) photographs, such information could be written directly on the image, so there is no chance the image could be separated from its metadata (see Figs. 1-4 and 1-5). A more common approach is to place metadata on the margins, back, or frame of the photographic medium (Fig. 2-11). Still this approach is often omitted or incomplete for SFAP, and years later nobody would remember the where, when, or what aspects for a photograph. Some camera-related metadata (EXIF header) is built into image files collected with most modern digital cameras. The image file contains metadata, under file properties, such as image dimensions, file size in bytes, date and time the image was taken, type of data compression, and camera model as well as shutter speed, f-stop, and ISO setting. Some camera models also may include GPS data and look direction (Fig. 2-12).
Fig. 2-11 Example of 35-mm color film mounted in a plastic frame (slide). Metadata written on the frame include location, date of acquisition, and picture number. Kite photo taken with a compact analog camera, Estonia.
Fig. 2-12 Close-up view of radio-controlled rig for Nikon Coolpix 16-megapixel digital camera. The AW110 model is waterproof, dustproof, and shockproof. It has built-in GPS, world map, and compass functions, which are recorded in metadata for each image. This setup is designed for use with a tethered and relatively stationary platform, such as kite, blimp, or balloon.
Image file sizes have increased substantially in the first decades of digital photography, resulting in increased storage requirements. Consider a typical 15-min UAS survey with a rather modest 12-megapixel camera. An exposure interval of 5 s, excluding starting and landing phase, would result in about 150 images of ~5½ MB
18
2. Basic Principles of SFAP
jpg file-format size—a total storage space exceeding that of a 700 MB CD, which was a substantial format only 10 years ago. Although the capacity of contemporary storage media continues to increase as well as their miniaturization, their perpetuity remains an important issue. Both digital images and analog photographs are stored in some type of physical medium, the properties of which determine how long the image is likely to survive. The potential longevity of panchromatic (b/w) film and prints is on the order of one century or more; color film and prints have only about half these lifetimes or less (Jensen 2015; Rosenthaler 2007). In contrast, magnetic media, such as disks, tapes, and solid-state devices, last only one or two decades at most, before they degrade under the Earth’s magnetic field. Optical disks (CD, DVD) are thought to survive for a century or more, although they have not been in service long enough to really know their lifespan in practice. Another, perhaps more serious, long-term issue for digital preservation concerns the computer hardware and software necessary to read, transfer, and display digital files stored in a particular medium. Medium types, file storage formats, and computer operating systems have changed rapidly since digital data became commonplace. So, although the magnetic tape, optical disk, external hard drive, or hardware behind the cloud storage service may survive intact, an operational reading device and software to interpret the file format may no longer exist in many cases. High-capacity external hard drives and removable solid-state memory have become commonplace; however, like all electronic media they are subject to failure. Serial failures of computers and memory devices, in fact, have cost more than one researcher substantial datasets representing months or years of work. The option to store data “in the cloud” is also subject to the same potential failures as well as security risks. Large technology companies encourage users to link their devices and sync their personal data via wireless connections. While this may be quite convenient, it means that commercial enterprise now controls much of a person’s private information, which is a scary and dangerous development (Pogue 2014). Burge (2007) emphasized the importance of data migration periodically as new digital media are established and old media become obsolete; such migration is necessary about every 5 years (Rosenthaler 2007). Large companies and governmental agencies may have the capability to transfer and reformat digital files from one medium to another periodically. In one geospatial analysis laboratory, for example, digital image files were updated from original 9-track tapes, to compact cartridge tapes, to zip disks, and finally to optical disks as well as external hard drives. Nearly all people who work with digital imagery and data have experienced at least one hard-drive crash or data loss due to mechanical failure, viruses, or simple human error. Multiple backup files stored on different devices in separate locations provide
some insurance that critical datasets would not be lost. The investment in time and labor to accomplish this is certainly significant, yet necessary. Lawrence’s photograph of San Francisco in Ruins (see Fig. 1-4) has survived for more than a century. One could ask what the chances are for modern digital photographs to survive with their metadata intact for so long (Burge 2007). Recent history suggests that technical innovation and obsolescence will continue to happen rapidly for digital storage devices and file formats. Means of long-term storage for digital imagery is an issue yet to be fully resolved.
2-4 SUMMARY SFAP is based primarily on solar radiation reflected from the Earth’s surface in the visible and near-infrared portions of the spectrum. SFAP is a compromise between ideal photographic conditions and the reality of what is possible to accomplish under natural conditions within logistical constraints and financial limitations. SFAP may be taken in a range of viewing angles from high-oblique to vertical vantages. This facilitates depicting a study area in its broader landscape context as well as in map-like views best suited for accurate measurements. Photographic scale (S) and GSD are closely related, but distinct, concepts dealing with spatial aspects of photographs. Resolution relates to spatial, spectral, temporal, and radiometric aspects of images. Compared to other remote-sensing systems, SFAP offers particularly great advantages with regard to the degree and flexibility of spatial and temporal resolutions. A single photograph has geometric characteristics similar to the image sensed through a human eye, which creates a single-point perspective view. Two overlapping views of the same area result in stereoscopic imagery in which depth perception is apparent. This is the basis for much photointerpretation as well as photogrammetry. Photographic information consists of both the primary image data and metadata about the image. Metadata should include date of image acquisition, location, type of camera and lens, and other relevant information. These metadata should be saved in a manner that is physically difficult to separate from the image data. Various media and file formats have been used over the years for storing photographic images and digital datasets. Among the most long-lived are panchromatic film negatives and optical disks; the least stable are magnetic media. Storage of digital datasets is subject to rapid changes in media types and related computer hardware and software necessary for reading the digital files. Continued technical changes and obsolescence are likely to happen in the near future, which raises questions for long-term archival storage of photographs. Periodic data migration and multiple backup files stored on different devices in separate locations may provide some insurance that critical datasets would not be lost.