- Email: [email protected]
A smart phone-based system for post-earthquake investigations of building damage
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Contents lists available at ScienceDirect
International Journal of Disaster Risk Reduction journal homepage: www.elsevier.com/locate/ijdrr
A smart phone-based system for post-earthquake investigations of building damage ⁎
Zhen Xua, Xinzheng Lub, , Qingle Chengb, Hong Guanc, Li Dengb, Zongcai Zhanga a b c
Beijing Key Laboratory of Urban Underground Space Engineering, School of Civil and Resource Engineering, University of Science and Technology Beijing, Beijing, China Key Laboratory of Civil Engineering Safety and Durability of China Education Ministry, Department of Civil Engineering, Tsinghua University, Beijing, China Griffith School of Engineering, Griffith University Gold Coast Campus, Queensland 4222, Australia
A R T I C L E I N F O
A B S T R A C T
Keywords: Post-earthquake investigation Building damage Smart phones Building-oriented database Data exchange Reverse address retrieval of photos
Current post-earthquake investigations of building damage are mainly conducted manually, making the collection and management of the investigation data inefficient. To overcome this shortcoming, a professional system for post-earthquake investigations of building damage is proposed herein using smart phones. This system adopts a framework which uses smart phones for distributed data collection and a web browser for centralized data management. A building-oriented database is designed to satisfy the requirements of data organization in the investigations. In addition, a data exchange mechanism between smart phones and the web server is created, demonstrating a satisfactory level of fault-tolerance for various use cases of the system. Further, an algorithm of reverse address retrieval of photos based on multi-threads is designed to conveniently index photos by their addresses. The proposed system is used in a virtual investigation of Tsinghua University campus and a practical investigation of the Tibet area during the 2015 Nepal earthquake. These case studies demonstrate that the proposed system offers an intelligentized, efficient and networked tool for post-earthquake investigations of building damage.
1. Introduction Professional teams (e.g., academics and engineers specialized in building structures) are always sent to the earthquake-hit areas to investigate the seismic damage of buildings after an earthquake [1,2], such as the Wenchuan, Ya’an and Ludian earthquakes in China [3–5], the Canterbury earthquake in New Zealand [6], and the Tohoku earthquake in Japan [7]. Such post-earthquake investigations of building damage can collect disaster information and assess the damage of building structures timely, which is very important for earthquake engineering research and post-disaster recovery and reconstruction activities. Current post-earthquake investigations of building damage are conducted manually involving a combined effort of completing required forms and taking relevant photos [1–7]. For example, the United States published the ATC-20 placard system which provides the investigation forms for assessing post-earthquake safety of buildings [8]. Similar investigation forms and working procedure have also been adopted in Greece [9]. In addition, plenty of photos are required to be taken for each building to record the details of seismic damage. In view of the current practices [1–8], Xu et al. [10] has summarized their
⁎
drawbacks as follows: (1) the investigation forms have no direct link to the damage photos. Details of building damage are recorded in a large number of photos, which requires extensive additional work to be matched with the investigation forms. It is usual that many photos cannot be identified and are therefore wasted. (2) It is inconvenient to search a photo that is taken at a specific place or on a specific target due to the lack of addresses in the collected photos. (3) The investigation data cannot be timely shared. Effective planning of post-disaster recovery activities reply heavily on the investigation results; however paper-based forms are difficult to be shared as fast as the digital forms, which may delay the decision-making process. As described above, an intelligentized, efficient and networked investigation method is urgently needed. Smart phones can be a great alternative to solve the abovementioned problems for post-earthquake investigations of building damage. Firstly, smart phones can be used to complete the investigation forms and collect multi-media data of seismic damage of buildings (e.g., photos, audios and videos). In addition, these forms and the collected multi-media data can be easily linked by using smart phones, thereby reducing the time-consuming matching work. Secondly, the collected photos can be indexed by their locations because the global positioning
Correspondence to: Department of Civil Engineering, Tsinghua University, Beijing 100084, China. E-mail address: [email protected] (X. Lu).
http://dx.doi.org/10.1016/j.ijdrr.2017.10.008 Received 18 April 2017; Received in revised form 29 September 2017; Accepted 4 October 2017 2212-4209/ © 2017 Elsevier Ltd. All rights reserved.
Please cite this article as: Xu, Z., International Journal of Disaster Risk Reduction (2017), http://dx.doi.org/10.1016/j.ijdrr.2017.10.008
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 1. The framework of the proposed post-earthquake investigation system.
Zhao et al. [31] developed a mobile application for earthquake intensity survey, through which the form of intensity survey can be filled in online. However, the abovementioned researches [16–31] do not provide necessary connections between the collected data and the corresponding buildings, which will makes post-earthquake investigations very inefficient. Moreover, the existing researches only focused on collecting the disaster data, with seldom consideration of the subsequent management of the collected data (e.g., indexing and visualization), making it difficult for the collected data to be used directly for post-earthquake decision-making. In summary, the issues with data collection and management in post-earthquake investigations have not been fully addressed yet. In this study, a professional system for post-earthquake investigations of building damage is proposed using smart phones. This system adopts a framework which uses smart phones for distributed data collection and a web browser for centralized data management. A building-oriented database is designed in this system to establish the link between the collected data and the corresponding buildings, thereby avoiding the time-consuming data identification work. A data exchange mechanism between smart phones and the web server is also created, with a satisfactory level of fault-tolerance for various use cases of the system. Furthermore, an algorithm of reverse address retrieval of photos based on multi-threads is designed to conveniently index photos by their addresses. This system is used in a virtual investigation of Tsinghua University campus and a practical investigation of the Tibet area in the 2015 Nepal earthquake. These case studies demonstrate that the proposed system offers an intelligentized, efficient and networked tool for the post-earthquake investigations of building damage.
system (GPS) coordinates of the photos can be stored by smart phones [11]. According to the latest analysis report of GPS performance which is published by the University of Texas at Austin in 2017 [12], the position service of GPS is available in more than 99% of the earth's surface. Therefore, the seismic zones can be mostly covered by GPS. Finally, post-earthquake investigation data can be quickly uploaded to the servers through the network of smart phones, so that seismic damage data can be timely shared to support decision making for postdisaster recovery. It is worth noting that post-earthquake investigations of building damage are normally performed after the completions of emergency rescues, so the mobile networks of the disaster area can generally be recovered during the investigations. For instance, in the Wenchuan earthquake with a magnitude of M8.0, it only took four days to recover the mobile network [13]. In the recent Jiuzhaigou earthquake (M7.0) occurring on Aug 8th, 2017, the mobile network was recovered within 24 h [14]. Therefore, post-earthquake investigations of building damage for the abovementioned earthquakes can be performed using mobile networks. In addition, the mobile network to date is fast enough to meet the transfer requirements of multi-media data in the investigation. For instance, the theoretical downlink and uplink rates of TD-LTE (Time Division-Long Term Evolution), a widely-used 4G network for smart phones, can be up to 100 Mbps and 50 Mbps [15], respectively. The applications of smart phones for collecting disaster data in earthquakes have been reported in the literatures [16–26]. For example, in the United States, Federal Emergency Management Agency (FEMA) also released an application (i.e., FEMA App) with a function of “submit disaster photos” [27], through which the post-earthquake photos can be collected from different app users. The UC Berkeley iShake project designed a mobile client-backend server architecture that uses sensor-equipped mobile devices to measure earthquake ground shaking [28]. In Japan, Lwin and Murayama developed a smart phone application based on web geographic information systems (GIS) [29], which can timely display photos from the disaster area. In China, Peng et al. [30] developed a smart phone software called "E-Explorer" to discover the positions of survivors for rescue workers. More recently,
2. Framework The framework of the proposed system includes five components: smart phones, a web browser, a web server, a database, and cloud services, as shown in Fig. 1. Among them, smart phones and the database are used to collect and store the seismic damage data of buildings 2
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
The architecture of the database is determined according to the relationship of the aforementioned six types of data. If the number of the multi-media data related to a building is N, the relations between the multi-media data and the building are N:1. A building has only one location, so the relation of the location and the building is 1:1. Since a user can investigate many buildings and a building can also be investigated by many users, so the relation between buildings and users is L:M, where L and M are the numbers of the buildings and users, respectively. Given that the network database [32] is mainly suitable for a corresponding relationship of 1:N, such a network database cannot be adopted in the proposed system. Also, hierarchical database [33] is not suitable for this system either because the investigation data have no top-down hierarchical relationship. As such, a relational database [34,35] is chosen for this system. In the relational database, each type of data is created as an entity and each entity has its own storage scheme. Using the relational database, complex relationships of all data can be described by several two-entity corresponding relationships, which is quite efficient for data indexing. (2) Implementation Being one of the most widely-used relational databases, MySQL [36] is adopted in the proposed system. The designed data structure of database is shown in Fig. 2. Entity building is the center of all entities. The relationships between entities building and video, photo and audio are all 1:N. The relationships between entities building and user is L:M. In MySQL, an entity building is established to store the data from the forms of post-earthquake investigations. It should be noted that each building has a unique ID. The entities photo, audio and video are established to store the collected photos, audios and videos in an investigation. These entities include three common items of key information: file ID, file path and building ID. Every file is named with a unique ID. The files are stored in a certain path in the computer in which the database is located. Thus, through the file ID and file path, the database can access any collected files. In order to establish the corresponding relation to buildings, the entities (i.e. photo, audio and video) must have a building ID. It should be noted that if a building's ID changes, all entities with this building ID should be changed accordingly. An entity user is created with a unique user ID for each user. To establish the L:M corresponding relationship between user and building, the entity building needs to include a group of user IDs and the entity user also needs to have a group of building IDs. In addition, the entity location with a unique ID is also created to store the GPS coordinates of buildings. It should be noted that photos also have their GPS coordinates, but these coordinates of photos are stored in the entity photo rather than the entity location. An entity building has a unique location ID, because a building corresponds to a specific location. According to the building IDs, The designed database can link all the collected data with buildings to form a structuralized dataset.
in post-earthquake investigations, respectively; the web browser is used to manage and display these collected data, while the web server is used for data exchange and processing; the cloud services will provide the map platform and address data for this system. The distributed data collection and centralized data management can be easily achieved in this framework. Specifically, users can use many smart phones to perform parallel data collecting tasks for a high efficient investigation; since all of the collected data are transferred to the server, the integrated investigation results can be timely provided to the earthquake emergency command center by using a web browser. In view of the above, this framework is very suitable for post-earthquake investigations of building damage. To implement the above framework, three critical data related challenges need to be addressed: (1) Data organization Post-earthquake investigations involve multi-source data of various types. For example, there may be a large number of users and each of whom may collect different types of data (e.g., building information, positions, photos, videos and audios). How to organize these complicated data is a rather challenging task. As buildings are the basic elements for data collection during the investigation, a building-oriented database is necessary to be established to address the data organization problem. (2) Data exchange Post-earthquake investigations may involve many users and each user may have different working scenarios of data exchanges between smart phones and the web server. Such complicated data exchanges can potentially cause data conflicts or repetitions. For example, two users may submit different names for the same building to the web server, or a user uploads the same photo twice. Therefore, a data exchange mechanism between smart phones and the web server with a satisfactory level of fault-tolerance need to be established. (3) Data indexing Photos are the most important data for recording building damage in a post-earthquake investigation. It is a challenging task to conveniently index photos by their addresses, because the GPS coordinates included in the photos are difficult to be used as search words for users. Generally, people are familiar with addresses rather than the GPS coordinates, hence a reverse address retrieval of photos is necessary to convert the GPS coordinates into addresses. In addition, since a large number of photos are taken in a postearthquake investigation, the process of the reverse address retrieval of photos must be efficient so that the investigation results can be shared timely with the earthquake emergency command center. 3. Key techniques 3.1. A building-oriented database (1) Database design The types of data that will be stored in the database are determined first. There are six types of data in the proposed system: building, user, location, photo, audio and video. Specifically, the building data includes the information of the investigation forms, while the user data includes the name, ID, account, passwords and so on. In this system, smart phones will send the GPS coordinates to the database when a building is investigated. Thus, the location data in the database includes the GPS coordinates and recording time, which are only used to record the locations of buildings. In addition, substantial amount of multi-media data of buildings (i.e., photos, audios and videos) are also generated during an investigation. These data includes files and their associated basic information (e.g., name, ID, stored path of files).
3.2. Data exchange mechanism The proposed system involves data exchange between smart phones, a web server, a web browser, a database, and cloud services, as shown in Fig. 1. The cloud service providers (e.g., Baidu map [37] and Amap [38]) have the fixed interface for data exchange with the web server. In addition, a widely-used third-party framework Primefaces of JavaServer Faces (JSF) [39] is used for communication between the web server and browsers. Therefore, the present work focuses on the data exchange mechanism between smart phones, the web server and a database. In this system, the web server is used as a data exchange hub between smart phones and database for fault-tolerant processing. Specifically, only the web server can access the database, while smart 3
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 2. The designed building-oriented database.
the buildings are checked to identify repeated buildings. Specially, if the distances between the new building and the existed buildings are smaller than 5 m, this new building is considered to be potentially repetitive. Subsequently, this event will also be sent to the pending list in the web server and await a manual check. Finally, if a building is confirmed to be repeated by the web server, this new building object thus adopts the existed building ID. Any building data conflicts (Case B) and any repetitions of the collected files (Case C) also need to be checked, as shown in Fig. 3. If the new building is confirmed not to be repeating through the web server, the new building will be assigned to a new building ID and the related data will be written to the database. If the data of a building need to be modified (Case B), it is necessary to check data conflicts. Firstly, the building object data should be traversed to identify conflicts; secondly, the conflict events will be added to the pending list in the web server; finally, after the manual judgment through the web server, the confirmed data will be written to the database, as shown in Fig. 3. When a new file is uploaded to the web server (Case C), it is necessary to check whether the uploaded file is repeated. The message-digest algorithm 5 (MD5), a widely-used method to verify data uniqueness [41], is used to uncover the repeated files. If the uploaded file is repeated, the file will be rejected to be stored in the database and a reminder message will be sent to the smart phone, as shown in Fig. 3. When a building object needs to be removed (Case D), the deleting event will be sent to the pending list and the web server will decide whether the building should be deleted. Through this mechanism, the uncertain events will be confirmed manually in the web server, while the certain events (such as file duplication) will be processed automatically. Therefore, this mechanism can balance the caution and efficiency in the process of data exchange. (2) Implementation A method suitable for transferring large files is adopted in the proposed system to guarantee highly-efficient transfer of photos, videos and audios from smart phones to the web server. Generally, the socket mechanism [42] is a widely-used method for the communications between smart phones and a web server. Due to the weakness in transferring large files, the socket mechanism is thus not suitable for the proposed system. As an alternative, an open-
phones are not allowed to access the database directly to avoid incorrect data operation. In general, smart phones firstly upload the collected data to the web server, and then the web server will process the uploaded data (e.g., the fault-tolerance judgment and reverse address retrieval of photos). Finally, the web server will write the processed data to the database. The above process is implemented by a data exchange mechanism considering fault-tolerance, as follows: (1) A data exchange mechanism considering fault-tolerance In the designed data exchange mechanism, the fault-tolerant algorithm is only executed in the web server but do not affect the data collection work in smart phones. A pending list including the events to be confirmed is built in the web server. All of the events that involve potential conflicts due to the uploaded data of smart phones will be added to this list. Before the events are confirmed in the pending list, the corresponding data will not be written to the database, but data collection of smart phones can continue without any influence. In addition, if the networks are unavailable for smart phones, the tasks of collecting data can still be performed, but the process of uploading data will be paused. As soon as the networks are recovered, the collected data in smart phones can then be uploaded to the web server with the fault-tolerant processing. After the conflict events are confirmed in the web server, smart phones and the database will be synchronized to update the confirmed data. There are four use cases for data exchange between smart phones and a web server: creating a new building object (Case A), modifying the building data (Case B), uploading the collected files (Case C) and deleting a building object (Case D), as shown in Fig. 3. When a new building object is established (Case A), a check is required as whether the same building object has already existed. In this system, the accuracy of distinguishing between different building locations is 5 m considering the positioning accuracy of GPS in a smart phone [40]. If the distance between the two buildings is smaller than 5 m, the buildings can be distinguished by their names in the investigation. Therefore, a double-check method using the names and locations of buildings is proposed to identify the repeated buildings. Firstly, the name of the new building is indexed to check whether there is a same name in the database. If yes, the event will be sent to the pending list for manual check. Otherwise, the locations of all 4
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 3. The data exchange mechanism with fault-tolerance.
send these coordinates to the interface of map servers (e.g., Baidu map [37] and Amap [38]) for requesting the addresses of photos; (c) write the returned addresses to the photo entity in the database. Although the map servers can directly provide the addresses according to the GPS coordinates, it still remains a challenging task to match the returned addresses with the photos. Note that the returning time of addresses is uncertain as it is determined by the status of the network and the map servers, so the sequencing of the returned addresses is not consistent with that of the requests, leading to potentially wrong matches between the addresses and the photos. To achieve an accurate match, a two-step strategy of retrieving the addresses of the photos is designed. The first step is to obtain the IDs of the photos from the database and send the GPS coordinates as well as the photo IDs to the map servers. The second step is to accept the returned addresses and IDs from the map servers, and write the addresses to the corresponding photo entities in the database according to their IDs. Employing the photo IDs can minimize the possibility of wrong matching caused by the uncertainties in the returning time of addresses. In addition, the reverse address retrieval of photos can be parallelized by multi-threads. Specifically, each retrieving task of a photo will be sent to the web server by a uniform resource locator (URL) request. In the web server, each
source framework AsyncHttpClient [43] is used to transfer data between smart phones and the web server. In smart phones, the hypertext transfer protocol (HTTP) requests of data transfer are sent to the web server using the Get and Post method in the AsyncHttpClient [43]; in the web server, many kinds of Servlet programs for various HTTP requests are developed to accept the uploaded data from smart phones. The Servlet programs of the AsyncHttpClient are suitable for large files and easy for multithread transfer [43], so they can meet the requirements of uploading large files by multiple users. In addition, a widely-used Java database connectivity (JDBC) standard [44] is adopted as the communication method between the web and the database in this system. 3.3. Reverse address retrieval of photos To obtain the addresses from the collected photos, an algorithm for reverse address retrieval of photos is proposed and the corresponding computer code is also implemented, as follows: (1) The algorithm for reverse address retrieval of photos Generally, there are three steps to obtain the addresses of photos: (a) obtain the longitude and latitude coordinates of the photos; (b) 5
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 4. The algorithm of reverse address retrieval of photos.
Fig. 5. Data collection based on the designed building-oriented database.
Fig. 6. Searching photos based on their addresses.
6
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 7. Distribution of building seismic damage in a smart phone and a web browser.
Fig. 8. Photos of seismic damage collected in the Tibet area by the proposed system.
Fig. 9. The cross searching with address and seismic damage level.
photo from the smart phone. The photo will be analyzed to have its basic data (i.e. GPS coordinates, user and shoot time) extracted from the exchangeable image file (EXIF) format which is used to store the basic attributes of the photos [45]. Then the thread will check if the uploaded photo is the same as the one already existed. If so, a reminder message will be sent to the user to terminate the thread. If not, the photo will be stored by the database with a
request will be assigned to a thread to obtain the addresses of the photos from the map servers. Such multi-thread method can process excessive number of photos to be uploaded by many users at the same time in post-earthquake investigations. The detailed algorithm of the reverse address retrieving of photos is shown in Fig. 4. In the web server, each HTTP request will be assigned to a thread which will perform the task of uploading the 7
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 10. The distribution of building seismic damage with photos displayed on the map.
maintained during the process of data exchange. Meanwhile, a satisfactory level of fault-tolerance was achieved by the proposed system. Therefore, the designed data exchange mechanism is proven to be suitable for data exchange in the investigation. Through the data processing in the web server, the addresses of the uploaded photos are obtained, which enables more convenient photo indexing. As shown in Fig. 6, all of the photos taken in Tsinghua campus can be found by searching the keyword “Tsinghua” as the address in this system. In addition, the photos can also be indexed by other options, e.g., structural type, user, shoot time and building names, which increases the efficiency of managing post-earthquake investigation data. In the investigation of building seismic damage, each building needs to be marked a damage level. The proposed system is able to visualize the distribution of building seismic damage level which is predicted by Lu and Guan [48] in a map using a web browser or smart phones, as shown in Fig. 7. This added information provides timely and global statistics of seismic damage to the decision-makers for their effective planning of post-disaster recovery.
unique Photo_ID. Finally, the GPS coordinates as well as the Photo_ID will be sent to the map server to obtain the required address. According to the Photo_ID, the returned address will be added to the corresponding photo entity in the database. (2) Implementation In the proposed system, the photos and other files are uploaded using the component FileUpload of Primefaces [39]. The class ImageMetadataReader in Java [46] is used to analyze the EXIF format of photos. In addition, the proposed algorithm of reverse address retrieval of photos involves multi-threads, the function synchronized () in Java [46] is applied before writing the database, to avoid the conflicts of threads. 4. Case studies In this study, two cases (a virtual earthquake in Tsinghua University campus and a real earthquake in the Tibet area) are investigated. It should be mentioned that Case 1 has a good network support for timely data sharing, whereas Case 2 has no 4G network. Case 1. The virtual investigation of Tsinghua University campus
Case 2. A practical investigation of the Tibet area in the 2015 Nepal earthquake
A typical Chinese university campus, Tsinghua University, is considered herein. The campus covers an area of 389.4 ha with more than 600 buildings of different structural types [47,48]. Assuming that the campus is hit by an earthquake, a virtual post-earthquake investigation of building damage can be performed using the proposed system. Firstly, the multi-media data of building seismic damage (e.g., photos, audios and videos) are collected through the app developed in smart phones. The specially designed building-oriented database is used. As shown in Fig. 5, when a building object is created on a smart phone, not only the investigation form can be filled in online, the collected multi-media data can also be automatically linked to the corresponding building, allowing a structuralized dataset to be generated. Comparing with the traditional post-earthquake investigation method, the proposed system eliminates the need of manual identification of the corresponding relationship between the data and the buildings, leading to an improvement in efficiency. Fast and stable 4G network at the Tsinghua campus facilitated a real-time data exchange between smart phones and the web server. The structured relationship between the buildings and data could be
The Tibet area was hit by the Nepal earthquake in April 2015. Building damage in this area has been investigated using the proposed system by the experts from the Ministry of Housing and Urban-Rural Development of China. Given the limited 4G network in the rural area of Tibet, the multi-media data of building damage collected by smart phones cannot be uploaded to the web server directly. The uploading job was done nightly after the experts returned to the base. Nevertheless, the collected data remained the correct matching relationship with the buildings (see Fig. 8), providing an important basis for subsequent data management. Moreover, through reverse address retrieval in the web server, the addresses of the uploaded photos were obtained. On this basis, cross searching of addresses and other relevant information can be performed. For example, as shown in Fig. 9, the building with a damage state of “extensive” in Dingri County can be listed by cross searching the address and the damage state in the system, which is very useful for the management of seismic damage data. In addition, the collected photos can also be displayed on the map according to their addresses, as shown 8
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
in Fig. 10, clearly illustrating the spatial distribution and intuitive details of the seismic damage of the buildings in the area concerned.
[15]
5. Conclusions
[16]
A new smart phone-based system for post-earthquake investigations of building damage is proposed in this study. A virtual case study of Tsinghua campus and a practical investigation of the Tibet area are performed, leading to the following conclusions:
[17]
(1) A framework consisting of smart phones, a web server, a web browser, a database and cloud services is proposed, which is highly suited for distributed collection and centralized management in post-earthquake investigations. (2) A building-oriented database is specially designed to provide a rational organization approach for the collection and storage of the investigation data. (3) A data exchange mechanism between the smart phones and the web server is established, which offers a satisfactory level of fault-tolerance for multiple use cases in an investigation. (4) An algorithm of reverse address retrieval of photos based on multithreads is also developed to conveniently index photos by their addresses, thereby improving the practicability and efficiency of the proposed system. (5) The proposed system provides an intelligentized, efficient and networked tool for seismic damage investigation of buildings, which can assist decision-making of post-disaster recovery and reconstruction activities.
[19]
[18]
[20] [21] [22] [23]
[24]
[25]
[26]
[27] [28]
Acknowledgments
[29]
The authors are grateful for the financial support received from the Beijing Natural Science Foundation (No. 8173057), the National Key Technology R & D Program (No. 2015BAK14B02), and Beijing Municipal Science and Technology Project (No. Z161100001116104).
[30]
[31] [32]
References [33] [1] S. Anagnostopoulos, M. Moretti, Post-earthquake emergency assessment of building damage, safety and usability—part 2: organisation, Soil Dyn. Earthq. Eng. 28 (3) (2008) 233–244. [2] S. Platt, B.D. Drinkwater, Post-earthquake decision making in Turkey: studies of Van and İzmir, Int. J. Disaster Risk Reduct. 17 (2016) 220–237. [3] X.Z. Lu, L.P. Ye, Y.H. Ma, D.Y. Tang, Lessons from the collapse of typical RC frames in Xuankou School during the great Wenchuan Earthquake, Adv. Struct. Eng. 15 (1) (2012) 139–153. [4] Q.X. Meng, W.Y. Xu, Ya’an earthquake of 20 April 2013: introduction and reflections, Nat. Hazards 70 (1) (2014) 941–949. [5] C. Xiong, X.Z. Lu, X.C. Lin, Z. Xu, L.P. Ye, Parameter determination and damage assessment for THA-based regional seismic damage prediction of multi-story buildings, J. Earthq. Eng. 21 (3) (2017) 461–485. [6] K. Elwood, F. Marquis, J. Kim, Post-earthquake assessment and repairability of RC buildings: lessons from Canterbury and emerging challenges, in: Proceedings of the Tenth Pacific Conference on Earthquake Engineering Building an EarthquakeResilient Pacific, 2015. [7] Y. Tsukimata, T. Mukai, H. Kinugasa, Damage evaluation for SRC apartment building suffered from earthquake and investigation on post earthquake functional use, AIJ J. Technol. Des. 22 (50) (2016) 105–108. [8] Applied Technology Council (ATC), ATC-20 Building Safety Evaluation Forms and Placards, USA, 2005. [9] S. Anagnostopoulos, M. Moretti, Post-earthquake emergency assessment of building damage, safety and usability—part 1: technical issues, Soil Dyn. Earthq. Eng. 28 (3) (2008) 223–232. [10] Z. Xu, X.Z. Lu, L. Deng, A.Z. Ren, M.W. Xie, A system for post-earthquake investigation of building damage based on smart phones, in: Proceedings of the 16th International Conference on Computing in Civil and Building Engineering, 2016. [11] R.M. Stokoe, Putting people at the centre of tornado warnings: how perception analysis can cut fatalities, Int. J. Disaster Risk Reduct. 17 (2016) 137–153. [12] B.A. Renfro, A. Terry, N. Boeker, An Analysis of Global Positioning System (GPS) Standard Positioning System (SPS) Performance for 2016, 2017, USA. [13] NetEase, China Mobile has Recovered the Mobile Network of Earthquake Disaster in Sichuan Province. 〈http://tech.163.com/08/0516/11/4C2FEBIQ00092MTQ. html〉. (Accessed 20 September 2017) (in Chinese). [14] Tecent, China Mobile Guarantees the Mobile Network for Jiuzhaigou Earthquake.
[34]
[35] [36] [37] [38] [39] [40]
[41] [42]
[43] [44]
[45] [46] [47] [48]
9
〈http://tech.qq.com/a/20170809/023907.htm〉. (Accessed 20 September 2017) (in Chinese). Wikipedia, LTE. 〈https://en.wikipedia.org/wiki/LTE_(telecommunication)/〉. (Accessed 13 April 2016). K. Yamazaki, A. Kawaguchi, R. Era, Information collecting measures during the earthquake disaster using smartphone and other tools, in: Proceedings of the 20th ITS World Congress, 2013. H.M. Bland, J.D. Frost, Opportunities and considerations for smartphone applications and mobile social media in post extreme event reconnaissance data collection, in: Proceedings of the Sixth Congress on Forensic Engineering, 2012. F. Finazzi, A. Fasso', Earthquake monitoring using volunteer smartphone-based sensor networks, in: Proceedings of the Joint Meeting of METMAVII and GRASPA14, 2014. H. Yuze, Y. Qian, N. Suzuki, Development of smartphone application for off-line use in case of disaster, in: Proceedings of the 27th IEEE International Conference on Advanced Information Networking and Applications Workshops, 2013. M.H. Cheng, S. Wu, T.H. Heaton, J.L. Beck, Earthquake early warning application to buildings, Eng. Struct. 60 (2) (2014) 155–164. Q. Kong, Smartphone-based networks for earthquake detection, in: Proceedings of the International Conference on Supercomputing, 2015. M. Kaigo, Social media usage during disasters and social capital: Twitter and the Great East Japan earthquake, Keio Commun. Rev., No. 34, 2012. Z. Zhu, S. German, I. Brilakis, Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation, Autom. Constr. 20 (7) (2011) 874–883. Q. Kong, R.M. Allen, L. Schreier, Y.W. Kwon, MyShake: a smartphone seismic network for earthquake early warning and beyond, Sci. Adv. 2 (2) (2016) e1501055. W. Shan, J. Feng, J. Chang, F. Yang, Z. Li, Collecting earthquake disaster area information using smart phone, in: Proceedings of the International Conference on System Science and Engineering, 2012. Y.R. He, D.J. Zhang, Y.H. Fang, Development of a mobile post-disaster management system using free and open source technologies, Int. J. Disaster Risk Reduct. (2017), http://dx.doi.org/10.1016/j.ijdrr.2017.08.007. Federal Emergency Management Agency (FEMA), FEMA App. 〈https://www.fema. gov/mobile-app/〉. (Accessed 5 May 2015). J. Reilly, S. Dashti, M. Ervasti, J.D. Bray, S.D. Glaser, A.M. Bayen, Mobile phones as seismologic sensors: automating data extraction for the iShake system, IEEE Trans. Autom. Sci. Eng. 10 (2) (2013) 242–251. K.K. Lwin, Y. Murayama, Web-based GIS system for real-time field data collection using personal mobile phone, J. Geogr. Inf. Syst. 3 (4) (2011) 382–389. D. Peng, X. Zhao, Q. Zhao, Y. Yu, Smartphone based public participant emergency rescue information platform for earthquake zone – "E-Explorer", in: Proceedings of the International Conference on Vibroengineering, 2015. X. Zhao, Y. Yu, Research on quick seismic damage investigation using smartphone, in: Proceedings of the SPIE, 2016. D. Papadias, J. Zhang, N. Mamoulis, Y. Tao, Query processing in spatial network databases, in: Proceedings of the International Conference on Very Large Data Bases, 2003. J. Black, T. Ellis, D. Makris, A hierarchical database for visual surveillance applications, in: Proceedings of the IEEE International Conference on Multimedia and Expo, 2004. C. Vicknair, M. Macias, Z. Zhao, X. Nan, Y. Chen, D. Wilkins, A comparison of a graph database and a relational database: a data provenance perspective, in: Proceedings of the Southeast Regional Conference, ACM, 2010. C. Tisell, K. Orsborn, A system for multibody analysis based on object-relational database technology, Adv. Eng. Softw. 31 (12) (2000) 971–984. Oracle, MySQL. 〈https://www.mysql.com/〉. (Accessed 6 January 2015). Baidu, Develop Platform of Baidu Map. 〈http://lbsyun.baidu.com/〉. (Accessed 6 February 2015) (in Chinese). Amap, Develop Platform of Amap. 〈http://lbs.amap.com/〉. (Accessed 6 February 2015) (in Chinese). Primefaces, PrimeFaces: Ultimate UI framework for Java EE. 〈http://primefaces. org/〉. (Accessed 23 February 2015). National Coordination Office for Space-Based Positioning, Navigation, and Timing, GPS Accuracy. 〈http://www.gps.gov/systems/gps/performance/accuracy/〉. (Accessed 23 February 2016). Wikipedia, MD5. 〈https://en.wikipedia.org/wiki/MD5/〉. (Accessed 11 January 2015). Javaworld, Sockets Programming in Java: A tutorial. 〈http://www.javaworld.com/ article/2077322/core-java/core-java-sockets-programming-in-java-a-tutorial. html〉. (Accessed 10 January 2015). Github, AsyncHttpClient. 〈https://github.com/AsyncHttpClient/async-http-client/ 〉. (Accessed 10 January 2015). Oracle, The Java Database Connectivity (JDBC). 〈http://www.oracle.com/ technetwork/java/javase/tech/index-jsp-136101.html〉. (Accessed 12 January 2015). Wikipedia, EXIF. 〈https://en.wikipedia.org/wiki/Exif/〉. (Accessed 4 January 2015). Oracle, Java SE Documentation. 〈http://www.oracle.com/technetwork/java/ javase/documentation/index.html〉. (Accessed 12 December 2014). X. Zeng, X.Z. Lu, T. Yang, Z. Xu, Application of the FEMA-P58 methodology for regional earthquake loss prediction, Nat. Hazards 83 (1) (2016) 177–192. X.Z. Lu, H. Guan, Earthquake Disaster Simulation of Civil Infrastructures: From Tall Buildings to Urban Areas, Springer, 2017.
Contents lists available at ScienceDirect
International Journal of Disaster Risk Reduction journal homepage: www.elsevier.com/locate/ijdrr
A smart phone-based system for post-earthquake investigations of building damage ⁎
Zhen Xua, Xinzheng Lub, , Qingle Chengb, Hong Guanc, Li Dengb, Zongcai Zhanga a b c
Beijing Key Laboratory of Urban Underground Space Engineering, School of Civil and Resource Engineering, University of Science and Technology Beijing, Beijing, China Key Laboratory of Civil Engineering Safety and Durability of China Education Ministry, Department of Civil Engineering, Tsinghua University, Beijing, China Griffith School of Engineering, Griffith University Gold Coast Campus, Queensland 4222, Australia
A R T I C L E I N F O
A B S T R A C T
Keywords: Post-earthquake investigation Building damage Smart phones Building-oriented database Data exchange Reverse address retrieval of photos
Current post-earthquake investigations of building damage are mainly conducted manually, making the collection and management of the investigation data inefficient. To overcome this shortcoming, a professional system for post-earthquake investigations of building damage is proposed herein using smart phones. This system adopts a framework which uses smart phones for distributed data collection and a web browser for centralized data management. A building-oriented database is designed to satisfy the requirements of data organization in the investigations. In addition, a data exchange mechanism between smart phones and the web server is created, demonstrating a satisfactory level of fault-tolerance for various use cases of the system. Further, an algorithm of reverse address retrieval of photos based on multi-threads is designed to conveniently index photos by their addresses. The proposed system is used in a virtual investigation of Tsinghua University campus and a practical investigation of the Tibet area during the 2015 Nepal earthquake. These case studies demonstrate that the proposed system offers an intelligentized, efficient and networked tool for post-earthquake investigations of building damage.
1. Introduction Professional teams (e.g., academics and engineers specialized in building structures) are always sent to the earthquake-hit areas to investigate the seismic damage of buildings after an earthquake [1,2], such as the Wenchuan, Ya’an and Ludian earthquakes in China [3–5], the Canterbury earthquake in New Zealand [6], and the Tohoku earthquake in Japan [7]. Such post-earthquake investigations of building damage can collect disaster information and assess the damage of building structures timely, which is very important for earthquake engineering research and post-disaster recovery and reconstruction activities. Current post-earthquake investigations of building damage are conducted manually involving a combined effort of completing required forms and taking relevant photos [1–7]. For example, the United States published the ATC-20 placard system which provides the investigation forms for assessing post-earthquake safety of buildings [8]. Similar investigation forms and working procedure have also been adopted in Greece [9]. In addition, plenty of photos are required to be taken for each building to record the details of seismic damage. In view of the current practices [1–8], Xu et al. [10] has summarized their
⁎
drawbacks as follows: (1) the investigation forms have no direct link to the damage photos. Details of building damage are recorded in a large number of photos, which requires extensive additional work to be matched with the investigation forms. It is usual that many photos cannot be identified and are therefore wasted. (2) It is inconvenient to search a photo that is taken at a specific place or on a specific target due to the lack of addresses in the collected photos. (3) The investigation data cannot be timely shared. Effective planning of post-disaster recovery activities reply heavily on the investigation results; however paper-based forms are difficult to be shared as fast as the digital forms, which may delay the decision-making process. As described above, an intelligentized, efficient and networked investigation method is urgently needed. Smart phones can be a great alternative to solve the abovementioned problems for post-earthquake investigations of building damage. Firstly, smart phones can be used to complete the investigation forms and collect multi-media data of seismic damage of buildings (e.g., photos, audios and videos). In addition, these forms and the collected multi-media data can be easily linked by using smart phones, thereby reducing the time-consuming matching work. Secondly, the collected photos can be indexed by their locations because the global positioning
Correspondence to: Department of Civil Engineering, Tsinghua University, Beijing 100084, China. E-mail address: [email protected] (X. Lu).
http://dx.doi.org/10.1016/j.ijdrr.2017.10.008 Received 18 April 2017; Received in revised form 29 September 2017; Accepted 4 October 2017 2212-4209/ © 2017 Elsevier Ltd. All rights reserved.
Please cite this article as: Xu, Z., International Journal of Disaster Risk Reduction (2017), http://dx.doi.org/10.1016/j.ijdrr.2017.10.008
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 1. The framework of the proposed post-earthquake investigation system.
Zhao et al. [31] developed a mobile application for earthquake intensity survey, through which the form of intensity survey can be filled in online. However, the abovementioned researches [16–31] do not provide necessary connections between the collected data and the corresponding buildings, which will makes post-earthquake investigations very inefficient. Moreover, the existing researches only focused on collecting the disaster data, with seldom consideration of the subsequent management of the collected data (e.g., indexing and visualization), making it difficult for the collected data to be used directly for post-earthquake decision-making. In summary, the issues with data collection and management in post-earthquake investigations have not been fully addressed yet. In this study, a professional system for post-earthquake investigations of building damage is proposed using smart phones. This system adopts a framework which uses smart phones for distributed data collection and a web browser for centralized data management. A building-oriented database is designed in this system to establish the link between the collected data and the corresponding buildings, thereby avoiding the time-consuming data identification work. A data exchange mechanism between smart phones and the web server is also created, with a satisfactory level of fault-tolerance for various use cases of the system. Furthermore, an algorithm of reverse address retrieval of photos based on multi-threads is designed to conveniently index photos by their addresses. This system is used in a virtual investigation of Tsinghua University campus and a practical investigation of the Tibet area in the 2015 Nepal earthquake. These case studies demonstrate that the proposed system offers an intelligentized, efficient and networked tool for the post-earthquake investigations of building damage.
system (GPS) coordinates of the photos can be stored by smart phones [11]. According to the latest analysis report of GPS performance which is published by the University of Texas at Austin in 2017 [12], the position service of GPS is available in more than 99% of the earth's surface. Therefore, the seismic zones can be mostly covered by GPS. Finally, post-earthquake investigation data can be quickly uploaded to the servers through the network of smart phones, so that seismic damage data can be timely shared to support decision making for postdisaster recovery. It is worth noting that post-earthquake investigations of building damage are normally performed after the completions of emergency rescues, so the mobile networks of the disaster area can generally be recovered during the investigations. For instance, in the Wenchuan earthquake with a magnitude of M8.0, it only took four days to recover the mobile network [13]. In the recent Jiuzhaigou earthquake (M7.0) occurring on Aug 8th, 2017, the mobile network was recovered within 24 h [14]. Therefore, post-earthquake investigations of building damage for the abovementioned earthquakes can be performed using mobile networks. In addition, the mobile network to date is fast enough to meet the transfer requirements of multi-media data in the investigation. For instance, the theoretical downlink and uplink rates of TD-LTE (Time Division-Long Term Evolution), a widely-used 4G network for smart phones, can be up to 100 Mbps and 50 Mbps [15], respectively. The applications of smart phones for collecting disaster data in earthquakes have been reported in the literatures [16–26]. For example, in the United States, Federal Emergency Management Agency (FEMA) also released an application (i.e., FEMA App) with a function of “submit disaster photos” [27], through which the post-earthquake photos can be collected from different app users. The UC Berkeley iShake project designed a mobile client-backend server architecture that uses sensor-equipped mobile devices to measure earthquake ground shaking [28]. In Japan, Lwin and Murayama developed a smart phone application based on web geographic information systems (GIS) [29], which can timely display photos from the disaster area. In China, Peng et al. [30] developed a smart phone software called "E-Explorer" to discover the positions of survivors for rescue workers. More recently,
2. Framework The framework of the proposed system includes five components: smart phones, a web browser, a web server, a database, and cloud services, as shown in Fig. 1. Among them, smart phones and the database are used to collect and store the seismic damage data of buildings 2
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
The architecture of the database is determined according to the relationship of the aforementioned six types of data. If the number of the multi-media data related to a building is N, the relations between the multi-media data and the building are N:1. A building has only one location, so the relation of the location and the building is 1:1. Since a user can investigate many buildings and a building can also be investigated by many users, so the relation between buildings and users is L:M, where L and M are the numbers of the buildings and users, respectively. Given that the network database [32] is mainly suitable for a corresponding relationship of 1:N, such a network database cannot be adopted in the proposed system. Also, hierarchical database [33] is not suitable for this system either because the investigation data have no top-down hierarchical relationship. As such, a relational database [34,35] is chosen for this system. In the relational database, each type of data is created as an entity and each entity has its own storage scheme. Using the relational database, complex relationships of all data can be described by several two-entity corresponding relationships, which is quite efficient for data indexing. (2) Implementation Being one of the most widely-used relational databases, MySQL [36] is adopted in the proposed system. The designed data structure of database is shown in Fig. 2. Entity building is the center of all entities. The relationships between entities building and video, photo and audio are all 1:N. The relationships between entities building and user is L:M. In MySQL, an entity building is established to store the data from the forms of post-earthquake investigations. It should be noted that each building has a unique ID. The entities photo, audio and video are established to store the collected photos, audios and videos in an investigation. These entities include three common items of key information: file ID, file path and building ID. Every file is named with a unique ID. The files are stored in a certain path in the computer in which the database is located. Thus, through the file ID and file path, the database can access any collected files. In order to establish the corresponding relation to buildings, the entities (i.e. photo, audio and video) must have a building ID. It should be noted that if a building's ID changes, all entities with this building ID should be changed accordingly. An entity user is created with a unique user ID for each user. To establish the L:M corresponding relationship between user and building, the entity building needs to include a group of user IDs and the entity user also needs to have a group of building IDs. In addition, the entity location with a unique ID is also created to store the GPS coordinates of buildings. It should be noted that photos also have their GPS coordinates, but these coordinates of photos are stored in the entity photo rather than the entity location. An entity building has a unique location ID, because a building corresponds to a specific location. According to the building IDs, The designed database can link all the collected data with buildings to form a structuralized dataset.
in post-earthquake investigations, respectively; the web browser is used to manage and display these collected data, while the web server is used for data exchange and processing; the cloud services will provide the map platform and address data for this system. The distributed data collection and centralized data management can be easily achieved in this framework. Specifically, users can use many smart phones to perform parallel data collecting tasks for a high efficient investigation; since all of the collected data are transferred to the server, the integrated investigation results can be timely provided to the earthquake emergency command center by using a web browser. In view of the above, this framework is very suitable for post-earthquake investigations of building damage. To implement the above framework, three critical data related challenges need to be addressed: (1) Data organization Post-earthquake investigations involve multi-source data of various types. For example, there may be a large number of users and each of whom may collect different types of data (e.g., building information, positions, photos, videos and audios). How to organize these complicated data is a rather challenging task. As buildings are the basic elements for data collection during the investigation, a building-oriented database is necessary to be established to address the data organization problem. (2) Data exchange Post-earthquake investigations may involve many users and each user may have different working scenarios of data exchanges between smart phones and the web server. Such complicated data exchanges can potentially cause data conflicts or repetitions. For example, two users may submit different names for the same building to the web server, or a user uploads the same photo twice. Therefore, a data exchange mechanism between smart phones and the web server with a satisfactory level of fault-tolerance need to be established. (3) Data indexing Photos are the most important data for recording building damage in a post-earthquake investigation. It is a challenging task to conveniently index photos by their addresses, because the GPS coordinates included in the photos are difficult to be used as search words for users. Generally, people are familiar with addresses rather than the GPS coordinates, hence a reverse address retrieval of photos is necessary to convert the GPS coordinates into addresses. In addition, since a large number of photos are taken in a postearthquake investigation, the process of the reverse address retrieval of photos must be efficient so that the investigation results can be shared timely with the earthquake emergency command center. 3. Key techniques 3.1. A building-oriented database (1) Database design The types of data that will be stored in the database are determined first. There are six types of data in the proposed system: building, user, location, photo, audio and video. Specifically, the building data includes the information of the investigation forms, while the user data includes the name, ID, account, passwords and so on. In this system, smart phones will send the GPS coordinates to the database when a building is investigated. Thus, the location data in the database includes the GPS coordinates and recording time, which are only used to record the locations of buildings. In addition, substantial amount of multi-media data of buildings (i.e., photos, audios and videos) are also generated during an investigation. These data includes files and their associated basic information (e.g., name, ID, stored path of files).
3.2. Data exchange mechanism The proposed system involves data exchange between smart phones, a web server, a web browser, a database, and cloud services, as shown in Fig. 1. The cloud service providers (e.g., Baidu map [37] and Amap [38]) have the fixed interface for data exchange with the web server. In addition, a widely-used third-party framework Primefaces of JavaServer Faces (JSF) [39] is used for communication between the web server and browsers. Therefore, the present work focuses on the data exchange mechanism between smart phones, the web server and a database. In this system, the web server is used as a data exchange hub between smart phones and database for fault-tolerant processing. Specifically, only the web server can access the database, while smart 3
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 2. The designed building-oriented database.
the buildings are checked to identify repeated buildings. Specially, if the distances between the new building and the existed buildings are smaller than 5 m, this new building is considered to be potentially repetitive. Subsequently, this event will also be sent to the pending list in the web server and await a manual check. Finally, if a building is confirmed to be repeated by the web server, this new building object thus adopts the existed building ID. Any building data conflicts (Case B) and any repetitions of the collected files (Case C) also need to be checked, as shown in Fig. 3. If the new building is confirmed not to be repeating through the web server, the new building will be assigned to a new building ID and the related data will be written to the database. If the data of a building need to be modified (Case B), it is necessary to check data conflicts. Firstly, the building object data should be traversed to identify conflicts; secondly, the conflict events will be added to the pending list in the web server; finally, after the manual judgment through the web server, the confirmed data will be written to the database, as shown in Fig. 3. When a new file is uploaded to the web server (Case C), it is necessary to check whether the uploaded file is repeated. The message-digest algorithm 5 (MD5), a widely-used method to verify data uniqueness [41], is used to uncover the repeated files. If the uploaded file is repeated, the file will be rejected to be stored in the database and a reminder message will be sent to the smart phone, as shown in Fig. 3. When a building object needs to be removed (Case D), the deleting event will be sent to the pending list and the web server will decide whether the building should be deleted. Through this mechanism, the uncertain events will be confirmed manually in the web server, while the certain events (such as file duplication) will be processed automatically. Therefore, this mechanism can balance the caution and efficiency in the process of data exchange. (2) Implementation A method suitable for transferring large files is adopted in the proposed system to guarantee highly-efficient transfer of photos, videos and audios from smart phones to the web server. Generally, the socket mechanism [42] is a widely-used method for the communications between smart phones and a web server. Due to the weakness in transferring large files, the socket mechanism is thus not suitable for the proposed system. As an alternative, an open-
phones are not allowed to access the database directly to avoid incorrect data operation. In general, smart phones firstly upload the collected data to the web server, and then the web server will process the uploaded data (e.g., the fault-tolerance judgment and reverse address retrieval of photos). Finally, the web server will write the processed data to the database. The above process is implemented by a data exchange mechanism considering fault-tolerance, as follows: (1) A data exchange mechanism considering fault-tolerance In the designed data exchange mechanism, the fault-tolerant algorithm is only executed in the web server but do not affect the data collection work in smart phones. A pending list including the events to be confirmed is built in the web server. All of the events that involve potential conflicts due to the uploaded data of smart phones will be added to this list. Before the events are confirmed in the pending list, the corresponding data will not be written to the database, but data collection of smart phones can continue without any influence. In addition, if the networks are unavailable for smart phones, the tasks of collecting data can still be performed, but the process of uploading data will be paused. As soon as the networks are recovered, the collected data in smart phones can then be uploaded to the web server with the fault-tolerant processing. After the conflict events are confirmed in the web server, smart phones and the database will be synchronized to update the confirmed data. There are four use cases for data exchange between smart phones and a web server: creating a new building object (Case A), modifying the building data (Case B), uploading the collected files (Case C) and deleting a building object (Case D), as shown in Fig. 3. When a new building object is established (Case A), a check is required as whether the same building object has already existed. In this system, the accuracy of distinguishing between different building locations is 5 m considering the positioning accuracy of GPS in a smart phone [40]. If the distance between the two buildings is smaller than 5 m, the buildings can be distinguished by their names in the investigation. Therefore, a double-check method using the names and locations of buildings is proposed to identify the repeated buildings. Firstly, the name of the new building is indexed to check whether there is a same name in the database. If yes, the event will be sent to the pending list for manual check. Otherwise, the locations of all 4
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 3. The data exchange mechanism with fault-tolerance.
send these coordinates to the interface of map servers (e.g., Baidu map [37] and Amap [38]) for requesting the addresses of photos; (c) write the returned addresses to the photo entity in the database. Although the map servers can directly provide the addresses according to the GPS coordinates, it still remains a challenging task to match the returned addresses with the photos. Note that the returning time of addresses is uncertain as it is determined by the status of the network and the map servers, so the sequencing of the returned addresses is not consistent with that of the requests, leading to potentially wrong matches between the addresses and the photos. To achieve an accurate match, a two-step strategy of retrieving the addresses of the photos is designed. The first step is to obtain the IDs of the photos from the database and send the GPS coordinates as well as the photo IDs to the map servers. The second step is to accept the returned addresses and IDs from the map servers, and write the addresses to the corresponding photo entities in the database according to their IDs. Employing the photo IDs can minimize the possibility of wrong matching caused by the uncertainties in the returning time of addresses. In addition, the reverse address retrieval of photos can be parallelized by multi-threads. Specifically, each retrieving task of a photo will be sent to the web server by a uniform resource locator (URL) request. In the web server, each
source framework AsyncHttpClient [43] is used to transfer data between smart phones and the web server. In smart phones, the hypertext transfer protocol (HTTP) requests of data transfer are sent to the web server using the Get and Post method in the AsyncHttpClient [43]; in the web server, many kinds of Servlet programs for various HTTP requests are developed to accept the uploaded data from smart phones. The Servlet programs of the AsyncHttpClient are suitable for large files and easy for multithread transfer [43], so they can meet the requirements of uploading large files by multiple users. In addition, a widely-used Java database connectivity (JDBC) standard [44] is adopted as the communication method between the web and the database in this system. 3.3. Reverse address retrieval of photos To obtain the addresses from the collected photos, an algorithm for reverse address retrieval of photos is proposed and the corresponding computer code is also implemented, as follows: (1) The algorithm for reverse address retrieval of photos Generally, there are three steps to obtain the addresses of photos: (a) obtain the longitude and latitude coordinates of the photos; (b) 5
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 4. The algorithm of reverse address retrieval of photos.
Fig. 5. Data collection based on the designed building-oriented database.
Fig. 6. Searching photos based on their addresses.
6
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 7. Distribution of building seismic damage in a smart phone and a web browser.
Fig. 8. Photos of seismic damage collected in the Tibet area by the proposed system.
Fig. 9. The cross searching with address and seismic damage level.
photo from the smart phone. The photo will be analyzed to have its basic data (i.e. GPS coordinates, user and shoot time) extracted from the exchangeable image file (EXIF) format which is used to store the basic attributes of the photos [45]. Then the thread will check if the uploaded photo is the same as the one already existed. If so, a reminder message will be sent to the user to terminate the thread. If not, the photo will be stored by the database with a
request will be assigned to a thread to obtain the addresses of the photos from the map servers. Such multi-thread method can process excessive number of photos to be uploaded by many users at the same time in post-earthquake investigations. The detailed algorithm of the reverse address retrieving of photos is shown in Fig. 4. In the web server, each HTTP request will be assigned to a thread which will perform the task of uploading the 7
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
Fig. 10. The distribution of building seismic damage with photos displayed on the map.
maintained during the process of data exchange. Meanwhile, a satisfactory level of fault-tolerance was achieved by the proposed system. Therefore, the designed data exchange mechanism is proven to be suitable for data exchange in the investigation. Through the data processing in the web server, the addresses of the uploaded photos are obtained, which enables more convenient photo indexing. As shown in Fig. 6, all of the photos taken in Tsinghua campus can be found by searching the keyword “Tsinghua” as the address in this system. In addition, the photos can also be indexed by other options, e.g., structural type, user, shoot time and building names, which increases the efficiency of managing post-earthquake investigation data. In the investigation of building seismic damage, each building needs to be marked a damage level. The proposed system is able to visualize the distribution of building seismic damage level which is predicted by Lu and Guan [48] in a map using a web browser or smart phones, as shown in Fig. 7. This added information provides timely and global statistics of seismic damage to the decision-makers for their effective planning of post-disaster recovery.
unique Photo_ID. Finally, the GPS coordinates as well as the Photo_ID will be sent to the map server to obtain the required address. According to the Photo_ID, the returned address will be added to the corresponding photo entity in the database. (2) Implementation In the proposed system, the photos and other files are uploaded using the component FileUpload of Primefaces [39]. The class ImageMetadataReader in Java [46] is used to analyze the EXIF format of photos. In addition, the proposed algorithm of reverse address retrieval of photos involves multi-threads, the function synchronized () in Java [46] is applied before writing the database, to avoid the conflicts of threads. 4. Case studies In this study, two cases (a virtual earthquake in Tsinghua University campus and a real earthquake in the Tibet area) are investigated. It should be mentioned that Case 1 has a good network support for timely data sharing, whereas Case 2 has no 4G network. Case 1. The virtual investigation of Tsinghua University campus
Case 2. A practical investigation of the Tibet area in the 2015 Nepal earthquake
A typical Chinese university campus, Tsinghua University, is considered herein. The campus covers an area of 389.4 ha with more than 600 buildings of different structural types [47,48]. Assuming that the campus is hit by an earthquake, a virtual post-earthquake investigation of building damage can be performed using the proposed system. Firstly, the multi-media data of building seismic damage (e.g., photos, audios and videos) are collected through the app developed in smart phones. The specially designed building-oriented database is used. As shown in Fig. 5, when a building object is created on a smart phone, not only the investigation form can be filled in online, the collected multi-media data can also be automatically linked to the corresponding building, allowing a structuralized dataset to be generated. Comparing with the traditional post-earthquake investigation method, the proposed system eliminates the need of manual identification of the corresponding relationship between the data and the buildings, leading to an improvement in efficiency. Fast and stable 4G network at the Tsinghua campus facilitated a real-time data exchange between smart phones and the web server. The structured relationship between the buildings and data could be
The Tibet area was hit by the Nepal earthquake in April 2015. Building damage in this area has been investigated using the proposed system by the experts from the Ministry of Housing and Urban-Rural Development of China. Given the limited 4G network in the rural area of Tibet, the multi-media data of building damage collected by smart phones cannot be uploaded to the web server directly. The uploading job was done nightly after the experts returned to the base. Nevertheless, the collected data remained the correct matching relationship with the buildings (see Fig. 8), providing an important basis for subsequent data management. Moreover, through reverse address retrieval in the web server, the addresses of the uploaded photos were obtained. On this basis, cross searching of addresses and other relevant information can be performed. For example, as shown in Fig. 9, the building with a damage state of “extensive” in Dingri County can be listed by cross searching the address and the damage state in the system, which is very useful for the management of seismic damage data. In addition, the collected photos can also be displayed on the map according to their addresses, as shown 8
International Journal of Disaster Risk Reduction xxx (xxxx) xxx–xxx
Z. Xu et al.
in Fig. 10, clearly illustrating the spatial distribution and intuitive details of the seismic damage of the buildings in the area concerned.
[15]
5. Conclusions
[16]
A new smart phone-based system for post-earthquake investigations of building damage is proposed in this study. A virtual case study of Tsinghua campus and a practical investigation of the Tibet area are performed, leading to the following conclusions:
[17]
(1) A framework consisting of smart phones, a web server, a web browser, a database and cloud services is proposed, which is highly suited for distributed collection and centralized management in post-earthquake investigations. (2) A building-oriented database is specially designed to provide a rational organization approach for the collection and storage of the investigation data. (3) A data exchange mechanism between the smart phones and the web server is established, which offers a satisfactory level of fault-tolerance for multiple use cases in an investigation. (4) An algorithm of reverse address retrieval of photos based on multithreads is also developed to conveniently index photos by their addresses, thereby improving the practicability and efficiency of the proposed system. (5) The proposed system provides an intelligentized, efficient and networked tool for seismic damage investigation of buildings, which can assist decision-making of post-disaster recovery and reconstruction activities.
[19]
[18]
[20] [21] [22] [23]
[24]
[25]
[26]
[27] [28]
Acknowledgments
[29]
The authors are grateful for the financial support received from the Beijing Natural Science Foundation (No. 8173057), the National Key Technology R & D Program (No. 2015BAK14B02), and Beijing Municipal Science and Technology Project (No. Z161100001116104).
[30]
[31] [32]
References [33] [1] S. Anagnostopoulos, M. Moretti, Post-earthquake emergency assessment of building damage, safety and usability—part 2: organisation, Soil Dyn. Earthq. Eng. 28 (3) (2008) 233–244. [2] S. Platt, B.D. Drinkwater, Post-earthquake decision making in Turkey: studies of Van and İzmir, Int. J. Disaster Risk Reduct. 17 (2016) 220–237. [3] X.Z. Lu, L.P. Ye, Y.H. Ma, D.Y. Tang, Lessons from the collapse of typical RC frames in Xuankou School during the great Wenchuan Earthquake, Adv. Struct. Eng. 15 (1) (2012) 139–153. [4] Q.X. Meng, W.Y. Xu, Ya’an earthquake of 20 April 2013: introduction and reflections, Nat. Hazards 70 (1) (2014) 941–949. [5] C. Xiong, X.Z. Lu, X.C. Lin, Z. Xu, L.P. Ye, Parameter determination and damage assessment for THA-based regional seismic damage prediction of multi-story buildings, J. Earthq. Eng. 21 (3) (2017) 461–485. [6] K. Elwood, F. Marquis, J. Kim, Post-earthquake assessment and repairability of RC buildings: lessons from Canterbury and emerging challenges, in: Proceedings of the Tenth Pacific Conference on Earthquake Engineering Building an EarthquakeResilient Pacific, 2015. [7] Y. Tsukimata, T. Mukai, H. Kinugasa, Damage evaluation for SRC apartment building suffered from earthquake and investigation on post earthquake functional use, AIJ J. Technol. Des. 22 (50) (2016) 105–108. [8] Applied Technology Council (ATC), ATC-20 Building Safety Evaluation Forms and Placards, USA, 2005. [9] S. Anagnostopoulos, M. Moretti, Post-earthquake emergency assessment of building damage, safety and usability—part 1: technical issues, Soil Dyn. Earthq. Eng. 28 (3) (2008) 223–232. [10] Z. Xu, X.Z. Lu, L. Deng, A.Z. Ren, M.W. Xie, A system for post-earthquake investigation of building damage based on smart phones, in: Proceedings of the 16th International Conference on Computing in Civil and Building Engineering, 2016. [11] R.M. Stokoe, Putting people at the centre of tornado warnings: how perception analysis can cut fatalities, Int. J. Disaster Risk Reduct. 17 (2016) 137–153. [12] B.A. Renfro, A. Terry, N. Boeker, An Analysis of Global Positioning System (GPS) Standard Positioning System (SPS) Performance for 2016, 2017, USA. [13] NetEase, China Mobile has Recovered the Mobile Network of Earthquake Disaster in Sichuan Province. 〈http://tech.163.com/08/0516/11/4C2FEBIQ00092MTQ. html〉. (Accessed 20 September 2017) (in Chinese). [14] Tecent, China Mobile Guarantees the Mobile Network for Jiuzhaigou Earthquake.
[34]
[35] [36] [37] [38] [39] [40]
[41] [42]
[43] [44]
[45] [46] [47] [48]
9
〈http://tech.qq.com/a/20170809/023907.htm〉. (Accessed 20 September 2017) (in Chinese). Wikipedia, LTE. 〈https://en.wikipedia.org/wiki/LTE_(telecommunication)/〉. (Accessed 13 April 2016). K. Yamazaki, A. Kawaguchi, R. Era, Information collecting measures during the earthquake disaster using smartphone and other tools, in: Proceedings of the 20th ITS World Congress, 2013. H.M. Bland, J.D. Frost, Opportunities and considerations for smartphone applications and mobile social media in post extreme event reconnaissance data collection, in: Proceedings of the Sixth Congress on Forensic Engineering, 2012. F. Finazzi, A. Fasso', Earthquake monitoring using volunteer smartphone-based sensor networks, in: Proceedings of the Joint Meeting of METMAVII and GRASPA14, 2014. H. Yuze, Y. Qian, N. Suzuki, Development of smartphone application for off-line use in case of disaster, in: Proceedings of the 27th IEEE International Conference on Advanced Information Networking and Applications Workshops, 2013. M.H. Cheng, S. Wu, T.H. Heaton, J.L. Beck, Earthquake early warning application to buildings, Eng. Struct. 60 (2) (2014) 155–164. Q. Kong, Smartphone-based networks for earthquake detection, in: Proceedings of the International Conference on Supercomputing, 2015. M. Kaigo, Social media usage during disasters and social capital: Twitter and the Great East Japan earthquake, Keio Commun. Rev., No. 34, 2012. Z. Zhu, S. German, I. Brilakis, Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation, Autom. Constr. 20 (7) (2011) 874–883. Q. Kong, R.M. Allen, L. Schreier, Y.W. Kwon, MyShake: a smartphone seismic network for earthquake early warning and beyond, Sci. Adv. 2 (2) (2016) e1501055. W. Shan, J. Feng, J. Chang, F. Yang, Z. Li, Collecting earthquake disaster area information using smart phone, in: Proceedings of the International Conference on System Science and Engineering, 2012. Y.R. He, D.J. Zhang, Y.H. Fang, Development of a mobile post-disaster management system using free and open source technologies, Int. J. Disaster Risk Reduct. (2017), http://dx.doi.org/10.1016/j.ijdrr.2017.08.007. Federal Emergency Management Agency (FEMA), FEMA App. 〈https://www.fema. gov/mobile-app/〉. (Accessed 5 May 2015). J. Reilly, S. Dashti, M. Ervasti, J.D. Bray, S.D. Glaser, A.M. Bayen, Mobile phones as seismologic sensors: automating data extraction for the iShake system, IEEE Trans. Autom. Sci. Eng. 10 (2) (2013) 242–251. K.K. Lwin, Y. Murayama, Web-based GIS system for real-time field data collection using personal mobile phone, J. Geogr. Inf. Syst. 3 (4) (2011) 382–389. D. Peng, X. Zhao, Q. Zhao, Y. Yu, Smartphone based public participant emergency rescue information platform for earthquake zone – "E-Explorer", in: Proceedings of the International Conference on Vibroengineering, 2015. X. Zhao, Y. Yu, Research on quick seismic damage investigation using smartphone, in: Proceedings of the SPIE, 2016. D. Papadias, J. Zhang, N. Mamoulis, Y. Tao, Query processing in spatial network databases, in: Proceedings of the International Conference on Very Large Data Bases, 2003. J. Black, T. Ellis, D. Makris, A hierarchical database for visual surveillance applications, in: Proceedings of the IEEE International Conference on Multimedia and Expo, 2004. C. Vicknair, M. Macias, Z. Zhao, X. Nan, Y. Chen, D. Wilkins, A comparison of a graph database and a relational database: a data provenance perspective, in: Proceedings of the Southeast Regional Conference, ACM, 2010. C. Tisell, K. Orsborn, A system for multibody analysis based on object-relational database technology, Adv. Eng. Softw. 31 (12) (2000) 971–984. Oracle, MySQL. 〈https://www.mysql.com/〉. (Accessed 6 January 2015). Baidu, Develop Platform of Baidu Map. 〈http://lbsyun.baidu.com/〉. (Accessed 6 February 2015) (in Chinese). Amap, Develop Platform of Amap. 〈http://lbs.amap.com/〉. (Accessed 6 February 2015) (in Chinese). Primefaces, PrimeFaces: Ultimate UI framework for Java EE. 〈http://primefaces. org/〉. (Accessed 23 February 2015). National Coordination Office for Space-Based Positioning, Navigation, and Timing, GPS Accuracy. 〈http://www.gps.gov/systems/gps/performance/accuracy/〉. (Accessed 23 February 2016). Wikipedia, MD5. 〈https://en.wikipedia.org/wiki/MD5/〉. (Accessed 11 January 2015). Javaworld, Sockets Programming in Java: A tutorial. 〈http://www.javaworld.com/ article/2077322/core-java/core-java-sockets-programming-in-java-a-tutorial. html〉. (Accessed 10 January 2015). Github, AsyncHttpClient. 〈https://github.com/AsyncHttpClient/async-http-client/ 〉. (Accessed 10 January 2015). Oracle, The Java Database Connectivity (JDBC). 〈http://www.oracle.com/ technetwork/java/javase/tech/index-jsp-136101.html〉. (Accessed 12 January 2015). Wikipedia, EXIF. 〈https://en.wikipedia.org/wiki/Exif/〉. (Accessed 4 January 2015). Oracle, Java SE Documentation. 〈http://www.oracle.com/technetwork/java/ javase/documentation/index.html〉. (Accessed 12 December 2014). X. Zeng, X.Z. Lu, T. Yang, Z. Xu, Application of the FEMA-P58 methodology for regional earthquake loss prediction, Nat. Hazards 83 (1) (2016) 177–192. X.Z. Lu, H. Guan, Earthquake Disaster Simulation of Civil Infrastructures: From Tall Buildings to Urban Areas, Springer, 2017.