A real-time map merging strategy for robust collaborative reconstruction of unknown environments

Expert Systems With Applications 145 (2020) 113109

Contents lists available at ScienceDirect

Expert Systems With Applications journal homepage: www.elsevier.com/locate/eswa

A real-time map merging strategy for robust collaborative reconstruction of unknown environments Carlos Alberto Velásquez Hernández∗, Flavio Augusto Prieto Ortiz Universidad Nacional de Colombia, Bogota, Colombia

a r t i c l e

i n f o

Article history: Received 5 February 2019 Revised 19 October 2019 Accepted 28 November 2019 Available online 29 November 2019 Keywords: Map merging MRSLAM Feature extraction Decision-making Collaborative reconstruction

a b s t r a c t The development of collaborative techniques for exploring and mapping environments has been rising in the last decade. These techniques, known as multi-robot SLAM (MRSLAM), aim to extend the use of autonomous mobile robots to autonomous multi-agent systems. The MRSLAM technique presented here consists mainly of a robust map merging algorithm and a decision-making algorithm that controls agents in the field. On the one hand, the proposed merging algorithm performs a consistent and robust map fusion in real time. It consists of an own corner detector, a cylindrical descriptor, a matching technique and the RANSAC algorithm. On the other hand, once the fusion of maps is performed, the decision-making algorithm is responsible for controlling the robot operation in the field, based on the general current state of the multi-robot system. The main contribution of this MRSLAM technique is the robust map merging algorithm, since it was implemented and validated in simulated and real scenarios, resulting in collaborative maps that are consistent with the environment and obtained in less than 280 ms. This technique also achieves a significant decrease in reconstruction time when two or three robots are used: up to 35% in a simulated scenario and up to 49% in a real one. The proposed MRSLAM technique shows important similarities to expert multi-agent systems, as it is able to control and organize a team of robots in order to collaboratively explore and map an unknown environment. This approach was developed under the ROS framework to be used and tested by the scientific and academic community. © 2019 Elsevier Ltd. All rights reserved.

1. Introduction In the past decade, one of the challenges of extensive study in robotics focused on developing techniques for locating and reconstructing environments autonomously. These techniques are called SLAM algorithms and several approaches have been proposed (Bahraini, Bozorg, & Rad, 2018; Balcılar, Yavuz, Amasyalı, Uslu, & Çakmak, 2017; Cho, Kim, & Kim, 2018; Engelhard, Endres, Hess, Sturm, & Burgard, 2011; Havangi, Nekoui, & Teshnehlab, 2012; Jeong, Yoon, & Park, 2018; Kohlbrecher, Stryk, Meyer, & Klingauf, 2011; Moratuwage, Vo, Wang, & Wang, 2012; Solà , VidalCalleja, Civera, & MartÃnez-Montiel, 2012; Wang & Wang, 2017). With these techniques, mobile robots could perceive their environment and perform interaction and recognition tasks without human (or user) intervention. Recent works in SLAM have focused on visual techniques (Labbé & Michaud, 2014; Li, Zhang, Gao, Wang, & Xian, 2019; Mur-Artal, Montiel, & Tardós, 2015; Yao, Zhang, Xu, Song, & Zhang, 2018) and have been used in aerial

∗

Corresponding author. E-mail addresses: [email protected] [email protected] (F.A. Prieto Ortiz). https://doi.org/10.1016/j.eswa.2019.113109 0957-4174/© 2019 Elsevier Ltd. All rights reserved.

(C.A.

Velásquez

Hernández),

(Yang, Scherer, Yi, & Zell, 2017) or underwater (Ozog, JohnsonRoberson, & Eustice, 2017) applications. In Wang, Zhang, and An (2017) a survey of SLAM techniques is presented in three major categories, while in Gaspar, Nunes, Pinto, and Matos (2018) a benchmarking of common visual techniques based on a public dataset is exposed. However, after testing the functionality of SLAM approaches, the need for the development of collaborative reconstruction systems arose in recent years, because the use of two or more agents (robots) in more complex tasks (such as search and rescue of people) has become necessary. This field is currently known as multi-robot SLAM (MRSLAM) and the developed approaches focus on obtaining a consistent method that allows any number of agents to explore and map an environment. Specifically, the MRSLAM methods encompass three major problems (Liu, Fan, & Zhang, 2013; Sun D. & Wendt, 2009; Zhou & Roumeliotis, 2006) to solve and make them computationally usable to real-time collaborative tasks: the scalability of processed SLAM data, the loop closure condition and the map merging problem. This last problem is the most studied and important one (Sun D. & Wendt, 2009; Topal, Erkmen, & Erkmen, 2010; Xu, Jiang, & Chen, 2012; Zhou & Roumeliotis, 2006), as it directly impacts the general performance of the technique and allows the interac-

2

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

tion among robots by sharing and merging their information. In Kshirsagar, Shue, and Conrad (2018), a review related to SLAM, MRSLAM and their proposed techniques is presented. This work introduces the main concepts, their applications and challenges of these fieldsMRSLAM. Similarly, the authors in Lee, Lee, Lee, Kim, and Lee (2012b) presented a review focused on different MRSLAM techniques that address the map merging problem. The authors also introduced a classification into direct (DMM) and indirect (IMM) map merging algorithms. The DMM approaches focus on the mixture of maps with information coming directly from the robots, while IMM techniques are focused on algorithms that monitor and merge the data provided by the robots in the environment. However, the proposed techniques only focus on the map merging problem and not on the agent control in the environment to complement their usefulness and potential. With this additional algorithm, a MRSLAM technique can be considered as an expert multi-robot system, which is able to take actions based on the analysis of the current state of the whole system. These actions (or decisions) make possible to cover a scenario in a shorter time by preventing physical encounters in the field or by avoiding remapping areas and get an organized agent distribution by monitoring their movements or targets, which shows the characteristics of an expert human system. The MRSLAM technique presented here has similarities to human rescue teams in USAR (urban search and rescue) environments, whose main goal is to quickly cover an area in order to search victims or wounded people, so these teams are usually deployed in the area sharing information such as position, descriptions of scenes or landmarks, avoiding remapping areas and reporting their findings to a centralized control position. A suitable MRSLAM technique must be able to reconstruct an environment in the shortest possible time, which means that the reconstruction time should significantly be reduced with the use of multi agent systems (two or more agents). This article presents the implementation of a MRSLAM technique capable of obtaining a consistent global map from an unknown environment with the use and field control of three agents. The proposed technique is divided into a SLAM map merging stage and a decision-making algorithm. The first step is responsible for analyzing and merging the maps reported by the agents, while the second one is responsible for controlling the agents to improve the efficiency of the reconstruction. The maps reported by the agents are occupancy grid maps (OGM), generated by the Hector SLAM algorithm (Kohlbrecher et al., 2011). The technique was tested in a real semistructured environment, where the reconstruction time and accuracy data were also measured to analyze the efficiency of the proposed technique. The remainder of this article is divided into the following sections. Section 2 describes the related works to the MRSLAM field, while Section 3 shows the merging algorithm, disclosing the stages that comprise it and the tests performed. Section 4 describes the decision-making algorithm and details the most relevant parameters for making a decision. Section 5 focuses on the tests and results obtained from the MRSLAM technique. Finally, Section 6 explains the conclusions and future work related to this work. 2. Related work MRSLAM is the extension of SLAM techniques applied to a group of robots that communicate with each other, mainly, in order to give autonomy to each agent in their task of exploring and mapping the environment. This technique arises as a need to solve certain SLAM flaws, due to the limitations of using a single robot in a large environment. On the one hand, the time that a robot requires to recognize an environment is proportional to the size of the environment to be mapped. On the other hand, the use of a single robot in the field is limited by the autonomy, sensors and

memory that it has to map and recognize an environment. This implies the use of expensive components to guarantee the proper functioning of the robot in the field. Based on the foregoing, first MRSLAM techniques were proposed using a unique global reference system, where the agents shared the information of their location and map since the beginning (Kshirsagar et al., 2018). With this approach, two variants emerged: MRSLAM with a common (or nearby) starting point for all agents (Stachniss, 2009) and the other one with different, but well-known, starting points (Yamauchi, 1998). However, these approaches restrict its use, as they restrict the starting point and starting time of the agents. In addition, this technique is sensitive to an incorrect initialization of the position of the robots with respect to the common global reference, which can cause a wrong merging of the partial map explored by each robot. Hence, there exists the need to propose flexible MRSLAM techniques, which overcome the inconveniences. As mentioned before, in Lee et al. (2012b), the authors present a division of these flexible MRSLAM techniques into direct (DMM) and indirect (IMM) map merging algorithms. The DMM approaches focus on the map mixture with information coming directly from the robots. Within this category, there are the Rendezvous techniques (Howard, 2006; Lee, Lee, Choi, & Lee, 2012a; León et al., 2009; Zhou & Roumeliotis, 2006) and the approaches based on the detection and fusion of feature points (landmarks) or objects in the environment (Cortés & Serratosa, 2016; Lee & Lee, 2009; Tungadi, Lui, Kleeman, & Jarvis, 2010). In Howard (2006), a MRSLAM technique based on the Rendezvous case is presented. The author exposes an approach that emulates the single SLAM, but applied to a robot network. The technique considers a start of each agent with their own map and state vector. Once an encounter between two or more robots is made, a single SLAM algorithm starts merging the information collected by each agent. Similarly in León et al. (2009), another Rendezvous variant is proposed, where each robot carries a camera that serves to detect other agents in the field. When the visual tracking algorithm accurately determines the presence of another robot, it estimates their relative position taking into account the encounter point. Once these calculations are done, the robots can exchange their maps and any other available information. Despite showing a breakthrough in the MR-SAM field, these techniques depend on the robustness of the vision algorithms to detect agents in fields and depend on such encounters in the environment, which do not prevent areas from being remapped or well merged. Regarding the IMM techniques, they are focused on algorithms that monitor and merge the data provided by the robots in the environment. Within this category, there are map merging techniques based on features points (Wang, Jia, Li, Li, & Guo, 2012) and the spectral information (Carpin, 2008; Lee & Lee, 2011) merging techniques. This last category is the most challenging, as spectral information entails the processing of large volumes of data in real time, though it offers the necessary distinctiveness of objects to make correct mergers. In the IMM category, there are few studies that deal with the merging problem, since it is generally assumed that the collaborative reconstruction system does not depend on the agents or on the a priori information (established points or references) of the environment. Some techniques perform the fusion of states generated by EKF-SLAM algorithms (Kojima, Okawa, & T., 2013; Sasaoka, Kimoto, Kishimoto, K., & Nakashima, 2016), however, those approaches require a lot of processing time and memory. Some techniques use bio-inspired algorithms such as PSO (Lee, Roh, & Lee, 2016), while others rely on map analysis to detect overlaps or common points (Blanco, Gonzalez-Jimenez, & Fernandez-Madrigal, 2007; Blanco, González-Jiménez, & Fernández-Madrigal, 2013). All these approaches look for the map transformation matrix (MTM),

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

which allows to merge two data sets from common points found among the maps. The authors of Blanco et al. (2007) present an important design of a map merging technique using the traditional methodology of Computer Vision (CV) algorithms. They also present a rigorous study and new improvements of their technique in Blanco et al. (2013). Although these works achieve map merging times between 600-807ms, they do not address the problem of agent control in the field and is not tested in real operating conditions, i.e., with maps being explored and reported in real time. Similarly, the authors of Andre, Neuhold, and Bettstetter (2014) present an integration of ROS (Quigley et al., 2009) packages (ad hoc communication protocol, map merger and exploration strategy) to implement a MRSLAM technique. As these ROS packages are designed for a single robot, their tight integration, including the extension of the map merger to online usage (during exploration process), is the main contribution. To prove their technique, different tests in a simulated scenario were carried out and metrics related to the map reconstruction error and the time required to complete the mission were measured. As result, the authors found that the exploration with three robots obtained a better performance than the same task performed by four robots. An explanation for this result is the cumulative error of the map merger and exploration process when the task is performed by four robots. There are also recent studies that introduce guidelines to help build a multi-robot system based on several parameters (Sullivan, Grainger, & Cazzolato, 2018) and make improvements in communications architecture based on ROS (Tardioli, Parasuraman, & Ögren, 2019). Despite not being related to the MRSLAM field, these works are important for the area as they present new solutions and guidelines that will help the design and implementation of new MRSLAM algorithms. Finally, it is important to mention that only in Battistelli, Chisci, and Laurenzi (2017) a decentralized technique, based on random finite sets (RFS) maps, is proposed. This method uses the peer-topeer robot approach to solve the cooperative slam problem, which means an advance in the MRSLAM field by eliminating the dependence of the multi-agent system on a single central agent; nevertheless, this technique is only tested in a simulated scenario and the agents since the beginning have a common reference frame, which constrains the usefulness of the technique by having initial conditions for its operation. Moreover, the vast majority of these techniques have only been tested in controlled or simulated scenarios, which is a disadvantage since, from the best of our experience, real environments differ greatly from simulated environments. 3. OGM merging algorithm The occupancy grid maps (OGM) merging technique is responsible for finding the right map transformation matrix (MTM) between two maps reported by two agents in the environment. This type of map (OGM type) was used in the map merging algorithm, since it has a graphical representation, so the resulting maps can be treated as standard images in CV algorithms. Although approaches of this class have been proposed (Blanco et al., 2007; Blanco et al., 2013), our algorithm is able to perform robust mergings in real scenarios without any initial condition and, according to all the tests, it can find the optimal MTM in an affordable time (less than 280 ms for maps of 1024 × 1024 cells), making it suitable for real time applications. This improvement is mainly due to the design of the merging algorithm, where every step was analyzed thoroughly in order to get the best performance and computation time. In Fig. 1, the MRSLAM technique is detailed. Within this scheme, it can be noted the OGM merging algortihm that follows the sequence of steps of a traditional CV algorithm: extraction, de-

3

scription, matching and MTM calculation (RANSAC and MTM refinement). The extraction step makes the greatest contribution in this algorithm as it was specially designed for this algorithm.

3.1. Feature extraction This stage looks for the feature point detection on the map reported by an agent. Given the OGM topology, the corner detector can ignore pre-processing steps, since the map is represented in 3 gray scale values: white (free state), black (occupied state) and gray (unknown state). This means that the map can be considered as a thresholded image. Additionally, filtering, thresholding or smoothing steps are not necessary as they directly affect the information contained in the maps, i.e. points of the environment could disappear in these steps, which means a loss of valuable information of the obstacles for the robot. For this stage, an exhaustive study of the existing and most known feature extraction techniques in CV was made: Harris detector (Harris & Stephens, 1988), Shi-Tomasi detector (Tomasi & Kanade, 1991), Trajkovic detector (Trajkovic & Hedley, 1998), SIFT extractor (Lowe, 1999) and ORB detector. With these techniques, a new feature extractor was developed and presented in Velasquez and Prieto (2015). As main result, this extractor proved to be more robust and efficient (in computation time) than the analyzed techniques in OGM (Velasquez & Prieto, 2015) and standard images (Velasquez & Prieto, 2019). For the images shown in Fig. 2, the average computation time was obtained from 20 tests performed on each image (Table 1). Images shown in Fig. 2 and results in Table 1 belong to a forty image dataset used in this study. The results show that the developed technique presents an outstanding performance for the feature extraction in OGM. Furthermore, the selected technique is robust in feature point detection in areas with a high degree of noise, thus this property is an advantage to carry out an adequate merging of maps. This technique was also proved to be a good performance in standard (RGB or color) images. Results related to the aforementioned work are reported in Velasquez and Prieto (2019).

3.2. Feature description As in the previous step, this one was carried out through a comparative analysis of the most known feature description techniques. For this stage, the algorithms of SIFT (Lowe, 1999; 2004), SURF (Baya, Essa, Tuytelaars, & Van Gool, 2006), ORB (Rublee, Rabaud, Konolige, & Bradski, 2011) and the cylindrical descriptor (Blanco et al., 2007) were studied. These techniques were chosen given their great recognition and use in scientific studies and applications related to CV. SIFT and SURF are the main references in the detection and description of feature points, while ORB is recognized for its great performance in computation time. The comparative analysis of the techniques was clearly based on the average computation time of the descriptors associated to each feature point detected in the extraction step. Table 2 shows the results obtained in this stage. It is noteworthy that the algorithm with the best performance is the cylindrical descriptor, since it follows the main goal in the design of the merging algorithm, i.e. improving performance in terms of computation time. In addition, the cylindrical descriptor has the advantage of reducing the search space by calculating the point descriptor in polar coordinates (r and θ ) instead of cartesian ones (x, y and θ ). With this change, the search space for pairing descriptors is reduced to the θ variable, which has a resolution of 22.5◦ , due to the feature orientation assigned by our feature ex-

4

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

Fig. 1. General scheme of the MRLSAM technique. It can be noted the two main procedures of the approach: the OGM merging algorithm and the decision-making method.

Fig. 2. Test images used for the analysis and performance measurement of the analyzed feature extraction techniques. Map images were scaled 1:2 : Map 1 with a size of 146 × 144 pixels, Map 2 with a size of 113 × 113 pixels, Map 3 with a size of 398 × 402 pixels, Map 4 with a size of 348 × 399 pixels, Map 5 with a size of 400 × 388 pixels and Map 6 with a size of 387 × 376 pixels.

Table 1 Comparative table of the computation times of features detected by each technique in each test image. Technique

Map 1 Time (μs)

Map 2 Time (μs)

Map 3 Time (μs)

Map 4 Time (μs)

Map 5 Time (μs)

Map 6 Time (μs)

Harris detector Shi Tomasi detector Trajkovic detector SIFT Keypoint extractor ORB extractor Our proposed detector

960 803 643 10120 949 157

536 475 347 6503 583 113

8296 6929 6765 75719 5976 964

5629 4372 1828 63510 3601 843

6137 4605 2616 72431 4474 857

5990 5151 2519 66667 4414 930

Table 2 Computation times of the descriptors of the analyzed techniques. Technique

Map 1 Time (μs)

Map 2 Time (μs)

Map 3 Time (μs)

Map 4 Time (μs)

Map 5 Time (μs)

Map 6 Time (μs)

Cylindrical descriptor SIFT descriptor SURF descriptor ORB descriptor

874 9345 9632 923

471 5607 4111 498

10628 62063 82711 9279

10470 48082 56610 6711

7969 52235 83628 8027

10490 58852 74670 10806

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

Fig. 3. Set of corresponding points that fulfill the selection criterion of similarity distance.

5

Fig. 4. From the feature matching step, Ransac searches and tests the best pairs of points to build a MTM, which allows the merging of the analyzed maps.

tractor. Eq. (1) describes the calculation of the descriptor.

fa [i, j] =

 φ j+1  φj

ri+1



m

ri

xa + r cos θ ya + r sin θ



drdθ

ri = Rmin + ir

φ j = j φ

(1)

A formal and extended presentation of this descriptor can be found in Blanco et al. (2007). 3.3. Feature matching Following the scheme shown in Fig. 1, the next step of the merging algorithm is responsible for pairing or matching the feature points of the maps. Based on how the cylindrical descriptor works, a feature matching is done when a pair of point descriptors has a high degree of similarity. This matching is done according to the similarity function shown in Eq. (2), where dij is the similarity distance, on a normalized scale, between the descriptors of the points i and j; N is the number of interest points associated with the mapA ; and M is associated with the number of interest points on the mapB .

 p=N r=M p=0

di, j =

r=0



mapA

Desci ( p, r ) ==mapB Desc j ( p, r ) else N·M



1 0

(2)

The pairs of the corresponding points are selected if dij > 0.7. This similarity threshold is initially set to 0.7, since it was experimentally observed that, below this value, the MTM solution may contain a large fusion error between the maps (Fig. 5a). In contrast, above this threshold, the maps have the potential to be merged into a single one. Fig. 3 shows the pair of corresponding points that have a similarity distance larger than the selection criterion. From this set of corresponding points, the MTM is calculated using the RANSAC algorithm. 3.4. RANSAC The RANSAC algorithm allows the alignment between two data sets (Fischler & Bolles, 1981). Since the pairs of points selected in previous steps have a high degree of similarity, it is necessary to calculate the MTM from this set of corresponding points. For this purpose, RANSAC was used, as it selects a pair of points and then calculates a MTM that is evaluated with the other pairs

Fig. 5. Merging of maps: a) without refinement of the MTM and b) fter refinement in θ .

of points to validate the proposed model (calculation of inliers). If the proposed MTM shows that more than 4 points comply with the model, this MTM is accepted as a possible solution until a model with a larger number of inliers can be found. Fig. 4 shows a model with 7 pairs of points that fits in the MTM found by the implemented RANSAC algorithm. Additionally, in this case, RANSAC does not have the stop condition by number of iterations, since it evaluates all the corresponding pairs in order to calculate the best model. This has no effect on the calculation time of the MTM as the number of pairs does not usually exceed 100 pairs for maps of 1024 × 1024 cells. 3.5. Refinement of MTM After the calculation of the MTM found by RANSAC, there is a refinement step in θ in order to improve the distance of similarity among the selected corresponding points i and j, because an increase in this distance value improves the map merging result. This refinement step is performed throughout the range [−7◦ , 7◦ ] in θ with angular resolution of 0.5◦ . In the RANSAC step, the MTM was calculated from a pair of points i and j, whose normalized similarity distance is greater than 0.7. Nevertheless, this similarity mea-

6

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

Fig. 6. Decision tree designed for the proposed MRSLAM technique.

sure needs to be improved and can be corrected by refining the MTM solution around θ . Therefore, after applying the refinement step, the similarity threshold is increased to 0.9 in order to guarantee a better merging with the analyzed maps, that is, the MTM is readjusted and finally accepted if dij > 0.9. Fig. 5.b shows the incidence of this step in the MTM calculated by RANSAC. The acceptance threshold was set at 0.9, considering that similarity values greater than 0.95 or close to 1 (perfect mixture) were never experimentally achieved. Once the calculated MTM is accepted, the merging algorithm constructs the frame tree to merge the OGM reported by the robots. With this frame tree, agents can know their positions with respect to a single reference frame. Additionally, the decisionmaking algorithm with a merged map can guide the exploration of the agents in the environment in order to improve the time reconstruction.

4. MRSLAM decision-making algorithm Once the merging algorithm gets the global map of the environment, it is necessary to improve the effectiveness of the cooperative navigation strategy by using a decision-making algorithm capable of controlling the agents in the environment. This algorithm

is responsible for avoiding physical encounters among robots or remapping known areas of the environment. The decision-making algorithm analyzes a set of agent variables to take an appropriate decision based on the current state of the collaborative reconstruction task. The system therefore prioritizes unknown points of the environment or tasks that require higher priority in order to minimize the time required to collaboratively map the environment. The variables analyzed by the decisionmaking algorithm are: 1. Robot state (robsta ): it contains the robot state during its operation. The states considered by this parameter are: operation, pause, stopped, failure. 2. Distance to current target (drest ): it is measured in meters and provides the remaining distance to the current target. 3. Known map percentage (mapkno ): since OGM have three states (occupied, free and unknown), this variable counts the percentage of cells in occupied and free status with respect to the total number of cells in the map. 4. Distance traveled (dtrav ): it provides in meters the distance that the robot has traveled along its navigation and reconstruction task in the environment. Fig. 6 shows the decision tree built for the developed decisionmaking algorithm.

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

From Fig. 6, the parameter robsta is highlighted as the most relevant, since it defines if the robot is able to receive orders from the decision-making algorithm. Nonetheless, it is noteworthy that for the paused and stopped status, the robot decides whether or not to execute an order, since it is the only one who knows the cause of its current state. The second most important parameter for the decision tree is drest . This parameter analyzes the possibility of redirecting one of the agents if two or more robots are going, for example, to the same target in the environment. The decision-making algorithm can also allow both agents to finish their current goal and, after an update of the global map, give them new targets. It is also remarkable that with this variable the decision-making algorithm reduces the collaborative exploration time of an unknown environment since it is able to redirect the agents in the field according to the current state of the merged map. The parameters mapkno and dtrav are used to know the reconstruction time of the environment. The parameter mapkno indicates the percentage of exploration of the map associated with a robot, however, it can be affected by the stochastic nature of SLAM algorithms, the noise provided by the sensors or by robot enclosures with dynamic obstacles in the environment. This deficiency is complemented by the second parameter, dtrav , since it counts the distance traveled by the robot in the environment: the greater the distance traveled is, the greater is also the percentage of exploration of the map. In this way, the decision-making algorithm always prioritizes exploration in agents with a low percentage in these parameters, as this indicates that the robot has a higher energy level. Besides, this parameter helps to explore a larger area of the environment, as agents with low percentage of exploration have more capabilities to contribute to the global map than those with high values in these parameters. Once again, the decision-making algorithm optimizes the exploration time by assigning tasks according to the current knowledge of the environment of each agent in operation. Finally, note that when the collaborative task is carried out with more than two robots, the MRSLAM technique creates an organized list based on the names of the robots, e.g. Ro1 , Ro2 , Ro3 and so on. The MRSLAM technique selects the first two robots (e.g. Ro1 and Ro2 ) and gets their maps (OGMRo1 and OGMRo2 ).Then, it continues with the map merging algorithm (Section 3) to try to merge them. If the merging between OGMRo1 and OGMRo2 is done, the decision-making algorithm receives the merged map (OGMRo1 _Ro2 ) and requests the poses of the robots (PRo1 and PRo2 ) and the internal parameters of the robot (robsta , drest , mapkno and dtrav ) in order to make a decision. Once a decision is made, the merged map (OGMRo1 _Ro2 ) is considered as a partial map of a robot called Ro1 _Ro2 and enters the created list of robots until only one map (the global one), which includes all the robots in operation, is found. However, in case the merging between the maps OGMRo1 and OGMRo2 is not possible, the technique continues with the second and third robot (e.g. Ro2 and Ro3 ) of the list and so on. This explanation is summarized in the pseudo-code showed in Algorithm 1.

Algorithm 1 MRSLAM technique. MRSLAM Inputs: OGMRoi and OGMRoi+1 Internal robot parameters: robsta , drest , mapkno and dtrav MRSLAM Outputs: OGMRoi _Roi+1 and F rRoi _Roi+1 Definitions: - OGM: occupancy grid map - MTM: map transformation matrix Synopsis: the MRSLAM technique performs collaborative reconstructions with a team of robots using an OGM merging method and a Decision-making algorithm. 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25: 26: 27: 28: 29: 30: 31: 32: 33:

Create a list of robot. i ← length(RobotList). OGMRoi _Roi+1 , F rRoi _Roi+1 ← null. while i > 1 do Select two robots Roi and Roi+1 . OGMRoi , OGMRoi+1 ← Maps of Roi and Roi+1 . goto OGM merging. if OGMRoi _Roi+1 = null then goto Decision-making. if OGMRoi _Roi+1 = null then goto Decision-making. procedure OGM merging(OGMRoi , OGMRoi+1 , OGMRoi _Roi+1 , F rRoi _Roi+1 ) Feature extraction of OGMRoi and OGMRoi+1 . Feature description of OGMRoi and OGMRoi+1 . Feature matching between OGMRoi and OGMRoi+1 . Finding MT MRoi _Roi+1 with RANSAC. Compute and refine MT MRoi _Roi+1 . if MT MRoi _Roi+1 is accepted then Create a frame F rRoi _Roi+1 from MT MRoi _Roi+1 . OGMRoi _Roi+1 ← Merging(OGMRoi , OGMRoi+1 ). Update RobotList. goto step 4. procedure Decision-making(OGMRoi _Roi+1 , F rRoi _Roi+1 , Roi , Roi+1 ) Analyze parameters: robsta , drest , mapkno and dtrav . Rodecision ← MakeDecision(Roi , Roi+1 ). Compute NewT arget for Rodecision based on OGMRoi _Roi+1 . if NewT arget == null then Exploration Ended. if NewT arget is better than Current Robot T arget then Assign NewT arget to Rodecision . goto step 4. Do not change Current Robot T arget . goto step 4.

5.1. Validation in the simulated scenario The scenario was built in Gazebo 2.0 and is shown in Fig. 7. For this test, 3 Kobuki mobile robots with Hokuyo laser sensor URG04LX-UG01 were used. The robots were called Jade, Luna and Mars. The tests were: •

5. Results and discussion •

To measure the performance of the MRSLAM technique, tests were carried out to quantify the performance and precision of the merged map. Tests in simulated and real environments were both performed. The MRSLAM technique was coded in C++, using the ROS Indigo framework in O. S. Ubuntu 14.04. The algorithm ran on a computer with a Intel core i5 @2,2GHz processor with 8 GB of RAM and 1 TB of hard disk.

7

•

Simple exploration: each robot explores the scenario shown in Fig. 7. This task was performed 3 times. Two robots exploration: two robots go through the scenario shown in Fig. 7. Tests were performed combining the robots on the scenario and each combination was tested 3 times. Three robots exploration: robots perform three times the cooperative reconstruction of the scenario shown in Fig. 7.

The results for the simulation scenario of Fig. 7 are shown in Fig. 8, which shows one map for each exploration test: simple exploration, two robots exploration and three robots exploration.

8

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109 Table 3 Estimated reconstruction error of the environment for each test. Simple exploration Robot Jade ± 3.77 cm

Two robots exploration

Robot Luna

Robot Mars

± 3.75 cm

± 3.47 cm

Robot Jade-Mars ± 3.56 cm

Three robots exploration

Robot Jade-Luna ± 3.82 cm

Robot Luna-Mars

Robot Jade-Luna-Mars

± 4.00 cm

± 6.13 cm

Table 4 Average times for every test carried out and its respective reduction percentage. Time (s)

Time reduction (%)

370.30 280.30 239.00

0.00 24.30 35.50

Simple exploration Exploration 2 robots Exploration 3 robots

Fig. 8. Results of the merging of maps for the simulated validation test. Table 5 Estimated error of the reconstruction of the real environment in each test. Test 1

Fig. 7. Simulation scenario used for the validation of the MRSLAM technique.

In Fig. 8, the quality of the merged maps can be observed in relation to the simple exploration test. In the 3 cases, the map consistency is significant and, therefore, the performance of the proposed MRSLAM technique is remarkable. Additionally, the results in terms of accuracy and time of the reconstruction are shown in Tables 3 and 4. The accuracy in this test measures the reconstruction error of the map with respect to the real scenario. For this calculation, the error was taken in centimeters from each segment (wall) of the real scenario with respect to the equivalent segment in the obtained map. These measurements were averaged to obtain the result shown in Table 3. The reconstruction error increases considerably when the merging is made with more agents in the environment. However, due to the estimation error associated with the SLAM algorithm ( ± 1.5cm) and the resolution of the map cell (3.0cm), that error has no affect on the consistency of the reconstructed environment. Additionally, note in Table 4 that the reconstruction time decreases as the number of agents in the environment increases (up to 35% when three robots operate simultaneously in the field), which validates the usefulness of these systems in collaborative tasks.

5.2. Validation in a real scenario The selected scenario is an apartment since it represents a semi-structured, indoor and uncontrolled environment. Fig. 9 shows the robots in the scenario. For this test, two mobile robots, called Luna and Mars, were used. Each robot was equipped with a Kobuki mobile base, a Hokuyo URG-04LX-UG01 laser sensor and a laptop with Intel Pentium N3710 @ 1.60 GHz processor, 4 Gb of RAM and Ubuntu 14.04 with ROS Indigo framework. The tests performed in the scenario shown in Fig. 9 are listed below:

Test 2

Robot Luna

Robot Mars

± 3.83 cm

± 4.14 cm

Robot Luna-Mars ± 5.64 cm

Robot Mars-Luna ± 6.04 cm

Table 6 Average times in each test and reduction percentage of the cooperative reconstructions regarding individual reconstruction.

Mars-Luna Luna-mars Mars-Luna Luna-Mars

•

•

A-B A-B A-A A-A

Time (s) 155.67 195.33 115.33 127.67

Luna time (s)

Mars time (s)

219.50 29.08% 11.01% 47.46% 41.84%

227.33 31.52% 14.08% 49.27% 43.84%

Test 1: each robot explores the environment in a single way, from starting points A and B. Each robot performs the task three times: twelve tests in total (six per robot). Test 2: two robots move through the scenario, starting from the same point A or from different points (points A and B, see Fig. 9). Tests were performed switching the starting positions of the robots. Each configuration was executed three times: twelve tests in total.

Tests carried out in this scenario seek to test the performance of the proposed technique in real operating conditions, validate the use of multi-robot systems in real time applications and test the flexibility of the system as the robots do not start at the same time. This behavior is common when deploying robots in the environment, since normally they are put into operation individually. The test results are shown in Fig. 10, where it can be clearly seen that the algorithm manages to merge the maps of all the tested configurations. Table 5 and Fig. 11 detail the information of the reconstruction error associated with the obtained map in each test and the reconstruction times, respectively. The reconstruction error was calculated following the idea presented in the simulated scenario test (Table 5). In these tests, the reconstruction times showed a reduction of the cooperative exploration time with respect to the individual exploration time. These results are presented in Table 6.

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

9

Fig. 9. Scenario of real tests with mobile robots in operation. Fig. 7.b. shows the robots in their initial positions A (magenta-colored robot Luna) and B (yellow-colored robot Mars).

Fig. 10. Results of the individual reconstruction of the test scenario (Test 1) and the cooperative reconstruction performed by the proposed MRSLAM technique (Test 2).

Fig. 11. Results associated with the reconstruction times for each of the tests performed. The table shows the start delays between the robots for the cooperative tests.

Note that the merging and decision-making algorithms developed for the MRSLAM technique achieve an outstanding performance in all the performed tests. The main purpose of MRSLAM techniques (reduce the exploration time) is validated, although the reductions obtained showed different results. An explanation for this result is the stochastic nature of SLAM algorithms, which directly affect the estimated pose, the constructed map and the other variables analyzed by the decision-making algorithm. On the other hand, the reduction of the exploration time when the robots start from the same point (at different starting times) is, as expected, the one with the best performance. The reason is that

once the second robot comes into operation, the merging algorithm is able to detect the maps overlappig. The latter is explained by the fact that robots have similar maps as they start from the same point, thus, the feature points should be similar and the merge should therefore be feasible. In addition, with the decisionmaking algorithm acting since the beginning, robots make the reconstruction more efficient by being directed to different unknown areas of the environment. Additionally, the reconstruction error associated with the tests performed (Table 5) had no effect on the consistency of the merged map obtained. The resulting map suffered no deformations or bad overlaps due to this estimated error.

Centralized technique. Scalability of the MRSLAM system. Robust and consistent map merging. Tested in simulated and real environment. Decision-making algorithm for field agent control.

Common reference frame to all robots. Technique tested in simulated scenario.

Robust, consistent and real-time map merging with field agent control. Merging without start conditions

Distributed multi-vehicle SLAM

Common Starting Position

Scalability of the MRSLAM system communication network. Integration of map merger, exploration and ad hoc communication algorithms to its online usage. How to collaboratively explore environments. Known Initial Positions

Control of reconstruction task by agents. Local map merging. Verification through Rendezvous technique Optimization of the energy consumed and time of the MRSLAM system. Decentralized or centralized approach. Easy configuration and use through ROS framework. Decentralized approach. Robots cooperate in a peer-to-peer fashion. Map merging based on Monte Carlo Localization hypotheses. Rendezvous Case

Multi-hypotheses approach. Robust merging algorithm.

Advantages

Merging technique based on OGM overlapping. Merging without start conditions

A robust, multi-hypothesis approach to matching occupancy grid maps (Blanco et al., 2007; Blanco et al., 2013) A Fast Map Merging Algorithm in the Field of Multi-robot SLAM (Liu et al., 2013) Team Size Optimization for Multi-robot Exploration (Yan et al., 2014) Coordinated Multi-Robot Exploration: Out of the Box Packages for ROS (Andre et al., 2014) Random Set Approach SLAM to Distributed Multivehicle SLAM (Battistelli et al., 2017) Proposed MRSLAM technique

Problem Approach Title

Table 7 Qualitative comparison of different MRSLAM techniques.

No multi-agent control algorithm. Technique tested with a bank of maps (off-line). Agents can share information from the beginning. Technique tested in simulated scenario. Odometry noise not considered in tests. Technique tested in simulated scenario. Common Starting Position for all robots. Technique tested in simulated scenario.

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

Disadvantages

10

In addition, the MRSLAM technique managed to merge the maps reported by the agents and managed to control the exploration trajectories of the robots in all the tests performed, which undoubtedly results in an improvement of the cooperative reconstruction times in semi-structured indoor environments. The technique is summarized in Table 7, with other MRSLAM techniques, in order to compare qualitatively the main characteristics and relevant information of the techniques. All the techniques have their own advantages and disadvantages. The proposed technique provides notable characteristics, since it does not depend on any initial condition to explore and map unknown environments. However, this technique is centralized, which means that it depends on the central server and the robustness of the communication protocol to perform the collaborative task. These drawbacks can be solved by integrating some properties that have other techniques such as the decentralized approach proposed by Battistelli et al. (2017), the scalability strategy presented in Yan, Fabresse, Laval, Bouraqadi, and MacDonald (2014), the communication protocol developed in Tardioli et al. (2019), among others. Furthermore, in Table 8, the metrics and performances reported by some techniques are presented. These quantitative results include accuracy, reconstruction error, exploration time, among others. In this table, it can be observed that the proposed technique improves the reduction time (with respecto to Yan et al. (2014) when using two agents in the environments. In addition, a notable difference in the map merging time is observed in the proposed technique with respect to Blanco et al. (2007, 2013). The reason is that (Blanco et al., 2007; Blanco et al., 2013) uses the Harris corner detector in their approach, while the proposed technique uses a developed feature extractor that has a better performance than the Harris detector (Section 3.1). Despite these results, a thorough quantitative comparison cannot be made, as each technique has different experimental conditions, tests and metrics. To make a rigorous comparison, the techniques must be subjected to the same experimental conditions. Techniques with initial conditions may be faster in map merging, but this constrains the deployment of multi-robot systems in field because the agents must know accurately the pose of the other robots and that would be difficult to establish in real scenarios, as the environment is unknown and it is not possible to make measurements on it. Likewise, techniques that depend on real encounters in the field (Rendezvous approaches) must have additional algorithms (like visual tracking algorithms) to recognize other agents in the field. They strongly depend on those algorithms to guarantee the encounter because if there is no recognition in field, there will be no merging. Then, the proposed technique seems to be a suitable approach to the map merging and field agent control problems as it does not depend on any restrictive condition. Finally, it is important to mention some findings when carrying out tests in real scenarios. First of all, real experiments undoubtedly differ from the simulated ones, as reality is difficult to emulate computationally. This can be observed in Tables 3 and 5, where the reconstruction errors differ significantly in both experiments despite tests were performed in different scenarios. Additionally, the communication is a determining aspect when testing multi-agent systems in real conditions, since maps, poses and related data from robots must be shared in real time, thus, the network becomes inefficient given its limited bandwidth. Therefore, researchers and developers are encouraged to propose new approaches and extend their works to real scenarios, as it is important to make them useful in real applications

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

11

Table 8 Metrics and performance of different MRSLAM techniques tested in simulated scenarios. Title

Metrics

Results

A robust, multi-hypothesis approach to matching occupancy grid maps (Blanco et al., 2007; Blanco et al., 2013) Team Size Optimization for Multi-robot Exploration (Yan et al., 2014)

Map matching accuracy (loop-closure detection) Map merging computation time

97% of accuracy for 1711 matchings between local maps. 600-807 ms to merge two maps.

Exploration time reduction (single agent vs two agents) Reduction of energy consumed (single agent vs two agents) Distance traveled reduction (single agent vs two agents) Reconstruction map error (in explored map pixels) Map merging computation time Reconstruction map error (cm) Exploration time reduction (single agent vs two agents)

22.50% reduction in exploration time. 17.85% reduction of energy consumed.

Coordinated Multi-Robot Exploration: Out of the Box Packages for ROS (Andre et al., 2014) Proposed MRSLAM technique

6. Conclusions The technique described above represents a contribution to the field of MRSLAM, since it presents an efficient and robust performance in the merging of OGM provided by agents in a real unknown environment. The technique shows a computation time in map merging (each map has 1024 × 1024 cells) of approximately 280 ms, while others approaches have reported a merging time of 600-807 ms (Blanco et al., 2007), which indicates an advance in the use of MRSLAM techniques in real-time applications. This outstanding performance is due to the meticulous design of the entire algorithm, which resulted in the development of a new corner detector. However, this time only reports the processing time on a central computer that executes the map merging algorithm. Additional tests must be carried out to measure the time (and also the bandwidth) related to the communication between the central computer and the robots. The development of this technique allowed to glimpse the usefulness of a decision-making algorithm, since it improves the time performance of the entire system, i.e. the exploration time decreases according to the number of agents, as verified in the simulated (12 in total) and real tests (12 in total). Nevertheless, this decrease is not proportional to the number of agents because of the stochastic nature of the SLAM algorithm and the robot navigation system. This is reflected in the fact that times, maps and reconstruction errors presented in this article correspond to the averages of the tests carried out and not to a repeatable and exact result of the tests. For future work, it is intended to extend this technique to a decentralized approach, because the proposed MRSLAM technique has the disadvantage of being centralized. Additionally, including 3D, RGB or environment texture information could improve the accuracy of the merging algorithm as this data can produce a smaller accumulated reconstruction error. Likewise, the extension of this technique to the use of five or more agents in the field is proposed to validate its scalability. Finally, it is important to point out some future lines of research that the MRSLAM field requires to have real and reliable implementations and allow its use in relevant tasks such as the USAR field, warehouse logistics and cooperative multi-agent systems. The first line of research is the 3D data fusion as this entails the merging of different data sources (3D lidar, RGB, RGB-D, among others) in order to get a consistent and unique 3D map. Additionally, 3D data involves handling and processing large volumes of data in real time, so the scalability and network bandwidth of the system must be considered. The second research line is the use of different mobile platforms (aerial and ground vehicles) in a single multi-robot system, as 3D data fusion allows it. Depending on the mobile platform used, the autonomous navigation system changes, hence the decision-making algorithm has to deal with this issue as com-

25.3% reduction in distance traveled. 19% of total explored map pixels 280 ms to merge two maps. ± 4.00 cm. 24.30% reduction in exploration time.

mands and targets must be well generated and sensor data must be managed according to the mobile platform and sensor. The third line of research is related to decentralized approaches that avoid the dependence on a central server. Centralized decision-making methods have the latent problem that if the central administrator goes out of operation, the cooperative task cannot be carried out. Decentralized methods solve this limitation as each agent acts individually, interacts with the other agents when it is possible and decides on its own how to contribute to the collaborative reconstruction task. The fourth research line that has to be mentioned is the network and communication protocols, since scalability is one of the biggest issues in MRSLAM, where communications play a relevant role. The more agents the multi-robot system has, the more data must be processed and sent through the network. In conclusion, the aforementioned research lines constitute the ideal MRSLAM technique, thus researchers are encouraged to consider them when developing new and modern algorithms in this field. Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Supplementary materials Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.eswa.2019.113109. References Andre, T., Neuhold, D., & Bettstetter, C. (2014). Coordinated multi-robot exploration: Out of the box packages for ros. In Proceedings of the IEEE Globecom workshops (GC Wkshps) (pp. 1457–1462). doi:10.1109/glocomw.2014.7063639. Austin, TX, USA. Bahraini, M. S., Bozorg, M., & Rad, A. B. (2018). Slam in dynamic environments via ml-ransac. Mechatronics, 49, 105–118. Balcılar, M., Yavuz, S., Amasyalı, M. F., Uslu, E., & Çakmak, F. (2017). R-slam: Resilient localization and mapping in challenging environments. Robotics and Autonomous Systems, 87, 66–80. Battistelli, G., Chisci, L., & Laurenzi, A. (2017). Random set approach to distributed multivehicle slam. International Federation of Automatic Control, 50, 2457–2464. doi:10.1016/j.ifacol.2017.08.410. Baya, H., Essa, A., Tuytelaars, T., & Van Gool, L. (2006). Speeded-up robust features (surf). Computer Vision and Image Understanding, 110, 346–359. Blanco, J. L., Gonzalez-Jimenez, J., & Fernandez-Madrigal, J. A. (2007). A new method for robust and efficient occupancy grid-map matching. Technical Report. 3rd. Iberian Conference on Pattern Recognition and Image Analysis (IBPRIA’07). Blanco, J. L., González-Jiménez, J., & Fernández-Madrigal, J. A. (2013). A robust, multi-hypothesis approach to matching occupancy grid maps. Robotica, 31, 687– 701. doi:10.1017/s02635747120 0 0732. Carpin, S. (2008). Fast and accurate map merging for multi-robot systems. Autonomous Robots, 25, 305–316.

12

C.A. Velásquez Hernández and F.A. Prieto Ortiz / Expert Systems With Applications 145 (2020) 113109

Cho, H., Kim, E. K., & Kim, S. (2018). Indoor slam application using geometric and icp matching methods based on line features. Robotics and Autonomous Systems, 100, 206–224. Cortés, X., & Serratosa, F. (2016). Cooperative pose estimation of a fleet of robots based on interactive points alignment. Expert Systems With Applications, 45, 150–160. Engelhard, N., Endres, F., Hess, L., Sturm, J., & Burgard, W. (2011). Real-time 3D visual Slam with a hand-held RGB-D camera. Technical Report. European Robotics Forum. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with application to image analysis and automated cartography. Communications of the ACM, 24, 381–395. Gaspar, A. R., Nunes, A., Pinto, A. M., & Matos, A. (2018). Urban@cras dataset: Benchmarking of visual odometry and slam techniques. Robotics and Autonomous Systems, 109, 59–67. Harris, C., & Stephens, M. (1988). A combined corner and edge detector. In Proceedings of the of fourth alvey vision conference (pp. 147–151). Havangi, R., Nekoui, M., & Teshnehlab, M. (2012). An improved fastslam framework using soft computing. Turkish Journal of Electrical Engineering & Computer Sciences, 20, 25–46. Howard, A. (2006). Multi-robot Simultaneous Localization and Mapping using Particle filters. Technical Report. Jet Propulsion Laboratory, NASA. Jeong, J., Yoon, T. S., & Park, J. B. (2018). Multimodal sensor-based semantic 3d mapping for a large-scale environment. Expert Systems with Applications, 105, 1–10. Kohlbrecher, S., Stryk, O., Meyer, J., & Klingauf, U. (2011). A flexible and scalable slam system with full 3d motion estimation. In Proceedings of the international symposium on safety, security, and rescue robotics (ssrr). Kyoto, Japan. Kojima, T., Okawa, Y., & T. , N. (2013). Multi-robot slam for large scale map building using relative information of local maps. In Proceedings of the SICE annual conference 2013 (pp. 164–169). Nagoya, Japan. Kshirsagar, J., Shue, S., & Conrad, J. M. (2018). A survey of implementation of multirobot simultaneous localization and mapping. In Proceedings of the Southeastcon. doi:10.1109/secon.2018.8478985. Labbé, M., & Michaud, F. (2014). Online global loop closure detection for large-scale multi-session graph-based slam. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems. Lee, H., Lee, S., Choi, M., & Lee, B. (2012a). Probabilistic map merging for multi-robot RBPF-slam with unknown initial poses. Robotica, 30, 205–220. Lee, H., Lee, S.-H., Lee, T.-S., Kim, D.-J., & Lee, B. (2012b). A survey of map merging techniques for cooperative-slam. In Proceedings of the 9th international conference on ubiquitous robots and ambient intelligence (URAI) (pp. 285–287). Daejeon, South Korea Lee, H.-C., & Lee, B.-H. (2011). Improved feature map merging using virtual supporting lines for multi-robot systems. Advanced Robotics, 25(13-14), 1675–1696. Lee, H. C., Roh, B. S., & Lee, B. H. (2016). Multi-hypothesis map merging with sinogram-based pso for multi-robot systems. Electronics Letters, 52(14), 1213–1214. Lee, H. S., & Lee, K. M. (2009). Multi-robot slam using ceiling vision. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 912–917). St. Louis, MO, USA León, A., Barea, R., Bergasa, L. M., Lopez, E., Ocaña, M., & Schleicher, D. (2009). Slam and map merging. Journal of Physical Agents, 3, 13–23. Li, S.-p., Zhang, T., Gao, X., Wang, D., & Xian, Y. (2019). Semi-direct monocular visual and visual-inertial slam with loop closure detection. Robotics and Autonomous Systems, 112, 201–210. Liu, Y., Fan, X., & Zhang, H. (2013). A fast map merging algorithm in the field of multirobot slam. The Scientific World Journal, 2013, 1–8. doi:10.1155/2013/169635. Lowe, D. G. (1999). Object recognition from local scale-invariant features. Technical Report. International Conference on Computer Vision. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60, 99–110. Moratuwage, D., Vo, B., Wang, D., & Wang, H. (2012). Extending Bayesian RFS slam to multi-vehicle slam. In Proceedings of the control automation robotics & vision (ICARCV). Guangzhou, China. Mur-Artal, R., Montiel, J. M. M., & Tardós, J. (2015). Orb-slam: a versatile and accurate monocular slam system. In Proceedings of the IEEE transaction robotics: 31 (pp. 1147–1163).

Ozog, P., Johnson-Roberson, M., & Eustice, R. M. (2017). Mapping underwater ship hulls using a model-assisted bundle adjustment framework. Robotics and Autonomous Systems, 87, 329–347. Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., . . . Ng, A. (2009). Ros: an open-source robot operating system. In Proceedings of the IEEE ICRA workshop on open source software. Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. (2011). Orb: an efficient alternative to sift or surf. In Proceedings of the international conference on computer vision (iccv). Sasaoka, T., Kimoto, I., Kishimoto, Y. K. T., & Nakashima, H. (2016). Multi-robot slam via information fusion extended Kalman filters. International Federation of Automatic Control, 49, 303–308. SolÃ, J., Vidal-Calleja, T., Civera, J., & MartÃnez-Montiel, J. M. (2012). Impact of landmark parametrization on monocular EKF-SLAM with points and lines. International Journal of Computer Vision, 97, 155–160. Stachniss, C. (2009). Robotic Mapping and Exploration. Springer. Sullivan, N., Grainger, S., & Cazzolato, B. (2018). Analysis of cooperative localisation performance under varying sensor qualities and communication rates. Robotics and Autonomous Systems, 110, 73–84. Sun, K. A. D., & Wendt, T. (2009). Multi-robot range-only slam by active sensor nodes for urban search and rescue. In RoboCup 2008: Robot Soccer World Cup XII (pp. 318–330). Tardioli, D., Parasuraman, R., & Ögren, P. (2019). Pound: A multi-master ros node for reducing delay and jitter in wireless multi-robot networks. Robotics and Autonomous Systems, 111, 73–87. doi:10.1016/j.robot.2018.10.009. Tomasi, C., & Kanade, T. (1991). Detection and Tracking of Point Features. Technical Report. International Journal of Computer Vision. Topal, S., Erkmen, I., & Erkmen, A. M. (2010). A novel map merging methodology for multi-robot systems. In Proceedings of the World Congress on Engineering and Computer Science. Trajkovic, M., & Hedley, M. (1998). Fast corner detection. Image and Vision Computing, 16, 75–87. Tungadi, F., Lui, W. L. D., Kleeman, L., & Jarvis, R. (2010). Robust online map merging system using laser scan matching and omnidirectional vision. In 2010 ieee/rsj international conference on intelligent robots and systems (pp. 7–14). Taipei, Taiwan Velasquez, C., & Prieto, F. (2015). A new corner detector approach for occupancy grid map merging. In Memorias iv congreso internacional de ingeniería mecatrónica y automatización - ciima 2015 (pp. 60–71). Envigado, Colombia Velasquez, C., & Prieto, F. (2019). A corner detector algorithm for feature extraction in simultaneous localization and mapping. Journal of Engineering Science and Technology Review, 12, 104–113. Wang, K., Jia, S., Li, Y., Li, X., & Guo, B. (2012). Research on map merging for multi-robotic system based on RTM. In Proceedings of the IEEE international conference on information and automation (pp. 156–161). Wang, X., & Wang, J. (2017). Detecting glass in simultaneous localisation and mapping. Robotics and Autonomous Systems, 88, 97–103. Wang, Y., Zhang, W., & An, P. (2017). A survey of simultaneous localization and mapping on unstructured lunar complex environment. In Proceedings of the AIP conference proceedings: 1980. Xu, W., Jiang, R., & Chen, Y. (2012). Map alignment based on plicp algorithm for multi-robot slam. In Proceedings of the IEEE international symposium on industrial electronics. Yamauchi, B. (1998). Frontier-based exploration using multiple robots. In Proceedings of the second international conference on autonomous agents. Yan, Z., Fabresse, L., Laval, J., Bouraqadi, N., & MacDonald, B. A. (2014). Team size optimization for multi-robot exploration. In Proceedings of the simulation, modeling, and programming for autonomous robots.: 8810 (pp. 438–449). doi:10.1007/ 978- 3- 319- 11900- 7_37. Yang, S., Scherer, S. A., Yi, X., & Zell, A. (2017). Multi-camera visual slam for autonomous navigation of micro aerial vehicles. Robotics and Autonomous Systems, 93, 116–134. Yao, E., Zhang, H., Xu, H., Song, H., & Zhang, G. (2018). Robust rgb-d visual odometry based on edges and points. Robotics and Autonomous Systems, 107, 209–220. Zhou, X. S., & Roumeliotis, S. I. (2006). Multi-robot slam with unknown initial correspondence: The robot rendezvous case. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (pp. 1785–1792). Beijing, China.