A real-time data acquisition and processing framework for large-scale robot skin

Robotics and Autonomous Systems 68 (2015) 86–103

Contents lists available at ScienceDirect

Robotics and Autonomous Systems journal homepage: www.elsevier.com/locate/robot

A real-time data acquisition and processing framework for large-scale robot skin✩ S. Youssefi ∗ , S. Denei, F. Mastrogiovanni, G. Cannata Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Via Opera Pia 13, 16145, Genoa, Italy

highlights • • • • •

Hard real-time data acquisition and distribution algorithms. Modular and scalable design which ensures data coherency and consistency. Provides skin technology abstraction, supporting heterogeneous technologies. Enables portability of skin-based algorithms among different technologies. Extensive validation and performance tests are performed.

article

info

Article history: Received 15 May 2014 Received in revised form 9 January 2015 Accepted 19 January 2015 Available online 9 February 2015 Keywords: Robot skin Tactile Middleware Skinware Software Framework

abstract Large-scale tactile sensing applications in Robotics have become the focus of extensive research activities in the past few years, specifically for humanoid platforms. Research products include a variety of fundamentally different robot skin systems. Differences rely in technological (e.g., sensory modes and networking), system-level (e.g., modularity and scalability) and representation (e.g., data structures, coherency and access efficiency) aspects. However, differences within the same robot platform may be present as well. Different robot body parts (e.g., fingertips, forearms and a torso) may be endowed with robot skin that is tailored to meet specific design goals, which leads to local peculiarities as far as technological, system-level and representation solutions are concerned. This variety leads to the issue of designing a software framework able to: (i) provide a unified interface to access information originating from heterogeneous robot skin systems; (ii) assure portability among different robot skin solutions. In this article, a real-time framework designed to address both these issues is discussed. The presented framework, which is referred to as Skinware, is able to acquire large-scale tactile data from heterogeneous networks in realtime and to provide tactile information using abstract data structures for high-level robot behaviours. As a result, tactile-based robot behaviours can be implemented independently of the actual robot skin hardware and body-part-specific features. An extensive validation campaign has been carried out to investigate Skinware’s capabilities with respect to real-time requirements, data coherency and data consistency when large-scale tactile information is needed. © 2015 Elsevier B.V. All rights reserved.

1. Introduction The sense of touch is essential for human beings [1] and has been shown to be of the utmost importance in Robotics for manipulation tasks [2,3], safety issues [4], and social behaviours [5]. Tactile sensors have been the subject of extensive research for three decades [6,7]. However, only recent activities in physical hu-

✩ A preliminary version of the work discussed in this article has been presented in Youssefi et al. (2011) [70]. ∗ Corresponding author. E-mail addresses: [email protected] (S. Youssefi), [email protected] (S. Denei), [email protected] (F. Mastrogiovanni), [email protected] (G. Cannata).

http://dx.doi.org/10.1016/j.robot.2015.01.009 0921-8890/© 2015 Elsevier B.V. All rights reserved.

man–robot interaction [8] have stressed the importance of using large-scale tactile systems, specifically in the form of robot skin [9]. Focusing on robot skin systems, possibly covering large surfaces of a robot body, different solutions as far as technological, system-level and data representation aspects are concerned, have been investigated. From a technological perspective, transduction, distributed communication as well as networking issues must be considered [9]. At the system-level design, novel issues arise when adopting large-scale robot skin. Such issues include scalability and modularity [10], conformance to robot surfaces [11,12], as well as how to scale wiring patterns [13] and embedded communication networks [14,15], just to name but few. According to these design principles, many examples of robot skin systems have been presented during the past few years, either based on modular designs [11,12,16,17], or on non-modular but

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

flexible and highly conformable sensor arrays [18–25]. Besides the adopted sensing mode [9], these robot skins have adopted both industrial and ad hoc solutions in actual robots, such as VMEbus [18], PCI bus [26], CAN bus [19,12,16], SMBus [11], EtherCAT [15], UART bus [17] or other methods [20–24,27,25]. However, at the representation level, it is necessary to identify strategies for data acquisition and processing, which must be independent from both technological and system-level solutions. Furthermore, huge amounts of tactile data originating from distributed sources are of limited use if not processed in time and according to well-defined contact models [28]. Tactile data representation and processing is the basis to implement tactile-based robot behaviours, e.g., exploiting local information related to the distribution of force exerted on the robot surface. At a first glance, it may be argued that tactile data processing can be tackled by extending established computer vision techniques. However, there is no shortage of reasons to conclude that these two scenarios are significantly different.

• Tactile elements (i.e., the discrete sensing locations of robot skin systems, henceforth referred to as taxels) are distributed over robot body parts with varying shape and curvature. As a consequence, taxel locations usually do not form a regular square grid. This renders the concept of a bitmap not directly applicable, as it assumes data to be organized on a regular and well-defined 2D grid. On the contrary, tactile images are hardly structured [29,30] in a regular arrangement. • Tactile images depend on specific robot–environment configurations [31]. Contacts may arise as a consequence of robot postures originating from control actions. Moreover, such interaction phenomena as robot self-touch need to be considered, where the tactile image strictly depends on the robot configuration only. • Different robot body parts may demand different requirements from a robot skin system in terms of tactile sensor density, resolution and sensitivity. Thus, tactile images need to be processed taking into account the peculiar characteristics of the body area originating tactile data. • In analogy with biological skin, robot skin may be designed to embed different types of transducers, e.g., temperature or vibration [17], whose values may be correlated over time and space [32]. At a representation level, different tactile images referring to different sensing modes may need to be jointly processed, e.g., to better describe the contact event [17]. In order to properly manage the acquisition and the efficient processing of tactile data, the need for a robot skin middleware arises, i.e., a software framework capable of providing (i) realtime data access strategies such as best effort, periodic and eventbased, and (ii) data abstraction features for tactile-based robot behaviours, specifically with respect to the robot skin hardware and mechanical implementation, the robot kinematic configuration and the specific robot body part originating tactile data. The focus of this article is to propose such a software framework, providing a hardware abstraction layer for tactile-based robot behaviours, able to guarantee a consistent, common interface towards different technological and system-level solutions. To the best of the authors’ knowledge, the need for such a software framework is solely acknowledged in [33], where a method for organizing tactile data originating from a sensory body suit is presented. The development of software frameworks for Robotics is topic of active research. Relevant examples of such frameworks include both non real-time and real-time systems. The first class includes, in chronological order, Ethnos/SeART [34–36], OROCOS [37], CLARAty [38], Player/Stage [39,40], Orca [41], RT-Middleware [42], YARA [43], YARP [44] and ROS [45]. The second class includes OpenDDS (an open source implementation of DDS [46]),

87

Fig. 1. Taxels, modules, patches and the network in the considered robot skin technology.

MiRPA [47] and RDDS [48]. However, these frameworks provide general-purpose mechanisms to design and implement concurrent and distributed software robot components. On the contrary, the proposed software framework as a whole, which will be henceforth referred to as Skinware, can be used as a specialized component in any of these frameworks, none of which provides any comparable alternative. It is noteworthy that although the same term middleware is used to both refer to the proposed software framework and the aforementioned Robotics software frameworks, Skinware is placed at a layer closer to hardware than that of those frameworks and does not try to provide an alternative to them. The main contribution of the article is two-fold. On the one hand, the definition of a software architecture able to manage large-scale robot skin systems in real-time and to provide users with data structures and communication protocols which are independent from the actual robot skin hardware and robot configuration. On the other hand, a detailed account of the choices that have been made to implement and validate the defined software framework in real-world scenarios. The implementation of Skinware is available online1 as free software. A preliminary version of the work discussed in this article has been presented in Youssefi et al. (2011) [70]. The article is organized as follows. Section 2 presents the terminology associated with the Skinware software framework. Section 3 gives an overview of the conceptual architecture of Skinware as well as the implemented structure. The main features of Skinware are discussed in Section 4. Relevant algorithms are discussed in Section 5. Tests performed on Skinware are presented in Section 6. Section 7 contains further discussions and future directions. Conclusion follows. 2. Terminology 2.1. Robot skin hardware In this article, we will adopt as a reference, the robot skin technology proposed in [12] (see Fig. 1) which was further developed in [10], and finally tested on several various robots [49]. This kind of robot skin system is an example of modular and scalable design, which is aimed at covering large-scale robot surfaces. Since mostly all the available modular and scalable designs for robot skin systems discussed in the literature share many similarities, it is noteworthy that application of the concepts discussed in this article to other skin designs is straightforward. Nonetheless, few examples are provided following the definitions. The typical robot skin system based on modular and scalable design choices [11,12,16,17] is hierarchically organized on the basis of the following elements. Taxel. A single tactile transducer, i.e., a discrete sensing point on the robot surface.

1 https://github.com/maclab/skinware/.

88

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Fig. 2. Mapping between the considered robot skin technology to abstract entities provided by the proposed middleware. Arrows specify the relation between hardware elements and the abstract concepts.

Module. A set of taxels closely located, usually managed by local hardware interfaces. Patch. A network of modules, and each patch is typically associated with a micro-controller, which is responsible for managing its associated modules and, in particular, for acquiring, routing and possibly local processing of tactile information. Embedded Network (or simply network). The infrastructure connecting various robot skin patches to the processing unit as well as the communication protocol used by the particular robot skin technology. Remark. This view of the elements in a robot skin is arguably generic and all robot skin technologies known to the authors are compatible with the above taxonomy. This is also valid for skins with a simpler hierarchy. For example a non-modular skin design can be viewed as having one module per taxel. For comparison, here are possible mappings for a few other robot skins:

• The modular robot skin in [11] has mechano-electrical sensors as taxels on H-shaped modules which are grouped together in •

•

•

•

patches. These patches are handled by a microcontroller which is slave to an in-house built SMBus network. The non-modular robot skin in [22] has piezoresistive sensors as taxels, each can be assumed to be an individual module, sheets of which as a patch are handled by a microcontroller and the data is acquired through an unspecified bus as the network. The non-module robot skin in [23] has multi-vibrator capacitive sensors as taxels, which again can be assumed to constitute an individual module, that are placed in sensor arrays viewed as a patch. These patches are themselves connected with an unspecified bus as the network. The modular robot skin for a humanoid glove in [27] has resistive sensors as taxels on highly branched modules which form a single patch per glove whose microcontroller transmits data through an unspecified network. The modular robot skin in [17] has multiple sensing modalities including temperature. The next section introduces sensors to be used instead of taxels to more accurately refer to these sensing modalities. These sensors are grouped together in hexagonal modules (dubbed HEX-O-SKIN) and together they form patches that transmit data through a UART network.

2.2. Abstract concepts In order to define a robot skin middleware, a number of abstract elements at the robot skin representation level need to be introduced, which are mapped to real hardware instances inside the software framework (Fig. 2). As previously pointed out, this mapping is fundamental. On the one hand, design and development of tactile-based algorithms are grounded on the assumption that tactile data can be processed independently from the particular robot

skin hardware implementation. On the other hand, abstract entities can enforce software portability. Abstract entities include the following elements. Sensor. Although current robot skin implementations are mostly designed to take only pressure information into account, it is possible to design more general robot skins containing other sensor modes, e.g., temperature or vibration [17]. Henceforth, the term sensor will be used regardless of the considered sensing mode. Sensor type/layer. Similarly to how the human skin hosts different types of sensors in a layered fashion [32], a robot skin middleware should be capable of managing sensor data originating from different sensor types. The terms sensor type, sensor layer and simply layer refer to one such sensor type, as if arranged in layers with respect to other sensor types. In principle, each robot skin networking infrastructure may carry information originating in multiple conceptual, if not physical layers. Region. A region is an area of interest in the robot body surface. This abstract concept represents a set of sensors in the robot skin that may be conceptually grouped. Examples of such regions include hand palm, forearm and torso. Regions may also overlap if the same set of sensors belongs to two different conceptual groups, e.g., forearm and robot front. Fig. 2 shows how it is possible to map these general concepts to virtually any robot skin hardware implementation, e.g., to the taxel, module, patch and network structure of the considered robot skin technology. Skinware is the proposed software framework that provides such elements, abstracting the robot skin from the physical hardware implementation. 3. Skinware architecture Tactile data may arrive at an application unprocessed or processed several times based on the application’s needs. As an example, consider three applications. A low level application that detects touches on the robot skin may directly process raw tactile data. An application that detects movements of the touch along the surface of the robot skin may provide data based on the results of the previous application. A high level application that detects certain movement patterns as gestures would in turn use the results of the second application. To support such process hierarchy, Skinware is designed as a layered architecture (Fig. 3). Two software layers are always present between the robot skin hardware and user applications, namely the driver handler layer (responsible for hardware management and communication) and the user handler layer (in charge of communicating with user applications). Furthermore, multiple instances of another layer, namely the service handler layer (i.e., for low-level software services), can be optionally present between the driver and the user handler layers, but in a different data path. The main components introduced by Skinware are as follows. Driver. A driver is responsible for communicating both with the robot skin networking infrastructure and Skinware. Such drivers allow Skinware to communicate with heterogeneous skin technologies simultaneously, thereby obtaining data from different sensor types connected to different networking infrastructures. Application. The interchangeable terms application and user both refer to a software program acquiring tactile data through the Skinware framework. Service. A user of tactile data providing already processed data to other users, whether as its main functionality or as a side effect. Writer. A real-time thread in the driver handler component responsible for acquiring data from the robot skin hardware (through a driver) and writing them to internal buffers. Reader. A real-time thread in either the user handler or service handler components that reads data from buffers in Skinware and exposes them to users.

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

89

A driver first provides Skinware with the topology of the layers it manages and then the sensor data at runtime. Skinware allows the drivers to have full control over the network hiding all the details and possible complexities regarding the transfer of tactile data. Remark. Such complexities include synchronization and sharing of the data. Furthermore, to ensure liveness of the system, each driver must be able to shut down and later restart at runtime, whether it is for power saving or due to a system fault, without interrupting other drivers, users or services.

Fig. 3. The layered Skinware architecture. Unprocessed data from the hardware layer to applications pass through two layers, whereas processed data reach the same applications passing through one extra layer. As this figure depicts, the driver handler is the bottom-most layer while the user handler is the topmost. The service handler is in between.

Service handler. A real-time thread in the service handler component that manages the provided service, e.g., storing processed data provided by that service. Data Frame. A set of tactile data acquired from sensors of a certain layer in one acquisition period. Each data frame is a tactile image representing the activity in the layer. According to the Skinware design (Fig. 3), the driver handler component is close to the robot skin hardware, interacting with the drivers, whereas the user handler component provides data to applications, and finally the service handler component interacts with both standalone services and application-based services, defined further in this Section. All these three components interact using shared locking mechanisms and write/read data to/from shared memory. In particular, the Skinware architecture is designed to fulfil the following properties (as well as a number of features discussed in Section 4).

• Hard real-time acquisition. Skinware must be able to perform data acquisition at all times. Specifically, writers must never miss a deadline due to the synchronization mechanism with readers. • Independence from users. Data acquisition in Skinware must remain sound and unblocked regardless of the existence of readers or their number. This property guarantees predictable behaviour in systems where the number of readers can dynamically change. According to Fig. 3, the robot skin hardware is the physical device represented by multiple abstract sensor layers (as defined in Section 2.2), which is connected to the main processing unit by means of different networks. Each network is handled by a driver that is specifically designed to manage the related communication interface. At the driver level, all tactile data are raw (i.e., depending on the sensing mode), organized by layers, patches and modules.

Communicating with drivers, writer threads are created, each corresponding to a single skin layer. They synchronize with readers using an algorithm introduced in Section 5.1 in the process of data transfer to higher levels. The choice of multiple concurrent threads managing robot skin data results in robustness and graceful degradation performance. For example, in case of a driver fault, this mechanism would allow disabling solely those layers that are managed by the faulty driver and recovering from the fault by a restart of that driver only; it would also allow selected parts of robot skin to be activated or deactivated at runtime, without affecting the execution of users. Data frames gathered by the writers (as well as relevant data structures introduced in the next section) are stored within the Skinware memory space and are accessible by and shared with applications and services through readers. This unique source of data ensures data consistency among different users. Readers in Skinware are spawned corresponding to writers, which were in turn spawned corresponding to data layers. Each layer’s reader is responsible for creating a local copy of that layer’s data at a selected frame rate. Then, each application can process its own local copy without conflicting with other applications. Applications based on Skinware can also register services with it. For each registered service, a service handler thread is spawned, which handles its output. Depending on the task, the application may also choose to create local copies of other service results. A standalone service may request readers for only some of the layers. Similar to application-based services, a service handler thread that manages the service output is created. Service results, similarly to raw tactile data, are stored in Skinware memory and shared with applications or other services along with their own synchronization locks. Using services, Skinware takes responsibility of any complexity regarding communication with various sub-components. Therefore, through the service interface introduced in Section 4, the service complexity is kept at a minimum. Each service selectively reads data from skin layers and provides arbitrary data. These are not interpreted by Skinware itself, but by the users of each service. Therefore, the responsibility for providing the users with an understanding of their output is left to services. Skin applications are the end users of Skinware. These may be standalone applications or components of a robot software framework, such as ROS or YARP. Two special applications implemented for Skinware are the calibrator and the regionalizer, which are described in the next section. 4. Skinware features Hard real-time data acquisition. Real-time data acquisition from large-scale robot skin is a key requirement for several tactile-based applications, for instance tactile-based robot control algorithms. This property guarantees that data acquisition is performed within a given deadline, thereby ensuring responsiveness of tactile-based applications. Skinware has adopted RTAI [50] as the real-time infrastructure. Appendix A discusses this choice.

90

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Concurrency among Skinware components. Skinware supports concurrency among all its components, and ensures data coherency and consistency for end-users while incurring minimum overhead. The tests presented in Section 6 measure the effect of implemented synchronization algorithms on the acquisition process. Data coherency. Skinware enables end-users to access data coherently. Indeed, a fundamental prerequisite is that any data acquisition method assure data being accessed by end-users not to be updated during their processing. In other words, Skinware ensures that data read by end-users belong to the same data frame. As an example, in tactile texture detection data coherency could be fundamental. Tactile texture detection algorithms may rely on filtering tactile information with differential filters, i.e., they may depend on mutual differences between one sensor value and the values of its neighbours. If data coherency were not ensured, data would be overwritten during the application of the filter, resulting in incorrect output that is accentuated especially by such high-pass filters. Data acquisition methods. In Skinware, writers are expected to read sensor data periodically, each one with a rate that depends on the low-level part of the whole architecture. On the other hand, end-users may require tactile data at a different pace. Skinware provides different tactile-based robot tasks with different strategies as far as data acquisition is concerned, based on their specific timing requirements.

• Periodic tasks. A tactile-based robot task may require tactile data periodically and not necessarily with the same period as data are acquired from the low-level network. Most of robot tactilebased tasks fall in this category, such as reflexive behaviours or control algorithms [51]. These tasks can choose a periodic acquisition method. • Aperiodic tasks. Tactile-based tasks requiring large and likely unknown execution time, such as high-level or intelligent robot behaviours, can choose a sporadic acquisition method. According to this strategy, tactile data acquisition is only performed upon request. • Soft real-time tasks. Soft real-time tasks, such as for monitoring or as a ROS module, may require data as they arrive with a best effort. These tasks may choose an as soon as possible (ASAP) acquisition method. While readers with sporadic acquisition mode acquire data upon user request, readers with ASAP mode acquire data when signalled by the corresponding writers, as soon as they are scheduled. Skinware allows for having different acquisition methods for different sensor types at the same time. Furthermore, it is also possible to have multiple instances of readers in a single application, each one reading data of the same sensor type at different rates. Data indices. Indices are aimed at entailing structures over data. The use of indices is very common in database applications where large amount of data exist and access to them needs to be optimized for the task at hand. Indices help reducing search time for a specific data instance or property. Since a typical configuration of robot skin contains tens of thousands of sensors, few indices have been introduced to optimize searching for active robot skin regions, to inspect an area of interest or to generally structure the data according to different logics. The Entity-Relationship Diagram (ERD) [52] of the indices integrated in Skinware is presented in Fig. 4 and described as follows.

• Sensor/module/patch/type structure. Skinware allows users to inspect tactile data through the hierarchy structure entailed by the hardware design. For a software application wishing to treat contact phenomena in a more abstract way, the use of this structure may not be desirable, whereas a hardware monitoring task may greatly benefit from such a data perspective. This

Fig. 4. An Entity-Relationship diagram of Skinware indices. Entity attributes are omitted for brevity.

structure is implemented as shown in Fig. 5 on the top left hand side. Different data structures, which correspond to the various hardware design levels, can be queried at the same time. • Sensor/region structure. As previously discussed, regions are user-defined areas of interest on the robot surface, which are independent from the actual hardware sensors configuration. By using the sensor-region relation, it is possible to inspect the status of certain areas of the robot skin without any knowledge of its physical structure. For example, one may require access to tactile data related to the robot left palm, without any actual knowledge about which micro-controller is in charge of acquiring data. Furthermore, skin regions may be redefined at runtime without changing the underlying hardware configuration. This structure is implemented as shown in Fig. 5 on the top right hand side. The Figure also introduces sub-regions, i.e., sets of sensors belonging to multiple regions. Sub-regions are introduced merely for a performance gain and can be invisible to users. In Section 6.3, this performance gain is thoroughly demonstrated. • 2D transformation map. In order to ground tactile-based data processing algorithms in a common metric space (regardless of the actual robot shape), work has been done to map the 3D manifold embedding the sensor structure to a 2D flattened tactile image [30]. Algorithms necessary to perform this mapping are integrated with Skinware and are performed after skin spatial calibration, i.e., the process of determining where sensors are located in 3D space with respect to a robot-centred reference frame (see below) [29,53]. • Sensor proximity graph. During the skin spatial calibration process, a proximity graph over sensors is created. For each individual sensor (represented as a node), the graph indicates the neighbouring sensors (via links). This graph can be used as a shared data structure by higher-level tactile-based robot behaviours. Section 6 shows an example of usage of such a data structure: a simple behaviour able to track parameters related to the robot skin status over time, e.g., the history of sensor activations. Driver interface. Drivers are responsible for initializing each sensor network and the actual data acquisition processes. A specific, ad hoc communication protocol has been implemented, which allows each driver to communicate with Skinware. Specifically, the protocol allows a network driver to register itself with Skinware and control whether each supported sensor layer should be paused or running. To increase fault tolerance, drivers are allowed to detach themselves from Skinware at any time and reattach themselves later as they see fit, e.g., to restart or reconfigure a faulty network. The driver handler component is responsible for handling requests by the drivers and managing the acquisition threads. This

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

91

Fig. 5. The implementation of indices in Skinware. Top left. The sensor-module-patch-type indices. Top right. The sensor-region indices. Sub-regions consist of contiguous blocks of sensors spread over possibly many layers and are introduced solely for efficiency purposes. Bottom. The implementation described in UML.

includes thread creation, execution, watchdog and synchronization with end-users and services. Therefore, Skinware greatly facilitates the task of data acquisition by the drivers and simplifies their source code. This in turn reduces new driver implementation costs as well as the chance of introducing a bug in the system. Service interface. Similar to drivers, services are components that can attach to and detach from Skinware during its execution, thereby providing useful processed data to other users. The communication protocol between service components and Skinware is similar to that of drivers. The service handler component is similarly responsible for handling service requests and managing service threads. Unlike drivers, the threads created for the services are not writers, but are reader threads, i.e., services are a type of Skinware users. However, services may produce output that can be used by other users. Skinware is responsible for handling communication and synchronization between those users and the services. The service interface has been developed with the goal of enforcing Skinware extendibility. The use of services has multiple benefits, which are outlined in the following paragraphs.

• Code reuse. A service can be easily used for the task at hand and copied across different platforms as it ensures decoupling of the service process from the use of its output. Without services, each program would be inclined towards packaging both the preprocessing the service could offer and the final usage, making it impossible, or costly at best, to reuse the implemented preprocessing algorithm. • Data reuse. Certain data preprocessing may be beneficial to multiple end-users. Without the use of services, users are bound to either redo the preprocessing each on their own, or to implement a mechanism to share these data themselves, which would be the same functionality the service interface of Skinware is providing, with the exception that Skinware can handle this mechanism more proficiently.

• Focused users. Normally, Skinware users are applications acquiring data from all layers and processing them in an abstract way. However, there may be need for certain processes to focus on certain aspects of the robot skin. Examples of such services include one that checks for hardware failure and issues the relevant command, or a service monitoring robot skin for certain data patterns indicating a dangerous situation for the robot. Through the service interface, normal users can also register themselves as services, viewed by other users as if they were standalone services. Services whose main functionality is to provide data for other users are dubbed standalone, whereas services that provide data as a side-effect of an application are termed application-based.

• Standalone services are fast, specialized or data-independent, such as those generating reflexive responses, interested solely in a particular layer or area of interest, or watchdogs. They may not be users of skin data per se and therefore their readers, if spawned, may not keep a local data copy. Services themselves may choose to do so if necessary. • Application-based services are side effects of the applications that choose to share their computation results with other possible users. For instance, an application detecting a texture in contact with finger tips primarily produces these data for itself, but is also able to share the data with other applications through a service. Any application-based service may also be implemented as a standalone service and vice versa, the choice of which is made by the system developer. Robot skin spatial calibration. Before the robot skin is spatially calibrated, Skinware is already functional. This means that data acquisition is properly carried out and propagated to users. However, solely having the sensor responses would not be of much use if not coupled with spatial calibration data indicating each sensor’s 3D position and orientation.

92

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

During the calibration phase of Skinware, it is possible for a calibration program to start using robot skin data and, by controlling the robot, performing automated spatial self-calibration [29, 53,51]. After the calibration has occurred, calibration data is propagated to Skinware and through it, to all end-users of Skinware. During this phase, the 2D skin mapping, as well as the sensor proximity graph, are generated. Skinware is capable of caching and reloading the calibration data so that in case of robot shutdown and restart, or merely restart of Skinware, calibration would not need to be redone. Regionalization. After skin spatial calibration occurs, the next phase in initializing Skinware is to create areas of interest over the robot surface, a process which we refer to as regionalization. This task is carried out by grouping sets of sensors, each group denoting one specific area of interest. Similarly to what happens during the calibration process, a regionalization program is allowed to acquire data providing the application designer with a graphical interface to define skin regions, either by activating the region sensors on the robot or selecting the relevant sensors manually. Regionalization data are then propagated to Skinware, through which all end-users would have access to them. Skinware can also cache and reload the regionalization data so that it would not need to be repeated upon each restart.

its working buffer with the free one. If no free buffers were available, the writer would periodically retry swapping buffers until its execution period would be over. This procedure is presented in Algorithm 1. It is noteworthy that the proposed solution shares a number of similarities with Periodic Data Buffers (PDB) [55], otherwise known as Cyclic Asynchronous Buffers (CAB) [56]. However, while Skinware uses rwlocks to protect buffers, PDB relies on the fact that (on most systems) pointer writes are atomic. Nevertheless, usage of rwlocks results in certain abilities such as the ASAP acquisition mode, since readers can be blocked and immediately notified by writers as a buffer gets unlocked, instead of having to poll buffers’ availability. Furthermore, Buttazzo [57] only mentions that there could be lost messages if writers were faster than readers. However, this issue is not addressed in [57]. The notion of swap skips (later introduced in this Section) targets a similar issue and is properly investigated in this article. Algorithm 1 The writer algorithm 1: 2: 3: 4: 5:

5. Synchronization algorithms

6: 7:

This section presents a few of the more important algorithms implemented in Skinware. First, synchronization algorithms between readers and writers are discussed. Then, an inevitable issue arising by the use of such synchronization algorithms is discussed and resolved.

8: 9: 10: 11: 12:

5.1. Synchronization between readers and writers

13: 14:

Given the nature of robot skin hardware, i.e., a large amount of sensors producing data (possibly) at high frequency, the typical configuration includes one writer spawned per sensor layer, which periodically requests data. Various solutions to the readers–writers synchronization problem are available. A simple readers–writers lock (rwlock in short) is a classical solution, which is not adequate in our case since it violates both the hard real-time acquisition and independence from users requirements introduced in Section 3. Various other algorithms have been proposed in the literature, which are often characterized by a not negligible overhead [54]. McKenney and Slingwine propose the Read–Copy–Update (RCU) algorithm. However, for the purposes of Skinware, RCU has multiple drawbacks. First, RCU focuses on readers with lower latencies and performs poorly in update-heavy applications. In Skinware, writers have higher priority as they must acquire the most recent data with minimum latency e.g., for control applications. Second, RCU is characterized by a very dynamic memory usage, which introduces significant unpredictability and, as a consequence, is not suitable for real-time use. Finally, RCU dictates data copy on the writer side, which increases data acquisition latency and is harmful for certain latency-critical applications. Therefore, it is important for the Skinware framework to design its own synchronization algorithms. In order to ensure data coherency in users and to avoid blocking writers, the proposed solution exploits multiple buffers between readers and writers. This allows users to read from a free buffer while a writer is simultaneously acquiring data to be stored into another buffer. In this solution, a writer can use a rwlock to keep its working buffer locked while other buffers are free. Once the acquisition is done, a writer searches a buffer that is not being used by any reader and swaps

15: 16: 17: 18:

procedure writer(l) ◃ Writer for layer l b←0 write_lock(r w ll,b ) w bl ← b while running do t ← now acquire(l, b) ◃ The acquire callback of the corresponding driver while period time left > threshold do for i ← 1, . . . , Bl do if i ̸= b and try_write_lock(r w ll,i ) then lbl ← b ltl ← t write_unlock(r w ll,b ) b←i w bl ← b goto 21 end if end for

19:

sleep

20:

end while

21: 22: 23:

wait_period end while end procedure

The input to Algorithm 1 is a layer l for which the writer is created. Specifically, Bl is the number of buffers, b is the current buffer, r w l is an array of rwlocks shared with readers, whereas w bl , lbl and ltl are variables shared with readers corresponding, respectively, as the buffer being written, the last written buffer and its timestamp. The buffer being written lets readers know which buffer they must wait on. The last written buffer is free and available for readers to read. The timestamp is used by readers to know whether they have already read this buffer. Writers always have one buffer locked (line 3). Therefore, they are never blocked by this synchronization mechanism. This property guarantees both hard real-time acquisition and independence from users. Furthermore, they never write to buffers locked by users, thereby ensuring data coherency with respect to readers. If none of the other buffers were available to a writer, the buffer swap (lines 14 and 15) would be skipped, which is undesirable as the current buffer would be overwritten without a chance for users to read them. This issue is discussed later in this Section. For each writer and, as a consequence, for each sensor layer, a reader is spawned (for each user application) in one of the three acquisition methods foreseen by Skinware, namely as soon as

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Algorithm 2 The ASAP reader algorithm 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21:

procedure reader(l) ◃ Reader of layer l pb ← inv alid pt ← inv alid while running do b ← lbl t ← ltl if b = pb and t = pt then ◃ Buffer already read read_lock(r w ll,wbl ) b = lbl t = ltl else if try_read_lock(r w ll,b ) = fail then goto 5 ◃ A buffer swap has just happened end if end if Make local copy of data from buffer b read_unlock(r w ll,b ) pb ← b pt ← t end while end procedure

Algorithm 3 An extension to the ASAP reader algorithm to allow for periodic reads 1: 2: 3:

if periodic then

wait_period end if

possible (ASAP), periodic and sporadic. In the following paragraphs, the reader algorithm is presented, with each Section introducing a reading policy. 5.1.1. ASAP read Readers in ASAP mode are not aware of the time interval in which writers work. In order to be awakened, readers depend only on the synchronization mechanism. Basically, readers wait on the buffer being written to be unlocked and immediately perform acquisition afterwards. The procedure for ASAP readers is presented in Algorithm 2. In Algorithm 2, b is the buffer being read and t is its timestamp, whereas pb is the previously read buffer and pt is its timestamp. The algorithm first checks whether the last written buffer lbl has been read already (line 7). If so, it sleeps on the buffer being written w bl , effectively continuing once the writer has completed acquisition (lines 8–10). Otherwise, it locks the buffer (line 12) and continues with acquisition (lines 16–19). In both periodic and sporadic acquisition methods, this part of the algorithm is present, i.e., too fast invocation of a reader results in it synchronizing itself with the corresponding writer, thereby avoiding unnecessary locking and data copy. 5.1.2. Periodic read This acquisition method is similar to ASAP mode, with the difference that it periodically tries to read from a free buffer. This implies that the readers cannot have a period smaller than that of the corresponding writers. Algorithm 3 shows the lines that need to be added at the end of the main loop in Algorithm 2 (between lines 19 and 20) to allow a reader to also work in periodic mode. 5.1.3. Sporadic read In sporadic mode, readers wait on a request from users to start acquisition and then respond to users that are waiting for the results. Also in this case, the ASAP mode mechanism is present, i.e.,

93

Algorithm 4 An extension to the ASAP and periodic reader algorithm to allow for sporadic reads 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15:

while running do if sporadic then wait(request) end if ... if sporadic then u←1 while try_wait(request) do u←u+1 end while for i ← 1, . . . , u do signal(response) end for end if end while

too fast read requests result in a wait on the buffer being written. The additions to the reader algorithm for enabling sporadic reads are presented in Algorithm 4. This algorithm also introduces the request and response semaphores. Algorithm 4 waits for a user request (line 3). After acquisition is done, the reader signals the user (line 12). It is possible for multiple users in the same application to simultaneously request data acquisition. In this case, data acquisition needs not be repeated for each user. Therefore, before signalling the requesting user, the reader counts the number of additional requests u that have been made in the meantime (lines 8–10). It then signals as many users as there have been requests, therefore servicing all the simultaneous requests in a single read (lines 11 and 12). The request from users is simply done by signalling request and waiting on response. The complete writer and reader algorithms, including the additions further detailed in this Section are present in Appendix B. 5.2. Swap skips During the discussion of Algorithm 1, we referred to swap skips as an issue. It arises when a writer tries to lock a buffer not being used by any reader (and to unlock the one it has been writing to), but no such buffer exists. Assuming a single layer, Fig. 6 on the left hand side shows how a buffer swap may not be possible in a double-buffer instance of Skinware working on a single processing unit with preemption. The writer resolves to skip the buffer swap and carry on with overwriting its currently held buffer. Swap skips are most undesirable as they prevent a whole data frame from being accessed by any reader. The following paragraphs present a method to first predict swap skips and then to avoid it as well as other methods to reduce the chances of swap skips happening. In Section 6, the effectiveness of these algorithms is shown. It is noteworthy that higher execution times in readers increase the chance of swap skips. 5.2.1. Swap skip prediction For the sake of argument, let us assume there are only two buffers. A swap skip occurs when a reader starts reading data from the free buffer slightly before the next writer period starts, while the writer has not performed a buffer swap yet (due to other readers keeping the free buffer locked). Therefore, to predict the swap, the reader checks whether it can perform the data copy in time. On the one hand, the writer can greatly aid this prediction by providing the reader with its next expected swap. Algorithm 1 is thus augmented by the following line after the swap is done and before waiting for the next period: nsl ← now + Tw

94

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Fig. 6. Swap skip generation and avoidance. Execution times are greatly exaggerated for the sake of example. Numbers indicate the buffer data is being written to or read from. Left: how a swap skip can occur. Right: how a swap skip can be avoided.

In the new line, now is the current time, Tw is the writer period and nsl is the next expected swap time, which is a variable shared with readers. On the other hand, the reader can keep track of its own performance. If we assume tx to be the reader expected execution time, the reader can predict whether it will be able to perform the read before the writer tries to swap buffers. This is the case if the condition in (1) holds. now + tx ≤ nsl .

(1)

5.2.2. Swap skip avoidance The strategy to avoid swap skips is for readers to stall until the next time a writer swaps buffers and then continue with reading. This is shown in Fig. 6 on the right hand side. However, in order to prevent starvation, this delay is only made once. Therefore, if the reader task is scheduled in such a way that, once awaken, it still produces a swap skip, then the read is done regardless of consideration for swap skips. This can be the case when the system is heavily loaded. This strategy is a compromise between losing one data frame for all other readers and starvation of the current, possibly critical, reader. The algorithm for readers is thus modified and the changes are presented in Algorithm 5. Algorithm 5 Modifications to the reader algorithm to enable swap skip prediction and avoidance 1: . . . 2: tx ← 0 3: while running do 4: ... 5: if (b = pb and t = pt) or now + tx ≥ nsl then 6: ... 7: end if 8: t ← now 9: Make local copy of data from buffer b 10: tx ← max(tx , now − t ) 11: ... 12: end while

is high, it is still possible for readers to maintain a lock for a long time and to cause swap skips. Another effective method is to increase the number of buffers. As a matter of fact, the higher the number of buffers, the smaller the chance for a writer to fail in finding any free buffer. In the normal execution of Skinware, a writer has buffer w bl locked while readers have buffer lbl locked. The other buffers are thus free and available to the writer even if a few of the readers were not able to execute in expected time. A system overload or a reader fault may also cause a swap skip. Therefore, a higher number of buffers also increases fault tolerance. For example, if there are B buffers, it is possible for B − 2 readers to crash, leaving their buffers permanently locked, and the acquisition would still function, having been reduced to a double-buffered one. A watchdog can be responsible for unlocking these buffers and recovering the readers to further improve fault tolerance. The effectiveness of these methods are discussed in Section 6. 6. Experimental results This section discusses various tests performed on Skinware to characterize its performance and the effectiveness of its design as it has been discussed above. A test with actual robot skin is presented afterwards. Tests have been performed on a workstation with the following specifications.

• Operating System: Ubuntu 12.04 using the Linux kernel 2.6.35.9 patched with RTAI 3.9. Processor: Intel Core 2 Duo E8200 @2.66 GHz. RAM: Four 48spaces DIMM DDR2 Synchronous 667 MHz. Ethernet board: Realtek RTL-8169 Gigabit Ethernet. CAN board: PEAK-System Technik’s PCAN-PCI CAN-Bus controller. • Compiler: gcc and g++ version 4.6.3 with -O2 optimization level.

• • • •

The tests included in this article are mainly focused on three aspects.

• Performance. These tests measure the execution times of readers 5.2.3. Other methods In a special, yet common, case, swap skips can be avoided altogether by proper task scheduling. This is the case where a reader is periodic and has the same period as the corresponding writer, or an integer multiple of it. A real-time periodic task is scheduled based on its period and task offset. By assigning the task offset of readers to a time slightly after the next writer invocation, the system task scheduler will invoke readers shortly after the writer has been awakened, greatly reducing the chance of readers still working when the writer is awakened next. If the system load

and writers and compare them on the basis of the number of layers, users, buffers and regions. The utilization factor of Skinware on a target system, as well as its schedulability analysis, can be predicted by observing the effects of each of these parameters on the execution time. • Swap Skips. These tests measure the chances of swap skips happening, given different configurations. The tests are also aimed at showing the effectiveness of the methods introduced in Section 5.2. • Sub-regions. This test measures the impact of introducing subregions on the Skinware performance.

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

In all tests, Skinware has been initialized with 30 848 sensors, which correspond to a robot skin surface of roughly 1 m2 using the target ROBOSKIN technology [12]. It has been shown in [58] that 1 m2 is a reasonable coverage area for a full humanoid robot body. The target number of sensors has been computed as follows: #sensors =

1 m2 2 3  .89cm

×

12 

≃ 30 848.

sensor per module

module area

To remove the effects of hardware delays from the measurements, a simulation driver has been developed that generates arbitrary values as sensor responses. A driver for real robot skin would introduce delays that widely vary based on the skin technology. First, this delay is inevitable regardless of whether the driver is acquiring data for distribution by Skinware or otherwise and therefore it adds little value had it been included in the test results. Second, using the simulation driver allows analysing the behaviour of Skinware in isolation, which would have been otherwise impossible to separate from the effect of using a particular hardware. Finally, this driver allows the authors to stress-test Skinware with very large number of sensors that would have been otherwise unavailable as manufactured skin. For each test, the simulation driver mimics a robot skin divided in a configurable number of layers, with the following properties: (i) 12 sensors per module; (ii) 4–16 modules per patch; (iii) 30 848 sensors equally divided among the layers; (iv) each writer has a random acquisition rate ranging between 30 and 50 Hz; (v) each reader has a random and likely different acquisition rate between 10 and 100 Hz. Furthermore, in each test, the robot skin is regionalized in a configurable number of overlapping or disjoint regions. The choice of random acquisition rates has been taken to test Skinware in the most extreme cases. The disharmony between readers and writers greatly increases the chances of swap skips and it is therefore used to better test the effectiveness of the synchronization algorithms. In real-world applications, where readers often have a rate proportional to writers, Skinware is expected to perform better than the results of these tests actually show. It is noteworthy that execution times have been measured by a wall-clock time due to the lack of precise execution time calculation in the underlying real-time system. This implies that actual time spent by readers and writers is indeed smaller than what is reported, having taken into account also the time spent when they are blocked in the round-robin scheduling queue or on a synchronization lock. Particularly, certain configurations of the tests fit better with RTAI’s scheduling strategies on a dual-core CPU and introduce a smaller overhead while other configurations result in higher context switches and thus higher reported execution times. In the following tests, this effect can be viewed as regular peaks and troughs as the number of threads in the tests changes. This effect is less visible in worst-case execution times, which are in fact more valuable from a real-time schedulability point of view. 6.1. Performance In these tests, execution times of writers and readers have been measured during a set of 15-s Skinware executions. As a result, the number of samples in each test for each writer thread varies between 450 and 750 based on the randomly selected writer task periods and for each reader thread between 150 and 1500 based on the randomly selected reader task periods. Measurements have been acquired taking into account every possible combination of a number of configuration parameters (resulting in 8700 test result sets), namely the number of layers in the range 1–5, the number of internal buffers (specifically, 2, 3 and 5), the number of user

95

applications, each one with all readers working, in the range 1–15, the number of regions in the range 1–9 and 10, 20, 30, . . . , 200, as well as with either disjoint or overlapping regions, with each sensor assigned to three different regions in the later case. Each data point in the resulting graphs accumulates data samples for all variants of the test with the specified parameters. For example, a graph showing execution times versus number of users and number of layers for a given number of buffers has data accumulated for all configurations of regions. 6.1.1. Effects of layers In a first analysis, the effect of dividing the skin sensors among multiple layers is observed. Fig. 7 on the left hand side shows the average of accumulated execution times of writers against the number of layers in the presence of 1–15 user applications, while Fig. 7 on the right hand side shows the box plot of the maximum of those values, when three buffers are used. Similar results can be observed when using two and five buffers. Analysis. It can be observed that, as the number of layers increases, the overall execution time of writers increases as well, even though for a smaller number of user applications this variation is smaller. It can be observed that these values change more slowly as the number of layers increases. Furthermore, it can be observed that as the number of user applications increases, the execution time of writers increases as well. This latest observation is further discussed in Section 6.1.2. Rationalization. The following facts can be observed. 1. Increase in execution time as the number of layers increases. The execution of writers consists of two major parts, namely data acquisition and synchronization with users. On the one hand, as the number of layers grow, the acquisition time of each writer is reduced since the number of sensors managed by each layer is lower. However, the total acquisition time is only minimally affected since the total number of sensor values gathered by all writers is constant. On the other hand, the synchronization part of each writer is independent from the number of sensors managed by the thread. Therefore, as the number of layers increases and, accordingly, the number of writers increases, the total synchronization cost among all writers is increased. 2. Smaller increase in execution time with a small number of user applications. The synchronization algorithm of a writer periodically tries to swap buffers. Given the random acquisition rates of readers, as the number of user applications increases the chances of finding a free buffer gets smaller. Therefore, on average, a higher number of retries is required to successfully perform buffer swap. As the number of layers is increased, this cost is amplified since each layer separately performs synchronization and suffers the related cost. 3. Slower increase in execution time for a higher number of layers. The CPU of the workstation used for these tests has two cores. It is concluded that the observed behaviour is due to the fact that the writer in tests with 1 layer and the writers in tests with 2 layers are scheduled each on a single core and are not interrupted for rescheduling. However, with at least 3 layers, writers are blocked and resumed by the RTAI Round-Robin scheduler. Therefore, the cost of task switching is introduced. This fact results in the tests with 1 or 2 layers to have smaller execution times, whereas tests with a higher number of layers are run under similar conditions. Next, we analyse reader execution times against the number of layers. Fig. 8 on the left hand side shows the average of accumulated execution times of readers against the number of layers, whereas Fig. 8 on the right hand side shows the box plot of the maximum of those values. Similar to Fig. 7, Fig. 8 represents

96

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Fig. 7. Average (left) and worst-case (right) total execution time of writers when sensors are divided between 1 and 5 layers, in the presence of 1–15 user applications.

Fig. 8. Average (left) and worst-case (right) total execution time of readers in the same tests as in Fig. 7.

tests using three buffers, whereas tests using two or five buffers show similar results. Analysis. These Figures show that an increase in the number of layers does not dramatically affect reader execution times. It is apparent from these figures that when only one user application is present, reader execution times are distinctly smaller. Peculiarly, tests with an even number of layers show smaller average execution times. Finally, it can be observed that a higher number of user applications results in an increase in reader execution times. The effect of the number of user applications on the execution time is further discussed in Section 6.1.2. Rationalization. The following facts can be observed.

is only one reader per writer. This means that a writer will always immediately find a free buffer to swap, thereby greatly reducing the cost of synchronization. This explains why, in Fig. 7, execution times are almost constant when there is only one user application. Furthermore, this implies that there is a much smaller chance for a reader to wait for a new buffer to be unlocked, and therefore it decreases the reader execution times, as it is observed. 3. Smaller execution time for tests with an even number of user applications. This is considered to be caused by RTAI SMP scheduling on a dual-core CPU and its effect on measurement of execution time as explained in the beginning of this Section.

1. Slower increase in execution times as the number of layers increases. Although the synchronization mechanism in readers is more complex, as it allows for different acquisition modes, its execution is faster than that of writers. This is due to the fact that a writer has to wait and to search for a free buffer to swap, whereas a reader simply tries to allocate the lock on the latest updated buffer. This implies that the change in execution time as the number of layers increases is not as drastic as that of writers. 2. Smaller execution times for tests with one or two user applications. Given three buffers, there is always one buffer locked by a writer and, possibly, another one locked by a reader when there

Conclusion. These tests show that Skinware can handle multiple layers with minor penalty and it is indeed useable for hybrid robot skin designs, in which multiple networks are used for reliability and robustness. 6.1.2. Effects of user applications In this analysis, the effect of the number of user applications on Skinware performance is observed. Fig. 9 on the left hand side shows the average execution times of writers with respect to the number of user applications and number of regions the skin is divided in, whereas Fig. 9 on the right hand side shows the box plot of the maximum of those values. These Figures correspond

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

97

Fig. 9. Average (left) and worst-case (right) total execution time of writers in the presence of 1–15 user applications, where the skin is divided in different number of regions.

to tests with 5 layers and 2 buffers. Tests with 1–4 layers have produced similar results except for different ranges of values, as was also apparent from the previous test. Tests with 3 or 5 buffers have produced similar results, graphs of which have been omitted for brevity. Analysis. These Figures show an increase in writer execution times as the number of user applications increases. With very few user applications, execution time is particularly small and the growth rate with a higher number of user applications is low. It can be observed that neither the number of regions nor the number of buffers particularly affect execution times. Rationalization. The following facts can be observed. 1. Increase in execution times as the number of user applications increases. After each cycle, each writer periodically tries to swap buffers. As the number of user applications increases and, as a consequence, the number of readers do, the chances of finding a free buffer are reduced. Therefore, writers spend more time on synchronization. 2. Independence of execution time from the number of buffers. It is expected for an increase in the number of buffers to increase the chance of writers to find a free buffer earlier, thereby reducing their execution time. However, given the random periods of readers, it is still highly likely that, for all other buffers not used by a writer, these are locked by readers. Furthermore, if there are b buffers, a writer has to search among b − 1 buffers for a free buffer, which is slower as the number of buffers increases. As the results have shown, there is very slight difference between using two, three or five buffers. 3. Particularly smaller execution times with few user applications. As was discussed in Section 6.1.1, when there is only one user application and, as a consequence, only one reader per writer, it is always immediately possible to find a free buffer to swap when using three buffers. The chances of finding a free buffer are still very high even when using two buffers. With very few user applications, this chance is still high and therefore execution times are small as the writer spends less time trying to swap buffers. 4. Steep increase in execution times when there are few user applications and slow increase when there are many. As the number of user applications increases, readers have a higher chance of performing the read in parallel. Therefore, as the number of user applications increases, the chances of buffers being locked by readers is increased in smaller steps. This produces a steeper increase in execution times with few user applications and a more gradual one with more user applications. Although it was aimed

for writers to be independent from readers, as mentioned in Section 3, the cost of synchronization still affects writers. Nevertheless, this cost grows slowly and may be acceptable in practice. We then analyse execution times of readers against the number of user applications. Fig. 10 on the left hand side shows the average of accumulated execution times of readers against the number of user applications, whereas Fig. 10 on the right hand side shows the box plot of the maximum of those values. Similar to the Figures of writer execution times, these figures are for tests with 5 layers. Tests with 1–4 layers have produced similar results with a different range of values, as was also apparent for tests in the previous section. Tests with 3 or 5 buffers have produced similar results, graphs of which have been similarly omitted for brevity. Analysis. It can be observed that, as the number of user applications increases, reader execution times increase as well. Similar to Fig. 9, a change in execution times is more rapid when few user applications are present, whereas execution times are independent from the number of regions and buffers as well. Rationalization. The following facts can be observed. 1. Increase in execution times as the number of user applications increases. As the number of readers increases, it takes writers longer time to swap buffers. This in turn increases the time readers spend waiting for the buffer being written to be unlocked. 2. Independence of execution time from the number of buffers. Readers await on the buffer currently being used by a writer and, as a consequence, ignore the number of buffers. Since writer execution times are unaffected by the number of buffers, reader execution times are unaffected as well, i.e., readers have to spend the same amount of time waiting for a new buffer regardless of the number of buffers. This means that the minimum execution time of a reader is proportionally bound to the execution time of its corresponding writer. 3. Particularly smaller execution times with few user applications. As discussed in the previous case, readers can be only as fast as writers. Since writers have smaller execution times with fewer user applications, so do readers. Conclusion. According to these test results, there can be multiple users simultaneously acquiring skin data without a great penalty. It is noteworthy that in real applications, only few user applications will be requiring skin data at such high rates, e.g., algorithms implementing reflexive behaviours, whereas others would perform at a lower rate, e.g., high-level tactile-based

98

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Fig. 10. Average (left) and worst-case (right) total execution time of readers in the same tests as in Fig. 9.

reasoning algorithms. Therefore, in practice, Skinware would be able to handle a higher number of user applications than that reflected by the results of these tests. 6.1.3. Effects of regions In this analysis, the effect of robot skin regionalization on Skinware performance is investigated. Fig. 11 on the left hand side shows writer average execution times against the number of regions where the skin is divided in 1–5 layers, whereas Fig. 11 on the right hand side shows the box plot of the maximum of those values. The same values for readers are presented in Fig. 12. These Figures correspond to tests with 15 user applications. Tests with 1–14 user applications produce similar Figures, except for different ranges of values, as was previously observed. The graphs for average execution time correspond to divisions of the skin in distinct regions. Tests with overlapping regions produce similar results, graphs of which have been omitted for brevity. Analysis. It can be observed from these figures that the logical division of robot skin in regions does not affect neither writer nor reader execution times. Rationalization. As previously pointed out, the sensor/ region(/sub-region) structure is separate from the layer/patch/ module/sensor structure. As a result, writers and readers function regardless of how robot skin is divided in regions. In Section 6.3, the effect of skin regionalization on a Skinware application is discussed. Conclusion. Skinware performance is unaffected by a logical division of the robot skin in regions. This allows Skinware users to regionalize the robot skin according to their needs without consideration of any performance penalty. 6.2. Swap skips In this section, the effectiveness of methods discussed in Section 5.2 to reduce the number of swap skips is presented and discussed. 6.2.1. Swap skip prediction and avoidance This test has been performed with 30 848 sensors divided in three layers and uses only two buffers for communication. The test has been performed both with and without the swap skip prediction and avoidance strategy described in Section 5.2. Readers are periodic with random periods between 10 and 400 ms and up to 30 user applications have been run. Fig. 13 shows the average number of swap skips in these tests. It can be observed that the swap skip prediction and avoidance has almost completely removed swap skips.

6.2.2. Task offset alignment This test is similar to the previous one, with the difference that reader periods are now integer multiples of writer periods in the range 1–10. In this test, the task offset alignment mechanism described in Section 5.2.3 is used. As a result, swap skips are completely avoided. It is noteworthy that, even though swap skips are more likely to happen when reader periods are not in harmony with that of writers, in practice this is not the case. Therefore, this particular strategy resolves the swap skip issue to a great extent. Having both the methods enabled, the number of swap skips during the tests described in Section 6.1 have been logged. It has been observed that with any combination of the number of buffers, layers and users, the number of swap skips has constantly been zero. Conclusion. The results of these tests indicate that the data loss issue raised by swap skips can be avoided. Nevertheless, increasing the number of buffers reduces the chances of swap skips happening under extreme unforeseen circumstances. 6.3. Sub-regions A user application has been developed to test the effect of introducing sub-regions in Skinware. The simulation driver keeps all the sensors deactivated, only activating one different sensor in each data frame. During the test execution, a service first inspects skin sensors and propagates activity information to sub-regions and regions. Then, the test application exploits these data in searching regions in random order to identify active sensors. The test has been performed with 30 848 sensors divided in three layers and the robot skin divided in regions with any possible combination of the following configurations, namely the number of regions in the range 1–200, and either disjoint or overlapping regions with each sensor assigned to three different regions. The test has been run 30 848 times to detect each sensor activate exactly once. Fig. 14 on the left hand side shows the average execution times to search for active sensors when sub-regions are used. Fig. 14 on the right hand side shows the same values when sub-regions are removed from the sensor/region(/sub-region) structure. It can be seen that the introduction of sub-regions between sensors and regions decreased the search time by up to 50% when the robot skin is divided into at least 10 regions. The time needed to identify the active sensor is almost independent from the number of regions in both cases, with the test utilizing sub-regions having about 50%

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

99

Fig. 11. Average (left) and worst-case (right) total execution time of writers when the skin is divided between 1 and 200 regions, where the skin is divided in 1–5 layers.

Fig. 12. Average (left) and worst-case (right) total execution time of readers in the same tests as in Fig. 11. Table 1 Performance measures for the service allowing fast detection of active sensors and complete scanning of skin sensors.

Service execution Complete scan

Average (ns)

Worst-case (ns)

STD

114 979 57 282

136 572 222 646

14 556 16 629

Conclusion. Even though the use of sub-regions introduces some complexity in the Skinware implementation and a minor complexity in its usage, the added performance benefit is useful. Therefore, it has been decided to include sub-regions nonetheless. 6.4. Tests with the ROBOSKIN technology

Fig. 13. Average swap skips per second when swap skip prediction and avoidance mechanism is enabled or disabled. With prediction and avoidance, the number of swap skips is nearly always at zero.

This section provides test results when the target ROBOSKIN technology is used. This considered skin piece, shown in Fig. 15, consists of 5 contiguous patches. The patches cover a saddle-like surface. The following configuration is used.

• One patch consisting of 15 triangular modules, placed in a row smaller search times. It is worthwhile to compare the efficiency of using this service against scanning the whole robot skin to find the active sensor. Table 1 shows the service execution time used in this test and the execution time of a complete robot skin scan. According to this table, the use of the implemented service is justified if at least three users are present.

on the upper side of the saddle. This patch is connected to the test workstation using a CAN network. • One patch consisting of 12 modules, one of which is faulty, and placed on the right side of the saddle, connected with an EtherCAT network. • Three patches consisting of 16 modules each, connected to the same EtherCAT network, consisting of two EtherCAT slaves.

100

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Fig. 14. Search for active sensors in robot skin when sub-regions are introduced (left) or not used (right).

Fig. 15. The test hardware covers a saddle-like surface. Five robot skin patches are indicated in cyan (one patch in top-row, connected to CAN) and grey (four patches in bottom row, connected to EtherCAT) colours. The micro-controller interfacing with the CAN network and two micro-controllers connected to EtherCAT slaves are also visible. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

Therefore, a total of 888 sensors (all taxels) in 74 modules are visible to Skinware (Fig. 15). All skin patches share the same technology design. However, since the robot skin patches are connected through two different networks (namely, CAN and EtherCAT), and therefore handled by two drivers, they are considered to be part of two layers. A Skinware-based visualization application is included with Skinware. Furthermore, a simple motion detection service has been implemented, which identifies the direction of sliding motions over the skin surface. This test has been accompanied by a script that periodically checks the outputs of Skinware for device drivers that have reported errors and tries to remove and reload them. To better observe this process, the visualization program highlights layers that are not properly working (in red). During the test execution, the CAN and the EtherCAT network cables have been randomly detached and later reattached. Fig. 16 shows the continued execution of the visualization program and the motion detection service working without interruption while the CAN cable is removed without prior notification. Once reattached, Skinware continues acquisition from the newly restarted driver and the results are visible in the visualization program. This test demonstrates the robustness of Skinware and the simplicity of recovering from unexpected errors. During one hour test execution, execution time data has been gathered from the two writers as well as the two readers. Table 2 shows information on execution times for both layers.

Fig. 16. The visualization program included with Skinware. This application shows the responses received from the robot skin in Fig. 15. Additionally, it is attached to a simple motion tracking service to visualize its result, which is the red arrow showing the direction at which the pressing indenter is moving. In this screenshot, the CAN cable is disconnected during operation without previous notice to Skinware and as a result, the top row of modules are drawn in red. The sensors attached to the EtherCAT network are still functional and presume the colour blue. Once the CAN cable is reattached, the sensors attached by the CAN network will return to blue and continue producing responses as well as participating in the motion tracking service. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.) Table 2 Execution times of writers and readers of both the CAN layer (180 sensors) and the EtherCAT layer (708 sensors).

Writer EC Writer CAN Reader EC Reader CAN

Average (ns)

Worst-case (ns)

STD

23 915 20 108 10 996 6 703

87 315 44 077 86 635 47 102

4381 2951 3533 1733

7. Discussion Potential future directions for Skinware include higher portability and higher reconfigurability. The later implies reconfigurability at different levels, such as the possibility for dynamic changes in the skin structure or the possibility for dynamic changes in the transduction technology itself. The former would be useful for a novel robot skin with the ability to reduce or increase the number of reported sensors based on interest on the particular region. The later would be useful for more advanced skin hardware sensors with the ability to adjust sensitivity through attention as in biological systems [59]. Further possible future work could be providing means for the drivers to get feedback from applications, for exam-

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

101

Table A.3 Comparison between the five considered real-time systems.

RTAI RTLinux RT-Preempt QNX VxWorks

A

B

C

D

E

F

G

H

I

J

K

L

M

N

✓ ✓ ✓

✓

✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓

2 3 L 4 2

✓

✓ ✓ ✓ ✓ ✓

✓ ✓ ✓

✓ ✓ ✓ ✓ ✓

✓

3 1 L 6 2

5 3 L 5 5

✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓

✓ ✓ ✓

This table shows which platforms support the following features: (A) kernel-level hard real-time threads; (B) user-level hard real-time threads; (C) multiprocessing; (D) periodic tasks; (E) real-time device drivers; (F) number of supported schedulers; (G) dynamic memory allocation for real-time threads; (H) shared memory; (I) shared memory between kernel and user spaces; (J) synchronization locks; (K) shared synchronization locks between kernel and user spaces; (L) number of supported IPCs; (M) number of supported architectures; (N) active development. In the table, L means as many as supported by the Linux kernel.

ple to adjust acquisition rate if required or move to power-saving mode if tactile feedback is momentarily deemed unnecessary. Skinware has been designed with generality and extensibility in mind. However, currently it has only been possible to use and test it with the ROBOSKIN technology. It is of the authors’ great interest to verify this generality by experimenting with different skin technologies. It is worth noting that Skinware has not been tested to check its compliance with existing safety standards. 8. Conclusions This article introduces Skinware, a software framework for the real-time management of large-scale tactile information. When large-scale robot skin is used, novel issues emerge as far as technological, system-level and representation aspects are concerned. These issues lead to heterogeneous solutions at various levels. As a matter of fact, different sensory modes and networking solutions may be employed even within the single robot body structure. The goal of Skinware is to provide high-level tactile-based robot behaviours with abstract interfaces (at the software and representation levels), which are independent from the underlying hardware and mechanical solutions. On the one hand, the software architecture is able to provide the developer with a hardware-independent perspective of tactile information. On the other hand, it constitutes the basis for the implementation of higher-level algorithms for tactile data processing, such as those described in [60], which is of the utmost importance for benchmarking and comparison activities. The software framework performance has been evaluated according to a number of aspects, namely real-time requirements, data coherency and data consistency. The whole framework is released open source and under a free licence. Acknowledgement The research leading to these results has received funding from the European Community’s Seventh Framework Programme (FP7/2007–2013) under Grant 231500 (project ROBOSKIN) and Grant 288553 (project CloPeMa).

a deadline, which is of utmost importance in Skinware. RTAI and RTLinux also have the added benefit of being implemented as separate modules for the Linux kernel. Differently from RTLinux, RTAI is able to bring real-time performance to user-space as well, even though it lacks an Earliest Deadline First (EDF) scheduler [67]. As far as commercial products are concerned, QNX is characterized by a better support and a safer design than VxWorks, and is comparable to RTAI’s performance and features. RTAI has been preferred over QNX due to the fact that it is free software and it is based on Linux, which is more popular in the Robotics community. Furthermore, it has been shown that RTAI has less scheduling delay and smaller jitter, and therefore higher predictability than most of the real-time systems in question [68,69]. Nevertheless, it has been foreseen that Skinware may be ported to other platforms as well. The Unified Real-Time interface project4 has been created to address portability of real-time software including Skinware, and it is work in progress. Appendix B. Complete algorithms The complete writer and reader algorithms introduced in Section 5 are presented in Algorithms 6 and 7 respectively. Algorithm 6 The complete writer algorithm 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13:

Appendix A. Choice of real-time infrastructure

14:

Three open-source real-time extensions to the Linux kernel have been considered, namely RTAI2 [50], RTLinux [62] and RTPreempt3 [64], as well as the proprietary QNX operating system [65] and VxWorks 5.x [66] (Table A.3). According to Table A.3, RTAI supports most of the needed features. Although RT-Preempt supports a larger number of architectures, it is not considered a fully real-time kernel as it minimizes maximum kernel preemption delay, but does not guarantee

16:

15: 17: 18: 19: 20: 21: 22: 23: 24:

procedure writer(l) ◃ Writer for layer l b←0 write_lock(r w ll,b ) w bl ← b while running do t ← now acquire(l, b) ◃ The acquire callback of the corresponding driver while period time left > threshold do for i ← 1, . . . , Bl do if i ̸= b and try_write_lock(r w ll,i ) then lbl ← b ltl ← t write_unlock(r w ll,b ) b←i w bl ← b goto 21 end if end for

sleep end while nsl ← now + Tw

wait_period end while end procedure

2 The Xenomai project has similar features to RTAI [61]. However, RTAI aims at the lowest technically feasible latency and is therefore preferred over Xenomai. 3 The Linux Kernel Preemption Project has similar features to RT-Preempt [63]. However, the development of this project has been halted since 2001 and therefore RT-Preempt is preferred over it.

4 https://github.com/ShabbyX/URT/.

102

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103

Algorithm 7 The complete reader algorithm 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25:

procedure reader(l) ◃ Reader of layer l pb ← inv alid pt ← inv alid while running do if sporadic then wait(request) end if b ← lbl t ← ltl if (b = pb and t = pt) or now + tx ≥ nsl then ◃ Buffer already read, or cannot be read in time read_lock(r w ll,wbl ) b = lbl t = ltl else if try_read_lock(r w ll,b ) = fail then goto 5 ◃ A buffer swap has just happened end if end if t ← now Make local copy of data from buffer b tx ← max(tx , now − t ) read_unlock(r w ll,b ) pb ← b pt ← t if periodic then

wait_period

26: 27: 28: 29: 30: 31: 32: 33: 34: 35: 36: 37: 38:

end if if sporadic then u←1 while try_wait(request) do u←u+1 end while for i ← 1, . . . , u do signal(response) end for end if end while end procedure

References [1] G. Robles-De-La-Torre, The importance of the sense of touch in virtual and real environments, IEEE Multimedia 13 (2006) 24–30. [2] J. Morley, Neural Aspects of Tactile Sensation, Elsevier, Amsterdam, 1998. [3] A. Okamura, N. Smaby, M. Cutkosky, An overview of dexterous manipulation, in: IEEE International Conference on Robotics and Automation, 2000. Proceedings, ICRA’00, Vol. 1, 2000, pp. 255–262. [4] T. Dahl, A. Palmer, Touch-triggered protective reflexes for safer robots, in: New Frontiers in Human-Robot Interaction, 2011. [5] T. Noda, T. Miyashita, H. Ishiguro, N. Hagita, Super-flexible skin sensors embedded on the whole body, self-organizing based on haptic interactions, in: Proceedings of Robotics: Science and Systems, Vol. IV, 2008. [6] M. Lee, H. Nicholls, Review article tactile sensing for mechatronics—a state of the art survey, Mechatronics 9 (1999) 1–31. [7] V. Lumelsky, M. Shur, S. Wagner, Sensitive skin, IEEE Sens. J. 1 (2001) 41–51. [8] B. Argall, A. Billard, A survey of tactile human–robot interactions, Robot. Auton. Syst. 58 (2010) 1159–1176. [9] R. Dahiya, G. Metta, M. Valle, G. Sandini, Tactile sensing—from humans to humanoids, IEEE Trans. Robot. 26 (2010) 1–20. [10] A. Schmitz, P. Maiolino, M. Maggial, L. Natale, G. Cannata, G. Metta, Methods and technologies for the implementation of large-scale robot tactile sensors, IEEE Trans. Robot. 27 (2011) 389–400. [11] Y. Ohmura, Y. Kuniyoshi, A. Nagakubo, Conformable and scalable tactile sensor skin for curved surfaces, in: Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006, ICRA 2006, 2006, p. 1348. [12] G. Cannata, M. Maggiali, G. Metta, G. Sandini, An embedded artificial skin for humanoid robots, in: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, 2008, MFI 2008, 2008, p. 434. [13] M. Nilsson, Tactile sensing with minimal wiring complexity, in: 1999 IEEE International Conference on Robotics and Automation, 1999. Proceedings. Vol. 1, 1999, p. 293.

[14] H. Tanaka, K. Ohnishi, H. Nishi, Haptic communication system using FPGA and real-time network framework, in: 35th Annual Conference of IEEE Industrial Electronics, 2009, IECON’09, 2009, pp. 2931–2936. [15] E. Baglini, G. Gannata, F. Mastrogiovanni, Design of an embedded networking infrastructure for whole-body tactile sensing in humanoid robots, in: 2010 10th IEEE-RAS International Conference on Humanoid Robots, Humanoids, 2010, p. 671. [16] Y. Fujimori, Y. Ohmura, T. Harada, Y. Kuniyoshi, Wearable motion capture suit with full-body tactile sensor, in: IEEE International Conference on Robotics and Automation, 2009, ICRA’09, 2009, pp. 3186–3193. [17] P. Mittendorfer, G. Cheng, Humanoid multimodal tactile-sensing modules, IEEE Trans. Robot. 27 (2011) 401–410. [18] P. Allen, P. Michelman, Acquisition and interpretation of 3-D sensor data from touch, IEEE Trans. Robot. Autom. 6 (1990) 397–404. [19] O. Kerpa, K. Weiss, H. Worn, Development of a flexible tactile sensor system for a humanoid robot, in: 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2003, IROS 2003. Proceedings. Vol. 1, 2003, p. 1. [20] P. Lang, Design and prototyping of a fiber optic tactile array, in: Canadian Conference on Electrical and Computer Engineering, 2008, CCECE 2008, 2008, pp. 373–376. [21] B. Guo, L. Qin, J. Liu, Z. Liu, Tactile sensor array signal and data processing based on information fusion, in: 3rd International Conference on Innovative Computing Information and Control, 2008, ICICIC’08, 2008, p. 526. [22] O. Oballe-Peinado, J. Castellanos-Ramos, J. Hidalgo, F. Vidal-Verdu, H. Macicior, E. Ochoteco, Interface for tactile sensors based on direct connection to a FPGA, in: IEEE International Conference on Mechatronics, 2009, ICM 2009, 2009, pp. 1–6. [23] J. Ulmen, M. Cutkosky, A robust, low-cost and low-noise artificial skin for human-friendly robots, in: 2010 IEEE International Conference on Robotics and Automation, ICRA, 2010, pp. 4836–4841. [24] J. Ding, F. Xu, Y. Li, S. Ge, F. Shuang, Structures and information acquisition algorithms for three-dimensional flexible tactile sensor, in: 2010 IEEE International Conference on Robotics and Biomimetics, ROBIO, 2010, pp. 862–867. [25] D. Tawil, D. Rye, M. Velonaki, Touch modality interpretation for an EIT-based sensitive skin, in: Proceedings of the 2011 IEEE International Conference on Robotics and Automation, ICRA 2011, Shanghai, China, 2011. [26] H. Kawasaki, T. Komatsu, K. Uchiyama, Dexterous anthropomorphic robot hand with distributed tactile sensor: gifu hand II, IEEE/ASME Trans. Mechatronics 7 (2002) 296–303. [27] T. Sagisaka, Y. Ohmura, Y. Kuniyoshi, A. Nagakubo, K. Ozaki, High-density conformable tactile sensing glove, in: 2011 11th IEEE-RAS International Conference on Humanoid Robots, Humanoids, 2011, pp. 537–542. [28] K. Johnson, Contact Mechanics, Cambridge University Press, Cambridge, UK, 1985. [29] G. Cannata, S. Denei, F. Mastrogiovanni, Towards automated self-calibration of robot skin, in: 2010 IEEE International Conference on Robotics and Automation, ICRA, 2010, p. 4849. [30] G. Cannata, S. Denei, F. Mastrogiovanni, Tactile sensing: steps to artificial somatosensory maps, in: 2010 IEEE RO-MAN, 2010, pp. 576–581. [31] G. Cannata, S. Denei, F. Mastrogiovanni, A framework for representing interaction tasks based on tactile data, in: 2010 IEEE RO-MAN, 2010, p. 698. [32] J. Schwartz, E. Kandel, T. Jessell, Principles of Neural Science, Prentice-Hall International, 1991. [33] Y. Hoshino, M. Inaba, H. Inoue, Model and processing of whole-body tactile sensor suit for human–robot contact interaction, in: 1998 IEEE International Conference on Robotics and Automation, 1998. Proceedings. Vol. 3, 1998, p. 2281. [34] M. Piaggio, A. Sgorbissa, R. Zaccaria, A programming environment for real-time control of distributed multiple robot systems, Adv. Robot. 14 (2000). [35] M. Piaggio, A. Sgorbissa, R. Zaccaria, Pre-emptive versus non pre-emptive realtime scheduling in intelligent mobile robots, J. Exp. Theor. Artif. Intell. 12 (2000). [36] F. Mastrogiovanni, A. Paikan, A. Sgorbissa, Semantic-aware real-time scheduling in robotics, IEEE Trans. Robot. Autom. 29 (2013) 118–135. [37] H. Bruyninckx, Open robot control software: the OROCOS project, in: IEEE International Conference on Robotics and Automation, 2001. Proceedings 2001 ICRA, Vol. 3, 2001, pp. 2523–2528. [38] R. Volpe, I. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, The CLARAty architecture for robotic autonomy, in: IEEE Proceedings of Aerospace Conference, Vol. 1, 2001, pp. 1/121–1/132. [39] R. Vaughan, B. Gerkey, A. Howard, On device abstractions for portable, reusable robot code, in: Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 3, 2003, pp. 2421–2427. [40] B. Gerkey, R. Vaughan, A. Howard, The player/stage project: tools for multirobot and distributed sensor systems, in: Proceedings of the 11th International Conference on Advanced Robotics, ICAR 2003, Coimbra, Portugal, 2003. [41] A. Brooks, T. Kaupp, A. Makarenko, S. Williams, A. Oreback, Towards component-based robotics, in: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005, IROS 2005, 2005, pp. 163–168. [42] N. Ando, T. Suehiro, K. Kitagaki, T. Kotoku, W. Yoon, RT-Middleware: distributed component middleware for RT (Robot Technology), in: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2005, IROS 2005, 2005, pp. 3933–3938. [43] S. Caselli, F. Monica, M. Reggiani, YARA: a software framework enhancing service robot dependability, in: Proceedings of the 2005 IEEE International Conference on Robotics and Automation, 2005, ICRA 2005, 2005, pp. 1970–1976.

S. Youssefi et al. / Robotics and Autonomous Systems 68 (2015) 86–103 [44] G. Metta, P. Fitzpatrick, L. Natale, YARP: yet another robotic platform, Int. J. Adv. Robot. Syst. 3 (2006). [45] M. Quigley, B. Gerkey, K. Conley, J. Faust, T. Foote, J. Leibs, E. Berger, R. Wheeler, A. Ng, ROS: an open-source robot operating system, in: Proceedings of the IEEE International Conference on Robotics and Automation, ICRA, Workshop on Open Source Robotics, Kobe, Japan, 2009. [46] G. Pardo-Castellote, OMG data-distribution service: architectural overview, in: 23rd International Conference on Distributed Computing Systems Workshops, 2003. Proceedings. 2003, pp. 200–206. [47] B. Finkemeyer, T. Kröger, F. Wahl, A middleware for high speed distributed real-time communication, in: Robot Systems for Handling and Assembly, first ed., in: Springer Tracts in Advanced Robotics, vol. 67, Springer, Berlin, Heidelberg, Germany, 2010, pp. 193–212. [48] W. Kang, K. Kapitanova, S. Son, RDDS: a real-time data distribution service for cyber-physical systems, IEEE Trans. Ind. Inform. 8 (2012) 393–405. [49] A. Billard, A. Bonfiglio, G. Cannata, P. Cosseddu, T. Dahl, K. Dautenhahn, F. Mastrogiovanni, G. Metta, L. Natale, B. Robins, L. Seminara, M. Valle, The ROBOSKIN Project: Challenges and Results, Vol. 544, 2013, pp. 351–358. [50] P. Mantegazza, E. Dozio, S. Papacharalambous, RTAI: real time application interface, Linux J. 2000 (2000). [51] S. Denei, F. Mastrogiovanni, G. Cannata, Towards the creation of tactile maps for robots and their use in robot contact motion control, Robot. Auton. Syst. 63 (2015) 293–308. [52] P.P. Chen, The entity-relationship model—toward a unified view of data, ACM Trans. Database Syst. (TODS) 1 (1976) 9–36. [53] A. Del Prete, S. Denei, L. Natale, F. Mastrogiovanni, F. Nori, G. Cannata, G. Metta, Skin spatial calibration using force/torque measurements, in: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, 2011, pp. 3694–3700. [54] P. McKenney, J. Slingwine, Read-copy update: using execution history to solve concurrency problems, in: 10th IASTED International Conference on Parallel and Distributed Computing and Systems, 1998, pp. 509–518. [55] D. Clark, HIC: an operating system for hierarchies of servo loops, in: 1989 IEEE International Conference on Robotics and Automation, 1989. Proceedings. Vol.2, 1989, pp. 1004–1009. [56] G. Buttazzo, HARTIK: a real-time kernel for robotics applications, in: Real-Time Systems Symposium, 1993. Proceedings. 1993, pp. 201–205. [57] G. Buttazzo, Hard Real-time Computing Systems: Predictable Scheduling Algorithms and Applications, Springer-Verlag TELOS, Santa Clara, CA, USA, 2004. [58] C.-Y. Yu, C.-H. Lin, Y.-H. Yang, Human body surface area database and estimation formula, Burn.: J. Int. Soc. Burn. Inj. 36 (2010) 616–629. [59] C. Frings, R. Bader, C. Spence, Selection in touch: negative priming with tactile stimuli, Perception & Psychophysics 70 (2008) 516–523. [60] L. Muscari, F. Mastrogiovanni, L. Seminara, M. Capurro, G. Cannata, M. Valle, Real-time reconstruction of contact shapes for large area robot skin, in: Proceedings of the 2013 IEEE International Conference on Robotics and Automation, ICRA 2013, 2013. [61] P. Gerum, Xenomai: real-time framework for Linux, 2001. http://www.xenomai.org. [62] V. Yodaiken, The RTLinux Manifesto, in: Proceedings of the 5th Linux Expo, 1999. [63] R. Love, The Linux Kernel Preemption Project, 2001. http://kpreempt.sf.net. [64] L. Fu, R. Schewebel, Real-Time Linux, 2004. https://rt.wiki.kernel.org. [65] D. Hildebrand, An architectural overview of QNX, in: Proceedings of the Usenix Workshop on Micro-Kernels & Other Kernel Architectures, Seattle, 1992. [66] W.R. Systems, VxWorks, 1987. http://www.windriver.com/products/vxworks/. [67] C. Liu, J. Wayland, Scheduling algorithms for multiprogramming in a hard-realtime environment, J. ACM (JACM) 20 (1973) 46–61. [68] A. Barbalace, A. Luchetta, G. Manduchi, M. Moro, A. Soppelsa, C. Taliercio, Performance comparison of VxWorks, Linux, RTAI and Xenomai in a hard real-time application, in: 2007 15th IEEE-NPSS Real-Time Conference, 2007, pp. 1–5. [69] A. Siro, D. Emde, N. McGuire, Assessment of the realtime preemption patches (RT-Preempt) and their impact on the general purpose performance of the system, in: Proceedings of the 9th Real-Time Linux Workshop, 2007.

103

[70] S. Youssefi, S. Denei, F. Mastrogiovanni, G. Cannata, A middleware for whole body skin-like tactile systems, in: 2011 11th IEEE-RAS International Conference on Humanoid Robots, Humanoids, 2011, pp. 159–164.

S. Youssefi received his B.Sc. degree in Computer Engineering in the University of Tehran in 2009 and his M.Sc. degree in Robotics in the joint European Master on Advanced Robotics (EMARO) from the University of Warsaw and University of Genova in 2011. He will receive his Ph.D.degree in Robotics from the University of Genova in 2015. Shahbaz occupies himself with real-time and embedded software and on his spare time, free software. His main research interests include real-time systems, robotic software platforms, robotic sensors and embedded software design. He is also gaining interest in embedded system design and real-time networks.

S. Denei received the M.Sc. in Computer Engineering and the Ph.D. in Robotics from the University of Genova, Italy in 2009 and 2013. He is a Post-Doctoral Researcher with the Department of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genova, where he works on tactile sensors for humanoid robots and tactile systems representations. His main research interests are in the area of: humanoid robots, tactile sensor, automatic control systems, real-time software architectures and robotics middleware. His research interests include also robotics, robot control theory and embedded device development.

F. Mastrogiovanni is Assistant Professor at University of Genoa, Italy, where he teaches Software Architectures for Robotics as part of the joint European Master on Advanced Robotics (EMARO)—Robotics Engineering study curriculum. Currently, Fulvio’s research interests are in AI techniques for cognitive systems. Fulvio is working in Ubiquitous and Humanoid Robotics, with a specific emphasis on real-time SW architectures, realtime AI, knowledge representation and reasoning. Fulvio’s interests are rapidly evolving towards self-aware systems and robot evolution. Fulvio received the M.Sc. and the Ph.D. in Robotics from University of Genoa in 2003 and 2008. Fulvio has been Visiting Professor at the Asian Institute of Technology, Thailand, and the Jiao Tong University, China. Fulvio is managing coordinator of the FP7 EC-funded project ROBOSKIN, and principal investigator of the SHELL project, funded by the Italian Ministry of Economic Development.

G. Cannata received the Laurea degree in Electronic Engineering from the University of Genova, Italy, in 1988. From 1989 to 1995 he has been research scientist at the Naval Automation Institute (I.A.N.) of the Italian National Research Council, working in the area of underwater robotics. From 1995 to 1998 he has been Assistant Professor at the Department of Communication, Computer and System Sciences (DIST) of the University of Genova. He is currently Associate Professor of automatic and digital control at the Faculty of Engineering of the University of Genova. His main research interests are in the area of: humanoid robots, automatic control systems and control architectures for robotic and mechatronic systems. His research interests include also robotics and robot control theory, control of mechanical systems, dynamic simulation.