Opportunistic control mechanisms for ambience intelligence worlds

Expert Systems with Applications 41 (2014) 1875–1884

Contents lists available at ScienceDirect

Expert Systems with Applications journal homepage: www.elsevier.com/locate/eswa

Opportunistic control mechanisms for ambience intelligence worlds José M. Fernández-de-Alba a,⇑, Pablo Campillo b, Rubén Fuentes-Fernández a, Juan Pavón a a b

Dept. Software Engineering and Artificial Intelligence, Universidad Complutense de Madrid, Madrid, Spain Dept. Communications and Information Engineering, Universidad de Murcia, Murcia, Spain

a r t i c l e

i n f o

Keywords: Ambient intelligence Opportunistic control Context-awareness Virtual worlds simulation FAERIE UbikSim

a b s t r a c t Ambient intelligence (AmI) worlds consist of heterogeneous collections of interconnected devices that integrate smoothly in the environment to offer services to their users. These devices can be added, change or fail, modifying the system topology. Also, other circumstances may change, affecting users’ activities, e.g., time, location, or the presence of other users. These changes modify the information the system has available to satisfy users’ needs, i.e., the context. AmI systems need to adapt to these evolving conditions in order to be able to provide their services, but being as unobtrusive as possible for their users. There are also performance requirements that the system must fulfill to provide responses to environment stimuli in real time. Opportunistic control mechanisms address these issues by monitoring the context, and suspending or resolving goals when the appropriate conditions are met. This paper presents FAERIE, a software framework that supports the development of AmI applications with facilities for context management that rely on opportunistic control. The development of FAERIE systems uses a 3D simulator tool for testing and validation called UbikSim. A case study on an artistic installation illustrates the use of this infrastructure. Ó 2013 Elsevier Ltd. All rights reserved.

1. Introduction Ambient intelligence (AmI) worlds are composed of multiple devices that are interconnected and tightly integrated with the environment where they are embedded (Remagnino, Hagras, Monekosso, & Velastin, 2005). They provide services to their users based on the information they gather, what allows them minimizing the need of explicit actions by users. This information is called context and comes from different sources, e.g., sensors, users’ input, historical data, or external sources (Abowd et al., 1999). Building applications with this functionality implies dealing with the heterogeneity of devices and the information they manage. These applications have also to consider the evolution of user related features, such as location, time, or activities. There are also dynamic changes in system configurations. Device connections can change or fail, modifying the topology and configuration of the system. These changes, in turn, modify the available information, providing new or redundant data, or making impossible to get an accurate representation of the current context. There are also requirements of quality of service, such as performance, cost, or energy consumption, which are important for real-time applications, to improve device autonomy, or to build affordable systems (Baldauf, Dustdar, & Rosenberg, 2007). ⇑ Corresponding author. Tel.: +34 696633877. E-mail addresses: [email protected] (J.M. Fernández-de-Alba), [email protected] (P. Campillo), [email protected] (R. Fuentes-Fernández), [email protected] (J. Pavón). 0957-4174/$ - see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.eswa.2013.08.084

AmI systems must deal with the previous issues requiring as little as possible user intervention for their configuration and functioning. Users should not be aware of all the devices in the environment in order to use it (Kieffer, Lawson, & Macq, 2009). Therefore, the system itself must be able to adapt dynamically to all those changes, building context representations with available information and making decisions based on the reliability of such representations. Opportunistic control mechanisms can be used to support the design of AmI applications with these features. Following this approach, when the system receives a goal, it does not solve it immediately, but suspends it. Then, it monitors changes in the context to detect a suitable moment to solve the goal, i.e., needed resources and information are available, and the context fulfills certain conditions (Patalano & Seifert, 1997). The observation of the context continues during goal resolution, so the system can suspend the process if the goal becomes unreachable. This behavior implies the existence of mechanisms to facilitate contextawareness: components are able to determine when conditions depending on context change. Some recent works (Huang, chan Lan, & Tsai, 2008) have proposed platforms that support these opportunistic features. They are mostly focused on the layers at the lowest levels of abstraction. These layers are related with establishing opportunistic connections among the nodes in an unstable network (Arnaboldi, Conti, & Delmastro, 2011; Boldrini, Conti, Delmastro, & Passarella, 2010), and the opportunistic use of unknown remote abstract resources (Conti, Giordano, May, & Passarella, 2010; Kurz &

1876

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

Ferscha, 2010). A further step is to prepare systems for identifying abstract conditions on their current context in order to produce abstract interpretations and to execute opportunistic behaviors (Challa, Gulrez, Chaczko, & Paranesha, 2005). This opportunistic information fusion is relevant to AmI applications where there exist behaviors dependent on multiples inputs. Although there are architectural solutions for all these levels individually, there is a lack of architectures including opportunism in every level. The development of AmI systems does not only require infrastructure for applications, e.g., platforms or libraries, but also tools for that development. One of the most problematic issues is how to test and validate real applications due to their deployment costs. A solution to this problem is simulating most of the involved physical devices and their deployment with virtual spaces. There exist works that develop test simulations using virtual sensors (Park, Moon, Hwang, & Yeom, 2007), and others that apply 3D scenarios to manage (Shirehjini et al., 2005) or demonstrate (Bylund & Espinoza, 2002) smart spaces in specific settings. However, little work has been done regarding general platforms that use 3D scenarios for assisting in the design and validation of AmI systems. This paper presents an infrastructure that addresses the previous issues. It includes two elements. There is a library providing the mechanisms for the opportunistic management of context in AmI applications called FAERIE (Framework for AmI: Extensible Resources for Intelligent Environments) (Fernández-de Alba, Fuentes-Fernández, & Pavón, 2012b). This is complemented with UbikSim (Campillo-Sánchez & Botía, 2012), a platform for the simulation of 3D AmI worlds, which supports the testing of FAERIE-based applications. FAERIE provides different services for building AmI worlds (Fernández-de Alba et al., 2012b). AmI components are defined as context observers that observe and update shared context containers, which manage parts of the abstract representation of the real context. These context containers transparently coordinate among them to offer a virtual globally shared representation of the context, which is distributed among different nodes. When a context observer modifies a piece of the context representation, every other context observer interested in that piece of information is made aware of the change, which triggers successive behaviors. FAERIE uses these components to support the development of workflow-based context-aware applications. This implies that applications are designed around the definition of sets of interconnected activities involving different actors (Ardissono, Furnari, Goy, Petrone, & Segnan, 2007). UbikSim is a platform that allows running simulations of 3D intelligent environments with different configurations (CampilloSánchez & Botía, 2012). It supports the specification maps of the physical spaces and their sensors and actuators. Engineers can simulate the signals of these sensors or connect them with actual external sensors. Active elements are implemented as software agents, which can represent users and system controllers of actuators. Simulations can be executed in batch mode (i.e., agents representing users have predefined behaviors) or interactively (i.e., those agents are controlled via keyboard). The use of this overall infrastructure is shown with an application that belongs to an artistic installation. The application guides spectators through different rooms using sensors to find their positions and speakers to give them instructions. An initial version of this paper was presented at the IDEAL 2012 conference (Fernández-de Alba, Campillo, Fuentes-Fernández, & Pavón, 2012a). This paper offers more detailed information on system components and extended experimentation. The remaining paper is organized as follows. Sections 2 and 3 introduce the infrastructure. The former explains how FAERIE supports opportunistic control behavior, and how this support drives the design of AmI applications; the later presents UbikSim and

discusses its integration with FAERIE. Sections 4–6 elaborate the case study. Section 4 presents the problem addressed in it. Section 5 describes how to design the application as a context-aware workflow whose components follow an opportunistic control. Section 6 shows the actual behavior of the application and its adaption in different scenarios. Results are discussed and compared with related work in Section 7. Finally, Section 8 presents some conclusions and discusses future work. 2. Opportunistic control in FAERIE FAERIE adopts an opportunistic control to organize the behavior of its systems. It offers these features through components of the framework (see Section 2.1) whose functionality supports different kinds of opportunistic behavior (see Section 2.2). FAERIE also includes information on how to design systems that incorporate opportunism at the application level using the previous mechanisms (see Section 2.3). 2.1. FAERIE components FAERIE conceives an AmI system as a set of interconnected environments. Each one contains one or more devices running components of the framework that provide services for the AmI applications or other components. They also manage private context containers that maintain the context information. Examples of environments are a smart room populated with sensors and a computing node, or a mobile device with sensing and computational capabilities. The context is a representation of the system’s environment, which includes the physical environment and the computational environment. The first one represents the things existing in the real world, such as people and external devices, and the second one represents the things existing in the computer executing the framework, such as software components. Fig. 1 represents this idea. These virtual entities are known as context elements, and work as a snapshot of the actual context. The representation of the context follows a ‘‘context conceptual’’ approach to represent the context, i.e., the context represents entities and their relationships through time, as opposed to ‘‘context theoretic’’ approaches, in which the context describes assertions or circumstances (Anagnostopoulos, Tsounis, & Hadjiefthymiades, 2007). Among the alternatives to implement the context, FAERIE adopts an object-oriented approach (Strang et al., 2004), based on the observer pattern. The context elements that represent the context are updated by context observers, which represent the behaviors that rule the context changes for the fulfillment of objectives (Fernández-de Alba, Fuentes-Fernández, & Pavón, 2011). The way in which context elements and observers work together is what actually implements the opportunistic behavior of the framework. 2.2. Framework-level opportunistic behavior Opportunistic behavior (Patalano & Seifert, 1997) consists in being able to execute tasks when the context meets certain conditions, and doing it in an adapted way to those conditions. This behavior needs control mechanisms to evaluate the conditions, and to suspend and restart tasks depending on the changes. This kind of control is relevant to AmI applications because of their dynamic changes and mobile nature. These are the result of the use of technologies as common today as geographic localization or ‘‘plug and play’’ devices. There are different types of opportunistic control in AmI applications. At the application level, applications perform opportunistic planning. It consists of observing the context in order to exploit the

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

opportunities following the business logic, e.g., notifying the user for certain tasks when walking near the grocery store. At the framework level there are opportunistic computing and opportunistic information fusion. FAERIE does generic opportunistic actions independent of particular applications, e.g., exploit redundant channels of information to be more fault-tolerant. These opportunistic actions depend on policies, or preferences, defined by the system administrator. These preferences establish priorities or filters considering metrics taken from the monitoring of context observers. Examples of these metrics are availability, accuracy, freshness, power consumption, reboot cost and stability. These generic actions can be considered at two levels: network and computing. The network level includes the control mechanisms necessary to interconnect distributed components in an unstable network. This implies that the intermediate nodes in the communication are able to store the received messages and send them when the target addressees become reachable. These mechanisms in FAERIE are built using common dynamic connecting technologies such as UPnP (Jeronimo & Weast, 2003) or Zero-Conf (Steinberg & Cheshire, 2005). A more abstract vision of opportunism appears at the computing level. It implies the use of abstract resources by applications, so that the framework is able to withstand changes in the actual devices connected to the system as long as they serve to the same purposes. The framework at this level considers event types related with topology changes. They are necessary to reconfigure the information sources and targets in order to be able to provide continuity in the service provision. FAERIE considers the following types of context changes:  Addition of context observers to the system. The possible causes are: – Plug in: a new context observer has been connected into the system. – Reachable: a context observer in a remote environment becomes reachable, including its capabilities to calculate new pieces of context information. – Recovery: a context observer that was not properly working has recovered its normal functioning.

– – – –

–

1877

 Removal of context observers from the system. The possible reasons are: Plug out: a context observer is manually disconnected from the system. Unreachable: a context observer in a remote environment cannot be accessed. Error: a context observer crashes.  Change of priority, filtering preferences, or metrics. Change of metrics: the metrics monitored from the context observer return values that are incompatible with the system preferences. Change of preferences: modification of the system preferences render invalid the updates from certain context observers.

These types of context changes are managed by specific observers called computational-context observers. These have certain privileges to adapt the behavior of the system as the environment changes. Their privileges consist in the ability to manipulate directly the information paths, by means of activating or deactivating other context observers. This is necessary to improve the system performance in an opportunistic way by exploiting the different possibilities of the system. Figs. 2 and 3 respectively illustrate the components and behavior related to one of these observers. Fig. 2 shows the alternative observer, which is a computationalcontext observer. The context manager creates this alternative observer for each existing context element. The observer performs the activities shown in Fig. 3. It monitors certain metrics in the context in order to select among different observers the one that updates a context element. For this, it maintains a list of candidate context observers to manipulate the context element, and it periodically reconsiders this list. The list of candidate context observers can adopt different policies:  Single candidate: only one context observer is active after reading the metrics. Typically, it is that preferred according to the observed metrics.

Fig. 1. Schema of a FAERIE smart environment.

1878

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

 N candidates: a certain number of candidates are active, i.e., producing context values. This alternative produces redundant information, which has to be managed.  Everyone is candidate: using this alternative, there is no need of reconsideration. Every context observer is held active, and the necessary redundancy management is done. This alternative is expected to produce the worst resource consumption, but may have other advantages, such as lower response times. Maintaining a list of active candidates for a context element forces the framework to deal with redundant information. There are different potential policies to deal with this issue:  All updates are valid: all the updates performed by context observers are accepted, provided that they come temporally ordered.  Blacklist/Whitelist: all the updates performed by observers from the whitelist are accepted, while all performed by members of the blacklist are rejected.  Filter by metrics: only the updates performed by context observers that match the specified metrics preferences are accepted, e.g., filter the values with accuracy higher than a given value.  Prioritize by metrics: if multiple updates are received in the same time slot, those with more preferable values of metrics are chosen (for example, those more accurate).  Limit update frequency: the number of updates by time slot is limited, and all the extra updates are discarded.

The activities can take place in several environments. The use of workflows supports the abstract definition of activities that will be instantiated in different runtime activities depending on the actual context. For instance, it allows executing tasks when certain resources are available, or when the location changes. FAERIE applies opportunism control mechanisms to this kind of workflow. In a running application, context observers continually monitor the alternatives to update the context and choose between them according to events and metrics. When changes occur, their related events are used to trigger the instantiation of workflows or transitions between activities of existing workflows. The triggered activities can imply actions over the environment, and evaluations and processes internal to the system. In this way, the system is able to react in an opportunistic way to the evolution of its environment.

3. Testing and validation with UbikSim Validation of AmI applications by testing can be challenging and expensive. It may involve high numbers of users and devices, and require a long time to perform certain testing scenarios. Besides, it becomes impractical in some situations like a fire scenario or evacuations. UbikSim (Campillo-Sánchez & Botía, 2012) addresses these difficulties by providing a virtual environment where simulations

In summary, FAERIE provides concrete mechanisms and policies to deal with the identified opportunistic scenarios, which are applied to the lower abstraction layers of the system. These mechanisms manage the information flows of the system, monitoring the possible alternative resources and context changes and readjusting them to the new circumstances, possibly triggering alternative behaviors. 2.3. Application-level opportunistic behavior The mechanisms presented in the previous section make possible that FAERIE supports opportunism in the layers at the highest abstraction levels of the architecture. These layers are characterized by components that consider complex context conditions and behavior. The framework applications are built around the concept of workflow. A workflow defines its actors, resources, and activities, as well as the conditional transitions that connect these activities.

Fig. 2. Class diagram of the alternative observer.

Fig. 3. Activity diagram for the opportunistic behavior.

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

can be performed on AmI systems before its deployment. Probably, more validation tests will be needed in the actual target environment, but errors found at early stages of development are cheaper and easier to solve. The tool is currently being applied in a number of projects related to AmI, such as the Necesity and Caronte projects. Necesity (García-Valverde, Campuzano, Serrano, Villa, & Botía, 2012) aims to reduce automatically and non-intrusively the long waiting times until an elderly person is attended after suffering a fall. Caronte (car, 2011) deals with emergency management in nursing homes using social technologies and Web 2.0 techniques for simulating intelligent emergencies. UbikSim defines a SUT (System Under Test) as a software module, or a set of them, involved in a smart environment, and whose functional behavior is under test. The SUT behavior depends on its inputs. These inputs are mainly context information from the environment, in this case from a simulation. Examples of these inputs are the values given by distance and tile sensors. The specification of a simulation is composed by a scenario and configuration. The scenario is an environment which has a topology and a number of users and devices with different roles, properties (e.g., number and location of sensors), and behaviors. The configuration sets the start date and the seed of the simulation, which allow UbikSim to perform repeatable simulations. Thus, a test in UbikSim is equivalent to a specific simulation defined by its specification, because depending on the specifications the simulated context input will be different. An AmI application or service (i.e., a SUT) is correctly validated when it passes all the tests, i.e., all the desired simulation specifications. Checking the results of tests is a task not supported by UbikSim, so the tester has to do it by reviewing the states of the SUT and the received context. For the previous tasks, UbikSim includes several modules:  A 3D editor, UbikEditor, which is an extensible tool to model the AmI scenarios of the simulations. It provides facilities to define graphically the elements of an in-house intelligent environment and their distribution. Each element has properties that can be modified, such as name, role, accuracy, or range of a sensor.  A simulator, which is built on top of the Mason agent based modeling library Luke, Cioffi-Revilla, Panait, Sullivan, and Balan (2005). The use of the agent-based approach allows to model human behaviors (e.g., professors in universities and caregivers in hospitals), sensors (e.g., presence, pressure, and open door sensors) and actuators (e.g., air conditioner and light switch) in a common framework. This simulator takes as input the specifications of simulations developed with the editor.  Adapters to different context-aware platforms. An API allows external modules to receive context information from the simulator. This facility is useful for testing third-party modules in a simulated scenario. For example, simulated data of sensors can be used to test a location module. Fig. 4 shows an example of a component diagram corresponding to a FAERIE application. Every component in the middle layer is a context observer connected with the local context container. The integration with the UbikSim component is done by the FAERIEUbikAdapter component. It maps every device simulated in UbikSim to context elements. Then, it reads the events generated by UbikSim to update the attributes of the element in the context. A complete example of testing is included in Section 6.

1879

& Pavón, 2010). It considers an intelligent environment where spectators move through different rooms and interact with the talking agents. The agents are clay statues with speech recognition and dialog management abilities to communicate with the spectator. In order to guide the spectators’ interaction with the installation, the system has to determine the spectators’ position, guide their movements to the statues and room exits, and control the interactions with the statues. For the sake of simplicity, this paper only deals with people tracking. Fig. 5 shows an instance of this installation using the UbikEditor that was described in the previous section. This scenario includes several requirements: 1. Locations in rooms are defined as sectors of their ground. These divisions may change between deployments, and thus the potential spectators’ locations. 2. Spectators can be located at each room using different sensors. The system calculates the exact location when required and using the available sensors. The actual sensors used and their kind can therefore change at runtime. Two types of sensors are considered in this scenario: distance sensors and tile sensors. Distance sensors determine location by triangulation, using sonars, and tile sensors by detecting pressure on a certain area of the floor. 3. In case of redundancy or conflict between the locations calculated by different alternatives, the system selects the one with the highest confidence score. 4. Sensor failures and incomplete deployments may cause blind spots. The application will do its best to determine location at these circumstances. 5. During execution, new sensors or alternative mechanisms to determine location can be added. 5. Design of a FAERIE application FAERIE conceives AmI applications as context-aware workflows. Their design includes three main tasks: 1. Identifying the context elements and modeling the structure of the context representation. This consists in the identification of the relevant entities participating in the application (e.g., devices, users, activities, and locations) and their attributes and relationships. 2. Defining the existing activities as workflows, indicating the abstract sub-activities and their relationships with the context information they use and the involved actors. 3. Defining the rules that determine context changes as a function of the changes on other context elements or the physical environment. Rules are implemented as context observer components that work on the representation of the context as context elements, consuming their values. These rules can provide the values for other elements or, as part of workflows, provide the alternative implementations of abstract activities depending on the context conditions. The following subsections shows how the indicated tasks are used to implement the location and guiding application of the case study. 5.1. Context identification and modeling

4. Case study: an artistic installation The case study is an evolution of the ‘talking agents’ artistic installation. This is a convenient case for AmI experimentation because of its flexibility and comprehensibility (Fernández-de Alba

This phase determines several entities, attributes and relationships that are expected to be relevant for the application operation. The elements to take into consideration include:  Features of physical environments.

1880

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

Fig. 4. Component diagram of the UbikSim integration with FAERIE.

 Available components and capabilities.  Users’ characteristics.  Activities taking place. These elements make up the context. They are represented in FAERIE as context elements. An example of these elements in this case study is the information coming from a tile sensor. It is represented by a context element with the same name as the sensor and the following attributes and relationships:

 WeightDetected: attribute that holds the value produced by the sensor. This value will be used to detect if there is a person on the tile.  AdjacentTileSensors: relationship that represents adjacency with other TileSensors. This relationship can be defined either at deployment time (as specified in the UbikSim scenario proposed in this paper), or after an automatic learning phase. The complete graph of adjacency is contained in the MapGraph element.

Fig. 5. System structure of the Talking Agents application with FAERIE and UbikSim.

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

Both the attribute and the relationship will be used by the TileSensorLocator element in order to calculate the location of users in the environment. 5.2. Workflows definition The model from the previous phase is used to describe the activities taking place in the AmI system. An activity is specified as a workflow where different actors and entities participate in its subactivities. This process generates new elements to take into consideration for the context model. The main activity in this case study is interacting through the three rooms of the installation with users for them to experience the intended artistic effects. In each room, there exist in turn other sub-activities that represent the user walking into the room, the system giving indications, and the dialogs between the user and the clay statues. These activities raise new concepts to be considered in the context modeling, such as the user location, the indications given by the system, or the topography of rooms. 5.3. Context rules definition The rules represent how the context elements change in response to physical or system changes. They describe how the activities and other entities evolve or are adapted depending on the actual conditions. There are two types of updates: interpretation updates, which reflect actual changes in the real context (i.e., information fusion updates starting in sensors), and decision updates, which reflect the initiative of certain context observers to push the system behavior in certain direction (i.e., updates made by agents to complete running activities). An example of context rule is how the calculated user location changes as the weight detected by the tile sensors changes. This calculation needs information about the arrangement of the different tile sensors in the room and their data. Therefore, there will exist a context observer dedicated to calculate this location, which will request to the context the values of the different tile sensors, as well as the map graph containing their arrangement. 5.4. Application structure Context elements and context observers determine the application structure. For the case study, the diagram in Fig. 5 summarizes it. The types of context observers and the information they use are:  TileSensorLocator: a context observer that provides the user’s location. It uses the physical location of the TileSensor, and superimposes the MapGraph to detect position.  FAERIEUbikAdapter: it adapts the UbikSim component. This component simulates the external environment of a system and provides the MapGraph and DisanceSensor context elements. The MapGraph contains an attribute with a directed graph where each node represents a sector in a room. The DistanceSensor includes two attributes, one with the current state of the sensor and another with its location, orientation and action range. The data of this last attribute are provided at the moment of its deployment.  DistanceSensorLocator: a context observer that triangulates using the information from DistanceSensors on location, orientation, action range and current state, and matches the result with the MapGraph to determine the spectator’s Location. These elements allow the system to calculate spectator’s locations in an opportunistic way. FAERIE is designed to activate only the context observers that are currently being used. For instance, if there is a context observer already providing the location

1881

information, neither the TileSensorLocator nor the DistanceSensorLocator context observers will be activated, and the context elements needed by them will not be requested. In turn, the context observers responsible of calculating those context elements will not be activated either. The mechanism used to choose between observers in this case study is the confidence score. It is a number offered by each context observer to represent the precision of their measurement. As an improvement, the DistanceSensorLocator component only requests the Sensor elements necessary to determine the current location and those in sectors adjacent to it, where the spectator can move next. If the spectator is in a blind spot, every location next to it is observed.

6. Testing the application with UbikSim This section shows how the implementation of the ‘talking agents’ system with FAERIE presented in the previous section works. It discusses the operations performed by the system as reaction to certain events in the first room. Fig. 6 shows the representation of the entrance and the first room of the installation. This room has multiple tile sensors that cover the different sectors, as well as several distance sensors, covering only certain spots. White circles represent ‘‘blind spots’’, i.e., sectors covered by broken tile sensors. Though these are not blind spots when the installation starts functioning, sensors may not work properly sometimes or break down, and their sectors become blind spots. The black lines represent sector adjacency. Table 1 represents the events produced in the system, and how the different context observers change the context. The scenario runs as follows: 1. When the person is outside the room, s/he is considered within a virtual ‘‘blind spot’’. The TileSensorLocator requests every TileSensor located next to that blind spot, according to the information in the MapGraph. The request of the context elements produces the activation of the actual sensors in the environment. Since the TileSensorLocator cannot offer a Location due to lack of information, the alternative observer delegates on the DistanceSensorLocator. It requests then the DistanceSensors in the first room. As they do not cover the entrance, the Location cannot be provided, and thus its value is set to null. 2. The person moves into sectors 1, 2 and 3. The TileSensorLocator offers the correct Locations using the information obtained from the TileSensors that cover those sectors. The DistanceSensorLocator is not used, because the SensorLocator is already offering a Location with a higher confidence score. As a consequence, the system deactivates the DistanceSensorLocator, which releases the DistanceSensors. 3. The person enters into sector 6, which is a blind spot. The TileSensorLocator requests again every TileSensor next to that blind spot. As it cannot offer a Location, the alternative observer delegates on the DistanceSensorLocator, which requests again the DistanceSensor. Now, it offers the correct Location. 4. A new tile sensor is deployed in sector 3, which was previously a blind spot. This circumstance is detected thanks to the UPnP [18] feature of the component framework, as mentioned in Section 2.2. The TileSensorLocator reconfigures its map and gets a correct Location for the person. This makes the DistanceSensorLocator useless, and thus it is released again, along with the DistanceSensors, as they are not needed at the current person’s location. The previous functioning could change if noise in the room decreases making the confidence score of the DistanceSensorLocator higher. Then, the alternative observer would delegate on the

1882

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

Fig. 6. Map of the first room, indicating the sectors and their adjacency.

Table 1 Changes in system state as a result of environment events. The asterisk represent which value is selected for location. Environ. event

Outside room Go to sector 1 Go to sector 3 Deploy at Section 3 Room noise decrease

Distance sensor locator

Ubik adapter

Tile sensor locator

confid.

location

Distance sensors

confid.

location

Tile sensors

0.5 0.5 0.5 0.5 0.8

*null null *first room null *sector 3

First room none First room none First room

0.6 0.6 0.6 0.6 0.6

null *sector 1 null *sector 3 null

A, B, E A, B, D A, B, E B, C, E none

DistanceSensorLocator for Location. It would request again the DistanceSensors and get a correct Location. Hence, the TileSensorLocator would be deactivated, which would also release the TileSensors, as another component would be already offering a more reliable result.

7. Related work The concept of opportunism has been applied to different aspects of AmI applications, creating multiple research fields. The most relevant are discussed below, organized according to the level of abstraction in a system architecture where they apply. Opportunistic networks refer to networks where node connections are highly unstable, and information paths are established using the eventual connections with other nodes. For that reason, the intermediate nodes in an end-to-end connection need to buffer the information, and to wait for certain nodes to be connected in

Ubik adapter

order to transmit the information. In addition, certain applications are defined so that the destination nodes are not known beforehand, but discovered in the dissemination process. This constitutes the most basic form of opportunism, and has been studied in several applications and architectures (Huang et al., 2008). The selection at each node of to which nodes forward the information may rely on additional contextual information. The most common approach for this context management is the data-centric approach, which consists in driving the processing by the data requests. For example, the CAMEO project (Arnaboldi et al., 2011) and Boldrini et al. (2010) propose a middleware that infers social information from the users in order to predict the nodes most likely to become connected. FAERIE supports this mechanism using remote context observers (Fernández-de Alba et al., 2011), which have access to the context in order to make these decisions. Opportunistic computing is a further step in these mechanisms. It consists in the use of remote and possibly unknown resources in order to execute complex tasks. The idea is similar to opportunistic

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

1883

networks, but in this case the remote requests for information refer to abstract items that need to be resolved. The exact implementation for these requests is left to the destination nodes (Conti et al., 2010). This kind of behavior needs the definition of abstract information models and abstract sensor definitions. This way, the same abstract information may be implemented by different concrete processors in different circumstances. This approach transforms the opportunistic network in a cloud of abstract resources usable by any node of the network. An example of these works is (Kurz & Ferscha, 2010), which proposes a collection of sensor abstractions and a mechanism of self-description for them. This architecture can be extended by replicating the described processes in multiple abstraction layers, as it is done in the FAERIE framework architecture (Fernández-de Alba et al., 2012b). Opportunistic information fusion is defined at the next level of abstraction. This kind of mechanism consists in deciding the exact sources of information to be used in order to create information of a higher level of abstraction. This decision is based on the current context, and it is opportunistic, i.e., based on certain metrics taken on the available resources, user preferences, current activities, location. . . Using this principle, an application is not only able to obtain information from different abstract sources, but even to switch between different sources depending on the situation. Challa et al. (2005) propose concepts and alternatives for this aspect, which are taken into consideration in FAERIE, such as the modification of fusion rules as response to context changes (Fernández-de Alba et al., 2011). Opportunistic planning corresponds to the highest abstraction level. It consists on defining complex activities at the application level that are executed in a context-aware and opportunistic way. At this level, the selection of fusion rules is transparent. The objective is to have mechanisms that facilitate the detection of situations in which the execution of certain activities is convenient for some reasons. For example, this can be useful to recommend possible presents for a friend’s birthday when the date is coming and the user is near a mall. In this line, the OPPORTUNITY project (Roggen et al., 2009) presents some activity and context recognition systems with opportunistic behaviors. It also introduces an architecture focused on the information fusion mechanism, but does not specify the details on context modeling and handling. On the contrary, the FAERIE specification defines the way to model context and modify it in order to develop a complete contextaware system, which later can use the opportunistic features described in this paper. The previous types of opportunism do not represent isolated efforts. In order to have one of them, it is usually necessary to have an implementation of the lower levels. These types can be combined in multiple ways, what produces different architectures.

To implement these tasks, FAERIE includes an infrastructure to connect the context observers with the context information, and provides the patterns to specify their execution as a function of the context changes. This is what permits the designer of the application not to worry about the management of the behavior changes, as the framework handles the activation and change of the different information flows following the indicated preferences. These mechanisms are tested on an application with opportunistic evaluation. The application determines the user’s location using different alternatives, chosen at each moment according to available sensors and their confidence level. The test is performed on the UbikSim (Campillo-Sánchez & Botía, 2012) simulator for virtual smart environments. FAERIE integrates this software by providing wrappers for its sensor components. The integration of FAERIE and a simulation environment provides the basis to explore relevant extensions of our AmI architecture. A first issue is the implementation of automated learning mechanisms that reduce the need of providing deployment information to components, therefore simplifying the system configuration. These learning processes frequently need a training phase or redundant sources of information, which can be provided using the simulator. Another improvement is taking advantage of the automatic behaviors present in the UbikSim Kit to validate properties of FAERIE systems in ‘‘batch mode’’. The integration of privacy requirements into the architecture is another open issue. There are already works that deal with this (Kapadia, Triandopoulos, Cornelius, Peebles, & Kotz, 2008). The management of privacy has to consider the details regarding the distribution of information. Opportunistic networks relay on the use of eventual intermediate nodes to transmit the information. Control on the choice of these intermediate nodes has to be done to avoid compromising sensitive data. In addition, as opportunistic computing allows executing tasks requested from remote nodes, it is necessary to establish procedures to ensure that the processes does not produce undesirable effects. A final issue that FAERIE research is exploring is the development of AmI systems using model-driven methodologies. There are tools to create domain-specific visual languages and its associated semantic validators and code generators. An example is the INGENME tool (Pavón, Gómez-Sanz, & Paredes, 2011). This approach would facilitate the integration of systems developed following different approaches in opportunistic context-aware environments. There has been preliminary work in this line using FAERIE and the INGENIAS model (Gómez-Sanz, Fuentes, Pavón, & García-Magariño, 2008).

8. Conclusions

This work has been done in the context of the project ‘‘Social Ambient Assisting Living - Methods (SociAAL)’’, supported by the Spanish Ministry for Economy and Competitiveness, with grant TIN2011-28335-C02-01. Also, we acknowledge support from the ‘‘Programa de Creación y Consolidación de Grupos de Investigación’’ UCM-BSCH GR35/10-A.

The paper has described several mechanisms implemented in FAERIE to provide opportunistic behaviors at different abstraction levels of AmI applications. These are: the way it acts when the topology of the system changes; the policies it uses to resolve redundancy of information; and guidelines to make the most of these mechanisms, following a simple methodology. In addition, it illustrates the incorporation of testing and validation facilities in the development cycle. This effort meets several goals. With respect to application development, it offers flexibility, allowing designing context-aware processes that are executed differently depending on the dynamic changes of the system. It also reduces validation and test costs. With respect to the use of applications, it gives flexibility, minimizing the user configuration effort when the environment changes and reducing resource consumption in these processes.

Acknowledgments

References Caronte project (2011). http://innovation.logica.com.es/web/caronte/objetivos. Abowd, G. D., Dey, A. K., Brown, P. J., Davies, N., Smith, M., & Steggles, P. (1999). Towards a better understanding of context and context-awareness. In H. W. Gellersen (Ed.), Handheld and ubiquitous computing. Lecture notes in computer science (vol. 707, pp. 304–307). Berlin, Heidelberg: Springer. Fernández-de Alba, J. M., Campillo, P., Fuentes-Fernández, R., & Pavón, J. (2012a). Opportunistic sensor interpretation in a virtual smart environment. In H. Yin, J. Costa, & G. Barreto (Eds.), Intelligent data engineering and automated learning – IDEAL 2012. Lecture notes in computer science (vol. 7435, pp. 109–116). Berlin/ Heidelberg: Springer.

1884

J.M. Fernández-de-Alba et al. / Expert Systems with Applications 41 (2014) 1875–1884

Fernández-de Alba, J. M., Fuentes-Fernández, R., & Pavón, J. (2011). A dynamic context-aware architecture for ambient intelligence. In J. Cabestany, I. Rojas, & G. Joya (Eds.), Advances in computational intelligence. Lecture notes in computer science (vol. 6692, pp. 637–644). Berlin/Heidelberg: Springer. Fernández-de Alba, J. M., Fuentes-Fernández, R., & Pavón, J. (2012b). Dynamic workflow management for context-aware systems. In P. Novais, K. Hallenborg, D. I. Tapia, & J. M. C. Rodrguez (Eds.), Ambient intelligence – software and applications. Advances in intelligent and soft computing (vol. 153, pp. 181–188). Berlin/Heidelberg: Springer. Fernández-de Alba, J. M., & Pavón, J. (2010). Talking agents: A distributed architecture for interactive artistic installations. Integrated Computer-Aided Engineering, 17, 243–259. Anagnostopoulos, C. B., Tsounis, A., & Hadjiefthymiades, S. (2007). Context awareness in mobile computing environments. Wireless Personal Communications, 42, 445–464. Ardissono, L., Furnari, R., Goy, A., Petrone, G., & Segnan, M. (2007). Context-aware workflow management. In L. Baresi, P. Fraternali, & G. J. Houben (Eds.), Web engineering. Lecture notes in computer science (vol. 4607, pp. 47–52). Berlin/ Heidelberg: Springer. Arnaboldi, V., Conti, M., Delmastro, F. (2011). Implementation of cameo: A contextaware middleware for opportunistic mobile social networks. In World of wireless mobile and multimedia networks (WoWMoM). 2011 IEEE international symposium on a (pp. 1 –3). Baldauf, M., Dustdar, S., & Rosenberg, F. (2007). A survey on context-aware systems. International Journal of Ad Hoc and Ubiquitous Computing, 2, 263–277. Boldrini, C., Conti, M., Delmastro, F., & Passarella, A. (2010). Context- and socialaware middleware for opportunistic networks. Journal of Network and Computer Applications, 33, 525–541. Bylund, M., & Espinoza, F. (2002). Testing and demonstrating context-aware services with quake iii arena. Communications of the ACM, 45, 46–48. Campillo-Sánchez, P., & Botía, J. (2012). Simulation based software development for smart phones. In P. Novais, K. Hallenborg, D. I. Tapia, & J. M. C. Rodrguez (Eds.), Ambient intelligence – software and applications. Advances in intelligent and soft computing (vol.153, pp. 243–250). Berlin/Heidelberg: Springer. Challa, S., Gulrez, T., Chaczko, Z., Paranesha, T. (2005). Opportunistic information fusion: A new paradigm for next generation networked sensing systems. In Information fusion, 2005 8th international conference on. vol. 1, (pp. 720–727). Conti, M., Giordano, S., May, M., & Passarella, A. (2010). From opportunistic networks to opportunistic computing. Communications magazine (48. IEEE (pp. 126–139). IEEE. García-Valverde, T., Campuzano, F., Serrano, E., Villa, A., & Botía, J. A. (2012). Simulation of human behaviours for the validation of ambient intelligence services: A methodological approach. Journal of Ambient Intelligence and Smart Environments, 4, 163–181. Gómez-Sanz, J. J., Fuentes, R., Pavón, J., & García-Magariño, I. (2008). Ingenias development kit: A visual multi-agent system development environment. In Proceedings of the 7th international joint conference on autonomous agents and multiagent systems: demo papers AAMAS ’08. International foundation for autonomous agents and multiagent systems (pp. 1675–1676). Richland, SC

Huang, C. -M., chan Lan, K., & Tsai, C. -Z. (2008). A survey of opportunistic networks. In Advanced information networking and applications – workshops, 2008. AINAW 2008. 22nd international conference on (pp. 1672 –1677). Jeronimo, M., & Weast, J. (2003). UPnP design by example: A software developer’s guide to universal plug and play. Intel Press. Kapadia, A., Triandopoulos, N., Cornelius, C., Peebles, D., & Kotz, D. (2008). Anonysense: Opportunistic and privacy-preserving context collection. In J. Indulska, D. Patterson, T. Rodden, & M. Ott (Eds.), Pervasive computing. Lecture notes in computer science (vol. 5013, pp. 280–297). Berlin/Heidelberg: Springer. Kieffer, S., Lawson, J. -Y., & Macq, B. (2009). User-centered design and fast prototyping of an ambient assisted living system for elderly people. In Information technology: New generations, 2009. ITNG ’09. Sixth international conference on (pp. 1220–1225). Kurz, M., & Ferscha, A. (2010). Sensor abstractions for opportunistic activity and context recognition systems. In P. Lukowicz, K. Kunze, & G. Kortuem (Eds.), Smart sensing and context. Lecture notes in computer science (vol. 6446, pp. 135–148). Berlin/Heidelberg: Springer. Luke, S., Cioffi-Revilla, C., Panait, L., Sullivan, K., & Balan, G. (2005). Mason: A multiagent simulation environment. Simulation, 81, 517–527. Park, J., Moon, M., Hwang, S., & Yeom, K. (2007). Cass: A context-aware simulation system for smart home. In Software engineering research, management applications, 2007. SERA 2007. 5th ACIS international conference on (pp. 461 – 467). Patalano, A. L., & Seifert, C. M. (1997). Opportunistic planning: Being reminded of pending goals. Cognitive Psychology, 34, 1–36. Pavón, J., Gómez-Sanz, J., & Paredes, A. (2011). The sicossys approach to sos engineering. In System of systems engineering (SoSE), 2011 6th international conference on (pp. 179 –184). Remagnino, P., Hagras, H., Monekosso, N., & Velastin, S. (2005). Ambient intelligence. In G. Foresti, P. Remagnino, & T. Ellis (Eds.), Ambient intelligence (pp. 1–14). Springer. Roggen, D., Forster, K., Calatroni, A., Holleczek, T., Fang, Y., Troster, G., Lukowicz, P., Pirkl, G., Bannach, D., Kunze, K., Ferscha, A., Holzmann, C., Riener, A., Chavarriaga, R., del Millan, J. (2009). Opportunity: Towards opportunistic activity and context recognition systems. In World of wireless, mobile and multimedia networks workshops 2009, IEEE international symposium on a (pp. 1– 6). Incomplete. Shirehjini, A. A. N. (2005). A generic upnp architecture for ambient intelligence meeting rooms and a control point allowing for integrated 2d and 3d interaction. In Proceedings of the 2005 joint conference on smart objects and ambient intelligence: Innovative context-aware services: Usages and technologies sOc-EUSAI ’05 (pp. 207–212). New York, USA: ACM. Steinberg, D., & Cheshire, S. (2005). Zero configuration networking: The definitive guide. O’Reilly Media, Inc. Strang, T., Linnhoff-Popien, C. (2004). A context modeling survey. In Proceedings of the workshop on advanced context modelling reasoning and management at UbiComp 2004 (pp. 1–8).