- Email: [email protected]
Cost Oriented Humanoid Robot communication with IoT devices via MQTT and interaction with a Smart Home HUB connected devices
Available online at www.sciencedirect.com
ScienceDirect IFAC PapersOnLine 52-25 (2019) 104–109
Cost Oriented Humanoid Robot communication with IoT devices via MQTT and Cost Oriented Humanoid Robot communication with IoT devices via MQTT and Cost Oriented Humanoid Robot communication with IoT devices via MQTT and interaction with a Smart Home HUB connected devices Cost Oriented Humanoid Robot communication with IoT devices via MQTT and interaction with a Smart Home HUB connected devices interaction with a Smart Home HUB connected devices interaction with a Smart Home HUB connected devices **&*** Stefan Chivarov**, Peter Kopacek** and Nayden Chivarov**&*** Stefan Chivarov**,, Peter Peter Kopacek** and and Nayden Nayden Chivarov Chivarov**&*** **&*** Stefan Chivarov*, Peter Kopacek * **&*** Stefan Stefan Chivarov Chivarov*, Peter Kopacek Kopacek* and and Nayden Nayden Chivarov Chivarov**&*** Mechatronics, IHRT ** TU TU Wien, Wien, Institute Institute for for Mechanics Mechanics and and Mechatronics, IHRT Favoritenstrasse 9-11/E325 – 1040 Wien IHRT ** TU Wien, Institute for Mechanics and A4, A TU Wien, Institute for 9-11/E325 Mechanics A4, andAMechatronics, Mechatronics, Favoritenstrasse – 1040 Wien IHRT (e-mails: Favoritenstrasse 9-11/E325 A4, A [email protected], [email protected]) Favoritenstrasse 9-11/E325 A4, A – – 1040 1040 Wien Wien (e-mails: [email protected], [email protected]) (e-mails: [email protected], [email protected]) **Institute of Information and Communication Technologies, Bulgarian [email protected], [email protected]) **Institute of Information (e-mails: and Communication Technologies, Bulgarian Academy Academy of of Sciences, Sciences, Sofia, Sofia, Bulgaria Bulgaria (e-mail:Technologies, [email protected]) **Institute of Information and Communication Bulgarian Academy of Sciences, **Institute of Information and Communication Bulgarian Academy of Sciences, Sofia, Sofia, Bulgaria Bulgaria (e-mail:Technologies, [email protected]) *** European Polytechnical University (e-mail: (e-mail: [email protected]) [email protected]) *** European Polytechnical University *** Polytechnical University *** European European Polytechnical University Abstract: The article describes different approaches for connecting the teen sized humanoid robot Archie Abstract: The article describes different approaches for connecting the teen sized humanoid robot Archie Abstract: The article describes different approaches for connecting the sized robot Archie with IoT devices. Main goal of the project is to develop a cost oriented autonomous humanoid to Abstract: The article describes approaches for connecting the teen teen sized humanoid humanoid robotrobot Archie with IoT devices. Main goal of different the project is to develop a cost oriented autonomous humanoid robot to Archie robot’s natural habitat willautonomous be our homes and workplaces. with IoT devices. Main goal of the project is to develop a cost oriented humanoid robot assist humans in daily life tasks, thus with IoT devices. Mainlife goal of the project is robot’s to develop a cost oriented humanoid robot to to assist humans in daily tasks, thus Archie natural habitat willautonomous be our homes and workplaces. Methods for ROS integration of remote internet enabled sensors,will actuators and other devices are assist in life Archie robot’s natural be and workplaces. assist humans humans in daily daily life tasks, tasks,ofthus thus Archie robot’senabled natural habitat habitat be our our homes homes and devices workplaces. Methods for ROS integration remote internet sensors,will actuators and other are described.for Test withintegration real life use cases are performed. MQTT M2M protocol have been used are for Methods ROS of internet enabled sensors, actuators and other devices Methods for ROS of remote remote internet enabledMQTT sensors, actuators andhave other devices described. Test withintegration real life use cases are performed. M2M protocol been used are for connecting Test to internet enabled IoT cases devices. Smart home hubs, which connect either locally or to the described. with real life use are performed. MQTT M2M protocol have been used for described. Test with real life use areSmart performed. MQTTwhich M2Mconnect protocol havelocally been or used for connecting to internet enabled IoT cases devices. home hubs, either to the cloud, IoT devices, thatenabled use the Zigbee or Z-Wave and otherhubs, protocols, rather thaneither only locally Wi-Fi connected connecting to internet IoT devices. Smart home which connect or to the connecting to internet IoT devices. Smartand home whichrather connect or to the cloud, IoT devices, thatenabled use the Zigbee or Z-Wave otherhubs, protocols, thaneither only locally Wi-Fi connected devices, aredevices, disused.that Finally, we have or successfully implemented and tested 3 different approaches for cloud, use Zigbee Z-Wave other rather only connected cloud, IoT IoT use the thewe Zigbee Z-Wave and and other protocols, protocols, rather than than only Wi-Fi Wi-Fi connected devices, aredevices, disused.that Finally, have or successfully implemented and tested 3 different approaches for connecting Archie robot to IoTwe devices, which gives the robot the ability to interact with the environment devices, are disused. Finally, have successfully implemented and tested 3 different approaches for devices, areArchie disused. Finally, have successfully implemented and tested 3 different approaches for connecting robot to IoTwe devices, which gives the robot the ability to interact with the environment by receivingArchie data and thetoability to control the thousands of IoTthe devices available onwith the market. connecting robot IoT devices, which gives the robot ability to interact the environment connecting robot IoT devices, which gives the robot abilityavailable to interact the environment by receivingArchie data and thetoability to control the thousands of IoTthe devices onwith the market. by receiving data and to the of devices available on the market. by receiving and the the ability ability to control control the thousands thousands of IoT IoT devices available on All therights market. © 2019, IFACdata (International Federation of Automatic Control) Hosting by Elsevier Ltd. reserved. Keywords: Keywords: Humanoid Humanoid Robots, Robots, IoT, IoT, IoRT, IoRT, Industry Industry 4.0, 4.0, ROS, ROS, MQTT, MQTT, openHAB openHAB Keywords: Humanoid Robots, IoT, IoRT, Industry 4.0, ROS, MQTT, openHAB Keywords: Humanoid Robots, IoT, IoRT, Industry 4.0, ROS, MQTT, openHAB
integration gives Archie access to more than 1500 supported integration gives Archie access more thanterminology, 1500 supported 1. things. Things, based on the to openHAB are 1. INTRODUCTION INTRODUCTION integration gives Archie access to more 1500 integration gives based Archieon access to more than thanterminology, 1500 supported supported 1. INTRODUCTION things. Things, the openHAB are smart home IoT devices from different vendors, using 1. INTRODUCTION things. Things, based on the openHAB terminology, are The teen sized humanoid robot Archie is under development smart Things,IoTbased on the are home devices fromopenHAB different terminology, vendors, using The teen sized humanoid robot Archie is under development things. smart home IoT devices from different vendors, using different means of connection and different protocols for devices The teen humanoid robot is at and IHRT, home IoTof devices fromanddifferent different means connection differentvendors, protocolsusing for TheIntelligent teen sized sizedHandling humanoiddevices robot Archie Archie is under under ––development development at Intelligent Handling and Robotics Robotics IHRT, TU TU smart different means of connection and different protocols for connecting with the Smart home hub. There are over 200 (A. Byagowi et all, 2011).and Main goal of –theIHRT, projectTU at Handling devices Robotics WIЕN is different means different for with of the connection Smart homeandhub. There protocols are over 200 at Intelligent Intelligent Handling devices Robotics WIЕN (A. Byagowi et all, 2011).and Main goal of –theIHRT, projectTU is connecting connecting with the Smart home hub. There are over 200 technologies, APIs, and services that are supported directly in to develop a cost oriented autonomous humanoid robot to WIЕN (A. Byagowi et all, 2011). Main goal of the project is connecting with the Smart home hub. There are over 200 technologies, APIs, and services that are supported directly in WIЕN (A. Byagowi et all, 2011). Main goal of the project is technologies, to develop a cost oriented autonomous humanoid robot to APIs, and services that are supported directly in openHAB (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, assist humans in daily life tasks, thus Archie robot’s natural to develop a cost oriented autonomous humanoid robot to technologies, APIs, and services that are supported directly in openHAB (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, to develop a cost oriented autonomous humanoid to openHAB assist humans in daily life tasks, thus Archie robot’srobot natural (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, Zigbee, KNX, Google Home, Amazon Alexa, etc.). habitathumans will beinour homes and workplaces (D. Chikurtev, assist daily life tasks, thus Archie robot’s natural openHAB (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, Zigbee, KNX, Google Home, Amazon Alexa, etc.). assist humans daily life tasks, thus Archie (D. robot’s natural habitat will beinour homes and workplaces Chikurtev, Google Home, Amazon Alexa, etc.). habitat will be our homes and workplaces (D. Chikurtev, Zigbee, KNX, 2016). Home, use Amazon Alexa, etc.). habitat SMARTKNX, homeGoogle technology devices connected to the 2016). will be our homes and workplaces (D. Chikurtev, Zigbee, 2016). SMART home technology use devices connected to the 2016). Internet of things (IoT) to automate and monitor in-home SMART home technology use devices connected to There are currently 27 billion IoT connected devices and this Internet home technology use devices connected to the the of things (IoT) to automate and monitor in-home There are currently 27 billion IoT connected devices and this SMART systems. of Smart home technology, alsoand often referred to as Internet things (IoT) to monitor in-home There currently 27 billion IoT devices and this number expected increase to Internet of things (IoT) to automate automate and monitor in-home Smart home technology, also often referred to as There are areis billion IoT connected connected devicesby and2030 this systems. number iscurrently expected27 to to increase to 125 125 billion billion by 2030 systems. Smart also referred to home automation or technology, domotics (from the Latin "domus" 2019). number expected (Compelo, systems. Smart home home technology, also often often referred to as as automation or domotics (from the Latin "domus" number is is 2019). expected to to increase increase to to 125 125 billion billion by by 2030 2030 home (Compelo, home automation or domotics (from the Latin "domus" meaning home), provides homeowners security, comfort, (Compelo, 2019). home automation or domotics (from the Latin "domus" meaning home), provides homeowners security, comfort, (Compelo, 2019). The provides homeowners security, convenience and energy by them The predicted predicted household household penetration penetration (usage) (usage) of of Smart Smart home home meaning meaning home), home), homeowners security, comfort, comfort, convenience and provides energy efficiency efficiency by allowing allowing them to to technologies inhousehold US will reach 33.2% (usage) in 2019of(other sources The predicted penetration Smart home convenience and energy efficiency by allowing them to control smart devices, often by a smart home app on their The predictedinhousehold penetration Smartsources home control technologies US will reach 33.2% (usage) in 2019of(other convenience and energy efficiency by home allowing them to smart devices, often by a smart app on their predict up toin38%) and reach is expected to hit 53.9% bysources 2023. control technologies US will 33.2% in 2019 (other smart devices, often by a smart home app on their smartphone or other networked device. A part of the internet technologies US will 33.2% to in hit 2019 (otherbysources predict up toin38%) and reach is expected 53.9% 2023. smartphone control smart devices, often by a smart home app on their or other networked device. A part of the internet This trend is followed by is theexpected EU member states: for by example, predict up 38%) to 53.9% 2023. of things (IoT), smart home device. systemsA and devices often smartphone or networked part the predict up isto tofollowed 38%) and and to hit hit 53.9% 2023. of This trend by is theexpected EU member states: for by example, smartphone or other other networked device. A and part of of the internet internet things (IoT), smart home systems devices often This trend is followed by the EU member states: for example, operate together, sharing consumer usage data among in Germany the household penetration will be 19.9% in 2019 of (IoT), home systems and often This trend is the followed by the EU member for example, in Germany household penetration willstates: be 19.9% in 2019 operate of things thingstogether, (IoT), smart smart home systems usage and devices devices often sharing consumer data among in the penetration will 19.9% in themselves and automating actions basedusage on the homeowners' and is to by France the operate together, sharing consumer data among in Germany Germany the household household penetration will be be in 19.9% in 2019 2019 and is expected expected to hit hit 34.5% 34.5% by 2023, 2023, in France the themselves operate together, sharing actions consumer usage data among and automating based on the homeowners' and to 34.5% by 2023, in the household penetration will be 2019 is themselves preferences and is is expected expected to hit hit by in in France France the preferences household penetration will 34.5% be 13.7% 13.7% in2023, 2019 and and is expected expected themselves and and automating automating actions actions based based on on the the homeowners' homeowners' household penetration will be 13.7% in 2019 and is expected to hit 26.2% by 2023 preferences (https://internetofthingsagenda.techtarget.com/definition/sma household penetration to hit 26.2% by 2023 will be 13.7% in 2019 and is expected (https://internetofthingsagenda.techtarget.com/definition/sma preferences to hit 26.2% by 2023 (https://internetofthingsagenda.techtarget.com/definition/sma rt-home-or-building) to 2023 (https://internetofthingsagenda.techtarget.com/definition/sma Onhit the26.2% other by hand, one of the nine principal technologies that rt-home-or-building) rt-home-or-building) On the other hand, one of the nine principal technologies that rt-home-or-building) make up Industry 4.0 is the Industrial Internet of Things. On hand, one the nine technologies that On the theupother other hand,4.0 oneis of of nine principal principal make Industry thethe Industrial Internettechnologies of Things. that Smart Smart home home automation automation is is becoming becoming aa de-facto de-facto standard standard in in make up Industry 4.0 is the Industrial Internet of Things. Smart home automation is becoming a de-facto most new home offerings. make up Industry 4.0 is atheRobot Industrial InternetSystem of Things. Smartnew home automation is becoming a de-facto standard standard in in This paper describes Operating (ROS - most home offerings. new home offerings. This paper describes a Robot Operating System (ROS - most new home trends, offerings. Kinetic) (V.describes Ivanova, a 2012) based solutions for (ROS Archie- most This paper Robot Operating System Based on these in order to really assist people in their This paper Robot based Operating System Kinetic) (V.describes Ivanova, a 2012) solutions for (ROS Archie- Based on these trends, in order to really assist people in their devices over Kinetic) (V. Ivanova, based solutions for Archie daily tasks, a must have capability of assist the Archie robot is robot’s IoT Based on trends, in to people in Kinetic) (V. communication Ivanova, 2012) 2012) with based for MQTT Archie daily robot’s direct direct communication with IoTsolutions devices over MQTT Basedtasks, on these these trends, in order order to really really assist peoplerobot in their their aa must have capability of the Archie is Telemetrywith Transport protocol - MQTT Oasis, daily robot’s direct communication IoT devices over direct interaction with IoT devices and communication with (Message Queuing tasks, must have capability of the Archie robot is robot’s direct communication IoT devices over- MQTT (Message Queuing Telemetrywith Transport protocol Oasis, direct daily tasks, a must have capability of communication the Archie robot is interaction with IoT devices and with and Queuing for interaction with the (Message Telemetry Transport protocol Oasis, 2015) IoT devices connected to direct interaction with IoT devices and communication with the Smart Home HUB of their houses. (Message Telemetry Transport protocol - Oasis, 2015) and Queuing for interaction with the IoT devices connected to the direct interaction with of IoT devices and communication with Smart Home HUB their houses. openHAB (open Home Automation Busto- the Smart Home HUB of their houses. 2015) and for with the devices connected the open 2015) andsource, for interaction interaction with the IoT IoT devices connected the open source, openHAB (open Home Automation Busto- the Smart Home HUB of their houses. Smart home hub. (openHAB) the open source, openHAB (open Home Automation Bus openHAB 2) The openHAB the open source, openHAB (open(openHAB) Home Automation Bus -openHAB 2) Smart home hub. The openHAB openHAB 2) Smart home hub. (openHAB) The openHAB 2405-8963 2019, IFAC home (International Federation of The Automatic Control) Hosting by Elsevier Ltd. All rights reserved. openHAB© 2) Smart hub. (openHAB) openHAB Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.12.455
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
As Archie COHR (Cost Oriented Humanoid Robot) is still in the development phase and is not fully functional, we used another robot - Robco18 (N. Chivarov et all, 2018) as a testbed for the Internet-of-Things (IoT) functionality, which will be later used on Archie. This is the power of ROS (Robot Operating System) - the source code and packages we develop, can be reused on other robots. During our research we've put the emphasis on cost-oriented solutions for IoT connectivity and put them in to real life tests. As the intent of this article is to share the experience we gained while experimenting with technologies, different libraries and packages and to introduce the readers to the concepts on higher level, we’ve simplified the excerpts of source code and configuration files for ease of understanding. 2. CONECTING SENSORS AND ACTUATORS USING ROS SERIAL OVER Wi-Fi (TCP CONNECTION) The method described below provides ROS integration of remote internet enabled sensors, actuators and so on devices. We tested it in the following real-life use case: At robotics exhibitions children are approaching the stands and want to interact with the robots in some way. We decided make the robot serve small candy (with its articulated arm) to the children. The feature is activated by voice request or by the press of a “Give me some candy” button. Automated Candy dispensers are available on the market The dispenser operation is simple: a motion-sensor actuates a motor, which rotates a screw transport auger in order to deliver the candy. In our case we need the robot (instead of the motion sensor) to activate the candy dispenser (for a pre-set time interval), after positioning the robot arm with the scoop in front of the dispenser opening. For the task, we’ve chosen the ESP8266 NodeMCU v.3 microcontroller as a cheap (less than 2 €) Wi-Fi ESP-12 based module. Together with the Motor Shield Board, based on L293D driver, it is sold for about 3.30 €.
105
$ rosrun rosserial_python serial_node.py tcp 2.2 On the ESP8266 (NodeMCU) after the code for Wi-Fi setting up the network connection, we set the rosserial server IP and port: IPAddress server(192,168,1,2); const uint16_t serverPort = 11411; Then we instantiate the ROS publishers and subscribers: ros::Publisher &str_msg);
pub_feedback("/candy_dispenser/feedback",
ros::Publisher pub_candy_request("/candy_dispenser/candy_request", &str_msg); ros::Subscriber sub_start_motor("/candy_dispenser/start_motor", &startMotorCallback); ros::Subscriber sub_change_duriation("/candy_dispenser/motor_run_duriatio n", &changeDuriationCallback); ros::Subscriber sub_button_enable("/candy_dispenser/button_enable", &buttonEnableCallback); In the setup() function we initialize the Node and advertise the Publishers and Subscribers nh.getHardware()->setConnection(server, serverPort); nh.initNode(); nh.advertise(pub_feedback); nh.advertise(pub_candy_request); nh.subscribe(sub_start_motor); nh.subscribe(sub_change_duriation); nh.subscribe(sub_button_enable); 2.3 Now our robot, by subscribing and publishing to the corresponding ROS topics, can control the motor, set the time the motor should spin the auger and also receive button pressed event and feedback for the commands sent to the dispenser controller.
We used the rosserial ROS package (ROS) and the rosserial_arduino Arduino Library (Michael Ferguson, Joshua Frank). rosserial is a protocol for wrapping standard ROS serialized messages and multiplexing multiple topics and services over a character device such as a serial port or network socket. 2.1 On the ROS side (Robot’s computer) we start the rosserial server in TCP mode by:
3.
CONECTING TO INTERNET ENABLED IoT SENSORS AND ACTUATORS OVER MQTT M2M (Machine to Machine) PROTOCOL
There are thousands of IoT and Industry 4.0 Industrial IoT devices using MQTT M2M protocol.
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
106
MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. MQTT was invented by (Andy Stanford-Clark and Arlen Nipper, 1999).
- factory: mqtt_bridge.bridge:MqttToRosBridge
We use Mosquito MQTT broker (server) to interact with the MQTT enabled IoT devices (https://mosquitto.org/)
We launch the mqtt_bridge by:
For this experiment, we’ve used Sonoff Basic Smart Switch IoT relay (https://sonoff.itead.cc/en/products/sonoff/sonoffbasic), falshed with Tasmota - alternative open source firmware for ESP8266 based devices such as iTead Sonoff, Allterco Shelly and many, many more.
msg_type: std_msgs.msg:String topic_from: /stat/sonoff_01/POWER topic_to: /sonoff_01/state
roslaunch mqtt_bridge sonoff_control.launch 3.2 Now our Robot can control the relay by publishing “ON” or “OFF” string to the /sonoff_01/power ROS topic and get the state of the relay subscribing to /sonoff_01/state ROS topic.
(https://github.com/arendst/Sonoff-Tasmota/wiki)
3.1 On the Robot side (ROS) we use the mqtt_bridge package (http://wiki.ros.org/mqtt_bridge). in the config file /mqtt_bridge/config/config.yaml first we set the connection to the MQTT broker and serializer / deserializer: mqtt: client: protocol: 4
# MQTTv311
account: username: < user_name_for_the_broker > password: < user_name_for_the_broker > connection: host: < MQTT_broker_IP_address > port: 1883 keepalive: 60 private_path: device/001#
4. CONNECT TO THE THOUSANS OF IoT DEVICES AVAILABLE, TROUGH A SMART HOME HUB In the previous two approaches for robot interaction with IoT devices, we are limited to connecting only to IoT devices, which can either be programmed to be part of the ROS environment or connected and controlled over the internet using MQTT. There are thousands of other home automation devices which use different communication technologies, most popular of them being Z-Wave and ZigBee. These technologies use different radios, frequencies and communication protocols, rather than Wi-Fi. The Internet of Things (IoT) initiative connects smart home devices such as doorbells, lights, locks, security cameras, and thermostats has made it easy for everyone to install and use these gadgets in their homes, but we still need some sort of mechanism to control them (John Delaney). A Smart Home Hub is hardware or software that connects devices on a home automation network and controls communications among them. Smart home hubs, which connect either locally or to the cloud, are useful for internet of things (IoT) devices that use the Zigbee or Z-Wave and other protocols, rather than Wi-Fi.
serializer: json:dumps deserializer: json:loads Then we set the topics to be bridged in both directions RosToMqtt and MqttToRos: bridge: - factory: mqtt_bridge.bridge:RosToMqttBridge msg_type: std_msgs.msg:String topic_from: /sonoff_01/power topic_to: /cmnd/sonoff_01/POWER
Sometimes referred to as a smart home bridge, a smart home hub collects and translates various protocol communications from smart home devices. For example, if a smartphone, which does not use Zigbee to communicate, wants to "talk" with a smart lock, which only uses Zigbee and not the Wi-Fi or Bluetooth native to the smartphone, the smart home hub acts as a translator between the two. As such, a smart home hub acts as the heart of a smart home network, tying together various devices and systems in a centralized platform (Margaret Rouse). Many different Smart home hubs are commercially available (to name a few - Google Home Hub, Amazon Echo, Wink Hub 2, Samsung SmartThings, Vivint Smart Home).
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
107
Although these commercial hubs support multiple communication protocols and can control devices by other vendor, they are focusing on making the customer use the smart things ecosystem of devices provided by the vendor. Most of the Smart home (IoT) devices and Smart home hubs connect to (and are controlled trough) some kind of cloud service, this brings up the serious concern about privacy and not being able to locally control the devices in case of cloud or internet connection failure.
mqtt:mosquitto.user=
Here the open source hubs, which give you an independent local control over the connected Smart home devices, come into play (the top being openHab, Home Assistant and Domoticz).
Presence Item - which will report if the smartphone of the homeowner can be located on the home Wi-Fi network (by MAC address). This does not require a separate sensor, the check for presence is done using the openHAB server Wi-Fi connection.
We’ve based our research on openHab (short for Open Home Automation Bus), as one of the best-known home automation tools among open source enthusiasts, with a large user community and quite a number of supported devices and integrations.
Mailbox sensor item – this item will check the state of the mailbox sensor (programming of the mailbox sensor will be described below) and report if there is mail in the box.
OpenHAB runs on most popular platforms such as Linux, Windows and MacOS and on almost any hardware ranging from Raspberry Pis to desktop computers and server PCs. openHAB is very flexible and can support many DIY devices. openHAB 2 is developed in Java, Jetty is included as an HTTP server. Through the openHAB REST API most aspects of the openHAB system can be readily accessed by other programs (https://www.openhab.org/docs/). With its pluggable architecture, openHAB supports more than 200 different technologies and systems and thousands of devices! (https://www.openhab.org/addons/). For our research experiment we’ve connected two different IoT devices to the openHAB Smart home hub: an ESP8266 (NodeMCU) based mailbox sensor (alerting when new mail is placed in your mailbox by the postman) and a Sonoff Dual IoT switch controlling imaginary ceiling and reading lights in the living room of “Archie’s home” (https://sonoff.itead.cc/en/products/sonoff/sonoff-dual). There are few ways to configure openHAB, based on the level of knowledge of the user, the one which give us full control is via configuration files. There are three main configuration files: openhab.cfg,.items name>.sitemap
and
We will use Mosquito MQTT broker (server) to interact with the IoT devices connected to the openHab server, using the MQTT M2M (Machine to Machine) protocol. In order add the MQTT functionality we have to activate the openHab MQTT binding in the Bindings manager. openhab.cfg is the overall config file located in /etc/openhab/configurations/. In this file global config is set such as security authentication and MQTT server setting. E.g. to connect to our MQTT server we add the following: mqtt:mosquitto.url=tcp://:1883
mqtt:mosquitto.pwd= mqtt:mosquitto.qos=2 mqtt:mosquitto.retain=true For simplicity of explanation, we will define just few items in openHab:
Sonoff Dual Item which will allow control and report the state of the two relays of this IoT device, which will control the lights in the living room (https://sonoff.itead.cc/en/products/sonoff/sonoff-dual). The Sonoff Dual is flashed with Tasmota alternative open source firmware (https://github.com/arendst/Sonoff-Tasmota/wiki). The user will be able to manually switch on the lights as usual, on top of that user will be able to control and view the status of this 3 Items using the openHAB WEB based user interface or the dedicated openHAB apps for Android and IOS smart phones and tablets. In the same time Archie robot will be able to get the information from the sensors and control the Sonoff Item relays trough the MQTT M2M protocol. ➢
Items file:
The items file defines all the ‘things’ you might want to view or control in openHAB. Items can be bound to ‘bindings’ which are optional packages that used to extend functionality of openHAB. Our use case makes use of the MQTT and HTTP bindings. The items file is located in /etc/openhab/configurations/items. Inbound configurations allow us to receive MQTT messages into an openHAB item. Every item is allowed to have multiple inbound (or outbound) configurations. The structure of the inbound MQTT configuration string is: Item myItem {mqtt="[::: :], [:::], ..."} Below you can see the structure of the outbound MQTT configuration string. Outbound configurations allow us to publish (send) an MQTT message to the MQTT broker when an item receives a command or state update, and other MQTT clients that are subscribed to the given topic on the same broker, like Arduino devices for example, will receive those messages.
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
108
Item itemName { mqtt="[::::]" } The in inbound and outbound MQTT configuration string can be combined.
Switch item= Light_GF_Living_Ceiling Switch item= Light_GF_Living_Reading } Frame label="Is there mail in the mailbox?"
Excerpt of our archieshome.items file defining the 3 items described above looks like this:
{ Text item=MailboxSensor
……… Group gGF ["GroundFloor"]
"Ground Floor"
}
Frame label="Is the owner at home?"
Group GF_Living "Ground Floor Living ["GroundFloorLivingRoom"]
Room"
Group Garden
"Garden"
["Garden"]
Group Presence
"Presence"
["Presence"]
{ Text item=Presence_OwnersSmartPhone } }
Group ROS (All) On the ROS side Archie robot’s computer: String ROS_Status "ROS [%s]" /* Lights */ Switch Light_GF_Living_Ceiling "Ceiling light" (gGF, GF_Living, Lights ,ROS) ["Light"] { mqtt=">[broker:cmnd/sonoff_01/POWER1:command:*:defa ult], <[broker:stat/sonoff_01/POWER1:state:default]" } Switch Light_GF_Living_Reading "Reading light" (gGF, GF_Living, Lights ,ROS) ["Light"] { mqtt=">[broker:cmnd/sonoff_01/POWER2:command:*:defa ult], <[broker:stat/sonoff_01/POWER2:state:default]" } /* Presence */ Switch Presence_OwnersSmartPhone "Owners SmartPhone presence detected" (ROS) { channel="network:pingdevice:1d2fc7d4:online" } /* Mailbox */ Switch MailboxSensor "Got mail?: [%s]" (Garden, ROS) {mqtt="<[broker:stat/WiFi_Mailbox_sensor/:state:default]"} ….. ➢
Sitemap
Sitemaps are used to create elements of a user interface for making openHAB items accessible to the front-end interface. A simplified sitemap file for Archie’s Smart Home looks like this: sitemap ArchiesHome label=" Archie’s home" { Frame label="Living Room Lights" {
The power of openHab is that it can connect not only devices using the Internet (Wi-Fi or wired), but also devices using different radio frequencies and protocols like Z-Wave, Zigbee, Bluetooth and so on. Hopefully, there is a package available which allows ROS to communicate with the openHab server via Internet/MQTT and in that way to access all of the devices connected to the openHab server, regardless of the protocol and the type of physical connection used between the device and openHab. This allows a ROS robot to connect to a vast variety of IoT devices such as motion detectors, Z-Wave, Zigbee, Bluetooth devices, lighting, door locks, etc. The iot_bridge package provides a bi-directional bridge between ROS and the OpenHAB Home Automation system (M Herbert). After installing the iot_bridge package, we should update the host address and port to match the openHAB server in iot_bridge/config/items.yaml Then we need to make some changes in the openHAB configuration files (this is already done in the files mentioned above). We have to create a ROS group: Group ROS (All) and add the ROS group to each item, that should send status updates to ROS. We can send commands from ROS to any openHAB item regardless of what group it is in. We also need to add ROS_Status (String ROS_Status "ROS [%s]") to the openHAB's item file. Now we can launch the IoT Bridge node: roslaunch iot_bridge iot.launch When there is an item (belonging to the ROS group) state change in openHab, we receive this on /iot_updates topic. We can view this in the terminal by rostopic echo /iot_updates
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
For example, a when presence of the owner’s smartphone (as defined in the openHab items config file above) is detected/triggered in OpenHAB. The OpenHAB bridge will publish the following to the iot_updates topic in ROS: message type: diagnostic_msgs/KeyValue, key: Presence_OwnersSmartPhone, value: 1 (string). Our robot can control devices connected to openHab by publishing to /iot_command topic, in Pythonit looks like: pub = rospy.Publisher('iot_command', queue_size=10)
KeyValue,
pub.publish(item_name, item_value) So, in order for our Robot to switch the celling lamp on we need to publish: pub.publish(“Light_GF_Living_Ceiling”, “ON”) And to turn off we need to publish: pub.publish(“Light_GF_Living_Ceiling”, “OFF”)
5. CONCLUSIONS We’ve successfully implemented and tested 3 different approaches for connecting Archie robot to IoT devices. This gives the robot the ability to interact with the environment by receiving data and (potentially) controlling the thousands of IoT devices available on the market. An emerging trend in robotics is integrating robots with the Internet as a form of Internet-of-Things (IoT). This allows robots to be controlled and monitored through the Internet and is a must for cloud robotics applications. We already have experience using RoboWebTools (http://robotwebtools.org/), but we still need to test and evaluate other approaches like Roslink (Koubaa, Anis, Alajlan, Maram & Qureshi, Basit, 2017). ROSLink is a new protocol to integrate Robot Operating System (ROS) enabled-robots with the IoT. ACKNOWLEDGEMENT The research presented in this paper was supported partially by the Bulgarian National Science Fund under the contract № KP-06-М27/1–04.12.2019 “Research and Development of Innovative, Intelligent Information and Communication Technologies for Control of Service Robots”. REFERENCES A. Byagowi, P. Kopacek, J. Baltes, “Archie”: A Cost Oriented Humanoid Robot, Proceedings of the 18th IFAC World Congress Milano (Italy), 978-3-902661-937/11/$20.00 © 2011 IFAC
109
D. Chikurtev (2016), Indoor Navigation for Service Mobile Robots Using Robot Operating System (ROS), PROBLEMS OF ENGINEERING CYBERNETICS AND ROBOTICS, Vol 67, Sofia, , p. 61-70, ISSN 02049848. Compelo (2019): https://compelo.com/smart-device-trends2019/ Statista, (2019): https://www.statista.com/outlook/279/109/smarthome/united-states Ivanova et all, Application of Meta-Operating System Ros for Control of Service Mobile Robots, ADP 2012, p. 247254, ISSN 1310-3946. Oasis (2015): "MQTT 3.1.1 specification". OASIS. December 10, 2015. Retrieved April 25, 2017. openHAB, 2019 by the openHAB Community and the openHAB Foundation e.V., https://www.openhab.org/ Chivarov, N., Chikurtev, D., Markov, E., Chivarov, S., Kopacek, Cost Oriented Tele-Controlled Service Robot for Increasing the Quality of Life of Elderly and Disabled - ROBCO 18, P., Volume 51, Issue 30, 2018, Pages 192-197, ISSN: 24058963 ROS, http://wiki.ros.org/ Arduino, https://www.arduinolibraries.info/libraries/rosserialarduino-library Dr Andy Stanford-Clark of IBM, and Arlen Nipper of Arcom (now Eurotech), in 1999. (http://mqtt.org/faq) Contributing Editor for PCMag, John Delaney, https://www.pcmag.com/article/327457/what-is-a-smarthome-hub-and-do-you-need-one Margaret Rouse https://internetofthingsagenda.techtarget.com/definition/s mart-home-hub-home-automation-hub M Herbert http://wiki.ros.org/action/show/iot_bridge?action=show &redirect=openhab_bridge ] Koubaa, Anis & Alajlan, Maram & Qureshi, Basit. (2017). ROSLink: Bridging ROS with the Internet-of-Things for cloud robotics. 10.1007/978-3-319-54927-9_8.
ScienceDirect IFAC PapersOnLine 52-25 (2019) 104–109
Cost Oriented Humanoid Robot communication with IoT devices via MQTT and Cost Oriented Humanoid Robot communication with IoT devices via MQTT and Cost Oriented Humanoid Robot communication with IoT devices via MQTT and interaction with a Smart Home HUB connected devices Cost Oriented Humanoid Robot communication with IoT devices via MQTT and interaction with a Smart Home HUB connected devices interaction with a Smart Home HUB connected devices interaction with a Smart Home HUB connected devices **&*** Stefan Chivarov**, Peter Kopacek** and Nayden Chivarov**&*** Stefan Chivarov**,, Peter Peter Kopacek** and and Nayden Nayden Chivarov Chivarov**&*** **&*** Stefan Chivarov*, Peter Kopacek * **&*** Stefan Stefan Chivarov Chivarov*, Peter Kopacek Kopacek* and and Nayden Nayden Chivarov Chivarov**&*** Mechatronics, IHRT ** TU TU Wien, Wien, Institute Institute for for Mechanics Mechanics and and Mechatronics, IHRT Favoritenstrasse 9-11/E325 – 1040 Wien IHRT ** TU Wien, Institute for Mechanics and A4, A TU Wien, Institute for 9-11/E325 Mechanics A4, andAMechatronics, Mechatronics, Favoritenstrasse – 1040 Wien IHRT (e-mails: Favoritenstrasse 9-11/E325 A4, A [email protected], [email protected]) Favoritenstrasse 9-11/E325 A4, A – – 1040 1040 Wien Wien (e-mails: [email protected], [email protected]) (e-mails: [email protected], [email protected]) **Institute of Information and Communication Technologies, Bulgarian [email protected], [email protected]) **Institute of Information (e-mails: and Communication Technologies, Bulgarian Academy Academy of of Sciences, Sciences, Sofia, Sofia, Bulgaria Bulgaria (e-mail:Technologies, [email protected]) **Institute of Information and Communication Bulgarian Academy of Sciences, **Institute of Information and Communication Bulgarian Academy of Sciences, Sofia, Sofia, Bulgaria Bulgaria (e-mail:Technologies, [email protected]) *** European Polytechnical University (e-mail: (e-mail: [email protected]) [email protected]) *** European Polytechnical University *** Polytechnical University *** European European Polytechnical University Abstract: The article describes different approaches for connecting the teen sized humanoid robot Archie Abstract: The article describes different approaches for connecting the teen sized humanoid robot Archie Abstract: The article describes different approaches for connecting the sized robot Archie with IoT devices. Main goal of the project is to develop a cost oriented autonomous humanoid to Abstract: The article describes approaches for connecting the teen teen sized humanoid humanoid robotrobot Archie with IoT devices. Main goal of different the project is to develop a cost oriented autonomous humanoid robot to Archie robot’s natural habitat willautonomous be our homes and workplaces. with IoT devices. Main goal of the project is to develop a cost oriented humanoid robot assist humans in daily life tasks, thus with IoT devices. Mainlife goal of the project is robot’s to develop a cost oriented humanoid robot to to assist humans in daily tasks, thus Archie natural habitat willautonomous be our homes and workplaces. Methods for ROS integration of remote internet enabled sensors,will actuators and other devices are assist in life Archie robot’s natural be and workplaces. assist humans humans in daily daily life tasks, tasks,ofthus thus Archie robot’senabled natural habitat habitat be our our homes homes and devices workplaces. Methods for ROS integration remote internet sensors,will actuators and other are described.for Test withintegration real life use cases are performed. MQTT M2M protocol have been used are for Methods ROS of internet enabled sensors, actuators and other devices Methods for ROS of remote remote internet enabledMQTT sensors, actuators andhave other devices described. Test withintegration real life use cases are performed. M2M protocol been used are for connecting Test to internet enabled IoT cases devices. Smart home hubs, which connect either locally or to the described. with real life use are performed. MQTT M2M protocol have been used for described. Test with real life use areSmart performed. MQTTwhich M2Mconnect protocol havelocally been or used for connecting to internet enabled IoT cases devices. home hubs, either to the cloud, IoT devices, thatenabled use the Zigbee or Z-Wave and otherhubs, protocols, rather thaneither only locally Wi-Fi connected connecting to internet IoT devices. Smart home which connect or to the connecting to internet IoT devices. Smartand home whichrather connect or to the cloud, IoT devices, thatenabled use the Zigbee or Z-Wave otherhubs, protocols, thaneither only locally Wi-Fi connected devices, aredevices, disused.that Finally, we have or successfully implemented and tested 3 different approaches for cloud, use Zigbee Z-Wave other rather only connected cloud, IoT IoT use the thewe Zigbee Z-Wave and and other protocols, protocols, rather than than only Wi-Fi Wi-Fi connected devices, aredevices, disused.that Finally, have or successfully implemented and tested 3 different approaches for connecting Archie robot to IoTwe devices, which gives the robot the ability to interact with the environment devices, are disused. Finally, have successfully implemented and tested 3 different approaches for devices, areArchie disused. Finally, have successfully implemented and tested 3 different approaches for connecting robot to IoTwe devices, which gives the robot the ability to interact with the environment by receivingArchie data and thetoability to control the thousands of IoTthe devices available onwith the market. connecting robot IoT devices, which gives the robot ability to interact the environment connecting robot IoT devices, which gives the robot abilityavailable to interact the environment by receivingArchie data and thetoability to control the thousands of IoTthe devices onwith the market. by receiving data and to the of devices available on the market. by receiving and the the ability ability to control control the thousands thousands of IoT IoT devices available on All therights market. © 2019, IFACdata (International Federation of Automatic Control) Hosting by Elsevier Ltd. reserved. Keywords: Keywords: Humanoid Humanoid Robots, Robots, IoT, IoT, IoRT, IoRT, Industry Industry 4.0, 4.0, ROS, ROS, MQTT, MQTT, openHAB openHAB Keywords: Humanoid Robots, IoT, IoRT, Industry 4.0, ROS, MQTT, openHAB Keywords: Humanoid Robots, IoT, IoRT, Industry 4.0, ROS, MQTT, openHAB
integration gives Archie access to more than 1500 supported integration gives Archie access more thanterminology, 1500 supported 1. things. Things, based on the to openHAB are 1. INTRODUCTION INTRODUCTION integration gives Archie access to more 1500 integration gives based Archieon access to more than thanterminology, 1500 supported supported 1. INTRODUCTION things. Things, the openHAB are smart home IoT devices from different vendors, using 1. INTRODUCTION things. Things, based on the openHAB terminology, are The teen sized humanoid robot Archie is under development smart Things,IoTbased on the are home devices fromopenHAB different terminology, vendors, using The teen sized humanoid robot Archie is under development things. smart home IoT devices from different vendors, using different means of connection and different protocols for devices The teen humanoid robot is at and IHRT, home IoTof devices fromanddifferent different means connection differentvendors, protocolsusing for TheIntelligent teen sized sizedHandling humanoiddevices robot Archie Archie is under under ––development development at Intelligent Handling and Robotics Robotics IHRT, TU TU smart different means of connection and different protocols for connecting with the Smart home hub. There are over 200 (A. Byagowi et all, 2011).and Main goal of –theIHRT, projectTU at Handling devices Robotics WIЕN is different means different for with of the connection Smart homeandhub. There protocols are over 200 at Intelligent Intelligent Handling devices Robotics WIЕN (A. Byagowi et all, 2011).and Main goal of –theIHRT, projectTU is connecting connecting with the Smart home hub. There are over 200 technologies, APIs, and services that are supported directly in to develop a cost oriented autonomous humanoid robot to WIЕN (A. Byagowi et all, 2011). Main goal of the project is connecting with the Smart home hub. There are over 200 technologies, APIs, and services that are supported directly in WIЕN (A. Byagowi et all, 2011). Main goal of the project is technologies, to develop a cost oriented autonomous humanoid robot to APIs, and services that are supported directly in openHAB (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, assist humans in daily life tasks, thus Archie robot’s natural to develop a cost oriented autonomous humanoid robot to technologies, APIs, and services that are supported directly in openHAB (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, to develop a cost oriented autonomous humanoid to openHAB assist humans in daily life tasks, thus Archie robot’srobot natural (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, Zigbee, KNX, Google Home, Amazon Alexa, etc.). habitathumans will beinour homes and workplaces (D. Chikurtev, assist daily life tasks, thus Archie robot’s natural openHAB (Just to name a few: Bluetooth, Wi-Fi, Z-Wave, Zigbee, KNX, Google Home, Amazon Alexa, etc.). assist humans daily life tasks, thus Archie (D. robot’s natural habitat will beinour homes and workplaces Chikurtev, Google Home, Amazon Alexa, etc.). habitat will be our homes and workplaces (D. Chikurtev, Zigbee, KNX, 2016). Home, use Amazon Alexa, etc.). habitat SMARTKNX, homeGoogle technology devices connected to the 2016). will be our homes and workplaces (D. Chikurtev, Zigbee, 2016). SMART home technology use devices connected to the 2016). Internet of things (IoT) to automate and monitor in-home SMART home technology use devices connected to There are currently 27 billion IoT connected devices and this Internet home technology use devices connected to the the of things (IoT) to automate and monitor in-home There are currently 27 billion IoT connected devices and this SMART systems. of Smart home technology, alsoand often referred to as Internet things (IoT) to monitor in-home There currently 27 billion IoT devices and this number expected increase to Internet of things (IoT) to automate automate and monitor in-home Smart home technology, also often referred to as There are areis billion IoT connected connected devicesby and2030 this systems. number iscurrently expected27 to to increase to 125 125 billion billion by 2030 systems. Smart also referred to home automation or technology, domotics (from the Latin "domus" 2019). number expected (Compelo, systems. Smart home home technology, also often often referred to as as automation or domotics (from the Latin "domus" number is is 2019). expected to to increase increase to to 125 125 billion billion by by 2030 2030 home (Compelo, home automation or domotics (from the Latin "domus" meaning home), provides homeowners security, comfort, (Compelo, 2019). home automation or domotics (from the Latin "domus" meaning home), provides homeowners security, comfort, (Compelo, 2019). The provides homeowners security, convenience and energy by them The predicted predicted household household penetration penetration (usage) (usage) of of Smart Smart home home meaning meaning home), home), homeowners security, comfort, comfort, convenience and provides energy efficiency efficiency by allowing allowing them to to technologies inhousehold US will reach 33.2% (usage) in 2019of(other sources The predicted penetration Smart home convenience and energy efficiency by allowing them to control smart devices, often by a smart home app on their The predictedinhousehold penetration Smartsources home control technologies US will reach 33.2% (usage) in 2019of(other convenience and energy efficiency by home allowing them to smart devices, often by a smart app on their predict up toin38%) and reach is expected to hit 53.9% bysources 2023. control technologies US will 33.2% in 2019 (other smart devices, often by a smart home app on their smartphone or other networked device. A part of the internet technologies US will 33.2% to in hit 2019 (otherbysources predict up toin38%) and reach is expected 53.9% 2023. smartphone control smart devices, often by a smart home app on their or other networked device. A part of the internet This trend is followed by is theexpected EU member states: for by example, predict up 38%) to 53.9% 2023. of things (IoT), smart home device. systemsA and devices often smartphone or networked part the predict up isto tofollowed 38%) and and to hit hit 53.9% 2023. of This trend by is theexpected EU member states: for by example, smartphone or other other networked device. A and part of of the internet internet things (IoT), smart home systems devices often This trend is followed by the EU member states: for example, operate together, sharing consumer usage data among in Germany the household penetration will be 19.9% in 2019 of (IoT), home systems and often This trend is the followed by the EU member for example, in Germany household penetration willstates: be 19.9% in 2019 operate of things thingstogether, (IoT), smart smart home systems usage and devices devices often sharing consumer data among in the penetration will 19.9% in themselves and automating actions basedusage on the homeowners' and is to by France the operate together, sharing consumer data among in Germany Germany the household household penetration will be be in 19.9% in 2019 2019 and is expected expected to hit hit 34.5% 34.5% by 2023, 2023, in France the themselves operate together, sharing actions consumer usage data among and automating based on the homeowners' and to 34.5% by 2023, in the household penetration will be 2019 is themselves preferences and is is expected expected to hit hit by in in France France the preferences household penetration will 34.5% be 13.7% 13.7% in2023, 2019 and and is expected expected themselves and and automating automating actions actions based based on on the the homeowners' homeowners' household penetration will be 13.7% in 2019 and is expected to hit 26.2% by 2023 preferences (https://internetofthingsagenda.techtarget.com/definition/sma household penetration to hit 26.2% by 2023 will be 13.7% in 2019 and is expected (https://internetofthingsagenda.techtarget.com/definition/sma preferences to hit 26.2% by 2023 (https://internetofthingsagenda.techtarget.com/definition/sma rt-home-or-building) to 2023 (https://internetofthingsagenda.techtarget.com/definition/sma Onhit the26.2% other by hand, one of the nine principal technologies that rt-home-or-building) rt-home-or-building) On the other hand, one of the nine principal technologies that rt-home-or-building) make up Industry 4.0 is the Industrial Internet of Things. On hand, one the nine technologies that On the theupother other hand,4.0 oneis of of nine principal principal make Industry thethe Industrial Internettechnologies of Things. that Smart Smart home home automation automation is is becoming becoming aa de-facto de-facto standard standard in in make up Industry 4.0 is the Industrial Internet of Things. Smart home automation is becoming a de-facto most new home offerings. make up Industry 4.0 is atheRobot Industrial InternetSystem of Things. Smartnew home automation is becoming a de-facto standard standard in in This paper describes Operating (ROS - most home offerings. new home offerings. This paper describes a Robot Operating System (ROS - most new home trends, offerings. Kinetic) (V.describes Ivanova, a 2012) based solutions for (ROS Archie- most This paper Robot Operating System Based on these in order to really assist people in their This paper Robot based Operating System Kinetic) (V.describes Ivanova, a 2012) solutions for (ROS Archie- Based on these trends, in order to really assist people in their devices over Kinetic) (V. Ivanova, based solutions for Archie daily tasks, a must have capability of assist the Archie robot is robot’s IoT Based on trends, in to people in Kinetic) (V. communication Ivanova, 2012) 2012) with based for MQTT Archie daily robot’s direct direct communication with IoTsolutions devices over MQTT Basedtasks, on these these trends, in order order to really really assist peoplerobot in their their aa must have capability of the Archie is Telemetrywith Transport protocol - MQTT Oasis, daily robot’s direct communication IoT devices over direct interaction with IoT devices and communication with (Message Queuing tasks, must have capability of the Archie robot is robot’s direct communication IoT devices over- MQTT (Message Queuing Telemetrywith Transport protocol Oasis, direct daily tasks, a must have capability of communication the Archie robot is interaction with IoT devices and with and Queuing for interaction with the (Message Telemetry Transport protocol Oasis, 2015) IoT devices connected to direct interaction with IoT devices and communication with the Smart Home HUB of their houses. (Message Telemetry Transport protocol - Oasis, 2015) and Queuing for interaction with the IoT devices connected to the direct interaction with of IoT devices and communication with Smart Home HUB their houses. openHAB (open Home Automation Busto- the Smart Home HUB of their houses. 2015) and for with the devices connected the open 2015) andsource, for interaction interaction with the IoT IoT devices connected the open source, openHAB (open Home Automation Busto- the Smart Home HUB of their houses. Smart home hub. (openHAB) the open source, openHAB (open Home Automation Bus openHAB 2) The openHAB the open source, openHAB (open(openHAB) Home Automation Bus -openHAB 2) Smart home hub. The openHAB openHAB 2) Smart home hub. (openHAB) The openHAB 2405-8963 2019, IFAC home (International Federation of The Automatic Control) Hosting by Elsevier Ltd. All rights reserved. openHAB© 2) Smart hub. (openHAB) openHAB Peer review under responsibility of International Federation of Automatic Control. 10.1016/j.ifacol.2019.12.455
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
As Archie COHR (Cost Oriented Humanoid Robot) is still in the development phase and is not fully functional, we used another robot - Robco18 (N. Chivarov et all, 2018) as a testbed for the Internet-of-Things (IoT) functionality, which will be later used on Archie. This is the power of ROS (Robot Operating System) - the source code and packages we develop, can be reused on other robots. During our research we've put the emphasis on cost-oriented solutions for IoT connectivity and put them in to real life tests. As the intent of this article is to share the experience we gained while experimenting with technologies, different libraries and packages and to introduce the readers to the concepts on higher level, we’ve simplified the excerpts of source code and configuration files for ease of understanding. 2. CONECTING SENSORS AND ACTUATORS USING ROS SERIAL OVER Wi-Fi (TCP CONNECTION) The method described below provides ROS integration of remote internet enabled sensors, actuators and so on devices. We tested it in the following real-life use case: At robotics exhibitions children are approaching the stands and want to interact with the robots in some way. We decided make the robot serve small candy (with its articulated arm) to the children. The feature is activated by voice request or by the press of a “Give me some candy” button. Automated Candy dispensers are available on the market The dispenser operation is simple: a motion-sensor actuates a motor, which rotates a screw transport auger in order to deliver the candy. In our case we need the robot (instead of the motion sensor) to activate the candy dispenser (for a pre-set time interval), after positioning the robot arm with the scoop in front of the dispenser opening. For the task, we’ve chosen the ESP8266 NodeMCU v.3 microcontroller as a cheap (less than 2 €) Wi-Fi ESP-12 based module. Together with the Motor Shield Board, based on L293D driver, it is sold for about 3.30 €.
105
$ rosrun rosserial_python serial_node.py tcp 2.2 On the ESP8266 (NodeMCU) after the code for Wi-Fi setting up the network connection, we set the rosserial server IP and port: IPAddress server(192,168,1,2); const uint16_t serverPort = 11411; Then we instantiate the ROS publishers and subscribers: ros::Publisher &str_msg);
pub_feedback("/candy_dispenser/feedback",
ros::Publisher pub_candy_request("/candy_dispenser/candy_request", &str_msg); ros::Subscriber
We used the rosserial ROS package (ROS) and the rosserial_arduino Arduino Library (Michael Ferguson, Joshua Frank). rosserial is a protocol for wrapping standard ROS serialized messages and multiplexing multiple topics and services over a character device such as a serial port or network socket. 2.1 On the ROS side (Robot’s computer) we start the rosserial server in TCP mode by:
3.
CONECTING TO INTERNET ENABLED IoT SENSORS AND ACTUATORS OVER MQTT M2M (Machine to Machine) PROTOCOL
There are thousands of IoT and Industry 4.0 Industrial IoT devices using MQTT M2M protocol.
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
106
MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. MQTT was invented by (Andy Stanford-Clark and Arlen Nipper, 1999).
- factory: mqtt_bridge.bridge:MqttToRosBridge
We use Mosquito MQTT broker (server) to interact with the MQTT enabled IoT devices (https://mosquitto.org/)
We launch the mqtt_bridge by:
For this experiment, we’ve used Sonoff Basic Smart Switch IoT relay (https://sonoff.itead.cc/en/products/sonoff/sonoffbasic), falshed with Tasmota - alternative open source firmware for ESP8266 based devices such as iTead Sonoff, Allterco Shelly and many, many more.
msg_type: std_msgs.msg:String topic_from: /stat/sonoff_01/POWER topic_to: /sonoff_01/state
roslaunch mqtt_bridge sonoff_control.launch 3.2 Now our Robot can control the relay by publishing “ON” or “OFF” string to the /sonoff_01/power ROS topic and get the state of the relay subscribing to /sonoff_01/state ROS topic.
(https://github.com/arendst/Sonoff-Tasmota/wiki)
3.1 On the Robot side (ROS) we use the mqtt_bridge package (http://wiki.ros.org/mqtt_bridge). in the config file /mqtt_bridge/config/config.yaml first we set the connection to the MQTT broker and serializer / deserializer: mqtt: client: protocol: 4
# MQTTv311
account: username: < user_name_for_the_broker > password: < user_name_for_the_broker > connection: host: < MQTT_broker_IP_address > port: 1883 keepalive: 60 private_path: device/001#
4. CONNECT TO THE THOUSANS OF IoT DEVICES AVAILABLE, TROUGH A SMART HOME HUB In the previous two approaches for robot interaction with IoT devices, we are limited to connecting only to IoT devices, which can either be programmed to be part of the ROS environment or connected and controlled over the internet using MQTT. There are thousands of other home automation devices which use different communication technologies, most popular of them being Z-Wave and ZigBee. These technologies use different radios, frequencies and communication protocols, rather than Wi-Fi. The Internet of Things (IoT) initiative connects smart home devices such as doorbells, lights, locks, security cameras, and thermostats has made it easy for everyone to install and use these gadgets in their homes, but we still need some sort of mechanism to control them (John Delaney). A Smart Home Hub is hardware or software that connects devices on a home automation network and controls communications among them. Smart home hubs, which connect either locally or to the cloud, are useful for internet of things (IoT) devices that use the Zigbee or Z-Wave and other protocols, rather than Wi-Fi.
serializer: json:dumps deserializer: json:loads Then we set the topics to be bridged in both directions RosToMqtt and MqttToRos: bridge: - factory: mqtt_bridge.bridge:RosToMqttBridge msg_type: std_msgs.msg:String topic_from: /sonoff_01/power topic_to: /cmnd/sonoff_01/POWER
Sometimes referred to as a smart home bridge, a smart home hub collects and translates various protocol communications from smart home devices. For example, if a smartphone, which does not use Zigbee to communicate, wants to "talk" with a smart lock, which only uses Zigbee and not the Wi-Fi or Bluetooth native to the smartphone, the smart home hub acts as a translator between the two. As such, a smart home hub acts as the heart of a smart home network, tying together various devices and systems in a centralized platform (Margaret Rouse). Many different Smart home hubs are commercially available (to name a few - Google Home Hub, Amazon Echo, Wink Hub 2, Samsung SmartThings, Vivint Smart Home).
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
107
Although these commercial hubs support multiple communication protocols and can control devices by other vendor, they are focusing on making the customer use the smart things ecosystem of devices provided by the vendor. Most of the Smart home (IoT) devices and Smart home hubs connect to (and are controlled trough) some kind of cloud service, this brings up the serious concern about privacy and not being able to locally control the devices in case of cloud or internet connection failure.
mqtt:mosquitto.user=
Here the open source hubs, which give you an independent local control over the connected Smart home devices, come into play (the top being openHab, Home Assistant and Domoticz).
Presence Item - which will report if the smartphone of the homeowner can be located on the home Wi-Fi network (by MAC address). This does not require a separate sensor, the check for presence is done using the openHAB server Wi-Fi connection.
We’ve based our research on openHab (short for Open Home Automation Bus), as one of the best-known home automation tools among open source enthusiasts, with a large user community and quite a number of supported devices and integrations.
Mailbox sensor item – this item will check the state of the mailbox sensor (programming of the mailbox sensor will be described below) and report if there is mail in the box.
OpenHAB runs on most popular platforms such as Linux, Windows and MacOS and on almost any hardware ranging from Raspberry Pis to desktop computers and server PCs. openHAB is very flexible and can support many DIY devices. openHAB 2 is developed in Java, Jetty is included as an HTTP server. Through the openHAB REST API most aspects of the openHAB system can be readily accessed by other programs (https://www.openhab.org/docs/). With its pluggable architecture, openHAB supports more than 200 different technologies and systems and thousands of devices! (https://www.openhab.org/addons/). For our research experiment we’ve connected two different IoT devices to the openHAB Smart home hub: an ESP8266 (NodeMCU) based mailbox sensor (alerting when new mail is placed in your mailbox by the postman) and a Sonoff Dual IoT switch controlling imaginary ceiling and reading lights in the living room of “Archie’s home” (https://sonoff.itead.cc/en/products/sonoff/sonoff-dual). There are few ways to configure openHAB, based on the level of knowledge of the user, the one which give us full control is via configuration files. There are three main configuration files: openhab.cfg,
and
We will use Mosquito MQTT broker (server) to interact with the IoT devices connected to the openHab server, using the MQTT M2M (Machine to Machine) protocol. In order add the MQTT functionality we have to activate the openHab MQTT binding in the Bindings manager. openhab.cfg is the overall config file located in /etc/openhab/configurations/. In this file global config is set such as security authentication and MQTT server setting. E.g. to connect to our MQTT server we add the following: mqtt:mosquitto.url=tcp://
mqtt:mosquitto.pwd=
Sonoff Dual Item which will allow control and report the state of the two relays of this IoT device, which will control the lights in the living room (https://sonoff.itead.cc/en/products/sonoff/sonoff-dual). The Sonoff Dual is flashed with Tasmota alternative open source firmware (https://github.com/arendst/Sonoff-Tasmota/wiki). The user will be able to manually switch on the lights as usual, on top of that user will be able to control and view the status of this 3 Items using the openHAB WEB based user interface or the dedicated openHAB apps for Android and IOS smart phones and tablets. In the same time Archie robot will be able to get the information from the sensors and control the Sonoff Item relays trough the MQTT M2M protocol. ➢
Items file:
The items file defines all the ‘things’ you might want to view or control in openHAB. Items can be bound to ‘bindings’ which are optional packages that used to extend functionality of openHAB. Our use case makes use of the MQTT and HTTP bindings. The items file is located in /etc/openhab/configurations/items. Inbound configurations allow us to receive MQTT messages into an openHAB item. Every item is allowed to have multiple inbound (or outbound) configurations. The structure of the inbound MQTT configuration string is: Item myItem {mqtt="
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
108
Item itemName { mqtt="
Switch item= Light_GF_Living_Ceiling Switch item= Light_GF_Living_Reading } Frame label="Is there mail in the mailbox?"
Excerpt of our archieshome.items file defining the 3 items described above looks like this:
{ Text item=MailboxSensor
……… Group gGF ["GroundFloor"]
"Ground Floor"
}
Frame label="Is the owner at home?"
Group GF_Living "Ground Floor Living
Room"
Group Garden
"Garden"
["Garden"]
Group Presence
"Presence"
["Presence"]
{ Text item=Presence_OwnersSmartPhone } }
Group ROS (All) On the ROS side Archie robot’s computer: String ROS_Status "ROS [%s]" /* Lights */ Switch Light_GF_Living_Ceiling "Ceiling light" (gGF, GF_Living, Lights ,ROS) ["Light"] { mqtt=">[broker:cmnd/sonoff_01/POWER1:command:*:defa ult], <[broker:stat/sonoff_01/POWER1:state:default]" } Switch Light_GF_Living_Reading "Reading light" (gGF, GF_Living, Lights ,ROS) ["Light"] { mqtt=">[broker:cmnd/sonoff_01/POWER2:command:*:defa ult], <[broker:stat/sonoff_01/POWER2:state:default]" } /* Presence */ Switch Presence_OwnersSmartPhone "Owners SmartPhone presence detected" (ROS)
Sitemap
Sitemaps are used to create elements of a user interface for making openHAB items accessible to the front-end interface. A simplified sitemap file for Archie’s Smart Home looks like this: sitemap ArchiesHome label=" Archie’s home" { Frame label="Living Room Lights" {
The power of openHab is that it can connect not only devices using the Internet (Wi-Fi or wired), but also devices using different radio frequencies and protocols like Z-Wave, Zigbee, Bluetooth and so on. Hopefully, there is a package available which allows ROS to communicate with the openHab server via Internet/MQTT and in that way to access all of the devices connected to the openHab server, regardless of the protocol and the type of physical connection used between the device and openHab. This allows a ROS robot to connect to a vast variety of IoT devices such as motion detectors, Z-Wave, Zigbee, Bluetooth devices, lighting, door locks, etc. The iot_bridge package provides a bi-directional bridge between ROS and the OpenHAB Home Automation system (M Herbert). After installing the iot_bridge package, we should update the host address and port to match the openHAB server in iot_bridge/config/items.yaml Then we need to make some changes in the openHAB configuration files (this is already done in the files mentioned above). We have to create a ROS group: Group ROS (All) and add the ROS group to each item, that should send status updates to ROS. We can send commands from ROS to any openHAB item regardless of what group it is in. We also need to add ROS_Status (String ROS_Status "ROS [%s]") to the openHAB's item file. Now we can launch the IoT Bridge node: roslaunch iot_bridge iot.launch When there is an item (belonging to the ROS group) state change in openHab, we receive this on /iot_updates topic. We can view this in the terminal by rostopic echo /iot_updates
Stefan Chivarov et al. / IFAC PapersOnLine 52-25 (2019) 104–109
For example, a when presence of the owner’s smartphone (as defined in the openHab items config file above) is detected/triggered in OpenHAB. The OpenHAB bridge will publish the following to the iot_updates topic in ROS: message type: diagnostic_msgs/KeyValue, key: Presence_OwnersSmartPhone, value: 1 (string). Our robot can control devices connected to openHab by publishing to /iot_command topic, in Pythonit looks like: pub = rospy.Publisher('iot_command', queue_size=10)
KeyValue,
pub.publish(item_name, item_value) So, in order for our Robot to switch the celling lamp on we need to publish: pub.publish(“Light_GF_Living_Ceiling”, “ON”) And to turn off we need to publish: pub.publish(“Light_GF_Living_Ceiling”, “OFF”)
5. CONCLUSIONS We’ve successfully implemented and tested 3 different approaches for connecting Archie robot to IoT devices. This gives the robot the ability to interact with the environment by receiving data and (potentially) controlling the thousands of IoT devices available on the market. An emerging trend in robotics is integrating robots with the Internet as a form of Internet-of-Things (IoT). This allows robots to be controlled and monitored through the Internet and is a must for cloud robotics applications. We already have experience using RoboWebTools (http://robotwebtools.org/), but we still need to test and evaluate other approaches like Roslink (Koubaa, Anis, Alajlan, Maram & Qureshi, Basit, 2017). ROSLink is a new protocol to integrate Robot Operating System (ROS) enabled-robots with the IoT. ACKNOWLEDGEMENT The research presented in this paper was supported partially by the Bulgarian National Science Fund under the contract № KP-06-М27/1–04.12.2019 “Research and Development of Innovative, Intelligent Information and Communication Technologies for Control of Service Robots”. REFERENCES A. Byagowi, P. Kopacek, J. Baltes, “Archie”: A Cost Oriented Humanoid Robot, Proceedings of the 18th IFAC World Congress Milano (Italy), 978-3-902661-937/11/$20.00 © 2011 IFAC
109
D. Chikurtev (2016), Indoor Navigation for Service Mobile Robots Using Robot Operating System (ROS), PROBLEMS OF ENGINEERING CYBERNETICS AND ROBOTICS, Vol 67, Sofia, , p. 61-70, ISSN 02049848. Compelo (2019): https://compelo.com/smart-device-trends2019/ Statista, (2019): https://www.statista.com/outlook/279/109/smarthome/united-states Ivanova et all, Application of Meta-Operating System Ros for Control of Service Mobile Robots, ADP 2012, p. 247254, ISSN 1310-3946. Oasis (2015): "MQTT 3.1.1 specification". OASIS. December 10, 2015. Retrieved April 25, 2017. openHAB, 2019 by the openHAB Community and the openHAB Foundation e.V., https://www.openhab.org/ Chivarov, N., Chikurtev, D., Markov, E., Chivarov, S., Kopacek, Cost Oriented Tele-Controlled Service Robot for Increasing the Quality of Life of Elderly and Disabled - ROBCO 18, P., Volume 51, Issue 30, 2018, Pages 192-197, ISSN: 24058963 ROS, http://wiki.ros.org/ Arduino, https://www.arduinolibraries.info/libraries/rosserialarduino-library Dr Andy Stanford-Clark of IBM, and Arlen Nipper of Arcom (now Eurotech), in 1999. (http://mqtt.org/faq) Contributing Editor for PCMag, John Delaney, https://www.pcmag.com/article/327457/what-is-a-smarthome-hub-and-do-you-need-one Margaret Rouse https://internetofthingsagenda.techtarget.com/definition/s mart-home-hub-home-automation-hub M Herbert http://wiki.ros.org/action/show/iot_bridge?action=show &redirect=openhab_bridge ] Koubaa, Anis & Alajlan, Maram & Qureshi, Basit. (2017). ROSLink: Bridging ROS with the Internet-of-Things for cloud robotics. 10.1007/978-3-319-54927-9_8.