Smart shirt for obstacle avoidance for visually impaired persons

Smart shirt for obstacle avoidance for visually impaired persons

3

S.K. Bahadir 1,3 , V. Koncar 2,3 , F. Kalaoglu 1 1 ITU, Istanbul, Turkey; 2GEMTEX, ENSAIT, Roubaix, France; 3University of Lille North of France, Lille, France

3.1

Introduction

Lack of visual perception due to physiological or neurological factors is known as blindness. According to a world health report, about 314 million people are visually impaired and among them 45 million are blind. This means that approximately 45 million people are dependent on others around them for movement, information processing, and environmental interpretation [1]. Lack of visual perception is paralleled with a loss of independence. In today’s society where social independence is important, visually impaired people like everyone else want to live without depending on others. They want to travel without fear of getting hit or lost. They want to reach information individually as everyone else does. Therefore, those who do not have the benefit of sight require assistive devices to be independent, such as for navigation, for reading signs, etc. Navigation is the art and science of determining one’s position by directing oneself to a desired destination in a safe way. In particular, outdoor and indoor navigation has always been a challenging problem for visually impaired people for their mobility since navigation concerns restrict the visually impaired access to many buildings, often precludes their use of public transit, and makes their integration into local communities difficult. To overcome navigation concerns of visually impaired people, considerable research has been conducted, and thereby several mobility aids like walking sticks, electronic travel robot aids, etc. have been developed [2e4]. However, as far as overcoming navigation aids are concerned, there are several limitations in such devices. For instance, the most widely used mobility aid today is the long cane. The long cane has some limitations as its detection range is limited due to its length, or it has difficulties in detecting overhanging obstacles and storage in public places, etc. Additionally, the weight of cane is another critical problem as in those of other mobility aids. In order to overcome navigational concerns of the visually impaired, there is a significant need for a new assisted guidance system to help blind people more easily recognize the environment.

Smart Textiles and Their Applications. http://dx.doi.org/10.1016/B978-0-08-100574-3.00003-5 Copyright © 2016 Elsevier Ltd. All rights reserved.

34

Smart Textiles and Their Applications

In this chapter the development of an innovative smart shirt system based on an e-textile design approach that can assist visually impaired persons to navigate safely and quickly among obstacles in an indoor environment will be introduced. The smart shirt system is an initial prototype system that combines a garment with sensors, actuators, power supplies, and a data processing unit. The working principle of the system is based on two main functions: sensing the surrounding environment as well as detection of obstacles via sensors and guiding the user by actuators through a feedback process interpreted in a signal processing unit. Within this approach, the design of the smart shirt prototype consists of the review of theories considering both the visually impaired and smart clothing. The framework of the development process of the smart shirt prototype is shown in Fig. 3.1. Throughout the chapter, the design concept of a smart shirt avoidance system, which can be worn as a garment that is flexible, lightweight, and comfortable for the human body, will be described in detail. Firstly, e-textile architecture for obstacle avoidance with the integration of electronic components to textile structures will be given. Then, working performance of electronic components integrated to textile structure will be discussed. The chapter provides a smart shirt concept model including both electronic circuit and the circuit layout transfer to shirt design. The chapter then addresses the development of an algorithm for obstacle avoidance and microcontroller programming with respect to the developed algorithm. Finally, this chapter concludes with discussion of the sensing performance of smart shirt and ends with future challenges.

3.2

E-textile architecture for obstacle avoidance

Designing a smart shirt requires exploring the design parameters not only in software and hardware components but also in everyday wearing requirements [5].

3.2.1 3.2.1.1

Hardware/software basis and wearability requirements Software and hardware component requirements

Before starting to design a smart shirt, many questions related with design variables are considered. In electronic system architecture toward the objective of the research, the following questions are answered: • • • • • • • • •

What types of sensors are required? What types of actuators can be used? How will the data be processed, and which types of signal processing units are required? What will be the decision parameters in the signal processing unit? How many sensors of each type are required? How many actuators of each type are required? What is the optimum placement of sensors and actuators on the human body? What is the most useful placement for the microcontroller? What algorithms are needed to provide accuracy in analyzing data gathered by sensors?

Smart shirt for obstacle avoidance for visually impaired persons

Review on theories in sensor and smart clothing

35

Review on theories in visually impaired

Determination of system components: type of sensors, actuators, microcontrollers, etc.

Integration of sensors to textile structure

Analysis on signal quality of sensor

Integration of actuators to textile structure

Analysis on beam pattern of sensor

Analysis for object detection

Interactive garment design

Electronic circuit design

Development of algorithms for obstacle avoidance

Microcontroller programming

Development of prototype of smart clothing

Testing and controlling of protoype

Interactive garment development

Figure 3.1 Flow chart for the development process of the smart shirt.

Analysis of vibrotactile perception

36

• • •

Smart Textiles and Their Applications

What is the required power consumption for the system? Which types of power supplies are adequate? Which types of conductive fibers are suitable for this system architecture?

3.2.1.2

Wearability performance requirements

When designing a smart clothing system, apart from electronic hardware and software concepts, the wearability of the system is also a critical issue. In wearability concept, some performance requirements such as being lightweight, breathable, comfortable, easy to wear, etc. have to be taken into consideration. In the presented system, all of the wearability performance requirements that are expected are shown in Table 3.1.

3.2.2 3.2.2.1

Integration of electronics to textile structures Adaptation of sensor methodology to textile structure

Sonar is a kind of instrument used for detecting, locating, or determining objects or measuring the distance to an object through reflected sound waves. In order to detect obstacles within an e-textile circuit, the LV-MaxSonar®-EZ3™ (MaxBotix® Inc.) ultrasonic sensor was chosen due to its small dimensions, low power requirements (2.5e5.5 V), and optimal detection angle. The detection capacity of this ultrasonic sensor ranges from 6 to 254 inches and the sensor operates at 42 KHz [6]. During the insertion of conductive yarns to the textile structure, the distances between pins of the sensor are taken into account. To integrate the ultrasonic sensor to textile structures as well as to form electric circuits in the structures, silver-plated nylon yarn with a linear resistance of <50 U/m and with a yarn count of 312/34f
4 dtex is used. To prevent formation of short circuits in the textile-based electric circuit, conductive yarns are hidden in the structure. A fabric structure is considered as a double-woven fabric, and conductive yarns are placed in the middle layer of the structure. The set of warp yarns of the upper layer are linked to the set of weft yarns from the bottom layer, and thus the two layers are held together. A four-harness satin weave is chosen for both layers. Fig. 3.2 shows the diagram representing the drawdown, threading, and lift plan of the double-woven cloth together with the 3D-graphical representation of the woven fabric structure. In order to work with multisensors, sensors should be chained together as shown in Fig. 3.3. Therefore, to keep running and constantly loop sensors, three main things are done: Firstly, a resistor 1 kU is added between the last sensor’s TX back to the RX of the first sensor. Secondly, BW pin is pulled high. Thirdly, to “kick start” sensors, the RX pin on the first sensor is pulled high for at least 20 mS. This lets the microcontroller return its pin to a high impedance state so that the next time around the TX output from the last sensor makes its way to the RX of the first sensor. Thus, all of the sensors in the chain run in sequence. This “ring of sensors” cycle around and around provides constantly maintaining the validity of their analog values. Hence considering this electrical connection, it is necessary to use voltage, ground, AN, TX, RX, and BW pins of the sensor. Therefore, with reference to dimensions of

Wearability performance requirements of the proposed system

Functionality

Maintainability

Manufacturability

Wearability

Durability

Detection of obstacles

Launderable

Ease of fabrication

Comfortable

Strength

Obstacles

Easy drying

Suitable size ranges

• Tear-Tensile Burst

Color fastness

• No skin irritation • No pressure points

Repairable

Breathable (Air permeable)

Abrasion resistance

Guidance alert

Moisture absorption Lightweight

Smart shirt for obstacle avoidance for visually impaired persons

Table 3.1

Dimensional stability Easy to wear and take off Maintain operational mobility Maximize range of motion

37

38

Smart Textiles and Their Applications

(a)

Threading

Lift - plan

2.54 mm

12 11 10 Weft stuffer 9 yarns 8 7 6 5 4 Conductive yarns 3 2nd yarn 2 1

Drawdown

(b)

e ctiv ndu n o C yar

Figure 3.2 (a) The draft for double-woven cloth with weft stuffers and conductive yarn position; (b) 3D representation of the double-woven cloth.

an ultrasonic sensor’s output pins, six electrical connection points (voltage, ground, AN, TX, RX, and BW pins) at specified distances are taken into account. Thus, in each sample, conductive yarn is inserted six times in the weft direction at desired distances to satisfy six electrical connection points. To construct an electrical circuit and to connect sensors with fabric, loops are formed among conductive yarns, and snap fasteners are sewn onto these loops. The final fabric structure corresponding the sensors’ connection points is shown in Fig. 3.4. As seen in the figure, the conductive yarns are in gray color in the middle part of fabric and nonconductive polyester microfibers are in white color.

Smart shirt for obstacle avoidance for visually impaired persons

Wire to an AD input

Wire BW to logic high

Wire to an AD input

Wire BW to logic high

39

Wire to an AD input

Wire BW to logic high

1K Repeat to add as many sensors as desired

Figure 3.3 Chaining of ultrasonic sensors. MaxBotix® Inc., LV-maxsonar®-EZ3™ data sheet and chaining notes, avalaible at http://www.maxbotix.com/documents/LV-MaxSonar-EZ_Datasheet.pdf (accessed June 2015).

3.2.2.2

Adaptation of actuator methodology to textile structure

Toward the objective of the research, vibrotactile feedback is chosen in order to guide visually impaired people. Considering clothing system requirements and ultrasonic sensor power requirements, the type of vibration motor is selected. In order to

N GND

+

+S TX RX AN PW

H

+

M

Ultrasonic sensor

Figure 3.4 Fabric overview: conductive yarns corresponding to sensor ground, Vcc, TX, RX, analog voltage, and BW output points.

40

Smart Textiles and Their Applications

Loops

After weaving

Backside appearance

Frontside appearance

Figure 3.5 Vibration motor integrated to woven fabric to attach over hip bone area of the garment.

give vibrotactile perception, Arduino LilyPad Vibe Board® vibration motor is used as an actuator to ensure vibrotactile sensation due to its small dimensions (2 g, 0.8 mm PCB, 20 mm outer diameter), low power requirement (5.5 V), and easy implementation. To integrate a vibration motor to textile structures as well as to form an electric circuit in the structure, silver-plated nylon yarn with a linear resistance of <50 U/m and with a yarn count of 312/34f
4 dtex is used. To prevent formation of short circuits in the textile-based electric circuit, conductive yarns are again hidden into the structure with a double-woven fabric construction as shown in Fig. 3.2. The fabric structure with vibration motor integrated is shown in Fig. 3.5.

3.3

Smart shirt concept model

During the development of a prototype, firstly, the electronic circuit of the system according to electronic software and hardware requirements is designed and then by considering wearability requirements and comfort of the user, the layout of the system is devised.

3.3.1

Smart shirt electrical circuit

The whole smart shirt electrical circuit is designed mainly considering multiconnection of ultrasonic sensors as discussed in Section 3.3.1.4. Therefore, the schematic circuit concept of vibration motors, ultrasonic sensor, and microcontroller will first be introduced and then the schematic diagram of the whole smart shirt circuit will be given in detail.

Smart shirt for obstacle avoidance for visually impaired persons

3.3.1.1

41

Schematic circuit diagram of vibration motors

Fig. 3.6 shows the schematic circuit diagram of Ardunio LilyPad Vibe Board® [7]. Here the GND pin of the each vibration motor is connected with the GND of the circuit. The Vcc pin of the each vibration motor is connected with the microcontroller digital outputs.

3.3.1.2

Circuit of ultrasonic sensors

Ultrasonic sensor functions using active components consist of an LM234, a diode array, and PIC 16F676 microcontroller, together with a variety of passive components [6]. During the usage of multiple ultrasonic sensors in a single system, there can be interference (cross talk) from the other sensors. In order to avoid the issue of cross talk, a chaining method as suggested in [4] is used. Here, the out pins GND, 5V, TX, RX, AN, and PW of each sensor are connected with the whole circuit according to multiconnection principles of sensors as discussed in Section 3.3.1.4. GND and Vcc (operates on 2.5e5.5 V) pins are connected with a 4.8 V NiMH flat battery. Since ultrasonic sensor’s AN-output works with a scaling factor of Vcc/512 per inch, that means a supply of 5 V yields w9.8 mV/inch (5 V/512 z 9.8 mV) whereas 3 V (3 V/512 z 5.8 mV) yields 5.8 mV/inch; programming is done according to this information.

3.3.1.3

Schematic circuit diagram of microcontroller

The LilyPad Arduino® microcontroller board is used. The LilyPad Arduino has a circle shape, approximately 50 mm in diameter. The thickness of the board itself is 0.8 mm, and with the attached electronics it is approximately 3 mm. The LilyPad Arduino can be powered via USB connection or with an external power supply. In smart shirt circuit design, it is powered with a 4.8 V NiMH flat battery. The board is based on ATmega328 [8]. The schematic diagram of the board is shown in Fig. 3.7.

33

D1

R1

5V

M1 Vibe motor

GND

Figure 3.6 Schematic circuit diagram of Ardunio LilyPad Vibe Board®. http://www.robotshop.com/ PDF/arduino-lilypadvibe-board-schematic.pdf, (accessed on March 2011).

42

Smart Textiles and Their Applications

R3

10 K

U1 29

RST

AVCC VCC VCC

20

AREF

1 2

S1 GND

GND

Y1

PC6

18 4 6

PC0 PC1 PC2 PC3 PC4 PC5 ADC6 ADC7

PD0 RESONATORSMD PB6 PD1 PD2 8 PB7 PD3 PD4 PD5 PD6 PD7

5 3 21

A0 A1 A2 A3 A4 A5

30 31 32 1 2 9 10 11

RXI TXO D2 D3 D4 D5 D6 D7

12 13 14 15 16 17

D8 D9 D10 MOSI/D11 MISO/D12 SCK/D13

ATMEGA168

D2

Status

R1

GND

AGND GND GND

PB0 PB1<0C1A> PB2 PB3 PB4 PB5

23 24 25 26 27 28 19 22

330

0.1 μF

VCC

3 4

DTR

C4

VCC

GND

Figure 3.7 Schematic diagram of LilyPad Main Board microcontroller. http://arduino.cc/en/uploads/Main/LilyPad_schematic_v18.pdf, (accessed on February 2011).

With reference to Figs. 3.4 and 3.8, the pins for analog inputs A0(23), A1(24), A2(25), and A3(26) are connected with the analog output pins of ultrasonic sensors. Vcc and GND pins are connected with the power supply, and the pins for the digital outputs D2(32), D3(1), D4(2), D5(9), D6(10), and D7(11) are connected with the vibration motors’ input pins.

3.3.1.4

Schematic diagram of smart shirt whole circuit

The schematic diagram of the smart shirt whole circuit is shown in Fig. 3.8. The function of this circuitry is to digitize as well as transform analog signals acquired by sensors into vibration signal. It modulates analog signals into different levels of vibrations by identifying correlations between position of obstacle and required turning action (direction and angle) for the user. There are four key connections and elements for this circuitry: (1) one microcontroller, (2) four ultrasonic sensors, (3) eight vibration motors, and (4) two power supplies. Four sensors are used to detect obstacles, and eight vibration motors (each of four on the left and right) are used in order to guide the user by recommending turning direction and angle. As will be mentioned in the developed algorithm in Section 3.3.2.3, commands for required turning action, which are processed through the microcontroller by linguistic variables, namely turning left or right with an angle of small (S), medium (M), large (L), and very large (VL), are provided by eight vibration motors. For that purpose, the microcontroller is used in order to process as well as transform data into commands.

Vcc GND

GND

Vcc GND

D6

D1

D5

D2

D4

D3

Power supply

Microcontroller

A0 A1 A2 A3

Sensor 3

Vcc GND

Sensor 4

Smart shirt for obstacle avoidance for visually impaired persons

3 Layered electric circuit GND

GND Vcc Power supply Vcc GND GND Vcc

Sensor 1

GND Vcc

Sensor 2

Vcc GND Vcc GND

1k

Vibration motors GND Vcc

GND Vcc

Vibration motors

Trigger button

GND A: Analog input pins D: Digital output pins Vcc: Voltage GND: Ground

Outer connections of sensor s circuit Inner connections of sensor s circuit Outer connections of circuit Outer back connections of circuit (GND)

Figure 3.8 Schematic diagram of smart clothing system circuit. 43

44

3.3.2

Smart Textiles and Their Applications

Electrical circuit layout on the shirt

Considering Fig. 3.8, the circuit layout of the prototype is designed as seen in Fig. 3.10. The circuit design over the front and back of garment, and over the arms, are given in Fig. 3.10(a) and (b), respectively.

3.3.2.1

Sensor placement

The position of sensors on the garment are placed under the breast zone of the garment taking account of both women’s and men’s body posture. The distance between four sensors are adjusted to 20 cm depending on experiments reported in [9,10]. Indeed, the position of sensor plays a great role on the detection of obstacles. They should be placed in a region where the garment does not move so much during the walking. Within this concept, there can be two alternative zones: shoulders zone or zone under the breast. If the sensors are placed over the shoulders zone, then obstacles with higher height such as a wardrobe or wall will be detected. On the other hand, if the sensors are placed over the zone under breast, then the obstacles not only with higher height but also lower than that height, such as tables, can also be detected. Toward our aim and considering environmental conditions, since there are more obstacles with lower height, then the zone under the breast on the garment is chosen for position of sensors in order to avoid more obstacle collisions.

3.3.2.2

Actuator placement

According to experiments reported in [11], it is found that the highest level of vibrotactile sensation is perceived over the outer wrist and hip bone area of the evaluators’ body. Therefore, to guide the user, vibration motors are to be placed over the outer wrist and hip bone area of the garment. Three of the vibration motors are to be placed on the wrist of the left arm, whereas the other three are placed on the wrist of the right arm. The left and right hip bone areas are chosen for summer clothing usage. The garment is designed for both summer and winter periods. Therefore, when the arms of the proposed garment are taken out, the system is designed to be able to generate control by the vibration motors placed on the left and right hip bone areas of the garment. To sum up, eight vibration motors are to be placed as follows: three of the six vibration motors are placed on each of the left and right arm over the outer wrist of garment and the other two are on the left and right hip bone areas of the garment. Three vibration motors are used on one arm to give the user information about the location of obstacles as well as about required turning angles. For instance, in the case of a right turn with a small angle, only the first vibration motor on the right arm will act. Similarly, if the required turning action is a right turn with a large angle, then three vibration motors on the right arm will act.

3.3.2.3

Microcontroller and power supply placement

After the decision of the actuator’s and sensor’s placement, the positions of microcontroller and power supplies are planned out. Considering circuit and resistance

Smart shirt for obstacle avoidance for visually impaired persons

45

constraints, microcontroller and power supplies should be placed as close as possible to each other. Moreover, a critical point in microcontroller placement is that it is the network of inputs and outputs. Therefore, it should be placed in a region that is able to gather all analog outputs from sensors and send inputs to actuators without any overlapping. Therefore, the best possible position for microcontroller is in the center of the garment with regard to the whole circuit. Due to microcontroller position and circuit constraints, positions of power supplies are determined close to the microcontroller. Hence, they will be placed around the vertical centerline of the garment.

3.3.3 3.3.3.1

Structures of smart shirt prototype Base structure of smart shirt

In order to obtain the circuit layout design shown in Fig. 3.9 on the garment, production of seamless products is considered. Hence, a MBS Merz® single-jersey circular knitting machine with a cylinder diameter of 13 inches and E28 gage is used to produce a base structure of interactive garment as seen in Fig. 3.10.

Figure 3.9 Electrical circuit layout on the shirt.

46

Smart Textiles and Their Applications

Figure 3.10 Base structure of developed smart shirt.

Considering wearability and durability performance requirements such as comfortable, breathable, moisture absorption, lightweight, strength, etc., polyamide (PA) 66 yarns with a linear density of 78/68
2 dtex are used. Additionally, in order to get a tight fit in the garment, elastomeric yarns composed of PA (22 Denier) including Lycra® (16 Denier) are also used during the production of base structure of the smart shirt. As shown in that figure, conductive silver-plated nylon 66-4ply yarns with a linear density and resistance of 312/34
4 dtex and 50 U/m, are in gray color, whereas polyamide yarns are in white. This structure is also washable. The other electronic parts are removable and washing them is not recommended.

3.3.3.2

Removable structures of smart shirt

The total removable parts for sensor and actuator connections are shown in Fig. 3.11. There are four sensors and eight vibration motors integrated to woven fabrics as described in Section 3.3.1.4. The connection of these removable parts to a main circuit in the base structure of the interactive garment is provided by snap fasteners. The removable fabric for microcontroller connection is produced by sewing. Similarly, snap fasteners provided connections the among main circuit and microcontroller. The fabric used for producing the base structure of interactive garment is also used to produce both microcontroller connection and pockets for batteries (see Fig. 3.12). The conductive yarns are again inserted as well as hidden in the middle part of knitted fabric.

Smart shirt for obstacle avoidance for visually impaired persons

47

Sensors

Vibration motors for the over wrist of garment

Vibration motors for the hip bone area of garment

Figure 3.11 Removable fabrics for sensor and actuator connections to a main circuit. Microcontroller integrated to knitted fabric Pockets for batteries

Figure 3.12 Removable fabrics for microcontroller and pockets for batteries.

3.3.3.3

Smart shirt prototype

Finally, a smart shirt with its removable parts is shown in Fig. 3.13. Sensors are positioned in front of the garment under the breast zone. Vibration motors are positioned over the wrist and hip bone area of the garment. Microcontroller and batteries are positioned along the vertical centerline of the garment. Parts for ultrasonic sensors, vibration motors, and batteries are designed to be attached from the inner side of the garment, and, thus, they are not visible when they are attached. However, only the microcontroller is visible on the garment when attached. In this way, the user can open and close the system easily via a button on the microcontroller.

48

Smart Textiles and Their Applications

Battery

Microcontroller Sensor 3

Sensor 4

Battery Vibration motors

Vibration motors Sensor 1

Vibration motor

Sensor 2 Vibration motor

Figure 3.13 Smart shirt with its removable parts.

Fig. 3.14 shows the final smart clothing prototype worn by a mannequin. The weight of the final prototype without batteries is about 250 g, whereas the weight including two batteries is 458 g. This smart garment enabling detection and avoidance of obstacles is easy to handle, light enough to wear and carry, and washable when removable parts are detached from the main structure.

3.4

An algorithm for obstacle avoidance and data transfer in a smart shirt

For a visually impaired person to navigate through an environment cluttered with obstacles, a neuro-fuzzy logicebased obstacle avoidance control algorithm is developed for a smart shirt system.

3.4.1

Obstacle avoidance strategy

In the presence of obstacles, guiding a person becomes more and more difficult. To guide a user, first, three important things should be determined [12]: • • •

Target Obstacles Person’s position

Smart shirt for obstacle avoidance for visually impaired persons

49

Figure 3.14 Final smart shirt prototype for visually impaired people.

Then, a guidance strategy can be implemented as seen in Fig. 3.15. The notation of the observer used in this diagram is equivalent to the notation of the control system. Indeed, the corresponding motion state of a walking person can be summarized as in Table 3.2. The position of a walking person in an environment can be estimated by controlling his/her angular wb and linear vb velocities. Suppose that the workspace W is cluttered up with N stationary obstacles On, n ˛{1,.,N} and there is a target point as seen in Fig. 3.16.

Where is target? (xt ,yt ,θt)

Where is user? (xb ,yb ,θb)

Observer

Guidance (νb, wb)

Figure 3.15 Block diagram of guidance strategy.

Where are obstacles? (xOn ,yOn ,θ On )

50

Smart Textiles and Their Applications

Table 3.2

Motion state of walking person

wb

wb [ 0

wb > 0

wb < 0

Motion state

Go straight

Turn right

Turn left

W

y (x On,y On) (x O2,y O2)

Obstacle n

Obstacle 2

α2 α1

y b(i+1)

d t(

2(

vb

(xt, yt)

) i+1

i+ 1)

wb

d

Pb

Target

Obstacle 1

d 1(i+1) (x O1,y O1)

Pb

y b(i)

θb

xb(i)

x xb(i+1)

Figure 3.16 Navigation of walking person under multiobstacles environment.

In the case of n obstacles and one target point, the distance to the target point is computed as: dtðiþ1Þ ¼

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  2  2 xt  xbðiþ1Þ þ yt  ybðiþ1Þ

[3.1]

and the desired direction angle (f) is modified according to the person as follows: fiþ1 ¼ tan1

yt  ybðiþ1Þ xt  xbðiþ1Þ

! [3.2]

By considering Eqs. [1] and [2], it can be concluded that there is a relation between recommended direction angle (f) and velocity of person. Therefore, while guiding the person, the variables’ distance to target (dt) and direction angle (f) have to be controlled at each decision point regarding obstacles. For example, if there are two obstacles in front of the walking person with a distance of

Smart shirt for obstacle avoidance for visually impaired persons

51

d1(iþ1) and d2(iþ1) as shown in Fig. 3.17, then the distances to obstacles can be calculated as: d1ðiþ1Þ ¼

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  2  2 xO1  xbðiþ1Þ þ yO1  ybðiþ1Þ

[3.3]

d2ðiþ1Þ ¼

qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  2  2 xO2  xbðiþ1Þ þ yO2  ybðiþ1Þ

[3.4]

and the angle (a1) between obstacle 1 and target point, and similarly the angle (a2) between obstacle 2 and target point can be computed as:  ! !  yO1  ybðiþ1Þ   1 yt  ybðiþ1Þ  1 [3.5] a1ðiþ1Þ ¼ tan  tan   xt  xbðiþ1Þ xO1  xbðiþ1Þ  a2ðiþ1Þ

 ! !  yO2  ybðiþ1Þ   1 yt  ybðiþ1Þ  1 ¼ tan  tan   xt  xbðiþ1Þ xO2  xbðiþ1Þ 

[3.6]

Assume that ak represents the system’s detection range in terms of angle, and let akmax denote the maximum detection angledin other words, the border of detection rangedthen to avoid obstacles, the following rules should be taken into account [13]: 1. If (a1(i þ1) > akmax) ^ (a2(i þ1) > akmax), which represents that there is no obstacle in the detection range, then there will be no avoidance strategy. This means go straight (zero) or wb ¼ 0 (Table 3.1). 2. If ((a1(i þ1)  akmax) ^ (a2(i þ1) > akmax)) n ((a1(iþ1) > akmax) ^ (a2(iþ1)  akmax)), which represents only either one is in the detection range, then there is only one obstacle to be avoided. Therefore, the question is simplified as how to avoid just one obstacle; turn right or left, or in other words wb > 0 or wb < 0 (Table 3.1). 3. If ((a1(i þ1)  akmax) ^ (a2(iþ1)  akmax)), which represents both are in the detection range then, there are two obstacles to be avoided. Thus, select the obstacle that should be avoided by considering the minimum distance rule (min (d1(iþ1), d2(iþ1))) as follows [13]: a. If d1(i þ1) < d2(i þ1), then firstly avoid first obstacle (O1), secondly avoid second obstacle (O2). b. If d1(iþ1) > d2(i þ1), then first avoid second obstacle (O2), secondly avoid first obstacle (O1). c. If d1(iþ1) ¼ d2(iþ1), then compare a1(i þ1) and a2(i þ1) i. If a1(i þ1) < a2(i þ1), then select obstacle 1 (O1) as target obstacle to be avoided. ii. If a1(i þ1) > a2(i þ1), then select obstacle (O2) as target obstacle to be avoided. iii. If a1(i þ1) ¼ a2(i þ1), then select one of them randomly: obstacle 1 or obstacle 2 as target obstacle to be avoided.

According to the following obstacle avoidance strategy as seen in Fig. 3.17, a control system is considered and developed in order to guide the visually impaired person. During the design of a control system, fuzzy and neural network approaches are used.

52

Smart Textiles and Their Applications

Define start point

Define target point

Check the detection range are there any obstacles No

Yes Check the obstacles if they are on the way of target or not

No

Yes Turned on obstacle avoidance controller

No

Arriving to a target point Yes END

Figure 3.17 Basic flow diagram of obstacle avoidance strategy.

Direct arriving to a target point

Smart shirt for obstacle avoidance for visually impaired persons

Left sensors

53

Right sensors

4

3

1

2

Up sensors

Down sensors

Figure 3.18 Sensors’ position on the garment.

3.4.2

Neuro-fuzzy control algorithm for obstacle avoidance

In the smart shirt system, four sensors integrated to the front side of the garment perceive surroundings. While the wearer navigates in an unknown environment, ultrasonic sensors detect the presence of obstacles as well as measure the distance to obstacles. During the design process, four sensors are divided into two groups (Fig. 3.18). In order to differentiate heights of obstacles, two ultrasonic sensors are to be placed up position on the garment while the other two are placed down position. As well, in order to differentiate positions of obstacles, whether they are on the left side or right side due to the wearer’s position, two sensors are to be placed on left part of the garment and the other two on the right part. Thus, by considering two groups of four-sensor situations, probable cases for detection of obstacles are determined and obstacles’ potential positions with regard to person position are examined. Fig. 3.19 shows some cases for obstacles’ positions.

Figure 3.19 Different cases between obstacles and user.

54

Smart Textiles and Their Applications There is an object

There is no object

Filtered data (x < 2.5)

Eliminated data (x > 2.5)

Object at very far If at least one of sensor values is less than 2 m (x < 2m), then there is an object to be extremely avoided.

0

If all sensor values are between 2 < x < 2.5 m, then it is again interpreted as there is no object to be avoided quickly when user at 0 point; that means there is an object at very far

2m

2.5 m

Figure 3.20 Data elimination process.

3.4.2.1

Data filtration and preprocessing

Data gathered by sensors are either eliminated or transmitted to a controller. It is known that sensor detection range is up to 6.45 m [6]. In order to give a controller decision, a predefined value is first determined. A value of 2.5 m (predefined value) is considered for elimination of data that means the sensor detects the distance to an object larger than 2.5 m, or in other words, if the object locates 2.5 m or further away from the user’s location, then the data is eliminated and considered as there is no object in the way of the user. Thus, in the first algorithm all averaged input data larger than 2.5 m is being filtered and directly sent to go straight position (0), which is interpreted as no turning action. Secondly, for the averaged data smaller than 2.5 m, it is interpreted as there is an object/s on the way of user, and according to decision of position of the object, it is sent to an avoidance strategy to be processed. Fig. 3.20 explains the data elimination process. In fact, when all the sensor values are between 2 and 2.5 m, they are interpreted as there is an object very far and it is not necessary to avoid this obstacle quickly at this time interval. Thus, this situation is again assigned to go straight position (0) as if there is no obstacle that should be avoided. However, sometimes one, two, or three of the sensors may measure between 2 and 2.5 m because of detection of an obstacle far away or noisy data, while the other/s detects an obstacle within 2 m. In this case, if at least one of the sensor values is less than 2 m, it is interpreted as there is an obstacle that should be extremely avoided. In order to decide an object’s position, experiments are conducted with various objects’ position in x- and y-axis in a real-time environment. For each sensor, 9900 data points are obtained. In this concept, possible scenarios for detection of objects by using four sensors were formed as follows: if Xd1i > 2 & Xd2i > 2 & Xd3i > 2 & Xd4i > 2 there is no obstacle;

Smart shirt for obstacle avoidance for visually impaired persons

55

elseif Xd1i < 2 & Xd4i < 2 & (Xd2i > 2 j Xd3i > 2) obstacle at the left; elseif (Xd1i > 2 j Xd4i > 2) & Xd2i < 2 & Xd3i < 2 obstacle at the right; elseif (Xd1i < 2 j Xd4i < 2) & Xd2i > 2 & Xd3i > 2 obstacle at the left; elseif Xd1i > 2 & Xd4i > 2 & (Xd2i < 2 j Xd3i < 2) obstacle at the right; elseif Xd1i < 2 & Xd2i < 2 & Xd3i > 2&Xd4i > 2 if Xd1i > Xd2i obstacle at the right; else obstacle at the left; end elseif Xd1i > 2 & Xd2i > 2 & Xd3i < 2&Xd4i < 2 if Xd4i > Xd3i obstacle at the right; else obstacle at the left; end elseif Xd1i < 2 & Xd2i > 2 & Xd3i > 2& Xd4i < 2 obstacle at the left; elseif Xd1i > 2 & Xd2i < 2 & Xd3i < 2& Xd4i > 2 obstacle at the right; elseif Xd1i < 2 & Xd2i < 2 & Xd3i < 2& Xd4i < 2 obstacle at the front; else there is no obstacle; end

After determining object’s position with Algorithm 1 (Algorithm 1 is tested with real-time measurements, and the success of the algorithm is found 97.98% by using MATLAB), data is sent to one of the avoidance strategies: left, front, and right obstacle avoidance.

3.4.2.2

Neuro-fuzzy control algorithm

This time, neuro-fuzzy algorithm starts processing data. Neuro-fuzzy algorithm is composed of the following: 1. Input layer 2. Hidden layer (rule layer and consequence layer) 3. Output layer

In the input layer and hidden layer of algorithm, fuzzy inference system (FIS) takes place. To set up the fuzzy inference system, MATLAB® Fuzzy Logic Toolbox is used. As shown in Fig. 3.21, three types of fuzzy inference systems are developed, namely (1) left, (2) front, and (3) right obstacle avoidance fuzzy inference systems.

56

Input layer Sensor input nodes

Hidden layer

Output layer Left obstacle avoidance

M M

1 2 3

Xd1i 30

Xd3i

1 2 3

M M

Turn right 16

M M

Right obstacle avoidance

Xd4i 30

Fuzzy functions

Fuzzy rules output nodes

Figure 3.21 Proposed neuro-fuzzy control system for the smart clothing.

Weighted vectors

TRoutput

Smart Textiles and Their Applications

1 2 3

TLoutput

M M

Data filtration & pre-processing

Xd2i

Turn left Front obstacle avoidance

Smart shirt for obstacle avoidance for visually impaired persons

57

Fig. 3.22 shows the developed fuzzy inference system for left obstacle avoidance. Similarly, front and right obstacle avoidance fuzzy inference systems are also developed. The fuzzification procedure maps the crisp input values to the linguistic fuzzy terms with membership values between 0 and 1. In this layer, the inputs are the filtered data, and each of these inputs is classified to fuzzy set membership functions. The inputs of fuzzy inference system are “averaged measured distances to an obstacle” information from sensor 1, sensor 2, sensor 3, and sensor 4, which are described by three linguistic variables: near, far, and very far. The domain of functions is from 0 (minimum) to 2.5 m (maximum) for each sensor. The two linguistic variables near and far are described by triangular membership functions, whereas very far is described by trapezoidal membership function as shown in Fig. 3.23. Indeed, the input values of 2 < Xdi  2.5 are regarded as there are no detected obstacles neither at far nor near, thus they were interpreted as “very far” (see developed fuzzy rules).

Figure 3.22 Fuzzy inference system (FIS) for left obstacle avoidance.

58

Smart Textiles and Their Applications

Degree of membership

μ ( x) 1

0.2

0.4

Very far

Far

Near

0.6

0.8

1

1.2

1.4

1.6

1.8

2

2.2

2.4 2.5

Input variables Xd1i, Xd2i, Xd3i, Xd4i

Figure 3.23 The membership functions for input variables.

For instance, the measured distances by sensors to an obstacle located at (10, 60) cm are shown in Fig. 3.24. As seen in the figure, sensor 1 and sensor 2 detect the obstacle around 60 cm while sensor 3 and sensor 4 do not detect the obstacle. When the averaged values of these data are taken, first they are filtered, and then each of these inputs is classified to fuzzy set membership functions as follows: 1. Since the “averaged measured distances to an obstacle” information from sensor 1 and sensor 2 is about 60 cm, they are classified as “near.” 2. Since the “averaged measured distances to an obstacle” information from sensor 3 and sensor 4 is about 2.4 m, they are classified as “very far.”

The outputs of the fuzzy inference system are also described by fuzzy linguistic variables, which are turn left small (S), medium (M), large (L), and very large (VL),

Measured distance (m)

2.5

Data taken by sensors when there is an obstacle at (–10, 60) cm Sensor 1 Sensor 2 Sensor 3 Sensor 4

2

1.5

1

0.5 0

20

40

60 Time (s)

80

100

Figure 3.24 Measurement results when the obstacle is at (10, 60) cm.

120

Smart shirt for obstacle avoidance for visually impaired persons

VL

–90

–75

–60

59

Degree of membership μ (x) S Zero 1

L

M

–45

–30

–15 Output variables turning angle (φ )-turn left (TL)

90

Figure 3.25 The membership functions for output variables “turn left.”

and similarly turn right small (S), medium (M), large (L), and very large (VL) as shown in Figs. 3.25 and 3.26, respectively. The domain of functions is [e90, 90]. All the linguistic variables are denoted by triangular membership functions (MF). In the hidden layer in order to control the user’s motion in an environment as well as establish the relation between sensor values and turning angle, 77 rules are designed considering algorithm 1. Hence, the rules are defined by human knowledge by using observed data taken by real-time measurements and training of data is done off-line. According to an obstacle’s position, the turning angle of the user is decided. Table 3.3 shows the recommended turning angle for a user to avoid an obstacle concerning its position. In the table “R” and “L” indicate the turn right and turn left, respectively. Additionally, as mentioned earlier, {Z, S, M, L, VL} values denote the turning angle in terms of linguistic variables. Consequently, an algorithm for a smart clothing system is described by fuzzy neural approaches. The system inputs are evaluated in terms of fuzzy relations, and then the outputs that are recommended turning angles in order to avoid obstacles are deduced by using neural network architecture. As a result, the outputs of the described neuro-fuzzy controller will be processed by the microcontroller, and then they will be transmitted to vibration motors as signals defined by intervals of “S,” “M,” “L,” and “VL” for turning action in order to guide the user.

–90

–75

–60

–45

–30

Degree of membership μ (x) S Zero 1

M

L

–15

30

45

15

Output variables turning angle (φ )-turn right (TR)

Figure 3.26 The membership functions for output variables “turn right.”

VL

60

75

90

60

Smart Textiles and Their Applications

The relation between turning angle and detected object position

Table 3.3

Object at y-axis (cm)

Object at x-axis (cm)

3.4.3

Turning angle (ϕ )

–∞

–40 –30

–20

–10 0

10

20

30

40

–∞

0–100

Z

RS RS

RM

RL

RVL/ LVL

LL

LM

LS

LS

Z

100–200 Z

RS RS

RS

RM

RL/LL

LM

LS

LS

LS

Z

∞

Z

Z

Z

Z

Z

Z

Z

Z

Z

Z

Z

Data transfer and microcontroller programming

In the smart shirt system, the Lilypad Arduino® microcontroller board is used. The board is based on ATmega328 (20 MHz, 6-channel 10-bit ADC, 14-channel programmable input/output lines) for which the instruction set and technical specifications of the chip are given in the ATMEL® Technical Data Manual [14]. In order to program the microcontroller, Arduino developed a C-based software. Thus, in this study Arduino’s own software® is preferred against assembly language due to easy programming. The design of the program is aimed at analyzing signals acquired by the ultrasonic sensors and transforming them into different vibration intervals in the case of obstacles as mentioned in the earlier section for guiding users with recommended turning actions. Fig. 3.27 shows the flow diagram of the microcontroller’s main program used in the smart shirt. The main program works as follows: First, the program goes through an initialization phase where all variables are set, all input/output ports are initialized and the external devices are enabled. Next, the processor waits for calibration during 5 s. In the calibration phase, all sensor outputs assign to the same range. Thus, they are capable of measuring the same interval. Then, data acquisition and the sampling loop start. Signals acquired by ultrasonic sensors are processed within a sampling period. In that period, data processing is done in order to understand if there is an obstacle in the way of user or not. According to data assessment, decision output is given as mentioned in an earlier section such that if the obstacle is detected at the right, then actuation signals are transformed to left vibration motors in order to guide the person by turning left or vice versa. If data assessment results in there is no obstacle in the way of the user, then there is no decision output as turn left or right, which means no actuation signals are transformed to vibration motors (zero ¼ go straight) and in this way the next data acquisition and sampling loop takes place within a next time interval.

Smart shirt for obstacle avoidance for visually impaired persons

61

Start

Set variables, I/Os

Calibration

Data acquisition & sampling

Ultrasonic sensors Decision output

CPU

Vibration motors

User

Obstacle

Figure 3.27 Flow diagram of microcontroller’s main program.

In microcontroller programming, the critical point is the period for sampling of data acquisition and output order. For the research, in order to determine sampling period as well as output order to guide the user at a correct time interval before crashing into an obstacle, first walking speed of visually impaired people is considered. Some studies reported that walking speed of a normal pedestrian is between 1.22 m/s (younger pedestrians) and 0.91 m/s (older pedestrians) [15,16]. Considering this known fact and our observations, walking speed of visually impaired persons is assumed as 0.6 m/s. Furthermore, during walking the distance to be checked for obstacles is defined as 2.5 m in an earlier section. Hence, a maximum timing diagram for microcontroller programming including sampling loop and decision output period in a safety margin is shown in Fig. 3.28.

62

Smart Textiles and Their Applications

2.5 m 0.6 m

0.6 m

0.6 m

0 Data acquisition/ 1 s Output order 2 s Turning action vibration sensation sampling

0.7 m

3s

Safety margin

1st sampling starts +5 V 2nd sampling starts

0V Time starts

Obstacle detection 1st sampling 2nd sampling (10 ms) (10 ms) Decision output (100 ms)

Figure 3.28 Timing diagram for microcontroller.

At the first second, data acquisition as well as sampling are performed. Minimum sampling time of data is calculated and determined as approximately 10 ms (1 sampling loop 10 ms). Once data is assessed after the sampling, one element of the decision matrix is updated. Hence, for the new condition, the decision output can be given after 10 sampling processes at least approximately in 100 ms. As soon as the decision output is given, actuation signals are transferred to vibration motors, and, thus, users can sense vibration motions in 1 s time. Sensation of vibration motions can start before the second time interval (tstart-vib 1 s, tduration-vib 1 s) due to decision output. (The intervals given in Fig. 3.29 show the maximum timing including safety margins in

Figure 3.29 Measurements for detection capability of the developed system and layout of environment including obstacles.

Smart shirt for obstacle avoidance for visually impaired persons

63

order to be able to guide users before crashing into an obstacle). After the sensations, one more second is given to users to compensate for forward motion during turning action. In this manner, a user’s avoidance of an obstacle within a 2e2.5 m range is guaranteed. Thus, considering the timing diagram and previously mentioned information, microcontroller programming is done and embedded to the smart shirt system.

3.5 3.5.1

Sensing performance of the smart shirt Sensing capability of obstacles and detection range

For experimental purposes, the smart shirt placed on the mannequin is tested for its detection range as shown in Fig. 3.29. During measurements, obstacles are placed in front of the mannequin in different positions in order to find the maximum detection range. For instance, white drawings on the ground seen in Fig. 3.29 are obtained during operation in different time intervals. Moreover, before conducting experiments since lengths and widths of obstacles are critical issues for obstacle detection due to sensors’ locations on the garment, the following assumptions are made: (1) the widths of obstacles used in experiments were larger than 30 cm and (2) the heights of obstacles used in experiments were higher than 90 cm. According to measurement results, during the first 2 h the detection capability of the system is up to 2.5 m in y-axis as seen in Fig. 3.30. However, as the operation time increases, the detection range decreases. After 4 and 6 h working time, the detection range decreases to 2.2 and 1.8 m, respectively. This result can be attributed to a decrease in battery voltage. As the time passes, batteries run out. Thus, the feeding voltage going to sensors decreases. Since the analog voltage output of our ultrasonic sensor works with a scaling factor of Vcc (feeding voltage)/512 per inch, the measured distance values acquired by sensors decrease due to a decrease in Vcc, and the detection range as time passes also decreases. Moreover, as shown in Fig. 3.30, it is observed that detection ranges of left and right sensors are a little bit different. The areas detected by left and right sensors are not symmetrical. This may be related to different sensitivities of sensors. Overall, experimental results show that the developed system is able to identify obstacles’ positions without any failure within the detection range.

3.5.2

Avoidance strategy with vibration motions

In order to test the system’s reaction in case of obstacles during movement, the experimental set-up seen in Fig. 3.31 is used. For instance, an intelligent garment worn by the mannequin is moved with a speed of 0.6 m/s toward an obstacle (see Fig. 3.32). When the mannequin is between 3 and 2.5 m away from obstacle, there is no actuation on the vibration motors that means go straight position (Z). However, in the first case (Fig. 3.33(a)), when the mannequin reaches a distance of 2.5 m away from an obstacle located on the left side, only the first vibration motor on the right arm acts in order to satisfy warning action. The other vibration motors do not show any vibrations until reaching a distance of 1.25 m away from the obstacle. On the

64

Smart Textiles and Their Applications

Figure 3.30 Detection capability of the developed smart clothing.

Obtained maximum detection range y-axis (cm) 260 240 220 200 180 160 In 2 h In 4 h In 6 h

140 120 100 80 60 40 20 0 –150

–100

–50

x-axis (cm) 0

50

100

150

other hand, during the movement along 1.25 m and 15 cm, in addition to the first vibration motor, the second vibration motor on the right arm also shows vibration in order to present the proximity of the obstacle (see Table 3.3). Signals over the conductive yarns measured by the oscilloscope at the point of connection with first, second, and third vibration motors on the left and right arms during the movement toward an obstacle located at the left side are shown in Fig. 3.33. In the second case (Fig. 3.34(b)), when the obstacle is located in the front of the mannequin, at the beginning again there are no vibrations (between 3 and 2.5 m) neither at left nor right. But when the mannequin reaches 2.5 m and during the movement till 1.25 m, it is observed that first and second vibration motors on both different sides (both at left and right) are acting simultaneously. Later on, after passing the distance of 1.25 m, three vibration motors on both sides start to act concurrently. Fig. 3.34 clearly shows the differentiation of signals taken over the vibration motors during the movement of mannequin toward the front obstacle. Based on experiment results, it can be said that the developed smart shirt system is able to detect an obstacle’s position accurately and presents encouraging results in order to warn the user to avoid an obstacle. It is able to identify an obstacle’s position without any failure within the detection range. That means the system is capable of

Smart shirt for obstacle avoidance for visually impaired persons

65

Figure 3.31 Experiments during moving toward an obstacle.

detecting obstacles accurately such that when obstacles are at the right, the system gives an output turn left or vice versa. When obstacles are at the front, the system gives an output like turn right and turn left at the same time. Therefore, a user can choose his/ her way randomly by turning right or left in order to avoid an obstacle in the front. Thus, it can be concluded that the success and the robustness of the developed smart clothing system is really promising.

3.5.3

Heating behavior

Thermal analysis is carried out in order to find out whether the garment heats up above the level when the comfort of the user can be affected or it may provoke injuries. A thermal camera (Testo 880®, Testo Inc.) is used to take infrared images of the structure. Testo 880® Thermal Camera has a thermal resolution of <0.1 C at 30 C Obstacle 3m x

20 cm

x

Mannequin

υ = 0.6 m/s

1st case (object at the left)

3m x x

Mannequin

υ = 0.6 m/s

Obstacle

2nd case (object in the front)

Figure 3.32 Smart shirt worn by the mannequin moving toward an object located on the left (first case) and in the front (second case).

66

Smart Textiles and Their Applications Obstacle at left (measurement between 2.5 m and 1.25 m) 6 5

6

Right-1st Right-2nd Right-3rd Left-1st

5

Left-2nd

4

Left-2nd Left-3rd

1

3

Right side

2

Voltage (V)

3

2 1

1000

1500

2000

2500

Left side

500

1 2 0

500

1000

Time

1500

2000

2500

Left side

0

0 1 0

Right-1st Right-2nd Right-3rd Left-1st

Left-3rd

Right side

Voltage (V)

4

Obstacle at left (measurement between 1.25 m and 15 cm)

Time

Figure 3.33 Measured signals over the first, second, and third vibration motors on the left and right arms during the movement toward an obstacle located at the left.

and is set to record temperatures every 5 s. A multichannel DC power source (Keithley 2400 SourceMeter®, Keithley Instruments Inc.) is used as the power supply. Experiments are done in standard laboratory conditions (20 C, %65RH). The base structure of the interactive garment was placed on a plastic stand about 50 cm away from the thermal camera. Then, conductive parts of the garment were clamped with the probe of the DC power supply. It is observed that the temperature of the conductive yarns increases rapidly with the voltage increase (see Fig. 3.35). With five voltages 1, 4, 8, 12, and 16 V, the obtained average temperature values are 18.3, 20.8, 30.5, 46, and 64.4 C, respectively. Based on these results, the recommended voltage range to be applied on a smart shirt should not exceed 6e7 V in case of utilization of silver-plated conductive yarns (<50 U/m) in order to guarantee comfort and safety.

2 0

Left side

2 4 6

0

500

1000 1500 Time

2000

2500

4 Right-1st Right-2nd Right-3rd Left-1st Left-2nd Left-3rd

2 0 2

Left side

Voltage (V)

4

6 Right side

Right-1st Right-2nd Right-3rd Left-1st Left-2nd Left-3rd

Voltage (V)

6

Obstacle in the front (measurement between 1.25 m and 15 cm)

4 6

Right side

Obstacle in the front (measurement between 2.5 m and 1.25 m)

0

500

1000

1500 Time

2000

2500

Figure 3.34 Measured signals over the first, second, and third vibration motors on the left and right arms during the movement toward the front obstacle.

Smart shirt for obstacle avoidance for visually impaired persons

67

Temperature (ºC)

Obtained average temperatures over transmission lines 70 60 50 40 30 18.3 20 10 0 1 2

64.4

46 30.5 20.8

3

4

5

6

7 8 9 10 11 12 13 14 15 16 Voltage (V)

Figure 3.35 Average temperatures over the transmission lines on the garment versus voltage.

Table 3.4

Resistance variation along transmission line after washing Wash cycle

Resistance (U)

3.5.4

0

1

2

3

4

5

6

7

8

9

10

11.43

11.71

11.82

12.22

12.245

12.263

12.272

12.293

12.32

12.342

12.37

Washability

In order to test the changes in electrical conductivity of the system after home laundering, a sample of base structure (without microcontroller, sensors, and vibration motors) is washed for 10 cycles under AATCC Test Method 135-2004 [17]. Hence, the specimen is washed with a detergent of 66 g/L, dummy load of 1.8 kg, water level of 18  1 gal (washing time: 12 min and final spin time: 6 min). After each washing cycle, the conductivity of the transmission line is tested. No significant difference could be noticed after 10 washing cycles along the transmission line as shown in Table 3.4.

3.6

Conclusions

In this chapter the development of a smart shirt prototype enabling obstacle avoidance for visually impaired people has been presented. Considering a smart clothing system, an algorithm based on a neuro-fuzzy controller for obstacle avoidance is also introduced in order to navigate visually impaired people through this smart clothing system. In the algorithm, system inputs acquired by sensors have been evaluated in terms of fuzzy relations, and then the outputs corresponding to recommended turning angles, indicated by vibration motors, have been deduced by using neural network architecture. Afterward, microcontroller programming has been realized according to a control algorithm. The design and development concept of a smart clothing

68

Smart Textiles and Their Applications

prototype has involved four key areas of research, namely (1) electronics, (2) information technology, (3) control engineering, and (4) textiles. The prototype has been tested for detection capability, avoidance strategy with vibration motors, heating behavior, and washability. Results show that the developed system is able to identify an obstacle’s position without any failure within its detection range. The system is capable of detecting left, right, and front obstacles’ positions accurately and giving right output while detecting obstacles such that when obstacles are at the right, the system gives an output to turn left; when obstacles are at the left, the system gives an output to turn right. When obstacles are in front, the system gives an output like turn right and turn left at the same time, hence the user can understand that there is an obstacle just in front of them and they can randomly choose to turn right or left at that moment. Therefore, it may be concluded that the developed system is successful, reliable, and robust. Regarding power consumption of the system, it is found that the system can work for at least 1 day without any additional battery, however, depending on the environment this result can change. Therefore, it is recommended to users to have a spare battery for much longer usage and to overcome upsets. Concerning heating behavior of the system, during the operation mode (voltage z 5 V), it is found that temperature along transmission lines on the garment is around 22 C, which approximately corresponds with ASHRAE STANDARD 55-2010s thermal comfort degree (21.1 C). However, owing to earlier findings, it is highly notable that before designing a smart clothing system, since the applied voltage value affects heating behavior of the smart clothing system, the comfort level of the user should be taken into consideration. Comprehensive investigation on the developed smart shirt system with respect to sensing performance of the indoor environment and guiding the visually impaired accurately will provide a new scientific understanding of interactive garment design and development. This represents a great challenge and significant contribution to the sensor and actuator integration knowledge to textile structure. In addition, intelligent textiles are a recently developing area and there are still many to be invented, therefore successful implementation and integration of electronics used in a smart shirt system have significant value for smart textiles research.

3.7

Trends in future challenges

Within this chapter we have demonstrated the development of a smart shirt prototype enabling obstacle avoidance for visually impaired people. However, based on the study presented here, the following future research work is suggested: 1. In order to detect smaller obstacles, a system can be combined with the whole garment of the user such as trousers. In addition, in order to detect obstacles on the ground such as big holes, or in order to detect stairs, a system can be integrated with shoes as well. 2. This system currently offers accurate detection statically. To guide the user during walking to accurately avoid collisions, implementing newly developed neuro-fuzzy controller algorithms for obstacle avoidance into microcontroller programming should be extremely useful. Upgrading the microcontroller to one that has more memory and wireless connection and

Smart shirt for obstacle avoidance for visually impaired persons

69

upgrading programming language by implementing digital filters in the microcontroller in order to attenuate noise in signals that are processed inside the microcontroller are recommended. In this way, the degree of navigation and guidance accuracy during walking should increase. 3. For the outdoor environment, a developed system can be fully integrated with GPS, RFID, camera, and vocal guidance; not only can it track the user but it will also find a route to a specific destination and then guide the user to this destination using synthesized speech by ensuring localization information to the user such as the street address of the current location, etc. 4. Regarding power consumption of the developed system, a control to warn the user about battery level or the level of voltage supply can be implemented to the system, eg, voltage monitoring circuit. Due to information on voltage level decrease, some coefficients related to detection range capability have to be added to microcontroller programming in order to prevent a decrease in detection range as well. 5. Due to the technology miniaturization and reduction of costs in electronics and textile industry, new sensing elements, new flexible technologies, new actuators, and new functional yarns can be implemented to our developed system. For instance, flexible textile-based solar cells, which are expected to be thinner, lighter in weight and more powerful in the future, would be embedded to newly developed smart clothing systems as power supply for further improvement. As an actuator, artificial muscles would be interesting. Finally, a fully textile flexible sonar system may be developed in order to replace existing miniaturized rigid sensors.

References [1] World Health Organization, World Health Report, http://www.who.int/mediacentre/ factsheets/fs282/en/index.html, (accessed at 10.02.11). [2] M. Bousbia-Salah, A. Redjati, M. Fezari, M. Bettayeb, An ultrasonic navigation system for blind people, signal processing and communications, ICSPC 2007, in: IEEE International Conference, 24e27 November, 2007, pp. 1003e1006. [3] L. Xiaohan, H. Makino, S. Kobayashi, Y. Maeda, Design of an indoor self-positioning system for the visually impaired e simulation with RFID and bluetooth in a visible light, in: Communication System Engineering in Medicine and Biology Society 29th Annual International Conference of the IEEE, 22e26 August, 2007, pp. 1655e1658. [4] S. Kursun Bahadir, V. Koncar, F. Kalaoglu, Wearable obstacle detection system fully integrated to textile structures for visually impaired people, Sens. Actuators A Phys. 179 (07/2012) 297e311. [5] S.K. Bahadir, Wearable Obstacle Avoidance System Integrated With Conductive Yarns for Visually Impaired People (Ph.D. thesis), Universite Lille, Sciences et Technologies, Computer Engineering/Automation, France, 11/2011. [6] MaxBotix® Inc., LV-maxsonar®-EZ3™ data sheet and chaining notes, avalaible at http:// www.maxbotix.com/documents/LV-MaxSonar-EZ_Datasheet.pdf, (accessed June 2015). [7] http://www.robotshop.com/PDF/arduino-lilypad-vibe-board-schematic.pdf, (accessed on March 2011). [8] http://arduino.cc/en/uploads/Main/LilyPad_schematic_v18.pdf, (accessed on February 2011). [9] S. Kursun Bahadir, F. Kalaoglu, V. Koncar, Comparison of multi-connected miniaturized sonar sensors mounted on textile structure in a different position angle for obstacle detection, in: International Conference on Intelligent Textiles & Mass Customisation, Casablanca, Morrocco, 27e29 October, 2011.

70

Smart Textiles and Their Applications

[10] S. Kursun Bahadir, V. Koncar, F. Kalaoglu, Multi-Connection of Miniaturized Sonar Sensors Onto Textile Structure for Obstacle Detection, Autex, Mulhouse, France, 06/2011, 08.06.2011e10.06.2011. [11] S. Kursun Bahadir, F. Kalaoglu, V. Koncar, Analysis of vibrotactile perception via e-textile structure using fuzzy logic, Fibres Text. East. Eur. 95 (6) (11/2012) 91e97. [12] S. Kursun Bahadir, S. Thomassey, V. Koncar, F. Kalaoglu, An algorithm based on neuro-fuzzy controller implemented in a smart clothing system for obstacle avoidance, Int. J. Comput. Intell. Syst. 6 (3) (2013) 503e517. [13] Q. Liu, Fuzzy obstacle-avoiding controller of autonomous mobile robot optimized by genetic algorithm under multi-obstacles environment, in: 6th World Congress on Intelligent Control and Automation, 2006, pp. 3255e3259. [14] http://www.atmel.com/dyn/resources/prod_documents/doc8271.pdf, (accessed on February 2011). [15] R.L. Knoblauch, M.T. Pietrucha, M. Nitzburg, Field studies of pedestrian walking speed and start-up time, Transp. Res. Rec. 1538 (2006) 27e38. [16] T.J. Gates, D.A. Noyce, A.R. Bill, V. Nathanael, Recommended walking speeds for pedestrian clearance timing based on pedestrian characteristics, in: TRB 2006 Annual Meeting, 2006. Paper No. 06e1826. [17] AATCC Test Method 135-2004 Dimensional Changes of Fabrics After Home Laundering.