TELEMATIC BASED TRANSPORT DEVICE TRACKING AND SUPERVISION SYSTEM

SAMPLE PAGES TO BE FOLLOWED EXACTLY IN PREPARING SCRIPTS

TELEMATIC BASED TRANSPORT DEVICE TRACKING AND SUPERVISION SYSTEM Janusz Szpytko, Artur Kocerba, Mateusz Tekielak AGH University of Science and Technology Mickiewicza Ave. 30, PL 30-059 Krakow, Poland

Abstract: The subject of this article is a method and monitoring tool of transport device, based on telematic approach. Based on the device movement trajectory analysis is possible to conclude about the system tracking and technical state degradation. The developed EYE tool connected with CASIP digital platform make possible the real-time and online continuously monitoring of the system degradation and failure, the prognosis of the drift behaviour, as well as the aid in diagnosis on the basis of the localisation and the identification of the potential causes. The project application has been successfully tested on selected mobile devices. Copyright © 2006 IFAC Keywords: monitoring, diagnostics, transportation, telematic

1. INTRODUCTION Telematics is a field of science that integrates telecommunications and informatics in the range of information processing with their effective transfer to the points, where it can be used to achieve specified quality of the action. Modern telematics gained economic potential from increasing capabilities at drastically decreasing costs for telecommunication links, as well as for data processing. One of more essential areas of using telematics is the transportation. Telematics is focusing on information-orientated activities, considering possible restrictions. Rational working on an object (person or cargo) is possible only with using possessed information. Working on the information includes: getting the information from different sources (monitoring), processing the information in a proper way (and a useful way for a person who is undertaking decisions), recording the information on digital media types for archival purpose or prognoses (storing), sending the information to interested recipient (transmission) and presentation of the information for the decision necessities.

Using telematics in transportation is the subject of numerous publications, for example (Bartczak, 2000; Hecht, et al, 2000; Korver, and Harrell, 1999; Ochieng, and Sauer, 2002; Szpytko, and Kocerba, 2003; Szpytko, 2000; Szpytko, 2004a; and others). The authors are focused on following questions:
the identification of cargo and/or the means of transport,
supporting the decision process in managing means of transport and the special services (in case of unfavorable events, which can menace the operation safety of the transportation system),
supporting of the decision process in the range of managing the motion of transportation means, and management,
supporting of the transportation means service process, as well as transport control,
generating the information in the form of rescue code (in case of danger-event). Telematics essentially increases the transportation safety, optimizing the routes of carrying cargos, and in effect, reduce the costs of transportation, as well as enlarge the reliability of transportation systems. The methodology of transportation system modeling using telematics is presented in the book (Szpytko, 2004b).

Telematics is also used in manufacture transport devices: in using- and serving process. For example, the mobile vehicles (Ulatowski, and Masłowski, 2003) the automated guided vehicles (AGV), the racking machines applied in narrow-aisle systems, and container gantry cranes. There are also wellknown systems supervising the functions of Kone lifts, as well as the mobile robots (Schilling, 2002). Using telematics in many various fields of human activity enables the reduction costs of technical means exploitation and negative environmental impact, as well as increase transport safety and capacity. Techniques supporting telematics (including expert software and tools) are rapidly developed. There are increasing requirements for the intelligent transportation service and for the dynamic management of the transport devices operation, both in the long distances, and in the integrated/ so-called intelligent automated transportation-production companies. The subject of this article is a method and tool of displacements monitoring of characteristic transport device points (especially automated large dimensional rails' transport systems), with telematic use. Based on the device movement trajectory analysis is possible to conclude about the system tracking and technical state degradation (Szpytko, 2004b). The developed EYE tool connected with CASIP (Computer Aided Safety and Industrial Productivity) digital platform make possible the realtime and online continuously monitoring of the system degradation and failure, the prognosis of the drift behaviour, as well as the aid in diagnosis on the basis of the localisation and the identification of the potential causes. The CASIP environment allows managing preventive-type supervision of device exploitation process and provides process management (Leger, et al, 1999; Yu, et al, 2003). The project application has been successfully tested on automated cranes. 2.

DEVICE TRACK MEASURING

Kinematics of any device (system) with i - th number of units can be described by 6 matrix: position M (linear and angular co-ordinates), velocity W (linear and angular speed), accelerations H (linear and angular acceleration), impetus Γ (mass and velocity), motion Φ (force and rotatory momentum), inertia momentum J (co-ordinates of the centre of system mass and its total mass). Dynamics of the device is characterized by Newton – Euler or Lagrange II type equations for holonomic generalized systems. This treatment demands a mathematic formalism of dependences between elements of device structure. Reciprocal relations between k-th- and j-th system unit appear as below: (wz - relative, u - float): W = W wz + W u (1)

H = H wz + H u + 2 ⋅ H wz ⋅ H u

Complexity of the device and variety of effects of its use and unacquaintance with occurred relations make difficult to prosecute analysis of system dynamics in the classic way. In this range, it is helpful to use optoelectronic techniques, which enable remote- and non-contact system research (Szpytko, 1996). There are characteristic points marked out on the device; dislocation of its points is observed by CCD video camera in XPY coordinates system The observed Q(t) characteristics of changes in the time t of location of device specified points is a function of specified system matrix: Q = f (M, W, H, Γ, Φ, J)

(2)

Figure 1 shows the block schema of displacement analysis system of the research object. The system includes: research objects, monitoring system, PC computer with attachment and B1 and B2 databases. PCI type card

special

file *.avi type

analysis of investigation objective B2

monitoring tests visualization

COMPUTER PC type





B1

MONITORING camera CCD type data carrier

TEST OBJECTIVE G1

R1

external influences

influences result

Fig.1. The block scheme of the system of research object’s dislocations analysis There were characteristic points (G1 and R1) marked out on the research object, accordingly with (x1, y1) and (x2, y2) coordinates. The aim of this research is observation of dislocation of G1 and R1 points in t (extortion effects) in the result of external extortions (figure 2): (x1, y1) (t) → (x1, y1) (t + ∆t)

(x2, y2) (t) → (x2, y2) (t + ∆t) Y

t1

t2

t3

(3)

t4

X O Fig.2. Expositions of changes of G1 and R1 points positions in t time (YOX coordinates)

Dislocation of G1 and R1 points in time t was recording, by CCD video camera, on magnetic data media (B1). Using the appropriate computer card, the information recorded on the B1 data media was saved in the *.avi file. This file was a subject of analysis, using the expert instrument; in the result, we could present of the observation of G1 and R1 research object characteristic points, in the form of equations (3), and its record in B2 knowledge database. 3.

IMAGE ANALYSIS TECHNIQUE

Analysis of records (in specified i- th moments of ti time) of digital image of location of research object characteristic point was preceded by initial processing stage (figure 3). The purpose of initial picture processing was improvement of the quality, by the elimination of possible noises and disturbances, and by emphasize of essential elements. median filtration

morphology transformation:
erosion,
dilatation, histogram,
opening / closing thresholding INITIAL IMAGE PROCESSING

IMAGE PROCESSING

based on area

oriented segmentation: geometrical centre of gravity of analyzed

objects

Fig.3. Block scheme of expert instrument helping image processing The stage of initial image processing includes: median filtration, thresholding, histogram and the morphological transformations. Median filter (from nonlinear filters group) is applied to eliminate all pixels which values are highly deviating from the accepted average; it is also applied to remove any local noises and disturbances. The principle of median filter effect is changing the value of individual image pixels to values chosen by specified rule (in the neighbourhood of every considered pixel). After transformation, the specified value of the image point becomes one of the values already present in the picture (because it has been chosen from its neighbourhood). By using the filter, the median value of the point is obtained as the growing sequence of pixels from its (considered) neighbourhood. Advantages of median filter are (Tadeusiewicz, and Korohoda, 1997):
keeping the sharpness of the edge (no new values for the outcome image),
filtered picture is resistant to pixels which are highly deviating from the accepted average values; it is also resistant to any possible local noises. The inconvenience of median filter is long time of calculation and image erosion for windows (masks) with large dimensions.

The aim of thresholding (binarization) in vision systems is identification of image (scene) objects. There is an appropriate T thresholding value chosen during the binarization (automatic or manual). In the effect, the information included in the picture is radically reduced, because grey or multi- colour image is transformed into 0/1 value pixels. To find the right T value, there is an analysis of histogram of T value, to find the local minimum. (Sankur, and Sezgin, 2001). Local minimum point that T value, which be a base to image segmentation process (Kohut, 2002). The result of binarization is picture partition into regions. To avoid jagged and isolated fragments of picture objects, there is universal morphologic transformations (such as: erosion, dilatation or its composition) used. In image analysis, where 2 characteristic G1 and R1 points with 2 colours were specified, before the binarization, there is colour component 2-argument weight substraction applied. This operation helps to emphasize particular colour components, and to separate them to partition for every spot separately. 2-argument weight substraction implicates formation of holes in analyzed objects. Regarding that, after thresholding, before morphologic filtration, it is necessary to perform an additional operation of filling created holes. In this moment, imfill procedure (IPT) is applied; this procedure is changing ‘0’ background joint pixels values into ‘1’ (first-plan pixels values) as long, as the specified object border is achieved. The range of this procedure is defined by specified pixels joint type (4- or 8-neighbourhood). Morphologic transformations enable analyses of shape of specified picture objects and their mutual location (Malina, et al, 2002). Morphologic operations include:
erosion (E) applied to remove isolated points on the image and small specified regions, also to remove narrow ledges and to smooth objects edges; in some cases to division to smaller regions (which is adapted in jointed objects division, before the partition),
dilatation (D) applied to close small holes and narrow bays of observation object, also to increase the shape area (sometimes it causes jointing the neighbouring regions),
opening/ closing. Substantial inconvenience caused by erosion and dilatation is the change of the area of the transformed regions (erosion decreases, dilatation increases). To eliminate this inconvenience, there is a new transformation applied – composition of erosion and dilatation: opening (E+D) and closing (D+E). Those transformations allow to smoothing the edge, keeping the shape size. In case of binary pictures, opening and closing transformations show the following characteristics:
opening removes small objects and small details (peninsulas, ledges), with no change of shape size (it can also divide some objects with narrowing),
closing fulls narrow notches, bays and small holes inside the shape, with no change of shape size (it can also joint some near objects). Image binarization and filtration, using chosen morphologic transformations, is an initial phase of picture

processing (analysis). Image analysis is one of fields of computer sight, ‘picture interpretation’ oriented (Kette, and Zamperoni, 1996). The essence of picture analysis depends on finding the right description, which (in the shorten form) can inform about all the substantial objects’ attributes. This process includes:
partition – isolating image objects from the background,
attributes extraction – area, middle of weight, etc.,
definition of object position. The result of image analysis is not its digital representative (like after initial image processing), but its symbolic and quantitive description, in the form of specified objects’ attributes. The choice of attributes is a subjective decision taken by the system designer; it depends on tasks, which should be realized in the system. One of possible is description the attributes of analyzed objects, applying the method of first grade – momentum (geometrical positions of shape’s middle of weight), using region partition technique. Procedures of qualifying the coordinates of analyzed objects’ middles of weight, which are realizing the region partition (Uhl, and Kohut, 2001), are built in Image Processing Toolbox (IPT) in MatLab Simulink environment:

m pq = ∑∑ i p j q xij n

m

i =1 j =1

(4) [n,m] – picture size

for the binary image: m00 - means object area, m10, m01 - its middle of weight. Momentum of first grade attributes describe geometrical position of shape’s middle of weight:

xc =

m10 m , yc = 01 m00 m00

(5)

4. EXAMPLE OF DEVICE TRACKING APPLICATION Expert tool EYE (interface instrument) to image processing was made in MatLab Simulink environment. User’s graphic interface of instrument is shown on figure 4 (Szpytko, et al, 2003). User’s graphic interface allows to: getting *.avi file(s) and its preview, establishing of amount of processed frames, choice of transformations applied in initial image processing: mediana filtration and the size of filter, morphologic transformations (erosion, dilatation, closing/opening) and shape and size of structural element,
establishing of threshold(s) of binarization,
visualization of following stages of picture processing,
reporting and visualization of obtained results.




Fig.4. User’s graphic EYE interface of image tool

analysis

5. CASIP AN OPERATIONAL SAFETY SOFTWARE To implement on-line computer aided monitoring and degradation diagnosis of the cranes the computer aided safety software (platform) CASIP has been developed (Predict, 2005; Szpytko, et al, 2002). CASIP integrates both modules of design allowing to analyse degradations, their causes, effects, and symptoms (based on FMECA Failure Modes Effects and Criticality Analysis, HAZOP Hazard and Operability Studies, Fault-Tree Analysis methods) and the modules of Proactive Maintenance (including monitoring, diagnostics, prognosis). CASIP complete the Enterprise Resources Planning (ERP) to manage the risk of the plants and the Manufacturing Execution System (MES) to react in real-time regarding to malfunctioning. At the ERP level software is open to Computer Aided Manufacturing (CAM), Computerised Maintenance Management System (CMMS) and other existing systems. At the MES level, CASIP is open to SCADA (Supervisory Control And Data Acquisition), PLC (Programmable Logic Controller), Distributed Input/ Output, Data Acquisition System and other control systems. CASIP integrates a real-time database allowing acquirement of:
the real-time and online continuously monitoring of the system degradation and failure,
the aid in diagnosis on the basis of the localisation and the identification of the causes,
the prognosis of the drift behaviour. CASIP is also a distributed software platform making it possible to implement a remote monitoring, diagnosis and maintenance for multi-sites enterprise (figure 5).

tested device

monitored exploitation parameters of the device Server SAM

Real time database Process database

Device technical state assessment: - ability, - non-ability,

Exploitation parameters: - acceptable value MODULE - critical level

SAM

Cause and effect analysis: - what, - where, prognosis of - why, failure appear

MODULE FMECA/HAZOP Device structure: - system 1 Interaction between device - subsystem elements - unit - element - system 2 ...

Fig. 5. Multimode e-platform CASIP

The CASIP system, as well as the developed tracking tool has been successfully tested on automated crane examples. The system has been developed in the Department of Technological Devices and Protection of Environment AGH in Cracow, in Technological Transportation Group. Data from the device and from the environment were transmitted by wire link, by the Internet, and by the wireless broadcast. The data transmission system was using the wi-fi card (e.g. Wireless) with the bandwidth up to 11 Mb/s, in the range of over a dozen meters. The research station (figure 6) enables gaining the information about the device environment (by CCD cameras and other sensors), and about the crane characterized with selected exploitation parameters. The device control system and information transmission system include: the driver, digital recording card, local server, switch, network card, and two computers. Investigation results related to the CASIP implementation into the automated crane devices have been presented in publications (Kocerba, and Tekielak, 2005; Szpytko, et al, 2005).

Fig. 6. The block schema of the research object and device remote monitoring 5. CONCLUSIONS Telematics in the essential way improves safety and reliability of transport systems, enables optimizing cargo motion routes and reduces the costs. Telematics is currently the important parts of transportation infrastructure. The key question in transportation system safety and reliability is its ability to gain the output products (information), generated by the participating in transportation process, and then integration for quantity and quality evaluation, and processing to output products useful in the decision process. Telematics, using techniques such as informatics, optoelectronics, automatics and telecommunications, helps to reduce costs of transportation potential management, improves the security and reliability of the transportation service and the decision process automation. Modern telematics methods offer a huge application potential in teleservicing, having impact on most engineering disciplines.

Presented method of analysis of device dislocations, using the recording CCD camera and the image analysis’ expert EYE tool (instrument) could be used in mobile (transportation) devices monitoring. Advantage of presented method is possibility of observation in on-line systems, remote and noncontact. Based on the mobile device track analysis is possible to conclude about the system operation and technical state degradation. The developed tool connected with CASIP makes moreover possible online monitoring of the system degradation and failure, the prognosis of the drift behaviour, and diagnosis. The project application has been tested on mobile devices models, including automated cranes. Using presented tracking tool of mobile devices both with combination with the computer aided safety software allows fulfillment of the expectations in: reduction of transport supply management costs, increase of safety and reliability of transportation services, and automation of decision process. The research project is financed from the Polish Science budget for the years 2005-08. REFERENCE Bartczak, K. (2000). Technologia RFID według TSS. Logistyka, 2, 64-66. Hecht, M., M., Janik, T., Rieckenberg and D. Salz (2000). Telematyka w kolejowych przewozach towarowych-transport ładunkow niebezpiecznych. Technika Transportu Szynowego, 1, 613. IPT (2002). Image Processing Toolbox for use with MATLAB. The MathWorks Inc. Kette, R. and P. Zamperoni (1996). Handbook of image processing operators. John Wiley & Sons Ltd., New York. Kocerba, A., and M. Tekielak (2005). Application of FMECA causal-consecutive analysis with an example of IMS digital platform. In Engineering Achievements Across the Global Village, Szpytko J. (Ed.), Library of Maintenance Problems, 415-422, IT-NRI, Radom. Kohut, P. (2002). Prototypowanie ukladow sterowania wizyjnego z wykorzystaniem procesorow sygnalowych. Rozprawa doktorska, AGH, Krakow. Korver, W. and L. Harrell (1999). Definition of European Transport Systems, FANTASIE Deliverable 13, Delft, TNO-INRO. Leger, J-B., B., Iung, B.A., Ferro and J. Pinoteau (1999). An innovative approach for new distributed maintenance system: application to hydro power plants of the REMAFEX project. Computers in Industry, 38 (2), 133–150. Malina, W., S., Ablameyko and W. Pawlak (2002). Podstawy cyfrowego przetwarzania obrazow. EXIT, Warszawa. Ochieng, W. Y. and K. Sauer (2002). Urban road transport navigation: performance of the global positioning system after selective availability.

Transportation Research Part C: Emerging Technologies, 10 (3), 171-187. Predict (2005). CASIP e-platform guide. Sankur, B. and M. Sezgin (2001). Image techniques: a survey over categories. Pattern Recognition. Schilling, K. (2002). Tele-maintenance of industrial transport robots. Proceedings of 15th Triennial World Congress, Barcelona, IFAC. Szpytko, J. (Eds.) (2000). Reorganizacja technicznych procesow dzialania w przedsiebiorstwach w aspekcie globalizacji rynku. Oficyna Cracovia, Krakow. Szpytko, J., B., Iung, J.B., Leger and D.K. Harrison (2002). Computer aided safety and productivity system for cranes. Proceedings of 21st Conference on European Plant Engineering Committe (CETIS, VDEh). Szpytko, J., A., Kocerba and M. Tekielak (2005). System dozorowania procesu eksploatacji srodka transportu. Zastosowania Teorii Systemow (Red.E.Kochan), Monografie, 3, 387-394, AGH WIMIR, Krakow. Szpytko, J. and A. Kocerba (2003). Telematyka narzędziem jakosciowego ksztaltowania transportu. Automatyzacja i eksploatacja systemow sterowania i lacznosci, Kitowski Z., Lisowski J. Eds., 1, 169-176, AMW Gdynia. Szpytko, J., M., Tekielak and P. Kohut (2003). Optoelektroniczna metoda analizy przemieszczen urządzenia. Pomiary, Automatyka, Kontrola, 11, 5-7 Szpytko, J. (2004b). Integrated decision making supporting the exploitation and control of transport devices. Monografie, UWND AGH, Krakow. Szpytko, J. (2004). Ksztaltowanie procesu eksploatacji srodkow transportu bliskiego. Biblioteka Problemow Eksploatacji, ITE, Krakow - Radom. Szpytko, J. (1996). Zintegrowany system nadzorowania wybranych parametrow eksploatacyjnych wielkogabarytowych szynowych urządzen transportowych na przykladzie zautomatyzowanej suwnicy pomostowej. Rozprawy i monografie, 46, Wyd. AGH, Krakow. Tadeusiewicz, R. and P. Korohoda (1997). Komputerowa analiza i przetwarzanie obrazow. FPT, Krakow. Uhl, T. and P. Kohut (2001). The rapid prototyping of the visual servoing on Matlab/Simulink/dSPACE environment. Proceeding 7th IEEE International Conference on Methods and Models in Automation and Robotics, Miedzyzdroje, 672-677. Ulatowski, W. and A. Maslowski (2003). Pojazdy sterowane automatycznie. Pomiary Automatyka Robotyka, 10, 28-31. Yu, R., B., Iung and H. Panetto H. (2003). A multi-agents based E-maintenance system with case-based reasoning decision support. Engineering Applications of Artificial Intelligence, 16 (4), 321-333.