Development of a control system for a microrobot-based nanohandling station

Copyright © IFAC Robot Control, Wroclaw, Poland, 2003

ELSEVIER

PUBLICATIONS www.elsevier.comllocalelifac

DEVELOPMENT OF A CONTROL SYSTEM FOR A MICROROBOT-BASED NANOHANDLING STAnON Steran Garnica, Helge Hiilsen, Sergej Fatikow Division Microrobotics and Control Engineering Department ofComputing Science, University ofOldenburg Uhlhornsweg 84, D-26111 Oldenburg, Germany

Abstract: A microrobot-based nanohandling station, which is able to handle objects in the micrometer range and below, is introduced. Following a short description of the microrobot's mobile platform the station's control system is described. The application of an extended Kohonen network for the low-level pose control of the mobile platform is presented in detail as well as some results achieved. Copyright © 2003 IFAC Keywords: Microsystems, robot control, fuzzy control, neural control, actuating signals, microscope

INTRODUCTION

three small ruby balls glued to a segment of a piezoelectric disk (see Figure I). Its operation is based on the slip-stick principle: By applying voltages to the segments of the disks the corresponding ruby balls move up and down. When the voltage is slowly ramped up or down the ruby ball turns the steel ball (stick phase). When the voltage is changing fast, the ruby ball moves without turning the steel ball due to inertia (slip phase).

Microrobotics is seen as a key technology with high potential in medicine, manufacturing, communication, environmental and bio technology. A versatile microrobot-based nanohandling station is currently being developed (Fatikow et al., 2002). The control of processes in application fields like microassembly, cell handling or nanotesting (mechanical testing of nm-thin layers) is a challenge, especially seen from the lowlevel control point of view (Fatikow, 2000). Particularly, handling operations requiring two robots are aspired for biological cell injection and for coping with adhesion forces dominant in micro physics. In the following chapters, the mobile microrobot and the control system architecture for the nanohandling station are introduced. In chapter 4 an extended Kohonen network for the low-level positioning control of the mobile platform is presented.

2

Figure I: Bottom side of the mobile platform

MOBILE MICROROBOT

To control this type of mobile platform a compromise has been found between a low number of control channels and enough degrees of freedom for the control system to move the platform to the desired 3Dpose which consists of the 2D-position and the ID-

A mobile microrobot consists of a mobile platform and a manipulator unit with end-effectors. One of the mobile platforms which have been developed at the University of Oldenburg (Fatikow and Kortschack, 2002) is based on three steel balls that are each supported by 533

level part and a low-level part. The user can communicate with the system by means of a graphical user interface (GUI) for setup purposes and for monitoring automated operations. In the teleoperated mode the user receives visual feedback directly from the vision sensors or force feedback via a haptic device.

Translations: I-x, 2-x, 3-x Rotation: x-2 2-2

For measuring the states of the microrobot manipulators and end-effectors different sensors are applied: Force sensors measure mft applied to the objects. CCD-cameras

view and a scanning electron microscope (SEM) for local view deliver time-stamped images i CCD and iSEM via a screen to the human user and to the image processing module. The latter extracts the geometrical states of the robots and objects mroblobj and thus serves as a position sensor. This function is accomplished by object recognition algorithms and appropriate geometrical transformations from local image sensor coordinates to global coordinates. The data of position sensors embedded inside a fixed robot platform is combined with visual sensors for the mobile platforms in order to acquire the poses m rob of the robots.

2-1

Figure 2: Arrangement of control channels (top view) orientation. Figure 2 shows the arrangement of the six control channels which allow a translation and rotation of the mobile platform.

3

platforms, types of the forces for global

CONTROL SYSTEM ARCHITECTURE

The high-level control module determines the desired geometric states of robots and objects droblobj' Two modes of operation are provided: Semi-automatic planning algorithms and teleoperation within a multimodal user interface. An important part of this is formed by the haptic force feedback device described by St. Fahlbusch et a1.(2002).

Figure 3 presents an overview over the control system structure for the nanohandling station employing several microrobots. A modular approach was chosen to allow each module to be implemented in a flexible way by selecting different possible components. The control loop is formed by sensors, actuators, and the actual control system, which includes teleoperated and automatic modes of operation. It is split into a high-

Microrobot with Sensors

electr. signal High-level Control

Low-level Control

Actuator Driver

Signal Generator

Position

Sensor .~ )

Force Sensor

SEM

rn

--Image Processing

Haptic Interface'

J

iCCD

visual feedback

rn

force feedback

s

4-------~----------.I i-------------------------------~

: SEM : CCD I I

Figure 3: Control system structure 534

Scanning Electron Microscope CCD Camera

: : I

--------------------------~

A major challenge is the cooperation of two or more microrobots in order to perform complicated handling actions. For example, the dominant physics in micro scale require coping with adhesion forces, which can be simplified by applying two mechanically independent end-effectors, a gripper and a needle. Another aspired multi-robot operation is an injection into a biological cell held by a canula. A manual approach and an agent-based software approach will be compared in order to investigate to what extent both endeffectors can be controlled by independently acting units. A significant increase in the speed of handling operations is expected from a highly dynamical switching and cooperation of teleoperation and automated decisions, which is under development.

significant drawbacks when used for microrobots (Fatikow, 2000): a) For the positioning of platforms and manipulators in the micro and sub-micrometer range wear and changes of temperature or air humidity lead to time-variant parameters (Zhou et al., 2002). The controller must be able to adapt to these parameters during run-time. For a classical controller an adaptation algorithm would have to be found to compensate these temporal variations. b) Experiments measuring the dependency of the velocity of the microrobot from the control parameters have shown non-linear behaviour. Hence, the controller has to be able to compensate such non-linearities. An implementation with a linear controller would lead to a model of high order and complexity. c) Because physical relations and material constants are only partially known and, as descibed above, locally and temporally variant, exact knowledge about the microrobot's behaviour and thus a corresponding mathematical model could only be obtained with very high effort. Therefore it would be sensible to design the controller without using a mathematical model of the controlled system, which would be a requirement for the design of a classical controller. d) Sensor information from force or vision sensors in a nanohandling station is often vague or incomplete. Especially under real-time contraints higher precision leads to higher latencies. The control system has to be able to generalise nonexact sensor information and/or to predict the manipulated variable over a longer period of time. A classical controller would require exact sensor information and sampling times below the smallest time constant of the controlled system.

The hardware architecture includes several Personal Computers (for vision, for high-level and low-level control). Currently Microsoft Windows NT/2oo0 is used as operating system. The communication is mostly accomplished by means of the Internet Protocol (IP) in order to enable a maximum flexibility in selecting the modules' implementations. The low-level control module monitors the poses of the robots and objects mrob/obj' It compares them with the desired states

drob/obj/ ..

and reacts by generating

the corresponding parameters for the actuators P ocr to minimise this deviation. For the piezo actuators described above, these parameters are amplitude, frequency, waveform and duration of the signals. Different approaches for control algorithms which can be used are discussed in chapter 4. The low-level control task is accomplished by the control PC described above, which sends the actuator parameters to the signal generator via the PCI-bus. A commercial intelligent signal generator card by Goldammer is used, which buffers the data within a function generator and thus allows a continuous signal. It performs a DA-conversion and outputs the signals cocr for the actuators, which are sawtooth-shaped for the piezo-electric actuators using the slip-stick principle.

From the above discussion it can be concluded that classical controllers are not practicable for the pose control of a mobile microrobot. Some work has been done in the last years to prove that controllers using fuzzy logic and/or a neural network could solve this problem (See below for more details). While some of the neural network based controllers fulfil the requirements a) adaptive, b) non-linear, c) model-free and d) generalising and predictive, fuzzy logic based controllers are a) not adaptive and d) not predictive and not generalising. Another important difference is that neural network based controllers only need implicit knowledge of the process (pairs of actuator parameters and measured position) while fuzzy logic based controllers need explicit control knowledge (rules of when and how to react).

The actuator driver module converts the control signal cuer to the actuator-specific electrical signal by voltage amplification. The device developed at the University of Oldenburg is based on operational amplifiers by Apex. It converts the signal amplitude from 10V to l50V and is capable of outputting the signal ramps necessary for the actuators at 15kHz as well as a quasistatic signal for fme piezo movement.

4

4.1

Some research on the use of fuzzy logic and neural networks for the control of microrobots has already been done. Santa and Fatikow (Santa and Fatikow, 2000; Santa et aI., 1999) compared a PlO controller, a fuzzy logic based controller and a neural network based controller applied to control the pose of a mobile microrobot platform. A fuzzy logic based controller is currently implemented at the division to compare the performance of different approaches.

LOW-LEVEL CONTROL USING A LOCAL LINEAR MAP

Problems and ReqUirements

To implement the important process pnmltlve of moving the microrobot, classical controllers like PlO controllers might be used. However, they have some 535

The controller presented in this publication is based on a local linear map (LLM), which is an extended selforganising feature map (SOFM). It was used by Ritter et al. (Ritter et al., 1992) to control a stationary (macroscopic) robot with five degrees of freedom (DOFs): Three for the position and two for the orientation. Two cameras supplied sensor data as visual feedback.

typical input vector distribution, which means that the density of the vector weights g; is higher in areas where the input vectors gdQ have occurred more often in the previous steps. This behaviour can be understood from the learning method, the so-called Kohonen learning, which is performed after finding the output vector Pact' All input weights gi are updated in the following way at time step v: g~V) = g~V-I) + C, . h(;' ,i,r,)' (g~ _ g~V-I») (1)

Currently, an LLM is being applied to the 2D-position and ID-orientation control of the mobile platform of the station's microrobot; its performance is analysed in the micro- and nano-world. For the sensor feedback, a high-resolution CCD camera and an SEM are used.

Here, c, and h(;', i, r,) are learning constants, where the latter is non-zero for all neurons i that are inside the radius r, of the winner neuron ;' in the grid c (see Figure 4). By this method the weights gi of all

4.2

neurons i in the neighbourhood of the winning neuron ;' are dragged towards its weight g ,.. Therefore this

Selforganising Feature Maps

Throughout the following text all poses are local poses, which are desired or measured positions and orientations with reference to the measured pose at the beginning of each step.

learning method can be called "topology-conserving", which means that the neurons which are neighbours in grid c become neighbours in the weight vector space G, too.

A SOFM or Kohonen network consists of N numbered neurons arranged in a grid c (see Figure 4). This grid can be one- or multidimensional and can have different shapes like a rectangular grid or a hexagonal grid. In a simple SOFM (see Hagan et a/.,1996) each neuron i is connected to an input weight g; E G , e.g. a desired local position of the mobile platform. On the other hand, an output weight Pi E P is associated to

Given an output weight estimate Pen that belongs to the winning input weight g;, the output vectors Pi can be trained in the same way as gi: p~v) =p~v-I) +cp .h(;' ,i,rp)'(p~; _p~V-I»)

The set of Pi also shows the neighbourhood-property, which means that neighboured neurons in the grid c output similar vectors.

each neuron i , e.g. parameters for the actuators of the platform to reach that position, which in this case are voltage values for the six channels. When an input vector gdQ is presented to the network it outputs the output vector Pact

= p..,

(2)

4.3

of the Neuron ;' whose input

Extended Selforganising Feature Maps

The SOFM type mentioned above has the drawback that the output vectors Pi are constant for the subspaces of G around their corresponding input vectors gi' To overcome this problem Ritter et al. (Ritter et

weight g;, is closest to gdQ (see Figure 4). Such networks thus perform the mapping G HP, where N subspaces of G belong to N vectors of P. Beside the feature mapping, a SOFM has the property of organising itself into a state such that it best represents a

al., 1992) proposed to associate a matrix Ai to each neuron i. Ai is the Jacobian matrix at the point

Figure 4: Extended SOFM 536

This implies that for every neuron its preceding actuator parameters and measured pose have to be stored.

g; / p;, which transforms a small pose difference gdes -g; into a small parameter difference p_ -p;. The actuator parameters are thus calculated by P«t =P;' + A;, ·(gdes-gi') (3)

4.5

Simulation Results

The matrices A; can be seen as a third set of weights (see Figure 4) and, given an estimate for the matrix A.., that belongs to the winning input weight g;"

Simulation results show that the approach described above can be applied to fmding actuator parameters for a mobile microrobot's platfonn.

trained in the same way as g; and p;: A: V) = A: v- I) +&,1 ·h(t,i,rA)·(A~ _A: v- 1)

The simulations were conducted with a simple model of the system behaviour, which interprets the voltages applied to each of the six control channel vectors (see Figure 2) as velocity vectors. The resulting translation is calculated by the integration of the weighted sum of these velocity vectors. The resulting rotation is calculated by the integration of the weighted vector product of the velocity vector and its corresponding radius.

(4)

A; shows the neighbourhood-property, too.

4.4

Estimating p"" and A..,

For the calculation of p"" and A.., an algorithm is used which descends on the squared error function, i.e. it follows the negative gradient of that function.

The conditions for the simulation results presented in Figure 5 are as follows: The voltage-velocity dependency in the model is nonlinear to test whether the algorithm can compensate non-linearities. In addition, after 1500 iteration steps the model is changed in order to see whether the algorithm can deal with time variant parameters. The velocity vector of channel 2-2 is scaled up to 120% and changed in direction by 20 0 •

Estimation of p"". The squared error function of p. I

for the steps v, which has to be minimised during the algorithm, is E(P;,) =

~ L[(P~: -P;' )-A~ (g:., -g;.)J

(5)

v

We have to follow the negative gradient of E(p.) to I

The network was fed with random values of the local desired pose gdes' where the distance of the position in each dimension was inside the range of (-lOOmm ... lOOmm) and the orientation was set to O.

find its minimum. Provided that the matrix A.; is already a good approximation of the linear behaviour A~ of the system around g.. and knowing that I

I

p~: -P;' = A;. (g~ -gi')' the estimate Pes, for each

The neighbourhood-defming grid was a rectangular grid with l6x5 neurons. The dimensions represent the direction and the distance of the desired local position.

step v is: (V) = p(V-I) + b . A (v-I) (g(V) _ g(v) ) (6) P est t p t des m«1S with bp = I. This estimation, applied to a large num-

ber of steps, will lead to the minimum of the squared error function (delta rule).

Important for the convergence of the pose error and the neighbourhood property is the decrease of the radii rg , rp and rA and the learning constants & g, & p and

Please note that the Jacobian matrix A;, plays a lead-

&,1 . The decrease function used is defined as

y" f(,~} y,(;:l'

ing role in this algorithm since not only the estimation of p.., depends on it but also the calculation of p_ and the estimation of A.., .

with tno"" = 2000 and the values Yo and y, for

tftnonro = 0 and tftnonro = I, respectively, (see Table I). A series of simulations has been conducted to fmd these values, which result in a good convergence of the moving average (last 100 values) pose error.

Estimation of A..,. The squared error function of A., for the steps v, which has to be minimised during the algorithm, can be measured by E(A;,) =

~ L( Ap(V) -

A;, . Ag(V)

f

(7)

Table 1 Parameters for the decrease of learning rates and radii over time parameter Yo YI

v

with Ap(V)

= p~: -

p~-I)

and Ag(v)

=g~ -

g~)

We have to follow the negative gradient of E(A.) to I

find its minimum and, as for p"", this yields the estimate A"" for each step v:

A~

=

A:~-I) +0,1 .(Ap(V) -A:~-I)Ag(V»)'(Ag(V)r (8)

with b A =

(9)

I 2' IIAg(V)11

537

&g

0.2

0.005

&p' &,1 rg

0.5

0.05

1.5

0.2

rp' rA

6

0.3

3O

-.-

10·

-,.

~EnCf,..,_SIOP

--

..

I Nu_"'_a-po __

v _ _ . _ . ~•• -~_...,.-_. _.

._.

.M·~

~

own SOFMs for the desired position and orientation, respectively.

-··'"'tl

5

j,.

-'-,

~

.' ...

The control system of a versatile microrobot-based nanohandling station was presented. The system is modular and versatile and uses as much standard components as possible, in order to ensure acceptance of the developed system.

Z-

l

°0---- mo"

,l

.,-

.--·,-'tiio--·--.-··

'W

t. - , .

MiD~'

The presented local linear map based algorithm has been shown to fulfil the requirements for the control of mobile microrobots. In addition, it has the potential to allow the control of end-effector poses using camera coordinates directly without any coordinate transformations. It is currently being implemented and tested in the nanohandling station of the Division.

,." -. .- . i&

Figure 5: Decrease of moving average pose error (solid line) and number of iteration steps (dashed line) The graph in Figure 5 shows that the moving average pose error decreases over time in spite of the nonlinearities in the controlled system. It also shows that the algorithm adapts to the change of system parameters. The pose error is given as relative difference between desired and measured pose with respect to the ranges 200rnm (position) and 21t (orientation). After 2000 steps this error is about 2%. This algorithm was applied repeatedly until the error had reached 0.1%. The number of these approaching iteration steps also decreases over time (test iteration steps) and is about 5 after 2000 steps (see Figure 5).

4.6

6

7

REFERENCES

Fahlbusch, St., A. Shirinov and S. Fatikow (2002). AFM-based Micro Force Sensor and Haptic Interface for a Nanohandling Robot. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Lausanne, Switzerland, pp. 1772-1777 Fatikow, S., et al. (2002). Development of a versatile nanohandling station in a scanning electron microscope. In: Microfactory, (R. Hollis, B. Nelson (Ed», pp. 93-96, IWMF02, Minneapolis, U.S.A. Fatikow, S., A. Kortschack (2002). Smart materials for actuation in microrobotics. In: Int. Symp. on Smart Structures. Devices. and Systems, Melbourne, Australia Fatikow, S. (2000). Mikroroboter und Mikromontage, Teubner Verlag, Stuttgart, Germany Hagan, M., H. Demuth, M. Bale (1996). Neural Network Design, chapter 14, PWS Publishing Company, Boston, U.S.A. Ritter, H., T. Martinetz, K. Schulten (1992). Neuronale Netze, pp. 175-236, Addison-Wesley, Bonn, Germany Santa, K., S. Fatikow, G. Felso (1999). Control ofmicroassembly robots by fuzzy-logic and neural networks. In: Computers in Industry, 13, pp. 219227, Elsevier Science Publishers B. V. Amsterdam, The Netherlands Santa, K., S. Fatikow (2000). Control system for motion control of a piezoelectric micromanipulation robot. In: Advanced Robotics, pp. 577-590, VSP and Robotic Society of Japan Zhou, Q., et al. (2002). Environmental Influences on Microassembly. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Lausanne, Switzerland

It has been shown that the described algorithm can fulfil the requirements of controlling a microrobot's platform. The algorithm is able to adapt to changing parameters of the mobile microrobot during run-time and can compensate non-linearities of the microrobot's behaviour. Furthermore, it was possible to design the control system without developing a mathematically complex model of the physical behaviour. The controller is able to work with non-exact sensor information since the neighbourhood property of the network ensures that similar inputs (desired poses) lead to similar outputs (actuator parameters). The controller is also able to predict the actuator parameters so that even after a longer period of time without sensor feedback the pose of the microrobot is within a certain range of the desired pose. Because of the self-organisation

gdes

ACKNOWLEDGEMENT

This paper is based on research supported by the European Union, project ROBOSEM (GRDI-2oo1-41864). Financial support is gratefully acknowledged.

Discussion

property the areas with

CONCLUSION

vectors appearing more

often have a higher output vector resolution. This ensures an economic use of computing power and memory. The method described also has the potential of working directly with camera coordinates of end-effectors. This means that current pose and desired pose in camera coordinates are presented to the network, which calculates the actuator parameters with that information only. This approach could reduce the error and effort caused by the transformation between the camera frame, the world frame and the end-effector frame. The network could be implemented as a hierarchy of SOFMs, with the main SOFM representing the measured pose. Each neuron of that SOFM would have its 538