- Email: [email protected]
Implementation of natural sensory feedback in a portable control system for a hand grasp neuroprosthesis
Medical Engineering & Physics 26 (2004) 449–458 www.elsevier.com/locate/medengphy
Implementation of natural sensory feedback in a portable control system for a hand grasp neuroprosthesis Andreas Inmann , Morten Haugland Center for Sensory-Motor Interaction, Aalborg University, Aalborg, Denmark Received 26 June 2003; received in revised form 22 February 2004; accepted 9 March 2004
Abstract This paper presents the design and implementation of the first generation of a portable system for a hand grasp neuroprosthesis that is controlled by means of signals from natural sensors in the skin of the index finger. To reduce development time and costs, we based our design on readily available, standardised modules such as a 486DX100 compatible CPU, a data acquisition board, a flash disk storage unit, and a high-efficiency DC/DC switch-mode power supply. Additionally, we designed and built a telemeter to supply an implanted muscle stimulator with power and control data. The signal from the natural sensors was recorded with a cuff electrode implanted around the palmar digital nerve innervating the radial aspect of the index finger. For amplification of the recorded nerve signal, we added an external low-noise nerve signal amplifier. For pre-processing of the recorded nerve signal, an optimised band-pass filter was used. A data-recording unit allowed storage and off-line analysis of the stimulator command and the recorded nerve signal. The portable system was used by a tetraplegic volunteer to test the feasibility of including natural sensors in a hand grasp neuroprosthesis for activities of daily living. The flexibility of the presented system allows rapid prototyping of experimental FES hand grasp systems intended for portable use. # 2004 Published by Elsevier Ltd on behalf of IPEM. Keywords: Functional electrical stimulation; Portable system; Sensory feedback
1. Introduction Injury to the spinal cord at the cervical level results in loss of sensory and motor functions in both upper and lower extremities leading to tetraplegia. To restore basic hand function and enable individuals with tetraplegia to grasp and manipulate objects, several hand grasp systems based on functional electrical stimulation (FES) have been developed [1–4]. These systems usually control the grasp without any grasp-specific feedback information such as finger position or grasp force, so that the user has to rely on vision and experience to perform grasping safely. Artificial external sensors can be used to provide feedback information for such stimulation systems, but these sensors usually Corresponding author. Advanced Bionics Corporation, 25129 Rye Canyon Loop, Valencia, CA 91355. Tel.: +1-800-678-2575; fax: +1-661-362-1519. E-mail address: [email protected] (A. Inmann).
require frequent calibration and are often inaccurate and difficult for the average user to position on a daily basis [5,6]. At the Center for Sensory-Motor Interaction, a new technique of providing such systems with feedback information was developed by incorporating signals from natural sensors readily present in the skin of the index finger [7]. These signals can be recorded with a nerve cuff electrode around the palmar digital nerve that innervates the radial aspect of the index finger. Several studies show that nerve cuff electrodes provide an electrically and mechanically stable long-term interface to the nerve in both animals and humans [8–11]. The recorded nerve signal provides information about skin contact, force changes, and slips across the skin [7,12,13]. Our experience with natural sensory feedback was obtained with work on an experimental laboratory system based on a personal computer (PC) with additional data acquisition facilities [7,14]. To obtain more information for optimising such a complex system and
1350-4533/$ - see front matter # 2004 Published by Elsevier Ltd on behalf of IPEM. doi:10.1016/j.medengphy.2004.03.003
450
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
to gain knowledge about the performance of the system outside the laboratory, a portable system was required. A portable system would also allow tests and evaluations on a daily basis, while the user is performing activities of daily living such as eating with a fork or drinking from a cup. Further, we wanted to show that such a closed-loop hand grasp neuroprosthesis could function outside the laboratory without continuous maintenance. Portable FES systems are usually based on microcontrollers or microprocessors that have a more or less fixed design [2,15,16]. Implementing changes of the design requires a relatively large effort. To allow rapid prototyping and adaptability to different requirements, we based the first generation of our portable system on modules of the PC/104 embedded-PC standard. By using standardised hardware and software around the broadly supported PC architecture, development time and costs could be reduced significantly. A broad range of modules is commercially available and their standardised size (90 96 16 mm) is small enough to include a number of modules in a reasonably sized package. To adapt the portable system for our special requirements such as controlling an implanted muscle stimulator and recording a nerve signal, additional modules were custom-built and added to the system. This paper presents the implementation of a portable system for an FES hand grasp neuroprosthesis incorporating natural sensory feedback. We describe the different modules of the system, the recording and processing of the nerve signal, and the algorithm for including natural sensory feedback in the control of the electrically stimulated muscles. A data-recording unit allows long-term recording of important information about the neuroprosthesis during daily use. The flexibility of the chosen approach enables easy implementation of different types of control algorithms, nerve
signal recording and processing, communication to an implanted muscle stimulator, and customised user interfaces.
2. System hardware The hand grasp neuroprosthesis consisted of an implanted and an external system (Fig. 1). 2.1. Implanted system To restore grasp function in the left hand, a tetraplegic volunteer was instrumented with a commercial eight-channel muscle stimulator [17], which is part of the FREEHAND1 System (NeuroControl Corp., Cleveland, OH, USA). Eight epimysial electrodes were implanted on paralysed muscles in the hand and forearm to generate a functional grasp. In addition, a tripolar nerve cuff electrode was implanted on a branch of the palmar digital nerve deriving from the median nerve. The cuff electrode recorded activity from skin mechanoreceptors innervating the radial aspect of the index finger [7]. The wires of the cuff electrode were routed subcutaneously to an exit site on the volar side of the forearm where they were attached to a small external connector. Informed consent was obtained from the volunteer, and the local ethics committee approved the implantations. 2.2. External system The external system of the hand grasp neuroprosthesis was a combination of custom-built modules (stimulator-transmitter, nerve signal amplifier and filter, user interface) and readily available standard PC/ 104-modules (CPU, data recording, data acquisition) on small, stackable circuit boards. The main modules
Fig. 1. Block diagram of external and implanted system of the hand grasp neuroprosthesis.
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
were connected with the standardised PC/104 bus. The external system was fitted into a box with the dimensions 240 115 80 mm made of 1.5 mm thick aluminium. External appliances were custom-fit to the user. Two user control buttons were mounted on the headrest of the wheelchair, and the transmitter coil of the stimulator telemeter was attached to the user’s chest. The external system was mounted on the back of the electric wheelchair, and cables were routed to the user control buttons, to the batteries, and to the on/off button in the control unit on the armrest.
3. External system modules 3.1. Central processing unit (CPU) The CPU module had an onboard Intel 486DX100 compatible processor and standard PC interfaces such as communication ports (parallel/serial/keyboard) and storage unit interfaces (IDE hard drive, floppy disk drive). Additionally, the module provided a real time clock and a 2 MB solid-state disk memory. The parallel port was used for communication with the telemeter of the muscle stimulator. 3.2. Data acquisition We used a readily available eight-channel analogueto-digital converter module equipped with additional analogue output and digital input/output facilities. One analogue input channel was reserved for recording the nerve signal, leaving seven channels free for future applications. Two digital input channels and one digital output channel were used for the user interface (i.e. control buttons, power on/off). The digital input channels were equipped with optional Schmitt-trigger input circuitry for signal conditioning of analogue on/off signals.
451
reached, the card could be removed from the portable system and sent by regular mail to the laboratory. 3.4. Nerve signal recording and processing The nerve signal (ENG) recorded with the cuff electrode was a composition of nerve activity and noise pick-up from various sources. The noise sources were activity from nearby muscles (EMG), artefacts from nearby stimulation electrodes, electromagnetic line noise (50 Hz in Europe), and thermal noise. The amplitude of the recorded ENG was in the range of a few microvolts, while the EMG could be up to several orders of magnitude larger (a few millivolts). The stimulation artefacts were large enough to saturate the nerve signal pre-amplifier. Fig. 2 shows an example of the contribution of ENG and EMG to the spectrum of the recorded nerve signal. The ENG had a maximum around 1.4 kHz with most of the power concentrated between 800 Hz and 4 kHz. The EMG had a peak around 200 Hz and most of the power was below 800 Hz. EMG and stimulation artefacts were partially suppressed by a good fitting and tightly closed cuff, and the triphasic nature of the recording produced additional common mode rejection [8,18]. The nerve signal was obtained by means of an external nerve signal amplifier that was connected with percutaneous lead wires to the implanted cuff electrode. A low-noise amplifier based on an AD624BD (Analog Devices Inc., Norwood, MA, USA) with a gain of 100,000 was used. A step-up transformer at the input of the amplifier provided a zero-noise gain of 10, increasing the signal-to-noise ratio and matching the cuff electrode impedance (around 1 kX) to the high input impedance of the amplifier [19]. The amplified nerve signal was filtered with a combination of a fourth-order high-pass filter (cut-off
3.3. Data recording For long-term data recording, we implemented an exchangeable 64 MB flash memory card, fully compatible with the ATA PC-card standard. The memory card had the capability for zero-power data retention and had no movable parts, making it highly suitable for rugged environments. The control software, the control parameters, and the grasp templates were stored on the memory card. For later off-line analysis, the processed nerve signal and the stimulator command were recorded with a rate of 40 samples per second, resulting in 844 kB of data per hour. Additionally, start and stop times of the use of the neuroprosthesis were recorded. When the capacity of the memory card was
Fig. 2. Power spectral density (PSD, normalised to the maximum of the ENG above 1 kHz) of the signal recorded with the cuff electrode, when rubbing the skin of the index finger (ENG), when the subject generated some reflex-mediated muscle contractions (EMG), and when no activity on the skin was present (noise).
452
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
frequency 1 kHz) and a fourth-order low-pass filter (cut-off frequency 4 kHz) both with Butterworthcharacteristic. The filters were implemented with a MAX274 (Maxim Integrated Products, Sunnyvale, CA, USA). To obtain a maximum signal-to-noise ratio of the recorded nerve signal and eliminate EMG contamination, the cut-off frequencies of the filters were optimised following the procedure described by Christiansen and Jensen [20]. The filtered nerve signal was sampled at 10 kHz (14 bit, 5 V) and digitally full-wave rectified. It was then integrated in blocks of samples in the last 5 ms of each stimulation interval of 25 ms to exclude stimulation artefacts (bin-integration) [21]. The bin-integrated nerve signal (RBI-ENG) was further processed with a first-order auto-regressive filter to remove interference from slow changes in background activity and to enhance peaks in the signal [12]. The processing of the nerve signal was implemented in software to allow easy testing of different algorithms such as, for example, described by Upshaw and Sinkjær [22]. Mechanical events on the skin of the index finger were detected by comparing the processed nerve signal (P-ENG) to a fixed threshold. The threshold setting is relatively robust [14], so that setting the threshold to 20% higher than the maximum value of the background noise was sufficient for reliable detection of the events in the nerve signal. 3.5. Stimulator telemeter The muscles were stimulated at 20 Hz and modulation of the pulse duration between 0 and 200 ls was used to control the stimulation intensity. Individual muscles were controlled by a single command that was translated into specific stimulation intensities for each stimulation channel by using a pre-determined activation scheme (grasp template) [23]. The grasp was directly controlled by the command ranging from 0 (fully open hand) to 100 (fully closed hand) (see also Fig. 7). To supply the implanted stimulator with power and to transmit the control data, we built an external telemeter based on an AT90S8535 microcontroller (Atmel Corp., San Jose, CA, USA). The power stage of the telemeter consisted of a highly efficient class E amplifier (Fig. 3) that was optimised following the procedure described by Inmann [24]. Power was transmitted to the implanted stimulator with a 6.78 MHz carrier, and control data were transmitted by switching the carrier on and off. The CPU module updated the telemeter (via the parallel port) every 25 ms with the pulse duration and the current amplitude of each stimulation channel. The telemeter generated automatically all necessary control data for the implanted stimulator following a protocol modified from Smith et al. [17]. Four
Fig. 3. Class E amplifier to transmit power and control data to the implanted stimulator. Control data were transmitted by switching the 6.78 MHz carrier on and off.
current amplitudes (2, 8, 14, and 20 mA) and a pulse duration of 0–200 ls (with a resolution of 1 ls) could be selected separately for each of the eight stimulation channels (Fig. 4b). The stimulation pulses for the individual muscles were delivered sequentially at the beginning of the stimulation interval with a temporal spacing of 1 ms between channels (Fig. 4a).
3.6. User interface Two push buttons, mounted on the headrest of the wheelchair, were used to control the neuroprosthesis. Each button consisted of a force sensitive resistor (FSR) (Interlink Electronics Inc., Camarillo, CA, USA) with a disc of Styrofoam (1 cm diameter, 5 mm thick) glued onto it. The two buttons were mounted on a piece of epoxy (2 cm by 6 cm), spaced 4 cm apart, and covered with suede. Each FSR was part of a resistor network (voltage divider) supplied by a constant voltage of 5 V. With the two control buttons, the neuroprosthesis user could activate/deactivate the system and could control opening and closing of the hand by ramping the stimulator command up/down (see Fig. 5). When the user had adjusted the grasp with the control buttons, the system took over and the grasp force was automatically regulated to a level that was necessary to hold an object securely [25] (see also section 4). A piezoelectric audio transducer provided feedback to the user such as indication of system state, grasp pattern, and stimulation ramp up/down. The push buttons were used for control of the neuroprosthesis, because they provided a simple and intuitive interface for the user of the portable system. To fit the neuroprosthesis to the needs of other possible users, interfaces using voice [26], shoulder movement [27,28], wrist movement [29], respiration [30], EMG [31,32], or
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
453
Fig. 4. Control of the implanted stimulator. (a) Stimulation current (sequence 1) and pulse duration (sequence 2) were subsequently transmitted for each stimulation channel with a temporal spacing of 1 ms between channels. The recorded nerve signal was rectified and integrated in the last 5 ms of the stimulation interval (bin-interval), keeping it artefact-free. (b) On/off switching of the 6.78 MHz carrier to transmit the control data to the implanted stimulator, modified from Smith et al. [17].
EEG [33] could be implemented easily by providing the proper circuitry and software routines. 3.7. Power supply The portable system was supplied from the batteries of the user’s electric wheelchair. All required voltages for the system (+5 V, 12 V) were generated with a high-efficiency (>85 %) switch-mode DC/DC converter situated on a standard PC/104 module. The user could switch the portable system on and off with a button mounted on the control unit of the electric wheelchair.
4. Control algorithm The control software for the portable system was written in the programming language C/C++. The parameters of the control algorithm and the look-up tables for different grasp patterns were stored as separate ASCII text files. 4.1. System states The state diagram of the hand grasp neuroprosthesis is shown in Fig. 5. The user could turn the system on and off by pressing a dedicated button on the wheelchair control unit (WB). The two buttons on the headrest of the wheelchair (left button LB, right button RB) were used for selection of the grasp pattern and active control of the stimulator command with predefined linear ramps. When the user had determined the command and the respective button was released, the system entered the automatic command control state. 4.2. Automatic command control
Fig. 5. State diagram of the hand grasp neuroprosthesis. Arrows show state transitions initiated by the following actions: no button pressed (NoB), left button pressed (LB), right button pressed (RB), button on wheelchair control unit pressed (WB), stimulator command reached zero (Com ¼ 0).
The implementation of the automatic command control is illustrated in Fig. 6. The stimulator command was regulated automatically based on mechanical events on the skin derived from the processed nerve signal (P-ENG). A pre-defined minimum level of the
454
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
Fig. 6. Flow diagram of the automatic command control state. The algorithm was executed every 25 ms when the stimulator command was above a pre-defined minimum level (MinCom). Abbreviations: command (Com), stored command from previous stimulation interval (ComS), automatic decrease ramp (Dec), detection (Detect) of the P-ENG crossing a threshold (Th), ignoring further threshold crossings of the P-ENG (Ignore) during 0.5 s after detection (Ignore Period), reaction amplitude after detection (React), applying a null command (Zero fill).
command (MinCom), determined from the grasp template and defining a minimum grasp force (see also Fig. 7), was used to distinguish between two sub-states. When the user set the command to a level below
MinCom, the command was kept constant (no automatic regulation). In this case, the hand was either open or produced only a negligible grasp force. When the command was set above MinCom and no button
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
455
5. System usage
Fig. 7. Template for lateral grasp and produced grasp force. The muscles involved in the grasp were: extensor pollicis longus (EPL), extensor digitorum communis (EDC), extensor carpi ulnaris (ECU), flexor digitorum superficialis (FDS), flexor pollicis longus (FPL), and adductor pollicis (AdP).
was pressed, the command was automatically decreased with a slow linear ramp until it reached MinCom. Each time the P-ENG crossed a pre-set threshold (detection), decreasing the command was interrupted and the command was set to the maximum level (i.e., 100 on an arbitrary scale from 0 to 100) for the next stimulation interval. After this initial reaction on an increase in the P-ENG, the command was set to a higher level than before the detection, linearly depending on the amplitude of the P-ENG in the first sample after detection [7]. Further crossings of the threshold by the P-ENG were ignored during a period of 0.5 s after detection (Ignore Period) preventing a wind-up of the command. Pressing any control button in the automatic command control state could override the automatic regulation of the command and either increase or decrease the command depending on which button was pressed. The clock frequency of the system was 40 Hz. Hence, the P-ENG was calculated every 25 ms. Null commands, not resulting in muscle stimulation, were applied in every other stimulation interval to keep the effective stimulation frequency at 20 Hz. A null command was omitted when an event was detected in the P-ENG (i.e. crossing the threshold) permitting two consecutive stimulation commands with an inter-pulse interval of 25 ms.
The principal application for the portable system was to control hand function with an implanted FREEHAND1 stimulator in individuals with C5/C6 tetraplegia. A lateral grasp template (Fig. 7), determined with the laboratory-based system following the procedure described by Kilgore et al. [23], and the control algorithm were stored on the memory card and transferred to the portable system. The system was then set up at the user’s home, and the first day of use was supervised to accustom the user to the new system. The evaluation of the hand grasp neuroprosthesis incorporating natural sensory feedback was done with a tetraplegic volunteer, whose palmar grasp was not strong mainly due to an unstable metacarpophalangeal joint of the thumb. He hardly used the palmar grasp during daily activities. Hence, this grasp was not used in the present study. During the evaluation period, the stimulator command, the P-ENG, and the time of usage were stored on the memory card. When the memory card was full, it was sent by regular mail to our laboratory for offline data analysis. Additionally, the user’s caretaker filled out a form reporting the activity for which the neuroprosthesis was used. In this way, problems, wishes, possible changes, and improvements could be noted. The evaluation period was nine consecutive days (Fig. 8a). Most of the activities were concentrated in the second half of a day. Long activity periods mainly occurred for dinner (between 18:00 and 22:00), and short activity periods mainly occurred for picking up objects. Fig. 8b shows sample data from one activity (dinner) of day four of the evaluation period. The variation of the command during the activity indicated a succession of active and inactive periods during the task. Fig. 8c shows a two-minute sequence of the activity shown in Fig. 8b, illustrating the details of the automatic control of the command. Peaks in the P-ENG indicated mechanical activity on the skin of the index finger such as force changes or slips of the held object. The stimulator command was automatically regulated between the minimum (65) and the maximum level (100) depending on detected events in the P-ENG (i.e. crossing the threshold). A high activity in the P-ENG led to a high average of the stimulator command. This produced a sufficiently high grasp force to counteract the increased mechanical activity and to secure the held object in the grasp. A detailed description of the user activity and the performance of the neuroprosthesis is out of the scope of this paper, but will be summarised in future reports that are currently under preparation.
456
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
Fig. 8. Use of the hand grasp neuroprosthesis. (a) User activity over nine consecutive days. Black bars indicate when the neuroprosthesis was turned on. (b) One selected activity (dinner) of day four. Shown are the processed nerve signal (P-ENG, normalised to maximum amplitude) and the stimulator command (Com). (c) Two-minute sequence of the activity shown in (b).
6. Discussion and conclusion A rapid prototyping design approach was used to transfer the external system of an FES hand grasp neuroprosthesis incorporating natural sensory feedback from an experimental set-up to a portable system that was used at the user’s home. We assembled the hardware of the system, programmed the required software, and made the whole system functional. System set-up, grasp templates, and parameters of the control algorithm were determined with our laboratory-based system, transferred to the portable system, and tested in functional tasks of daily living both in the laboratory and at the user’s home. The presented first generation of a portable system has been a very useful tool to analyse the performance of the FES hand grasp neuroprosthesis incorporating natural sensory feedback. Automatic regulation of the stimulator command based on mechanical activity on the skin of the index finger showed that less stimulator command was used on average compared to a system without feedback [25]. In a hand grasp neuroprosthesis
without feedback, the stimulator command is usually held constant and locked at the maximum level [34]. The control of the neuroprosthesis with push buttons that were permanently mounted on the headrest of the wheelchair was highly accepted by the user. Donning and doffing of the system were only comprised of attaching the transmitter coil and connecting the cuff electrode. The daily task of attaching and positioning the shoulder transducer of the FREEHAND1 System was not needed. The portable system was powered from the batteries of the wheelchair. However, the power consumption of the portable system did not significantly affect the operating range of the wheelchair. An important step to decrease hygienic requirements, imposed by the use of percutaneous lead wires from the cuff electrode, will be the use of an implanted, externally powered nerve signal amplifier. Here, the recorded nerve signal is transmitted to the external system via a radio-frequency link [35]. The implanted nerve signal amplifier has been successfully used in a drop-foot patient [36] and is currently under test in a hand grasp neuroprosthesis user.
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
We will further analyse the performance of the FES hand grasp neuroprosthesis at the user’s home by using the signals stored with the data-recording unit. The main question to be answered is whether the idea of natural sensory feedback for hand grasp control is beneficial and acceptable for the user in everyday tasks such as eating with a fork or drinking from a cup. Although the portable system is still a prototype and the power consumption is relatively high for a practical battery-driven system, it will help us to answer this question. The presented system can be easily adapted for different purposes and requirements due to its open architecture and capability to implement various components such as data acquisition modules and customised user interfaces.
[11]
[12]
[13]
[14]
[15]
[16]
Acknowledgements [17]
The authors thank R. J. Wilkins for programming the microcontroller of the stimulator telemeter. This work was supported by the Danish National Research Foundation and the European Union under the Training and Mobility of Researchers programme NEUROS.
[18]
[19]
References [1] Buckett JR, Peckham PH, Thrope GB, Braswell SD, Keith MW. A flexible, portable system for neuromuscular stimulation in the paralyzed upper extremity. IEEE Trans Biomed Eng 1988;35(11): 897–904. [2] Handa Y, Ohkubo K, Hoshimiya N. A portable multi-channel FES system for restoration of motor function of the paralyzed extremities. Automedica 1989;11:221–31. [3] Nathan RH, Ohry A. Upper limb functions regained in quadriplegia: a hybrid computerized neuromuscular stimulation system. Arch Phys Med Rehabil 1990;71(6):415–21. [4] Prochazka A, Gauthier M, Wieler M, Kenwell Z. The bionic glove: an electrical stimulator garment that provides controlled grasp and hand opening in quadriplegia. Arch Phys Med Rehabil 1997;78(6):608–14. [5] Crago PE, Chizeck HJ, Neuman MR, Hambrecht FT. Sensors for use with functional neuromuscular stimulation. IEEE Trans Biomed Eng 1986;33(1):256–68. [6] Webster JG. Artificial sensors suitable for closed-loop control of FNS. In: Stein RB, Peckham PH, Popovic DB, editors. Neural prostheses: replacing motor function after disease or disability. New York, NY: Oxford University Press; 1992, p. 88–98. [7] Haugland M, Lickel A, Haase J, Sinkjær T. Control of FES thumb forces using slip information obtained from cutaneous electroneurogram in quadriplegic man. IEEE Trans Rehab Eng 1999;7(2):215–27. [8] Stein RB, Charles D, Davis L, Jhamandas J, Mannard A, Nichols TR. Principles underlying new methods for chronic neural recording. Can J Neurol Sci 1975;2(3):235–44. [9] Stein RB, Nichols TR, Jhamandas J, Davis L, Charles D. Stable long-term recordings from cat peripheral nerves. Brain Res 1977;128(1):21–38. [10] Hoffer JA. Techniques to study spinal-cord, peripheral nerve, and muscle activity in freely moving animals. In: Boulton AA,
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
457
Baker GB, Vanderwolf CH, editors. Neurophysiological techniques: applications to neural systems. Clifton, NJ: Humana Press; 1990, p. 65–145. Popovic D, Stein RB, Jovanovic K, Dai R, Kostov A, Armstrong WW. Sensory nerve recording for closed-loop control to restore motor functions. IEEE Trans Biomed Eng 1993;40(10):1024–31. Haugland MK, Hoffer JA. Slip information provided by nerve cuff signals: application in closed-loop control of functional electrical stimulation. IEEE Trans Rehab Eng 1994;2(1):29–36. Haugland MK, Hoffer JA, Sinkjær T. Skin contact force information in sensory nerve signals recorded by implanted cuff electrodes. IEEE Trans Rehab Eng 1994;2(1):18–28. Lickel A. Restoration of lateral hand grasp in a tetraplegic patient applying natural sensory feedback. PhD dissertation. Center for Sensory-Motor Interaction, Aalborg University, DK9220 Aalborg, Denmark,1998. Crook SE, Chappell PH. A portable system for closed loop control of the paralysed hand using functional electrical stimulation. Med Eng Phys 1998;20(1):70–6. Ilic M, Vasiljevic D, Popovic D. A programmable electronic stimulator for FES systems. IEEE Trans Biomed Eng 1994;2(4): 234–9. Smith B, Peckham PH, Keith MW, Roscoe DD. An externally powered, multichannel, implantable stimulator for versatile control of paralyzed muscle. IEEE Trans Biomed Eng 1987;34(7):499–508. Struijk JJ, Thomsen M. Tripolar nerve cuff recording: stimulus artifact, EMG, and the recorded nerve signal. Proceedings of the IEEE/EMBS 17th Annual International Conference 1995; Montreal, Canada. 1995, p. 1105–6. Nikolic ZM, Popovic DB, Stein RB, Kenwell Z. Instrumentation for ENG and EMG recordings in FES systems. IEEE Trans Biomed Eng 1994;41(7):703–6. Christiansen TG, Jensen BV. Optimised digital filtering of ENG signals recorded with a cuff electrode. MSc thesis. Center for Sensory-Motor Interaction, Aalborg University, 1999. Haugland MK, Hoffer JA. Artifact-free sensory nerve signals obtained from cuff electrodes during functional electrical stimulation of nearby muscles. IEEE Trans Rehab Eng 1994;2(1):37–9. Upshaw B, Sinkjær T. Digital signal processing algorithms for the detection of afferent nerve activity recorded from cuff electrodes. IEEE Trans Rehabil Eng 1998;6(2):172–81. Kilgore KL, Peckham PH, Thrope GB, Keith MW, GallaherStone KA. Synthesis of hand grasp using functional neuromuscular stimulation. IEEE Trans Biomed Eng 1989;36(7): 761–70. Inmann A. Optimisation of an RF-transmitter used for transcutaneous transmission of energy and digital data for medical implants (in german). MSc thesis. Department of Communications and RF-Engineering, Technical University of Vienna, Austria and Department of Biomedical Engineering and Physics, University of Vienna, Austria, 1997. Inmann A, Haugland M. Closed-loop control of an FES system incorporating natural sensory feedback used for restoration of hand grasp in tetraplegics. Proceedings of the IFESS 4th Annual Conference 1999; Sendai, Japan. 1999, p. 77–80. Handa Y, Handa T, Nakatsuchi Y, Yagi R, Hoshimiya N. A voice-controlled functional electrical stimulation system for the paralyzed hand. Jap J Rehabil Med 1985;23(5):292–8. Bayer DM, Lord RH, Swanker JW, Mortimer JT. A two-axis shoulder position transducer for control of orthotic/prosthetic devices. IEEE Trans IECI 1972;IECI-19(2):61–4. Johnson MW, Peckham PH. Evaluation of shoulder movement as a command control source. IEEE Trans Biomed Eng 1990;37(9):876–85. Hart RL, Kilgore KL, Peckham PH. A comparison between control methods for implanted FES hand-grasp systems. IEEE Trans Rehab Eng 1998;6(2):208–18.
458
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
[30] Hoshimiya N, Naito A, Yajima M, Handa Y. A multichannel FES system for the restoration of motor functions in high spinal cord injury patients: a respiration-controlled system for multijoint upper extremity. IEEE Trans Biomed Eng 1989;36(7):754–60. [31] Solomonow M, Baratta R, Shoji H, D’Ambrosia RD. The myoelectric signal of electrically stimulated muscle during recruitment: an inherent feedback parameter for a closed-loop control scheme. IEEE Trans Biomed Eng 1986;33(8):735–45. [32] Scott TR, Peckham PH, Kilgore KL. Tri-state myoelectric control of bilateral upper extremity neuroprostheses for tetraplegic individuals. IEEE Trans Rehabil Eng 1996;4(4):251–63.
[33] Lauer RT, Peckham PH, Kilgore KL. EEG-based control of a hand grasp neuroprosthesis. Neuroreport 1999;10(8):1767–71. [34] Burelbach JC, Crago PE. Instrumented assessment of FNS hand control during specific manipulation tasks. IEEE Trans Rehab Eng 1994;2(3):165–76. [35] Zhou L, Munih M, Haugland MK, Perkins TA, Donaldson NN. An implantable telemeter for E.N.G. signals. 6th Vienna International Workshop FES. Vienna, Austria; 1998. p. 327–30. [36] Hansen M, Haugland M, Sinkjær T, Donaldson N. Real time foot drop correction using machine learning and natural sensors. Neuromodulation 2002;5(1):41–53.
Implementation of natural sensory feedback in a portable control system for a hand grasp neuroprosthesis Andreas Inmann , Morten Haugland Center for Sensory-Motor Interaction, Aalborg University, Aalborg, Denmark Received 26 June 2003; received in revised form 22 February 2004; accepted 9 March 2004
Abstract This paper presents the design and implementation of the first generation of a portable system for a hand grasp neuroprosthesis that is controlled by means of signals from natural sensors in the skin of the index finger. To reduce development time and costs, we based our design on readily available, standardised modules such as a 486DX100 compatible CPU, a data acquisition board, a flash disk storage unit, and a high-efficiency DC/DC switch-mode power supply. Additionally, we designed and built a telemeter to supply an implanted muscle stimulator with power and control data. The signal from the natural sensors was recorded with a cuff electrode implanted around the palmar digital nerve innervating the radial aspect of the index finger. For amplification of the recorded nerve signal, we added an external low-noise nerve signal amplifier. For pre-processing of the recorded nerve signal, an optimised band-pass filter was used. A data-recording unit allowed storage and off-line analysis of the stimulator command and the recorded nerve signal. The portable system was used by a tetraplegic volunteer to test the feasibility of including natural sensors in a hand grasp neuroprosthesis for activities of daily living. The flexibility of the presented system allows rapid prototyping of experimental FES hand grasp systems intended for portable use. # 2004 Published by Elsevier Ltd on behalf of IPEM. Keywords: Functional electrical stimulation; Portable system; Sensory feedback
1. Introduction Injury to the spinal cord at the cervical level results in loss of sensory and motor functions in both upper and lower extremities leading to tetraplegia. To restore basic hand function and enable individuals with tetraplegia to grasp and manipulate objects, several hand grasp systems based on functional electrical stimulation (FES) have been developed [1–4]. These systems usually control the grasp without any grasp-specific feedback information such as finger position or grasp force, so that the user has to rely on vision and experience to perform grasping safely. Artificial external sensors can be used to provide feedback information for such stimulation systems, but these sensors usually Corresponding author. Advanced Bionics Corporation, 25129 Rye Canyon Loop, Valencia, CA 91355. Tel.: +1-800-678-2575; fax: +1-661-362-1519. E-mail address: [email protected] (A. Inmann).
require frequent calibration and are often inaccurate and difficult for the average user to position on a daily basis [5,6]. At the Center for Sensory-Motor Interaction, a new technique of providing such systems with feedback information was developed by incorporating signals from natural sensors readily present in the skin of the index finger [7]. These signals can be recorded with a nerve cuff electrode around the palmar digital nerve that innervates the radial aspect of the index finger. Several studies show that nerve cuff electrodes provide an electrically and mechanically stable long-term interface to the nerve in both animals and humans [8–11]. The recorded nerve signal provides information about skin contact, force changes, and slips across the skin [7,12,13]. Our experience with natural sensory feedback was obtained with work on an experimental laboratory system based on a personal computer (PC) with additional data acquisition facilities [7,14]. To obtain more information for optimising such a complex system and
1350-4533/$ - see front matter # 2004 Published by Elsevier Ltd on behalf of IPEM. doi:10.1016/j.medengphy.2004.03.003
450
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
to gain knowledge about the performance of the system outside the laboratory, a portable system was required. A portable system would also allow tests and evaluations on a daily basis, while the user is performing activities of daily living such as eating with a fork or drinking from a cup. Further, we wanted to show that such a closed-loop hand grasp neuroprosthesis could function outside the laboratory without continuous maintenance. Portable FES systems are usually based on microcontrollers or microprocessors that have a more or less fixed design [2,15,16]. Implementing changes of the design requires a relatively large effort. To allow rapid prototyping and adaptability to different requirements, we based the first generation of our portable system on modules of the PC/104 embedded-PC standard. By using standardised hardware and software around the broadly supported PC architecture, development time and costs could be reduced significantly. A broad range of modules is commercially available and their standardised size (90 96 16 mm) is small enough to include a number of modules in a reasonably sized package. To adapt the portable system for our special requirements such as controlling an implanted muscle stimulator and recording a nerve signal, additional modules were custom-built and added to the system. This paper presents the implementation of a portable system for an FES hand grasp neuroprosthesis incorporating natural sensory feedback. We describe the different modules of the system, the recording and processing of the nerve signal, and the algorithm for including natural sensory feedback in the control of the electrically stimulated muscles. A data-recording unit allows long-term recording of important information about the neuroprosthesis during daily use. The flexibility of the chosen approach enables easy implementation of different types of control algorithms, nerve
signal recording and processing, communication to an implanted muscle stimulator, and customised user interfaces.
2. System hardware The hand grasp neuroprosthesis consisted of an implanted and an external system (Fig. 1). 2.1. Implanted system To restore grasp function in the left hand, a tetraplegic volunteer was instrumented with a commercial eight-channel muscle stimulator [17], which is part of the FREEHAND1 System (NeuroControl Corp., Cleveland, OH, USA). Eight epimysial electrodes were implanted on paralysed muscles in the hand and forearm to generate a functional grasp. In addition, a tripolar nerve cuff electrode was implanted on a branch of the palmar digital nerve deriving from the median nerve. The cuff electrode recorded activity from skin mechanoreceptors innervating the radial aspect of the index finger [7]. The wires of the cuff electrode were routed subcutaneously to an exit site on the volar side of the forearm where they were attached to a small external connector. Informed consent was obtained from the volunteer, and the local ethics committee approved the implantations. 2.2. External system The external system of the hand grasp neuroprosthesis was a combination of custom-built modules (stimulator-transmitter, nerve signal amplifier and filter, user interface) and readily available standard PC/ 104-modules (CPU, data recording, data acquisition) on small, stackable circuit boards. The main modules
Fig. 1. Block diagram of external and implanted system of the hand grasp neuroprosthesis.
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
were connected with the standardised PC/104 bus. The external system was fitted into a box with the dimensions 240 115 80 mm made of 1.5 mm thick aluminium. External appliances were custom-fit to the user. Two user control buttons were mounted on the headrest of the wheelchair, and the transmitter coil of the stimulator telemeter was attached to the user’s chest. The external system was mounted on the back of the electric wheelchair, and cables were routed to the user control buttons, to the batteries, and to the on/off button in the control unit on the armrest.
3. External system modules 3.1. Central processing unit (CPU) The CPU module had an onboard Intel 486DX100 compatible processor and standard PC interfaces such as communication ports (parallel/serial/keyboard) and storage unit interfaces (IDE hard drive, floppy disk drive). Additionally, the module provided a real time clock and a 2 MB solid-state disk memory. The parallel port was used for communication with the telemeter of the muscle stimulator. 3.2. Data acquisition We used a readily available eight-channel analogueto-digital converter module equipped with additional analogue output and digital input/output facilities. One analogue input channel was reserved for recording the nerve signal, leaving seven channels free for future applications. Two digital input channels and one digital output channel were used for the user interface (i.e. control buttons, power on/off). The digital input channels were equipped with optional Schmitt-trigger input circuitry for signal conditioning of analogue on/off signals.
451
reached, the card could be removed from the portable system and sent by regular mail to the laboratory. 3.4. Nerve signal recording and processing The nerve signal (ENG) recorded with the cuff electrode was a composition of nerve activity and noise pick-up from various sources. The noise sources were activity from nearby muscles (EMG), artefacts from nearby stimulation electrodes, electromagnetic line noise (50 Hz in Europe), and thermal noise. The amplitude of the recorded ENG was in the range of a few microvolts, while the EMG could be up to several orders of magnitude larger (a few millivolts). The stimulation artefacts were large enough to saturate the nerve signal pre-amplifier. Fig. 2 shows an example of the contribution of ENG and EMG to the spectrum of the recorded nerve signal. The ENG had a maximum around 1.4 kHz with most of the power concentrated between 800 Hz and 4 kHz. The EMG had a peak around 200 Hz and most of the power was below 800 Hz. EMG and stimulation artefacts were partially suppressed by a good fitting and tightly closed cuff, and the triphasic nature of the recording produced additional common mode rejection [8,18]. The nerve signal was obtained by means of an external nerve signal amplifier that was connected with percutaneous lead wires to the implanted cuff electrode. A low-noise amplifier based on an AD624BD (Analog Devices Inc., Norwood, MA, USA) with a gain of 100,000 was used. A step-up transformer at the input of the amplifier provided a zero-noise gain of 10, increasing the signal-to-noise ratio and matching the cuff electrode impedance (around 1 kX) to the high input impedance of the amplifier [19]. The amplified nerve signal was filtered with a combination of a fourth-order high-pass filter (cut-off
3.3. Data recording For long-term data recording, we implemented an exchangeable 64 MB flash memory card, fully compatible with the ATA PC-card standard. The memory card had the capability for zero-power data retention and had no movable parts, making it highly suitable for rugged environments. The control software, the control parameters, and the grasp templates were stored on the memory card. For later off-line analysis, the processed nerve signal and the stimulator command were recorded with a rate of 40 samples per second, resulting in 844 kB of data per hour. Additionally, start and stop times of the use of the neuroprosthesis were recorded. When the capacity of the memory card was
Fig. 2. Power spectral density (PSD, normalised to the maximum of the ENG above 1 kHz) of the signal recorded with the cuff electrode, when rubbing the skin of the index finger (ENG), when the subject generated some reflex-mediated muscle contractions (EMG), and when no activity on the skin was present (noise).
452
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
frequency 1 kHz) and a fourth-order low-pass filter (cut-off frequency 4 kHz) both with Butterworthcharacteristic. The filters were implemented with a MAX274 (Maxim Integrated Products, Sunnyvale, CA, USA). To obtain a maximum signal-to-noise ratio of the recorded nerve signal and eliminate EMG contamination, the cut-off frequencies of the filters were optimised following the procedure described by Christiansen and Jensen [20]. The filtered nerve signal was sampled at 10 kHz (14 bit, 5 V) and digitally full-wave rectified. It was then integrated in blocks of samples in the last 5 ms of each stimulation interval of 25 ms to exclude stimulation artefacts (bin-integration) [21]. The bin-integrated nerve signal (RBI-ENG) was further processed with a first-order auto-regressive filter to remove interference from slow changes in background activity and to enhance peaks in the signal [12]. The processing of the nerve signal was implemented in software to allow easy testing of different algorithms such as, for example, described by Upshaw and Sinkjær [22]. Mechanical events on the skin of the index finger were detected by comparing the processed nerve signal (P-ENG) to a fixed threshold. The threshold setting is relatively robust [14], so that setting the threshold to 20% higher than the maximum value of the background noise was sufficient for reliable detection of the events in the nerve signal. 3.5. Stimulator telemeter The muscles were stimulated at 20 Hz and modulation of the pulse duration between 0 and 200 ls was used to control the stimulation intensity. Individual muscles were controlled by a single command that was translated into specific stimulation intensities for each stimulation channel by using a pre-determined activation scheme (grasp template) [23]. The grasp was directly controlled by the command ranging from 0 (fully open hand) to 100 (fully closed hand) (see also Fig. 7). To supply the implanted stimulator with power and to transmit the control data, we built an external telemeter based on an AT90S8535 microcontroller (Atmel Corp., San Jose, CA, USA). The power stage of the telemeter consisted of a highly efficient class E amplifier (Fig. 3) that was optimised following the procedure described by Inmann [24]. Power was transmitted to the implanted stimulator with a 6.78 MHz carrier, and control data were transmitted by switching the carrier on and off. The CPU module updated the telemeter (via the parallel port) every 25 ms with the pulse duration and the current amplitude of each stimulation channel. The telemeter generated automatically all necessary control data for the implanted stimulator following a protocol modified from Smith et al. [17]. Four
Fig. 3. Class E amplifier to transmit power and control data to the implanted stimulator. Control data were transmitted by switching the 6.78 MHz carrier on and off.
current amplitudes (2, 8, 14, and 20 mA) and a pulse duration of 0–200 ls (with a resolution of 1 ls) could be selected separately for each of the eight stimulation channels (Fig. 4b). The stimulation pulses for the individual muscles were delivered sequentially at the beginning of the stimulation interval with a temporal spacing of 1 ms between channels (Fig. 4a).
3.6. User interface Two push buttons, mounted on the headrest of the wheelchair, were used to control the neuroprosthesis. Each button consisted of a force sensitive resistor (FSR) (Interlink Electronics Inc., Camarillo, CA, USA) with a disc of Styrofoam (1 cm diameter, 5 mm thick) glued onto it. The two buttons were mounted on a piece of epoxy (2 cm by 6 cm), spaced 4 cm apart, and covered with suede. Each FSR was part of a resistor network (voltage divider) supplied by a constant voltage of 5 V. With the two control buttons, the neuroprosthesis user could activate/deactivate the system and could control opening and closing of the hand by ramping the stimulator command up/down (see Fig. 5). When the user had adjusted the grasp with the control buttons, the system took over and the grasp force was automatically regulated to a level that was necessary to hold an object securely [25] (see also section 4). A piezoelectric audio transducer provided feedback to the user such as indication of system state, grasp pattern, and stimulation ramp up/down. The push buttons were used for control of the neuroprosthesis, because they provided a simple and intuitive interface for the user of the portable system. To fit the neuroprosthesis to the needs of other possible users, interfaces using voice [26], shoulder movement [27,28], wrist movement [29], respiration [30], EMG [31,32], or
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
453
Fig. 4. Control of the implanted stimulator. (a) Stimulation current (sequence 1) and pulse duration (sequence 2) were subsequently transmitted for each stimulation channel with a temporal spacing of 1 ms between channels. The recorded nerve signal was rectified and integrated in the last 5 ms of the stimulation interval (bin-interval), keeping it artefact-free. (b) On/off switching of the 6.78 MHz carrier to transmit the control data to the implanted stimulator, modified from Smith et al. [17].
EEG [33] could be implemented easily by providing the proper circuitry and software routines. 3.7. Power supply The portable system was supplied from the batteries of the user’s electric wheelchair. All required voltages for the system (+5 V, 12 V) were generated with a high-efficiency (>85 %) switch-mode DC/DC converter situated on a standard PC/104 module. The user could switch the portable system on and off with a button mounted on the control unit of the electric wheelchair.
4. Control algorithm The control software for the portable system was written in the programming language C/C++. The parameters of the control algorithm and the look-up tables for different grasp patterns were stored as separate ASCII text files. 4.1. System states The state diagram of the hand grasp neuroprosthesis is shown in Fig. 5. The user could turn the system on and off by pressing a dedicated button on the wheelchair control unit (WB). The two buttons on the headrest of the wheelchair (left button LB, right button RB) were used for selection of the grasp pattern and active control of the stimulator command with predefined linear ramps. When the user had determined the command and the respective button was released, the system entered the automatic command control state. 4.2. Automatic command control
Fig. 5. State diagram of the hand grasp neuroprosthesis. Arrows show state transitions initiated by the following actions: no button pressed (NoB), left button pressed (LB), right button pressed (RB), button on wheelchair control unit pressed (WB), stimulator command reached zero (Com ¼ 0).
The implementation of the automatic command control is illustrated in Fig. 6. The stimulator command was regulated automatically based on mechanical events on the skin derived from the processed nerve signal (P-ENG). A pre-defined minimum level of the
454
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
Fig. 6. Flow diagram of the automatic command control state. The algorithm was executed every 25 ms when the stimulator command was above a pre-defined minimum level (MinCom). Abbreviations: command (Com), stored command from previous stimulation interval (ComS), automatic decrease ramp (Dec), detection (Detect) of the P-ENG crossing a threshold (Th), ignoring further threshold crossings of the P-ENG (Ignore) during 0.5 s after detection (Ignore Period), reaction amplitude after detection (React), applying a null command (Zero fill).
command (MinCom), determined from the grasp template and defining a minimum grasp force (see also Fig. 7), was used to distinguish between two sub-states. When the user set the command to a level below
MinCom, the command was kept constant (no automatic regulation). In this case, the hand was either open or produced only a negligible grasp force. When the command was set above MinCom and no button
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
455
5. System usage
Fig. 7. Template for lateral grasp and produced grasp force. The muscles involved in the grasp were: extensor pollicis longus (EPL), extensor digitorum communis (EDC), extensor carpi ulnaris (ECU), flexor digitorum superficialis (FDS), flexor pollicis longus (FPL), and adductor pollicis (AdP).
was pressed, the command was automatically decreased with a slow linear ramp until it reached MinCom. Each time the P-ENG crossed a pre-set threshold (detection), decreasing the command was interrupted and the command was set to the maximum level (i.e., 100 on an arbitrary scale from 0 to 100) for the next stimulation interval. After this initial reaction on an increase in the P-ENG, the command was set to a higher level than before the detection, linearly depending on the amplitude of the P-ENG in the first sample after detection [7]. Further crossings of the threshold by the P-ENG were ignored during a period of 0.5 s after detection (Ignore Period) preventing a wind-up of the command. Pressing any control button in the automatic command control state could override the automatic regulation of the command and either increase or decrease the command depending on which button was pressed. The clock frequency of the system was 40 Hz. Hence, the P-ENG was calculated every 25 ms. Null commands, not resulting in muscle stimulation, were applied in every other stimulation interval to keep the effective stimulation frequency at 20 Hz. A null command was omitted when an event was detected in the P-ENG (i.e. crossing the threshold) permitting two consecutive stimulation commands with an inter-pulse interval of 25 ms.
The principal application for the portable system was to control hand function with an implanted FREEHAND1 stimulator in individuals with C5/C6 tetraplegia. A lateral grasp template (Fig. 7), determined with the laboratory-based system following the procedure described by Kilgore et al. [23], and the control algorithm were stored on the memory card and transferred to the portable system. The system was then set up at the user’s home, and the first day of use was supervised to accustom the user to the new system. The evaluation of the hand grasp neuroprosthesis incorporating natural sensory feedback was done with a tetraplegic volunteer, whose palmar grasp was not strong mainly due to an unstable metacarpophalangeal joint of the thumb. He hardly used the palmar grasp during daily activities. Hence, this grasp was not used in the present study. During the evaluation period, the stimulator command, the P-ENG, and the time of usage were stored on the memory card. When the memory card was full, it was sent by regular mail to our laboratory for offline data analysis. Additionally, the user’s caretaker filled out a form reporting the activity for which the neuroprosthesis was used. In this way, problems, wishes, possible changes, and improvements could be noted. The evaluation period was nine consecutive days (Fig. 8a). Most of the activities were concentrated in the second half of a day. Long activity periods mainly occurred for dinner (between 18:00 and 22:00), and short activity periods mainly occurred for picking up objects. Fig. 8b shows sample data from one activity (dinner) of day four of the evaluation period. The variation of the command during the activity indicated a succession of active and inactive periods during the task. Fig. 8c shows a two-minute sequence of the activity shown in Fig. 8b, illustrating the details of the automatic control of the command. Peaks in the P-ENG indicated mechanical activity on the skin of the index finger such as force changes or slips of the held object. The stimulator command was automatically regulated between the minimum (65) and the maximum level (100) depending on detected events in the P-ENG (i.e. crossing the threshold). A high activity in the P-ENG led to a high average of the stimulator command. This produced a sufficiently high grasp force to counteract the increased mechanical activity and to secure the held object in the grasp. A detailed description of the user activity and the performance of the neuroprosthesis is out of the scope of this paper, but will be summarised in future reports that are currently under preparation.
456
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
Fig. 8. Use of the hand grasp neuroprosthesis. (a) User activity over nine consecutive days. Black bars indicate when the neuroprosthesis was turned on. (b) One selected activity (dinner) of day four. Shown are the processed nerve signal (P-ENG, normalised to maximum amplitude) and the stimulator command (Com). (c) Two-minute sequence of the activity shown in (b).
6. Discussion and conclusion A rapid prototyping design approach was used to transfer the external system of an FES hand grasp neuroprosthesis incorporating natural sensory feedback from an experimental set-up to a portable system that was used at the user’s home. We assembled the hardware of the system, programmed the required software, and made the whole system functional. System set-up, grasp templates, and parameters of the control algorithm were determined with our laboratory-based system, transferred to the portable system, and tested in functional tasks of daily living both in the laboratory and at the user’s home. The presented first generation of a portable system has been a very useful tool to analyse the performance of the FES hand grasp neuroprosthesis incorporating natural sensory feedback. Automatic regulation of the stimulator command based on mechanical activity on the skin of the index finger showed that less stimulator command was used on average compared to a system without feedback [25]. In a hand grasp neuroprosthesis
without feedback, the stimulator command is usually held constant and locked at the maximum level [34]. The control of the neuroprosthesis with push buttons that were permanently mounted on the headrest of the wheelchair was highly accepted by the user. Donning and doffing of the system were only comprised of attaching the transmitter coil and connecting the cuff electrode. The daily task of attaching and positioning the shoulder transducer of the FREEHAND1 System was not needed. The portable system was powered from the batteries of the wheelchair. However, the power consumption of the portable system did not significantly affect the operating range of the wheelchair. An important step to decrease hygienic requirements, imposed by the use of percutaneous lead wires from the cuff electrode, will be the use of an implanted, externally powered nerve signal amplifier. Here, the recorded nerve signal is transmitted to the external system via a radio-frequency link [35]. The implanted nerve signal amplifier has been successfully used in a drop-foot patient [36] and is currently under test in a hand grasp neuroprosthesis user.
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
We will further analyse the performance of the FES hand grasp neuroprosthesis at the user’s home by using the signals stored with the data-recording unit. The main question to be answered is whether the idea of natural sensory feedback for hand grasp control is beneficial and acceptable for the user in everyday tasks such as eating with a fork or drinking from a cup. Although the portable system is still a prototype and the power consumption is relatively high for a practical battery-driven system, it will help us to answer this question. The presented system can be easily adapted for different purposes and requirements due to its open architecture and capability to implement various components such as data acquisition modules and customised user interfaces.
[11]
[12]
[13]
[14]
[15]
[16]
Acknowledgements [17]
The authors thank R. J. Wilkins for programming the microcontroller of the stimulator telemeter. This work was supported by the Danish National Research Foundation and the European Union under the Training and Mobility of Researchers programme NEUROS.
[18]
[19]
References [1] Buckett JR, Peckham PH, Thrope GB, Braswell SD, Keith MW. A flexible, portable system for neuromuscular stimulation in the paralyzed upper extremity. IEEE Trans Biomed Eng 1988;35(11): 897–904. [2] Handa Y, Ohkubo K, Hoshimiya N. A portable multi-channel FES system for restoration of motor function of the paralyzed extremities. Automedica 1989;11:221–31. [3] Nathan RH, Ohry A. Upper limb functions regained in quadriplegia: a hybrid computerized neuromuscular stimulation system. Arch Phys Med Rehabil 1990;71(6):415–21. [4] Prochazka A, Gauthier M, Wieler M, Kenwell Z. The bionic glove: an electrical stimulator garment that provides controlled grasp and hand opening in quadriplegia. Arch Phys Med Rehabil 1997;78(6):608–14. [5] Crago PE, Chizeck HJ, Neuman MR, Hambrecht FT. Sensors for use with functional neuromuscular stimulation. IEEE Trans Biomed Eng 1986;33(1):256–68. [6] Webster JG. Artificial sensors suitable for closed-loop control of FNS. In: Stein RB, Peckham PH, Popovic DB, editors. Neural prostheses: replacing motor function after disease or disability. New York, NY: Oxford University Press; 1992, p. 88–98. [7] Haugland M, Lickel A, Haase J, Sinkjær T. Control of FES thumb forces using slip information obtained from cutaneous electroneurogram in quadriplegic man. IEEE Trans Rehab Eng 1999;7(2):215–27. [8] Stein RB, Charles D, Davis L, Jhamandas J, Mannard A, Nichols TR. Principles underlying new methods for chronic neural recording. Can J Neurol Sci 1975;2(3):235–44. [9] Stein RB, Nichols TR, Jhamandas J, Davis L, Charles D. Stable long-term recordings from cat peripheral nerves. Brain Res 1977;128(1):21–38. [10] Hoffer JA. Techniques to study spinal-cord, peripheral nerve, and muscle activity in freely moving animals. In: Boulton AA,
[20]
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
457
Baker GB, Vanderwolf CH, editors. Neurophysiological techniques: applications to neural systems. Clifton, NJ: Humana Press; 1990, p. 65–145. Popovic D, Stein RB, Jovanovic K, Dai R, Kostov A, Armstrong WW. Sensory nerve recording for closed-loop control to restore motor functions. IEEE Trans Biomed Eng 1993;40(10):1024–31. Haugland MK, Hoffer JA. Slip information provided by nerve cuff signals: application in closed-loop control of functional electrical stimulation. IEEE Trans Rehab Eng 1994;2(1):29–36. Haugland MK, Hoffer JA, Sinkjær T. Skin contact force information in sensory nerve signals recorded by implanted cuff electrodes. IEEE Trans Rehab Eng 1994;2(1):18–28. Lickel A. Restoration of lateral hand grasp in a tetraplegic patient applying natural sensory feedback. PhD dissertation. Center for Sensory-Motor Interaction, Aalborg University, DK9220 Aalborg, Denmark,1998. Crook SE, Chappell PH. A portable system for closed loop control of the paralysed hand using functional electrical stimulation. Med Eng Phys 1998;20(1):70–6. Ilic M, Vasiljevic D, Popovic D. A programmable electronic stimulator for FES systems. IEEE Trans Biomed Eng 1994;2(4): 234–9. Smith B, Peckham PH, Keith MW, Roscoe DD. An externally powered, multichannel, implantable stimulator for versatile control of paralyzed muscle. IEEE Trans Biomed Eng 1987;34(7):499–508. Struijk JJ, Thomsen M. Tripolar nerve cuff recording: stimulus artifact, EMG, and the recorded nerve signal. Proceedings of the IEEE/EMBS 17th Annual International Conference 1995; Montreal, Canada. 1995, p. 1105–6. Nikolic ZM, Popovic DB, Stein RB, Kenwell Z. Instrumentation for ENG and EMG recordings in FES systems. IEEE Trans Biomed Eng 1994;41(7):703–6. Christiansen TG, Jensen BV. Optimised digital filtering of ENG signals recorded with a cuff electrode. MSc thesis. Center for Sensory-Motor Interaction, Aalborg University, 1999. Haugland MK, Hoffer JA. Artifact-free sensory nerve signals obtained from cuff electrodes during functional electrical stimulation of nearby muscles. IEEE Trans Rehab Eng 1994;2(1):37–9. Upshaw B, Sinkjær T. Digital signal processing algorithms for the detection of afferent nerve activity recorded from cuff electrodes. IEEE Trans Rehabil Eng 1998;6(2):172–81. Kilgore KL, Peckham PH, Thrope GB, Keith MW, GallaherStone KA. Synthesis of hand grasp using functional neuromuscular stimulation. IEEE Trans Biomed Eng 1989;36(7): 761–70. Inmann A. Optimisation of an RF-transmitter used for transcutaneous transmission of energy and digital data for medical implants (in german). MSc thesis. Department of Communications and RF-Engineering, Technical University of Vienna, Austria and Department of Biomedical Engineering and Physics, University of Vienna, Austria, 1997. Inmann A, Haugland M. Closed-loop control of an FES system incorporating natural sensory feedback used for restoration of hand grasp in tetraplegics. Proceedings of the IFESS 4th Annual Conference 1999; Sendai, Japan. 1999, p. 77–80. Handa Y, Handa T, Nakatsuchi Y, Yagi R, Hoshimiya N. A voice-controlled functional electrical stimulation system for the paralyzed hand. Jap J Rehabil Med 1985;23(5):292–8. Bayer DM, Lord RH, Swanker JW, Mortimer JT. A two-axis shoulder position transducer for control of orthotic/prosthetic devices. IEEE Trans IECI 1972;IECI-19(2):61–4. Johnson MW, Peckham PH. Evaluation of shoulder movement as a command control source. IEEE Trans Biomed Eng 1990;37(9):876–85. Hart RL, Kilgore KL, Peckham PH. A comparison between control methods for implanted FES hand-grasp systems. IEEE Trans Rehab Eng 1998;6(2):208–18.
458
A. Inmann, M. Haugland / Medical Engineering & Physics 26 (2004) 449–458
[30] Hoshimiya N, Naito A, Yajima M, Handa Y. A multichannel FES system for the restoration of motor functions in high spinal cord injury patients: a respiration-controlled system for multijoint upper extremity. IEEE Trans Biomed Eng 1989;36(7):754–60. [31] Solomonow M, Baratta R, Shoji H, D’Ambrosia RD. The myoelectric signal of electrically stimulated muscle during recruitment: an inherent feedback parameter for a closed-loop control scheme. IEEE Trans Biomed Eng 1986;33(8):735–45. [32] Scott TR, Peckham PH, Kilgore KL. Tri-state myoelectric control of bilateral upper extremity neuroprostheses for tetraplegic individuals. IEEE Trans Rehabil Eng 1996;4(4):251–63.
[33] Lauer RT, Peckham PH, Kilgore KL. EEG-based control of a hand grasp neuroprosthesis. Neuroreport 1999;10(8):1767–71. [34] Burelbach JC, Crago PE. Instrumented assessment of FNS hand control during specific manipulation tasks. IEEE Trans Rehab Eng 1994;2(3):165–76. [35] Zhou L, Munih M, Haugland MK, Perkins TA, Donaldson NN. An implantable telemeter for E.N.G. signals. 6th Vienna International Workshop FES. Vienna, Austria; 1998. p. 327–30. [36] Hansen M, Haugland M, Sinkjær T, Donaldson N. Real time foot drop correction using machine learning and natural sensors. Neuromodulation 2002;5(1):41–53.