Haptics in Surgical Robots

15 G

Haptics in Surgical Robots Peter Culmer1, Ali Alazmani1, Faisal Mushtaq1, William Cross2 and David Jayne1 1

University of Leeds, Leeds, United Kingdom St James’s University Hospital, Leeds, United Kingdom

2

ABSTRACT This chapter addresses the use of haptics in robots for soft-tissue surgery, aiming to bring a cohesive consideration of the clinical context, underlying technologies, and state-of-the-art applications in this field. The fundamentals of haptics are first introduced, using the human sensory system as an example which serves as a benchmark for technological advances. The chapter then provides a description of the clinical context, describing surgical areas and procedures of particular relevance to robotics before introducing key clinical challenges that have the potential to be addressed by the introduction of haptic technology. The basic building blocks of haptics, sensing and feedback systems are then introduced to provide an understanding of the fundamentals together with an overview of state-ofthe-art in each area. From this foundation, a review of haptics applied to surgical robots is presented, highlighting key commercial systems together with research advances. The chapter concludes with a discussion of current trends in this field and considers the technological and clinical challenges which remain. Handbook of Robotic and Image-Guided Surgery. DOI: https://doi.org/10.1016/B978-0-12-814245-5.00015-3 © 2020 Elsevier Inc. All rights reserved.

239

240

Handbook of Robotic and Image-Guided Surgery

15.1

Introduction

Haptics, a broad term for the sense of “touch,” rivals vision for its importance to surgeons and is closely intertwined with surgical practice, from “physical” examination to precise interventions. It would seem logical that this relationship would be enhanced with step changes in surgical practice following the introduction of minimally invasive and robotically assisted minimally invasive surgery (RAMS). Yet the opposite is true; these innovations have resulted in haptic feedback being largely diminished or entirely removed from the hands of the surgeon. Given the perceived importance of haptics to surgery, and RAMS in particular, it is instructive to analyze how this arose, examine the state-of-the art, and look toward future innovations which promise to turn this situation around. Our aims in this chapter are to bring together a diverse range of research to provide a focused picture of haptics applied to RAMS. We seek to provide the reader with a solid background to the fundamental science relevant to this area, combined with insight into recent developments in the research field. We do so by drawing on key publications across the fields of engineering, psychology, and surgery and highlight important textbooks and reviews for further reading in each. In this chapter we begin by introducing the fundamental concepts of haptics. In particular we examine the human sense of haptics and explore its value in open surgical practice. We then consider the application of haptics on a generalized tele-operated surgical robot, highlighting key components and performance attributes and their relation to the human haptic system. This provides a common technical background which we use to examine the state of the art in clinically available RAMS, reviewing the technical attributes of commercial systems with haptic capabilities and then inspecting their clinical reach and virtue into different surgical specialties. Following this we look at the wealth of research that has been undertaken into haptics for surgical robotics, highlighting technological developments in sensors, feedback systems, and full robotic research platforms. To provide a holistic view of this research area we then consider the critical aspect of how well these systems perform, is there value in integrating artificial haptics into RAMS? The chapter concludes by looking toward the future. How are surgical needs for RAMS changing and how do these relate to technical requirements for haptic systems? How is surgical interaction with RAMS evolving and what are the implications of increasing levels of automation in these systems? While many questions are raised, it should become clear that haptics is set to become an increasing part of future surgical practice, driven by real clinical need and enabled by exciting advances in technology.

15.1.1

Fundamentals of haptics

The word haptics is derived from the Greek word “haptikos,” which refers to “a sense of touch,” and the Oxford English dictionary defines haptics as: “Relating to the sense of touch, in particular relating to the perception and manipulation of objects using the senses of touch and proprioception.” As shown in Fig. 15.1, “haptics” is a broad construct that encompasses a number of different sensory inputs arising from receptors embedded within the body. These receptors can provide information about the state of the body known as kinesthetic sensation (i.e., the angles formed by the

FIGURE 15.1 Key elements of the human haptic sensory system.

Haptics in Surgical Robots Chapter | 15

241

15.1.2

Surgery and haptics

Surgery presents a canonical example of where humans (and increasingly robots) need to skillfully interact with the environment to perform a task with a high degree of spatiotemporal accuracy and precision [15]. Accurate and precise sensorimotor behavior requires information for movement planning and feedback control. It is therefore useful to consider the situations in which haptic information is relevant to surgical practice.

15. Haptics in surgical robots

various joints within the arm), and the physical characteristics of objects within the world, known as tactile sensations. In contrast to vision, haptic information regarding an object’s physical properties requires the body to come into contact with the object. Mechanical interactions between an object and the body provide a rich source of information that is not readily available from vision, and the usefulness of this information to humans is a matter of common observation. The scientific examination of haptics started in the 1800s and this type of investigation can be attributed directly to two of the founding fathers of experimental psychology, Ernst Weber and his student Fechner [1]. Weber used calipers to measure two-point thresholds on the skin and found large variations in sensitivity (e.g., high sensitivity near the lips and low sensitivity on the trunk). From a series of observations on touch and other senses, Weber showed that the threshold at which a change in a physical stimulus could be detected is a constant ratio (percentage change) of the original (reference) stimulus. In haptic perception, Weber is also responsible for the phenomenon now known as the Weber
Fechner Law of perception, formally describing the sensitivity of the human haptic system (i.e., the smallest differences in haptic sensation it can discriminate), contributing to the birth of psychophysics (the scientific study of the relation between stimulus and sensation) and a finding which remains relevant almost 200 years later [2]. Since the time of Weber, psychophysics has highlighted the remarkable performance of human tactile sensation; the temporal resolution of touch is approximately 5 ms and the spatial resolution at the fingertip as low as 0.5 mm [3]. This spatial and temporal resolution of touch provides humans with rich data about the physical characteristics of objects contacted by the hand. The haptic information comes in part through mechanoreceptors (nerve endings which react to a mechanical stimulus), and a variety of different mechanoreceptor types are found across the body. These receptors provide specialized somatosensory information input (information regarding haptics) to the central nervous system (CNS). The mechanoreceptors work through changes in the physical properties of their plasma membranes. The rate of adaptation and threshold of activation varies according to the mechanoreceptor type. Human skin can be considered to contain four types of mechanoreceptor that are specialized to provide tactile input to the CNS (Meissner’s corpuscles, Pacinian corpuscles, Merkel’s disks, and Ruffini’s corpuscles). These high-sensitivity/low-threshold receptors can be found on glabrous (hairless) skin and the hand. Fingertips in particular have an intense distribution of low-threshold receptors and are thus one of the areas of the human body most sensitive to external contact with objects [4
6]. We have already drawn attention to the fact that the term “haptics” is also used to refer to the sensory system that provides the body with information about the relative position of body parts. This class of information relates to the sense of position and movement of the limbs in space and is often described as “kinesthesia” [7]. The source of this information arises from mechanoreceptors embedded in muscles, tendons, and joints. These receptors allow humans to sense the angular position of a joint and have a fundamental role in the control of human movement. An important aspect of haptic perception is that it is not processed by the CNS in isolation, haptic information is combined with vision (and to a lesser extent other senses) when humans interact with the external world. Indeed, it is well established that humans integrate information across multiple modalities when determining the characteristics of external objects and the state of the body [8
11]. The multimodal integration of information in humans suggests that the benefits of robot surgery might be best harnessed by systems that have the capacity to integrate multiple forms of information in order to arrive at the best possible state estimates. We will explore the consequences of a multimodal approach to surgical technologies in more detail later in this chapter, but here it is worth noting that integration of multiple streams of information in robotic systems is already yielding promising results. For example, a recent experiment evaluating the efficacy of integrating haptic feedback in robotic surgery showed that information from vision and haptics can be synergistically linked to reduce force application error [12]. Today, the study of haptics transcends philosophy and psychology, bringing together a diverse range of disciplines, from neurophysiology through to computer science. Haptics has become a “broad church,” covering the study of human and/or computer interactions with the environment and most often related to the manipulation of objects through touch and force feedback [13]. A large degree of research activity in haptics has focused on practical solutions, from supporting activities of daily living through to the incorporation of haptics in surgical technologies. The optimization of robotic surgery will require further practical considerations of how information can be best extracted and used to support the high levels of accuracy and precision required when physically interacting with tissue in surgery [14].

242

Handbook of Robotic and Image-Guided Surgery

Section 15.1 highlighted that mechanoreceptors in the body provide information that can contribute toward the performance of skilled sensorimotor control behaviors. The information can be used to guide arm movement (especially when visual information on arm position is not available) and provide rich data on the physical properties of an object of interest. Information on an object’s physical properties is an important precursor for interacting with the object in a skillful manner (e.g., ensuring that the appropriate fingertip forces are exerted when lifting or squeezing the entity). A move toward minimally invasive techniques and the associated degradation of haptic cues (or, in robot surgery, the removal of some forms of haptic cues), clearly reduces the information available to the surgeon. This means that current challenges in surgical technologies include: (1) restoration of normally available information through the creation of intelligent instruments that provide adequate substitutes for knowing effector position, producing appropriate forces and sensing deformation; (2) augmenting this information so surgeons can reach ever higher levels of sensorimotor performance and cognitive decision-making; and (3) presenting this information effectively so that surgeons can incorporate it into their sensorimotor control and decision-making processes, examined in Section 15.3.3. To address these three challenges, a consideration of the task is required in order to understand how information is used to perform a given task (termed “task analysis”). In the case of surgery, the surgeon needs to precisely monitor his or her limb position so that the controlled end point (e.g., the fingertip) is moved to the appropriate spatial location and applies the desired amount of force. If a surgeon is moving their finger to palpate tissue in open surgery then they are able to use a combination of visual and haptic information in order to guide their arm. The task is more complicated when the surgeon holds a tool, but there is a wealth of data showing that humans can use haptic information when holding a handheld object. The augmentation of such haptic information with vision enables humans to precisely control the end point of a handheld instrument (as if the instrument were part of their own body) [16]. In laparoscopic surgery, the surgeon loses the ability to directly monitor the end point of the laparoscopic tool through vision and must therefore rely on an indirect view of the instrument (via a camera feed), together with the haptic information obtained through wielding the laparoscopic device. The fact that surgeons can master laparoscopic surgery is a testament to the incredible learning capabilities of the human nervous system. It also demonstrates that humans are able to learn to produce skillful movement when alternative sources of information replace the normal information signals used within a task. The use of robotic surgical systems places further demands on the surgeon as the information available when holding a surgical tool is no longer directly available to the operator. This means that the surgeon no longer has access to haptic cues providing information on the end point position of the tool and so feedback control must rely entirely on a visual error signal. As a result, overall performance will be degraded, as highlighted by recent meta-analyses demonstrating the utility of including haptic information in teleoperative systems [17,18]. The other major use of haptic information in surgery relates to the information that a surgeon can glean about the physical characteristics of the tissues that are involved in the operation. Haptics can play a central role in driving surgical performance by allowing the surgeon to gain a much better understanding of the boundaries of a malignant growth, for example, and determining the forces required to complete a particular action such as manipulating tissue [16]. This highlights an important aspect of haptic perception (and indeed all of human perception): the process of obtaining information about the external world is not passive. Humans are information predators and we actively hunt to extract information from the world around us. In vision, humans scan with their eyes to extract information from the optical field. Indeed, humans make more eye movements in their lifetime than their heart beats. In haptics, humans actively interact with external objects and it is through this activity that information is obtained about the physical characteristics of the object. Active interaction with the world creates a haptic feedback loop, termed the “perception
action cycle,” of information flow. The manipulation of objects relies on “action for perception,” which in turn results in “perception for action” (i.e., the perceptual information needed to generate the skillful action that will allow the actor to apply the desired forces to the object to achieve a particular goal). To illustrate this perception
action cycle of information, consider the typical actions of a surgeon identifying a cancerous tumor through palpation in open surgery. The surgeon will actively probe the tissue of interest with their hands (prodding, pulling, squeezing, and stroking) to generate and observe sensory information (both visual and haptic) to inform assessment [19]. It can be seen that the challenge is to allow equivalent (or enhanced) functional information to be available to the surgeon within laparoscopic and robotic surgical devices where traditional haptic sources of information are degraded or absent.

15.1.3

Tele-operated surgical robot systems

Surgical robotics is becoming an increasingly broad field and the definition of what this constitutes can be somewhat nebulous. The SAGES-MIRA Robotic Surgery Consensus Group states that a “surgical robot” generally refers to systems which should strictly be considered as “remote telepresence manipulators,” dependent on the “control of a human

Haptics in Surgical Robots Chapter | 15

243

operator” and without autonomy. In fact, truly autonomous surgical robots remain an unfulfilled but exciting aspiration, as discussed later in the chapter, and hence here we consider robotic surgery as a surgical procedure or technology that adds a computer technology-enhanced device to the interaction between a surgeon and a patient during a surgical operation and assumes some degree of control heretofore completely reserved for the surgeon [20].

15.2 15.2.1

Surgical systems The surgical robotics landscape

The robotic revolution in the manufacturing industry is often used as an argument for similar transformations in robotic surgery. Among the numerous qualities afforded by robotic systems, precision, dexterity, repeatability, and cost efficiency have clear relevance to surgical practice [23,31]. Yet, in contrast to the manufacturing industry, surgical procedures represent a more complex and unstructured environment in which robotic systems must interact precisely with delicate, mechanically compliant structures and where the consequence of error is directly linked to human life. As a

15. Haptics in surgical robots

In this context, the key elements of a surgical robot with haptics are shown in Fig. 15.2. Foremost is the haptic measurement system, consisting of sensors (described in Section 15.3.1) used to obtain force and pressure information from the robot and operative field and which replace direct sensory feedback from the arms and hands of the surgeon. Next the robotic control system mediates the processing and return of information, emulating the nervous system of the body. Lastly, haptic information is “rendered” back to the surgeon by one or more haptic interfaces, as discussed in Section 15.3.2. This disconnect between the surgeon and the operative field brings associated technical challenges, together with opportunities to both change and enhance the surgical experience [21]. Design of a tele-operated system requires careful consideration of how to balance the conflicting requirements to maximize “transparency,” a term denoting the degree of telepresence achieved (i.e., the fidelity of the haptic information) while ensuring system “stability” (i.e., ensuring the robotic system behaves predictably) [22]. Transparency and stability have particular resonance in a surgical context; reliable behavior is crucial when the briefest erroneous movement could result in patient morbidity or death and appropriate telepresence is necessary if the haptic capabilities are to benefit, rather than hinder, surgical procedures. As shown in Fig. 15.2, the overall tele-operated system involves three interacting components, the surgical operator, the robotic system, and the surgical environment, each of which has its own dynamics [23]. Ensuring stability can be addressed by considering the flow of energy between components and ensuring that the robotic system is “passive” such that it can only absorb, rather than generate energy [24]. Promoting telepresence requires kinematic precision (the instruments should move to the surgeon’s commands accurately), kinetic fidelity (haptic information should be measured and rendered to minimize distortion when perceived by the user), and minimize temporal lags (specifically transport delays in the robotic system which occur at mechanical interfaces and in electronic communications). Transport delays also inhibit system transparency and in a surgical context have been demonstrated to slow procedures and affect confidence in the system [25]. For a detailed analysis of these factors the reader is directed to an excellent technical review [22]. Despite the inherent technical challenges, separating the surgeon’s hands from the operative environment can be used to positive effect. Without a direct physical connection between stimulus and sensory sensation, haptic information can be manipulated or artificially created prior to being displayed to the surgeon. Sensory substitution, in which haptic information is displayed in a different modality or form, emerged due to limitations in haptic displays, for example, displaying tactile pressure readings via a visual interface [26]. However, it can also be used to augment existing sensory information, like using variable-frequency vibration to convey the magnitude of a grasping force [27]. Just as endoscopic cameras provide magnification of small features, “force scaling” illustrates the potential for haptics to enhance human sensory capabilities by amplifying low-magnitude forces (e.g., tissue manipulations during microsurgery) to levels which can be better perceived by the surgeon [28]. Equally, just as surgical displays are increasingly augmented with additional information to help the surgeon (e.g., overlaying preoperative CT scans [29]), so haptic displays can be used to define “virtual fixtures” which help physically guide appropriate motion of the robotics system to avoid errors, for example, to protect sensitive tissue in neurosurgery [30], or alerting the surgeon when critical thresholds are approached/exceeded. These examples underline both the relevance of haptic technology to modern robotic surgery and its relative immaturity when compared to visual technology, as will be evident in the next section which examines extant haptic technology in commercial surgical robots.

244

Handbook of Robotic and Image-Guided Surgery

FIGURE 15.2 Key elements of artificial haptic feedback in a tele-operated surgical robot.

FIGURE 15.3 Prevalence of haptic-enabled surgical robots in clinical studies across different surgical specialties. Data from Amirabdollahian F, Livatino S, Vahedi B, Gudipati R, Sheen P, Gawrie-Mohan S, et al. Prevalence of haptic feedback in robot-mediated surgery: a systematic review of literature. J Robot Surg 2018;12:11
25. https://doi.org/10.1007/s11701-017-0763-4 [32].

result, to date there has been a limited uptake of robotic surgical systems, particularly outside general surgery, as shown in Fig. 15.3. A prime example can be found in general surgery where a step change in practice was brought about through the introduction of laparoscopy in the 1980s. Laparoscopic surgery avoids the need for large incisions (used in open surgery), instead using minimally invasive techniques to access abdominal tissues and organs using long instruments inserted through small incisions in the abdominal wall. The benefits of laparoscopy (i.e., improved cosmesis, lower infection rates, and faster recovery) are well recognized and have driven uptake of the technique, despite the increased

Haptics in Surgical Robots Chapter | 15

245

15.2.2

Commercial surgical robot systems

The range of commercially available surgical robots with haptic capabilities is increasing, with the current state of the art presented in Table 15.1 and illustrated in Fig. 15.4. The systems span a range of surgical specialties and there is evidence of the market developing more specialized systems. It should be noted that the information available is typically limited due to the highly competitive commercial market and many sources are based on the research systems upon which the commercial systems were developed. In tandem, a number of major surgical robot systems exist without TABLE 15.1 Clinically available surgical robots with haptic capabilities. System name

Manufacturer

Surgical area

Haptic capabilities

Clinical studies

Senhance (formerly ALF-X)

TransEnterix

General surgery

Force feedback [38,39]

Gynecology [39,40]

RIVO-I

Meere, Korea

General surgery

Force feedback [42]

Colorectal [41] Preclinical anastomosis [42] Preclinical cholecystectomy [43] Preclinical partial nephrectomy [44]

MiroSurge

NeuroArm

Medtronic (formerly Covidien)

General surgery

Flexible arm configuration

Laparoscopic surgery?

Open surgery (cardiac)

Bimanual force feedback

Preclinical heart studies [45]

MacDonald, Dettwiler and Associates

Microsurgery

Tooltip force feedback

Glioma [30,47]

Hansen Medical Inc.

Endovascular

Force scaling Virtual fixtures [30,46]

Sensei X and X2

DoFs, Degrees of freedom.

Catheter tip with three DoFs force sensor

Stent grafting [51]

Full force feedback system [48]

Catheter ablation [52,53]

Minimizes contact force [49,50]

Catheter ablation—robot versus manual [54]

15. Haptics in surgical robots

challenge faced by the surgeon (i.e., impaired visual and haptic feedback, loss of dexterity, and complex instrument control). This forms the justification for robotic-assisted-laparoscopic surgery which has been dominated by the da Vinci robot series from Intuitive Surgical. The da Vinci uses robotic technology to improve surgical dexterity, visualization, and precision while simplifying control to ease cognitive loading. The da Vinci has been a commercial and clinical success, enjoying an unchallenged market share and uptake across the world despite its high cost. However, the technology is notable for its lack of haptic feedback, which is frequently linked to an increased risk of inadvertent tissue injury [20,33]. While this may not necessarily manifest in higher clinical complication rates it does have the effect of limiting the utility of the system to those areas in which haptic feedback is not absolutely necessary. The specific example of the da Vinci system in general surgery highlights the broader trend that robotic surgical technology must be developed to fully meet the clinical needs and associated challenges of modern surgery and that current systems are notable for a lack of haptic feedback, described in a comprehensive review [32]. These limitations do not necessarily reflect a lack of appropriate technology (as will be shown later in this chapter) but rather the realities of getting such technology to market in the face of a highly restrictive and patent-encumbered commercial landscape [34]. However, the expiry of key patents and renewed investment by large multinationals (e.g., Johnson and Johnson) give cause for optimism of a more competitive market bringing technological advances including haptic feedback to broaden the spread of robotic surgery [35
37].

246

Handbook of Robotic and Image-Guided Surgery

FIGURE 15.4 Clinically available surgical robots with haptic capabilities: (A) Senhance (TransEnterix Inc., Morrisville, North Carolina, United States) [55]; (B) RIVO-I (Meere Company) [42]; and (C) the MiroSurge system with haptic control interface (inset) [DLR (CC-BY 3.0)] [56].

haptic capabilities at the time of writing, including Versius (Cambridge Medical Robotics), SPORT (Titan Medical), and the da Vinci (Intuitive Surgical).

15.2.2.1 General surgery: Senhance Senhance is a surgical robot platform developed for laparoscopy, originally conceived and developed by the Joint Research Centre of the European Commission in collaboration with SOFAR SpA (Italy) as the Telelap Alf-x and subsequently rebranded as Senhance surgical robot system when the technology was acquired by TransEnterix

Haptics in Surgical Robots Chapter | 15

247

15.2.2.2 General surgery: REVO-I The REVO-I is a general surgical platform developed through a collaboration between University College of Medicine, Korea, and Meere Company [42]. Development was initiated in 2007 and after several technical iterations and over 20 animal studies, the present system has received regulatory approval from the Korean Drugs Administration and was launched in March, 2018, aiming to provide cost-effective robotic surgery to reduce overall costs for resourcelimited healthcare systems with lower costs reported in comparison to use of an ALF-X robotic system for the same procedures [44]. The general configuration is deliberately similar to the da Vinci platform, with a central four-armed robotic cart used in conjunction with a surgeon’s control console, as shown in Fig. 15.4. The robotic arms are controlled by the surgeon console; one arm holds a high-definition (HD) stereoscopic laparoscope, the remaining three arms are equipped with instruments and selectively positioned and moved by the surgeon using a hand-operated grip-system [43,44]. Integration of haptic functionality into the RIVO-I has been reported as a key technical feature differentiating it from the ubiquitous da Vinci. Although technical details on the implementation are scarce, the system provides the surgeon with force feedback from surgical graspers and actively regulates instrument speed and force within specified “safe” limits [36,42].

15.2.2.3 General surgery: Medtronic MiroSurge MiroSurge (Medtronic, United States) is a commercialized version of the MiroSurge surgical robot for general and cardiac surgery, originally developed at DLR (German Aerospace Center), then licensed by Covidien before their acquisition by Medtronic who continued to develop MiroSurge and launched the system as a rival to systems like da Vinci (Intuitive Surgical, United States). MiroSurge was developed as a specific telerobotic instance of DLR’s broader Miro robotic surgery platform which consists of technologies for robotic manipulators, surgical instrumentation, user interfaces, and computer-assisted planning and registration. The MiroSurge system was developed to provide a flexible surgical system, in particular focusing on reducing the operative footprint in comparison to competing systems. To achieve this the system does not have a dedicated “base station” but instead uses a reconfigurable series of lightweight robot arms mounted directly onto the operative table, each holding a custom surgical instrument. The robot arms are based on DLR’s MIRO technology, a general-purpose robotic arm with seven DoFs, a lightweight (10 kg) structure, and integrated joint torque sensors, designed to achieve low-inertia and high-bandwidth (0.5 m/s) control, with a maximum payload of 3 kg. Instrument control in MiroSurge exploits this configuration by using combined force and position control to attain four DoFs redundant positioning of the tooltip within the body (to avoid external arm collisions) with movement about a variable fulcrum (in contrast to a fixed fulcrum employed by systems such as the da Vinci). MiroSurge instruments are attached to each robot arm to locate an additional two DoFs wrist mechanism inside the body, resulting in a functional endeffector with six DoFs, together with a single DoF functional movement (e.g., open/close grasping jaws) [56]. The result is a system that can dynamically adapt to, and compensate for, natural movement of the body (e.g., due to breathing) or enable robotic assistance in complex procedures such as open cardiac surgery [45,59]. The haptic capabilities of MiroSurge comprise seven DoFs force sensing at each instrument with a bimanual feedback interface. Measurements of tool
tissue interactions are made directly at the end-effector of each instrument, as shown in Fig. 15.4, using a miniaturized six DoFs force/torque sensor combined with a grasp force sensor integrated

15. Haptics in surgical robots

(Morrisville, North Carolina, United States). The system was approved for general surgical procedures in Europe in 2012 and obtained FDA approval for the United States in 2017 [55,57]. There is little detailed technical information on the Senhance system, but the capabilities of the system can be inferred from a series of publications evaluating its clinical efficacy (see Section 15.2.3). The system was designed to be both cost-effective and to minimize disruption to existing operating theater environments and workflows. Instruments are held by a series of up to four robot arms, each situated on individual mobile carts to allow a flexible configuration around the patient. Senhance promotes that its instruments are similar to those used in manual laparoscopy, designed to promote familiarity with the surgeon they also lack the additional “wristed” degrees of freedom (DoFs) found in competing systems like da Vinci [58]. The surgeon sits at a console providing three-dimensional (3D) visualization, eye-tracking control of the endoscope, and haptic feedback via two force feedback manipulators which resemble laparoscopic instruments [55,57]. The manipulators transmit grasp force, enabling tissue consistency, or object manipulation, to be felt by the surgeon [57]. It is reported that the capabilities are instructive during thread and needle manipulation when suturing [58].

248

Handbook of Robotic and Image-Guided Surgery

into the jaws of forceps and needle-driver tools. This configuration minimizes mechanically induced disturbances to enable precise measurement (0.04 N resolution, 10 N range) at the expense and complexity of requiring compatibility with autoclave sterilization processes [56]. MiroSurge is controlled using two haptic manipulators developed through a commercial partnership with DLR (sigma.7, Force Dimension), each of which provides seven DoFs movement control with full force feedback on all DoFs. Thus the surgeon is able to continuously perceive and control both the pressure applied to grasp tissue (0
8 N) with the forces (0
20 N) and torques (0
0.4 N m) necessary to move it during operative practice. Crucially, to ensure safe surgical operation with this level of robotic complexity, the MiroSurge control system conforms to requirements for passivity (see Section 15.1.2), providing confidence of stability and reliability [60].

15.2.2.4 Microsurgery: NeuroArm The NeuroArm system has been developed to exploit the ability for robotic systems to perform precise movements and to operate in challenging environments, in this case to perform robotic microsurgery in neurological procedures within an MRI scanner used for intraoperative imaging. The system combines intraoperative MRI expertise from University of Calgary with tele-operated robotic technology originally developed for the International Space Station by Macdonald Dettwiler and Associates (Aurora, Canada) [46]. As a consequence, the system was developed to meet both aerospace and medical standards, and the system was licensed to a spin-off which now trades as IMRIS-Deerfield (United States) and has reportedly delivered systems to clinical use in over 70 locations worldwide. The reader is recommended to refer to Ref. [61] for an enlightening insight into the evolution of the NeuroArm project and the challenges involved in commercializing medical robotics technology. Central to the design and operation of NeuroArm is the constraint to be MRI-compatible, which impacts on its configuration, use of materials, and sensory systems. In the operative field, a mobile base-unit supports a field camera and two anthropomorphic seven DoFs robot arms, each of which can hold a range of detachable microsurgical tools. A remote surgeon workstation then controls this assembly and integrates the various imaging and robotic technologies through an array of 2D and 3D display units and hand controllers [46]. The system can operate at a maximum speed of 200 m/s with a 750 g payload while achieving precise motion (50 μm spatial resolution) for sensitive tasks like microsurgical dissection [30,46]. The haptic functionality of the NeuroArm is centered on the use of force feedback from the robotic arms to the surgeon’s hand controllers. Precision force transducers (Nano17, ATI Industrial Automation Inc., Apex, North Carolina, United States) enable high resolution (0.149 g-force) feedback of translational forces at the tools, together with an additional grasping degree of freedom via two force feedback hand-controllers (Omega 7, Force Dimension, Switzerland) [62]. The control system features a combination of motion scaling, a 2 Hz low-pass “tremor” filter and force scaling, the latter enabling enhanced sensation such that soft tissues can appear stiffer, or vice versa [30]. In addition, NeuroArm uses its haptic capabilities to provide guidance using virtual fixtures [28], notably to define “no-go” zones informed by MRI information. In addition, a “haptic warning system” can be enabled to inform when definable force thresholds are exceeded to help avoid tissue trauma [47]. During typical operative use the tool forces fall within a 0
20 Hz band, signifying controlled, precise motion [62]. However, even with this range of features, surgical tool motion was found to be slower in the NeuroArm system in comparison to conventional microsurgery. This is thought to be a consequence of small delays in the control system, forcing surgeons to slow their movement such that their movement remained synchronized with the robotic system, highlighting the importance of “haptic transparency.” In addition, the team note that the system’s spatial precision exceeds its visual capabilities and that to fully exploit this requires future enhancement or automation [25].

15.2.2.5 Endovascular: sensei The Sensei X platform (Hansen Medical, Mountain View, California, United States) merits discussion for advancing haptic feedback in the growing surgical field of endovascular catheters. The Sensei X provides an open robotic platform for endovascular surgical procedures focused on catheter-based radiofrequency ablation [50]. The system supports specialized catheters which have been instrumented to measure contact force and global position, for example, the ThermoCool SmartTouch ablation catheter places a precalibrated spring at the catheter’s tip and measures its deflection with a magnetic coil-sensor pair to infer contact forces with a 1 g resolution [63]. When used in conjunction with the Sensei X platform, the surgeon is provided with visual feedback of contact force magnitude and vibration “alerts” when this exceeds predefined limits. In a clinical context this is crucial since insufficient contact force could lead to ineffective ablative treatment while excessive force risks tissue trauma or perforation. Interestingly, in a study focused on atrial

Haptics in Surgical Robots Chapter | 15

249

fibrillation (ablation is used to form curative lesions) the use of this system led to higher contact forces than in manually performed procedures, but crucially these were more consistently in the optimum treatment range and remained within “safe” limits ( . 40 g/cm2), ultimately resulting in improved clinical outcomes [50].

15.2.3

Surgical practice

The systems described in Section 15.2.2 have led the way in both defining and exploring the virtues of haptically enhanced robotic surgery in real clinical settings. The roles and demands required of these systems are closely linked to the surgical field in which they operate, although there are also commonalities which emerge, as we demonstrate here by examining clinical use of haptically enhanced surgical robots.

The key robotic system incorporating haptic feedback currently in use for general surgery is the Senhance system (TransEnterix, Morrisville, North Carolina, United States). The haptic capabilities of the system have been used clinically for estimation of structure consistency inside the peritoneal cavity by pulling/pushing maneuvers and for estimating tissue tension when grasping structures. Although the system is not perfect, with a small delay before resistance is felt, it is suggested that the haptic feedback may be beneficial in reducing the risk of injury to organs and delicate structures, particularly if the instruments move outside the optical field. So far, the Senhance robotic system has been evaluated in gynecology and colorectal surgery. Fanfani et al. analyzed a prospective cohort of 146 patients undergoing gynecological surgery and showed it to be safe and with a low risk of conversion [64]. In colorectal surgery, Spinelli et al. reported clinical outcomes on 45 patients undergoing a range of colorectal operations for benign and malignant disease, with results similar to what might be expected from conventional laparoscopic surgery [55]. Although the Senhance system has demonstrated its clinical capabilities across a range of different operative procedures, the additional advantage that its haptic capabilities bring remains to be proven. They are marketed largely on the basis of minimizing tissue trauma, but given the low incidence of this in general robotic surgery it requires further study to make a more conclusive cost
benefit argument.

15.2.3.2 Endovascular Haptic feedback is of great interest in the performance of endovascular procedures where excessive forces between the catheter tip and vascular wall can result in iatrogenic injury with the potential for perforation, vessel wall dissection, or pseudoaneurysm formation. Haptic sensation at the proximal end of the catheter can also help guidance-navigation of the catheter through the vasculature. Catheter systems can be divided into two categories: mapping catheters for profile visualization of the vessel or heart, and intervention catheters in which the tip of the catheter is equipped with a tool. To date, the evidence in support of the advantages of hepatic feedback in vascular catheters is derived mainly from experimental studies, with limited clinical data available. Russo et al. used the Sensei robotic navigation system with the ThermoCool SmartTouch catheter to treat patients suffering with atrial fibrillation and found that the use of the robotic system allowed larger catheter tip contact forces during atrial ablation therapy, thus improving treatment performance which resulted in lower rates of recurrence at clinical follow-up [50,65].

15.2.3.3 Neurosurgery Neurosurgery demands precision and accuracy to enable benign and malignant lesions to be resected with maximal preservation of normal brain tissue. Haptic feedback might be particularly advantageous in this scenario, given the inherent soft consistency of the brain and the proximity of vital structures [66]. Two robotic systems have been developed for neurosurgery and used on humans: NeuRobot and neuroArm. Clinical reports have described the use of the NeuRobot for a variety of human procedures, including third ventriculostomy, resection of tumors, as well as portions of intracranial microsurgical procedures such as Sylvian fissure dissection [67
69]. However, the only system that remains in clinical use is the NeuroArm, which has been used in a variety of clinical procedures, including stereotactic MRI-guided biopsy, blunt or microsurgical dissection, suturing, hematoma aspiration and irrigation, and cauterization [61,70]. The force scaling feature of NeuroArm provides a significant clinical advantage, allowing the surgeon to enhance their sense of touch to improve tissue interaction. Maddahi et al. used the NeuroArm in seven cases of glioma surgery to record the range of forces between the tool tips and brain tissue but were unable to correlate the forces exerted to the tumor pathology, showing that using haptic information for assessment can be challenging in a surgical environment [47].

15. Haptics in surgical robots

15.2.3.1 General surgery

250

Handbook of Robotic and Image-Guided Surgery

15.2.4

Emerging surgical needs

Within the field of surgery, the application of robotic technology is a relatively new event and still within its evolutionary infancy. Despite significant economic, regulatory, and technological barriers, robotic minimally invasive surgery (RMIS) has been embraced by enthusiastic and pioneering surgical teams to the benefit of patients worldwide. For example, the use of articulating instruments, immersive pseudo-3D operative fields, motion scaling, and elimination of tremor, all enhance surgical performance, leading to greater surgical precision. Robotic technologies have allowed surgeons to become proficient in traditional laparoscopic procedures more quickly (shorter learning curve), which has led to many experienced surgeons changing their open surgical practice to RMIS. These technologies have also facilitated the development and advancement of surgical techniques and procedures, which would not be possible with traditional laparoscopic equipment. But if RMIS is to develop and expand into new surgical areas and be regarded as the standard of surgical care, the next generation of RMIS systems will need to address the current technological limitations and opportunities [29]. One of the main shortcomings of surgical systems has been the lack of haptic feedback. Previous attempts to integrate haptics have often failed to realistically replicate the normal surgical experience and the cost of developing such systems, in the absence of an obvious clinical benefit, has been questioned by some manufacturers. Surgeons must therefore resort to using visual cues to estimate forces applied to tissues, sutures, and other materials, and misinterpretation of these cues can lead to irreversible tissue trauma and potentially patient morbidity. The routine use of haptic systems will likely first emerge in clinical applications that demand accuracy and fine dexterity. Such procedures include tasks such as suturing, where motor skills need to be combined with precise judgment of suture tension, and microdissection in proximity to vital structures, as shown in Fig. 15.5. These clinical needs have received increased attention from the surgical community as robotic technology has developed, as summarized in Table 15.2. A common theme is tissue assessment, in which the consistency of a tissue is often related to an underlying

FIGURE 15.5 Using a surgical robot (da Vinci, Intuitive Surgical) without the use of haptics to (A) dissect a blood vessel and (B) apply a vessel clip to temporarily prevent blood flow. These tasks are routine but tissue manipulation must be regulated through visual feedback alone. The full video of this procedure is available online.

TABLE 15.2 A summary of the opportunities and benefits of haptically enabled systems across different surgical specialties. Surgical specialty

Opportunities for haptics in RMIS

Urology

G

References [71]

G

Improved preservation of the neurovascular bundles, which are important for erectile function, during radical prostatectomy and cystectomy Enhanced tissue handling of the ureters during reconstructive urinary tract procedures

Gastrointestinal surgery

G

Reduced tissue trauma during bowel resection and anastomosis

[35]

Plastic and reconstructive surgery

G

Enhanced manipulation of nerves, vessels, and muscles in free-flap surgery Brachial plexus surgery

[72]

G

Neurosurgery

G

Improved stereotactic positioning and minimization of collateral damage to neural tissues

[73]

Orthopedics

G

Facilitate prosthesis manipulation and positioning

[74]

Ophthalmology

G

Retinal microsurgery

[75]

RMIS, Robotic minimally invasive surgery.

Haptics in Surgical Robots Chapter | 15

251

15.3

Research systems

While the integration of haptics into commercially available surgical robots is immature, with few examples moving beyond simple force feedback, it has received significant attention in the research community over the last 20 years, driven by both a recognition of clinical need and increasing technological capability [23]. Here we consider notable advances made in the research field which have particular relevance to the commercial sector, encompassing both the enabling technology in sensor and feedback systems and research-grade surgical robot platforms which have advanced our understanding of the benefits of using haptics in surgery.

15.3.1

Sensing systems

The foundation of any haptic robotic system is the sensory system used to acquire data from the environment with which it interacts. This has been a subject of longstanding interest in the robotics community to support dexterous interaction with the environment [77]. The opportunity for integration of haptics into surgical robotics has been long recognized, exemplified by pioneering work on remote tissue palpation [78]. With this opportunity comes the technical challenge of moving from structured and static industrial environments toward the uncertain and highly variable environments found in surgery [79]. This has catalyzed the research community with reviews showing the development of haptic sensing technologies specifically for surgical applications [80
83]. The sensory features of surgical robots are strongly influenced by a series of factors. Foremost is clinical need, which determines the fundamental sensing requirements; what type of haptic measurement is required for the surgical procedures performed by that system? Section 15.2.4 describes a spectrum of surgical tasks which can be ranked in haptic complexity, from suturing requiring “simple” force feedback to assessment of tissue margins demanding multipoint tactile feedback. Here we distinguish tactile sensors from force sensors as those systems designed to obtain information through direct interaction with the environment. Market factors and the surgical environment then shape the implementation of the sensing systems. Any elements placed within the operative field must be compatible with cleaning and sterilization procedures (e.g., high-temperature autoclave) if they are reusable, or designed for modular and single use, placing constraints on materials, packaging, robustness, and cost. In addition, modern surgery increasingly uses minimally invasive approaches which impose size constraints and considerations about the atraumatic tissue interaction. Integration of force sensing into surgical robot systems would seem more readily achievable given the ubiquity of commercially available systems and, indeed, commercial load-cells have been used to measure the tool
tissue interaction in open surgical procedures [84]. However, the increased demands of Minimally invasive surgery (MIS) have driven the development of bespoke load cell technology, early examples include a 5 mm diameter tri-axis force sensor using optical fiber transducers [85] and a six-axis force and torque sensor (0.04 N resolution, 10 N range) using miniature strain gauges in the DLR Miro [56]. More recently, advances using fiber Bragg grating have enabled precise (0.15 mN resolution, 0
25 mN range) tooltip sensing on microforceps with 0.9 mm diameter (Fig. 15.6) for use in

15. Haptics in surgical robots

disease process; for example, the majority of malignant tissues are harder than their normal counterparts due to an increase in cell and stromal components. The ability to sense changes in tissue consistency can therefore assist surgeons in diagnosing disease and determining disease margins intraoperatively and in real time. It also has the potential to allow the development of procedures in parts of the body that are restricted due to current optical technology, particularly through use of haptic guidance (e.g., active constraints) which can inform the surgeon of the proximity of vital structures, for example, blood vessels or nerves, and guide their movement appropriately. In addition to guiding surgical execution, positional, and force data collected during robotic surgery can contribute to the development of surgical stimulators. The inclusion of touch sensation in a surgical stimulator should enhance the operator experience, providing a more realistic training platform where skills can be gained in a fail-safe environment. Another major opportunity with the introduction of haptically enabled robotic systems is in surgical training. The acquisition and development of skills in RMIS is commonly obtained through a combination of computer surgical simulation, practice in animal labs, and experience in the operating theater. Advancements in virtual reality training platforms have allowed surgeons to become proficient in basic and advanced skills, through risk-free computer simulation, before applying them on real patients. Although objective data on the benefits of haptic feedback in virtual reality RMIS simulation are limited, there is a suggestion that haptics enhances perception of the surgical field with reduced errors, improved task-time and overall training performance [76]. In addition, closing the loop and using intraoperative haptic data from real procedures can be used to help define and improve the mechanical properties of virtual tissue models, assisting the development of better virtual environments for surgical training.

252

Handbook of Robotic and Image-Guided Surgery

FIGURE 15.6 Force and tactile sensor systems developed for use in surgical robotics: (A) the STIFF-FLOP soft surgical robot with integrated force sensing [91], (B) microforcep with shaft-force sensor for retinal surgery [86], and (C) instrumented forceps with five DoFs tactile sensing jaws [99].

retinal surgery [86]. An alternative approach to direct force measurement is to integrate load sensors within the drive systems of robots, thus allowing them to be located remotely, outside the constraints of the surgical environment. A proof of concept implementation for the da Vinci surgical robot measures torque at the cable drive pulleys but highlighted that mechanical losses and nonlinearities made calibration challenging [87]. Use of statistical estimation techniques helped to minimize these factors in the Raven II surgical platform (Fig. 15.6) such that grasping forces could be reliably determined [88]. Another approach used neural networks to determine the external forces acting on a surgical grasper, with indicative errors of 0.24 N across a 0
4 N range when grasping [89]. While these external measurement techniques are implemented on rigid robotic structures, they also have relevance to soft robotic systems which are growing in popularity and technical maturity. This is exemplified by the development of the STIFF-FLOP surgical manipulator (Fig. 15.6), a structure composed of multiple “soft” segments, each of which includes three-axis position and force sensing using pneumatic transducers [90,91].

Haptics in Surgical Robots Chapter | 15

253

15.3.2

Haptic feedback systems

In comparison to the ubiquity and advanced state of visual and sound display technologies, haptic feedback systems have remained rather underutilized because of their relative immaturity, limiting their use to meet the challenging requirements found in robotic surgery [106]. An important consideration here is the bidirectionality of haptic perception, which implies an interchange of mechanical energy between the sensory organ and the environment, unlike any other human sensing modality. Enayati et al. described bidirectionality as the root of many challenges that not only make the design, production, and implementation of haptic interfaces demanding, but also make evaluation of their performance nontrivial [23]. Nonetheless, despite substantial technological challenges in the implementation of effective haptic feedback, the spread of haptics appears to be inevitable in RMIS. Several multipurpose haptic devices are presented in the literature and some of them are commercially available. In general, the mechanism of haptic feedback in these technologies can be categorized into kinesthetic (force feedback) devices, skin deformation, and/or vibration devices and haptic surfaces [107]. The most commonly used kinesthetic haptic interface is arguably the PHANToM (originally SensAble Technologies, now Geomagic, United States), which uses a serial kinematic structure to provide three DoFs of force feedback, six DOFs of positional sensing, and low free-space impedance [108]. The largest PHANToM model has a range of motion equivalent to the human arm, with a maximum exertable force of 22 N in each axis. An alternative to serial manipulators is devices with parallel kinematic configurations which offer high stiffness and low inertia (locating motors in the base) at the expense of a more limited workspace. These devices are often augmented to provide haptic grasping, a common requirement in RMIS, by mounting an actuated griper onto the parallel device with an additional rotational DoF at the interface [109]. A popular example of this arrangement is the Sigma-7 (Force Dimension, United States), which provides seven DoFs force feedback using a delta-based parallel structure, with up to 20 N and 400 mNm for translational forces and torques, respectively, and a maximum grasping force of 8 N using a motorized active wrist. In comparison, the HD2 (Quanser, Canada) has a larger workspace but a reduced maximum continuous force and torque of 11 N and 0.950 N m, respectively, while the Virtuose 6D (Haption, Germany) has a workspace

15. Haptics in surgical robots

Despite the lack of tactile sensing in current commercial surgical robots, it is a vibrant research field driven by clinical need. Minimizing tissue trauma is a key challenge, particularly in the delivery of intravenous catheters and needles where distally mounted tactile sensors can provide valuable information. The inherent size constraints are suited to optical sensing techniques (the reader is directed to a comprehensive review of optical sensing in MIS for further examples by [92]), for example, a one DoF tactile sensor using an optical Fabry
Perot interferometry technique was integrated into the needle of an MRI-guided tele-robotic system for percutaneous prostate surgery. This approach enables MRI compatibility and high sensitivity (0.01% of 0
20 N range) at small scales (i.e., 0.7 mm diameter needles), although overall cost may limit application in other areas [93]. Fabrication of cost-effective sensor systems at this scale can be challenging, particularly considering single-use devices. The recent development of “pop-up” fabrication techniques offers one potential solution which is well suited for single-use surgical applications, highlighted by an instrumented catheter using a light-modulation force sensor for measurement of tissue contact force, as shown in Fig. 15.6 [94]. Measurement of contact pressures is also critical in general surgery, where tactile sensors play a dual role; first in facilitating safe and appropriate interaction, second used as an assessment tool to interrogate tissue properties [78]. These sensors can be broadly divided into single-point force sensors and tactile “arrays” which map an area. Singlepoint force sensors have been deployed for tissue assessment where a probe is scanned across target tissue by the robotic system, for example, using a “wheeled” optical probe to map liver disease [95] or a magnetic palpation probe for the da Vinci research platform [96]. While single-axis pressure sensing has been translated into a commercially available laparoscopic grasper using optical fiber Bragg sensing [97] it is multiaxis sensing which represents state of the art. Noteworthy examples include tactile forceps for the Raven II surgical platform with four DoFs force sensing [98] and similar forceps for the S-Surge platform (Fig. 15.6) with five DoFs force and torque [99]. Research into tactile arrays for surgery has sought to increase spatial resolution, measurement performance, and physical robustness through the use of innovative sensing methods which include piezoelectric elements [100], MRIcompatible optical fibers [101], varying resistivity in soft conductive polymers [81], and conductive liquid in microfluidic channels [102]. Despite their relative complexity, low-cost “disposable” tactile arrays have been developed using capacitive sensing [103]. Notably, the group has also combined this technology with an ultrasound transducer into a single multimodal probe, compatible with the da Vinci platform (Fig. 15.6) for improved assessment of tissue state [104]. Other work has combined piezoresistive and optical sensing systems [105] to similar effect, and highlights that multimodal tactile sensing is likely to be a key focus in future research.

254

Handbook of Robotic and Image-Guided Surgery

FIGURE 15.7 Commercially available haptic devices categorized based on their actuation mechanisms into parallel/serial kinematics and magnetic levitation: (A) Omega.6, (B) Sigma.7, (C) HD2 high-definition haptic device, and (D) Virtuose 6D TAO. Courtesy (A and B) Force Dimension, Switzerland, (C) Quanser, Canada, and (D) Haption, Germany.

matching that of the human arm and a corresponding maximum force and torque of 35 N and 3 N m. These systems were compared in a study to identify the most appropriate and comfortable system for completing a surgical task and the PHANToM Premium 3 was found to offer the best overall performance, potentially relating to the similarity of its kinematic structure to that of the human arm [110]. In recent years, research has explored the development of kinematic mechanisms to improve haptic manipulators. Lambert and Herder developed a combined parallel and serial device which allows haptic grasping and six DoFs motion while locating drive motors in the base [111]. Similarly, a new six DoFs coupled-parallel device, Delta Haptic, is able to provide a large, singularity-free workspace with the low inertial characteristics associated with parallel mechanisms [112]. In addition, advances in actuator technology have enabled the design of haptic devices which improve stability and transparency in master
slave teleoperation, in particular those devices employing magnetorheological fluids as a means to passively vary interface compliance [113
115], or using novel approaches, for instance the Maglev200 uses electromagnetic coupling to impart forces to the user (Butterfly Haptics, Pittsburgh, Pennsylvania, United States) (Fig. 15.7). In the last decade, researchers began to examine whether focusing on the tactile component of interface forces could bring advantages in terms of the size, complexity, cost, and wearability of haptic feedback systems. Tangential skin stretch (shear force) and vibration have been investigated by actuating a surface relative to the skin through friction and normal indentation. Devices have been developed to stretch the skin of the fingertip and thus convey tactile information to the user for stiffness discrimination in palpation [116] and to convey cues for navigation [117]. Prattichizzo et al. developed a wearable device that is worn on the fingerpad, and that applies a normal force on the fingerpad skin [118]. The device has been used to provide valuable feedback in surgical training tasks including needle insertion and pegtransfer [119]. Quek et al. developed a custom six DoFs tactile stylus which uses a combination of skin stretch and normal indentation to convey haptic information while avoiding the potential stability concerns found with direct force feedback systems [120]. It is important to consider how these technologies can be integrated into surgical robots, two

Haptics in Surgical Robots Chapter | 15

255

15.3.3

Human interaction

Section 15.1.1 highlights the virtues of haptic information within surgery and draws attention to the fact that humans combine multiple sources of information in order to obtain optimal state estimates and are able to learn to switch between different sources of information when undertaking a particular goal-directed action. In support of this conjecture, survey data show that surgeons do not necessarily miss the absence of haptic feedback [127], and it is demonstrably the case that robotic procedures are being completed successfully by surgeons across the world despite the absence of some forms of haptic information. These facts suggest that it should be possible to create laparoscopic and robotic systems that provide different information signals that can compensate for the degradation or removal of information normally obtained through the haptic modality. Moreover, augmented surgical devices may make it possible to provide more salient informational cues that possess a lower signal-to-noise ratio than those normally available to the surgeon. These signals could be fed to the human haptic system or be provided via another sensory modality. In order to develop “haptic-enabled robotic systems” (which we will define as systems that enable access to the information normally made available through haptics), it is essential to understand how information is processed by the human brain. In any given environment, our sensory systems are noisy and the incoming signals open to ambiguity. For the brain to resolve this uncertainty and interact effectively with the environment, it must make inferences about the state of the world from imperfect knowledge. Numerous models of cognitive processing propose that these inferences are resolved in a Bayesian fashion [128,129]. Whether the underlying neural processes are indeed Bayesian (or involve other mechanisms that result in outputs that merely look Bayesian in nature) is a matter of debate, but there is much evidence to suggest that information processing within the brain results in Bayesian type outcomes. Moreover, a Bayesian framework presents an instructive way of considering how surgeons use information in the operating theater. This can be illustrated through the example of a surgeon trying to decide whether tissue is malignant on the basis of a number of sources of information (e.g., visual and haptic “observations”), each of which is subject to noise. In the Bayesian approach, the imperfect knowledge of the exact parameter values is accounted for through probability distributions. The specification of prior information requires that knowledge about the parameters is expressed in

15. Haptics in surgical robots

prime examples are work by Culjat to relay tactile information using a pneumatic balloon actuator mounted onto the hand controls of a da Vinci robotic system [121], while Kuchenbecker has pioneered VerroTouch, a system providing vibration feedback in robotic surgery, demonstrated on the dVRK. VerroTouch measures high-bandwidth vibrotactile information from robotic instruments and relays this to the surgeon’s fingertips using voice coil actuators [122]. An interesting adjunct to these technologies is recent research into “haptic surfaces” which employ morphable physical surfaces to allow cutaneous feedback (by changing tactile properties based on location, e.g., a surface with variable friction) or simultaneous kinesthetic and cutaneous feedback [107]. The integration of haptic feedback into surgical robots remains largely experimental and it is a nontrivial task to ensure the combined sensory, feedback, and control elements of a system perform in a reliable and stable manner. It is therefore instructive to examine the state of research-grade and precommercial robotic systems to understand current challenges and opportunities. Raven II is an open-architecture platform developed for collaborative research on the advancement of RMIS. The system comprises two cable-driven seven DoFs arms and a surgeon interface console complete with haptic feedback to a pair of three-fingered control devices [123]. Preceyes is a platform for vitreoretinal microsurgery. The system features a master
slave approach, with tremor control, motion scaling, and haptic force feedback from the microsurgical instrument tips [75]. Little detailed information is available on the system, likely due to its commercialization through a company which aims to obtain regulatory approval in 2019 and commercial availability in 2020. Currently it is undergoing a clinical trial and was the first system to perform robotic eye surgery [124,125]. Eye Robot is a cooperatively controlled hand-over-hand system for retinal and vitreoretinal microsurgery which senses forces exerted by the surgeon on the tool handle, and moves to comply, filtering out any tremor [126]. While haptic feedback systems can be considered a limiting factor, preventing greater uptake of haptics in surgical robotics, it is also evident that there is a diverse range of technology being developed. Force feedback systems are more common than tactile displays, but the latter offer the promising opportunity to augment and enhance surgical robot control without the stability concerns associated with direct force feedback. However, with increasing demand for haptic displays across other sectors, it seems likely that they will change from specialist subsystems toward modular commodities, greatly facilitating their uptake in future surgical robots.

256

Handbook of Robotic and Image-Guided Surgery

terms of a prior distribution, which is formulated independently of the observations. This probability density is then used, together with the observations, to obtain the posterior distribution using Bayes theorem: pðθjxÞ 5 Ð

f ðxjθ Þ π ðθÞ f ðxjθ Þ π ðθÞ d ðθÞ

where x 5 (x1, x2, . . ., xn) is the vector of information sources, π(θ) the prior probability density of the parameters, f(x|θ) the likelihood of the observations, and p(θ|x) the posterior probability density of the parameters given the observations. There is much evidence to show that the human sensory system is able to flexibly integrate different sources of information in a (Bayesian) statistically optimal manner [130]. It has been demonstrated empirically that the accuracy with which we can make perceptual judgments increases with the number of independent sensory signals [131,132] and, conversely, uncertainty increases as information sources reduce [133]. Increasing uncertainty places larger pressures on the offline cognitive systems underpinning human decision-making and can reduce the capacity of working memory and attentional resources [134]. It follows that increasing the availability and quality of information has the potential to improve the perceptual judgments of surgeons and this, in turn, might improve their surgical performance. Nevertheless, the Bayesian nature of human perception flags the potential complexities involved with providing multiple information sources. The perceptual judgments of the actor will not just be a function of the information provided but will also be a function of the prior knowledge that the human has about the reliability of that information. This means that the human can erroneously place high confidence in a source of information that is not warranted in the environment in which it is operating. For example, visual information is likely to provide precise estimates of hand position under conditions where there is good lighting and a rich abundance of visual cues (e.g., Ref. [135]). Thus it can be seen that sensorimotor performance can be affected because of the “priors” that exist in a human operator. It is also the case that the decisions made by a surgeon will be a function of the information provided by the system and the surgeon’s existing knowledge (beliefs). This is again captured by Bayes’ theorem. The strength of a surgeon’s belief that a patient has a cancerous lump can be defined probabilistically as a real number between zero and one. The surgeon’s prior belief that the probability (p) of the event that a lump is cancerous (θ) can therefore be expressed as p(θ). The surgeon will have a prior belief about the probability of the lump being cancerous but will seek to test that belief by obtaining more information during the surgery. To obtain more information, the surgeon may start the process of palpation and, in open surgery, through the mechanoreceptors located with the hands receive information about the likelihood of θ. The surgeon can use this information to update his or her beliefs about the proposition that the lump is cancerous. Formally, Bayes’ theorem expresses this as: pðθjxÞ 5

pðxjθÞ pðθÞ pðxÞ

where p(θ|x) is the likelihood of observing a given level of tissue stiffness given that the lump is cancerous, and p(x) is the sum of the probabilities of observing cancerous tissue. In short, the available haptic information coupled with the prior expectation of finding cancerous tissue determines the probability that the tissue will be classified as cancerous. This highlights that strong priors can dominate perceptual experiences and also indicates a separation between offline cognitive judgments and lower level sensorimotor control decisions. It is clear that careful consideration of such subjective biases is required when designing systems that enable the provision of additional sensory information [136,137]. Given that the human decision-making process is bounded by our information processing capacities, it is crucial to consider how information should be fed back to the user. For example, augmenting the visual display on a master console with real-time data streams could increase access to information, but the benefits may be negligible for surgeons at the sharp end of the learning curve because cognitive resources are allocated to other fundamental aspects of the task. This consideration of how a human interacts with a robotic surgical system shows that it is important for careful thought to be given to the ways in which information is presented to the end-user. It is reasonable to conjecture that robotic systems which enable access to the information normally made available through haptic perception will allow improved surgical performance. Nevertheless, further work is required to understand how to effectively integrate this type of feedback into robot systems so that utility outweighs possible costs.

15.4

Future perspectives

The opportunity for haptics in surgical robots has long been recognized but to date it remains largely confined to research studies. However, there is good reason to believe that now a combination of clinical, technical, and commercial factors have emerged to create the right conditions for widespread commercial translation of the technology.

Haptics in Surgical Robots Chapter | 15

257

The clinical needs, and equally expectations, for surgical robots have evolved since their inception early in this millennium. From the first systems which focused on robotics for general and orthopedic surgery, the market for surgical robots has now expanded dramatically to encompass a wide array of specialties [32]. Within each specialty robotic systems are being used to support an ever-expanding range of procedures. With these clinical advances come the recognition and demand for provision of haptic feedback, from urology [138], colorectal surgery [139], gastroenterology [140], vascular [65], plastics [72], neurosurgery [73], and microsurgery [75]. While each specialty brings its own particular requirements, key trends are evident which have particular resonance to haptics.

These must be considered in the context of ongoing healthcare challenges, particularly the need for efficiency in the surgical environment. Robotic systems should demonstrate a cost
benefit advantage to justify the increased complexity they often bring to the operating theater. Similarly, it is critical that haptic technology promotes adoption of surgical robots, rather than acting as a barrier due to lengthier learning curves and procedural complexity [23]. Fortunately, haptic technology is now maturing to the point at which these clinical requirements and challenges can be reasonably addressed, as demonstrated by the introduction of the first commercial systems with haptic capabilities. The burgeoning surgical robotics research field has catalyzed this process, both through the development of individual enabling technologies in sensing and feedback, and in research platforms like the Raven surgical system. The move toward micromanipulation is being supported by advances in haptic sensing technology, with examples of both force and tactile sensors at the millimeter scale. Equally, the development of high-fidelity haptic feedback systems is necessary to improve delicate and dexterous tasks such as manipulation of soft-tissue structures. Close attention must be paid to providing haptic feedback in a form and function which integrates with, and improves, the surgical experience. Inappropriate implementations of haptic feedback risk cognitively overloading the surgeon with streams of potentially unnecessary or misleading information. The emergence of research into increasing (robotic) autonomy in surgical robotic systems will likely have a significant impact on the form and function of haptic feedback. Autonomy can be described as a spectrum in which control is ceded to robotic systems, from “low-autonomy” (e.g., virtual fixtures) through task autonomy (suturing) and beyond to surgeons acting in a supervisory capacity [141]. This may initiate a move away from high-bandwidth, low-latency feedback to support real-time “surgeon in the loop” control, toward intermittent but higher fidelity feedback to inform more complex decisions (e.g., identification of tumor margins). These needs will require further exploration of haptic sensing, particularly the application of multimodal sensing systems combined with data fusion and analysis techniques to provide rich data for improved feedback. Accompanying these shifts in clinical need and technological capability is a commercial market more receptive to haptics. This is perhaps best exemplified in general surgery where competition has recently flourished, ending the market dominance enjoyed by Intuitive Surgical and driving the adoption of innovative features like haptics to differentiate between competing systems [36]. However, there must be a strong health-economic argument for the technology, which demonstrates that the clinical benefit obtained from integrating haptic technology justifies the added expense and complexity. This is particularly apt as the surgical robotics industry focuses on reducing traditionally high capital and maintenance costs [142], thus reducing margins for technology costs. In addition, it is important to recognize that obtaining regulatory approval for innovative technology is a key part of translating research into clinical practice. This encompasses metrics, defining consistent measures to demonstrate your technology meets relevant medical device standards, to which systems must comply before receiving regulatory approval (e.g., FDA or CE marking). Surgical robots, and the inclusion of haptics, are now being explicitly recognized and adopted into these practices, for example, the US National Institute of Standards and Technology identified “. . . critical performance metrics for force and haptic feedback” and “Development of performance metrics for evaluating the overall input/output motion of teleoperated surgical robots” as key priorities for its surgical robotics governance, with similar initiatives being made in European bodies [29]. It should be hoped that these standards can help guide the development of research into haptic systems for surgical robots and speed up their adoption into clinical practice so they can bring patients the benefits they have long promised.

15. Haptics in surgical robots

1. Robots are operating at smaller scales. For example, supporting microsurgical procedures like submillimeter vessel anastomoses and nerve repair necessitates both precise movement and regulation of force [72]. 2. Interacting with delicate anatomical structures. For example, performing dissection in neurosurgery [73]. 3. Enhancing the surgical experience. Robotic systems have utility beyond making precise movement and should support more advanced functionality. For example, providing information to support improved assessment and decision-making [37]. 4. Robotic systems should bring improved patient outcomes. For example, using haptics to reduce surgical errors [31].

258

Handbook of Robotic and Image-Guided Surgery

15.5

Conclusion

This chapter has taken a necessarily broad view of haptics in surgical robotics in which we have considered technical, psychological, clinical, and commercial facets of this flourishing field, from the concept of haptics through to their potential to bring clinical advances in a new generation of surgical robotics. We have seen that haptics is an umbrella term encompassing a complex human sensory process, and the study of how this behaves (the field of psychophysics) is as important for surgical robotics as the technological advances necessary to realize these features in clinical practice. The surgical robotic landscape has reached a point of expansion into different features, and surgical fields. There is related clinical demand for haptic technology to underpin this innovation. This has the potential to improve surgical performance and even enable new approaches, particularly in microsurgical procedures. However, to be commercially competitive and clinically viable these new systems must demonstrate a clear cost
benefit. Many challenges remain in the general field of surgical robotics and haptics technology can play a role in addressing these. Increasing levels of task automation and intraoperative tissue assessment are two key areas in which haptic feedback will play a key role. Consequently, although the role of the surgeon may change with increased automation, the use of haptics seems likely to increase to match the current ubiquity of vision systems.

Acknowledgment This chapter is supported by the UK NIHR MedTech Co-operative in Surgical Technologies (a research network linked to the UK NHS) to provide insight into current surgical practice and associated clinical challenges.

References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20]

Fechner GT. Elements of psychophysics. Holt, Rinehart and Winston; 1966. Heidelberger M. Nature from within: Gustav Theodor Fechner and his psychophysical worldview. University of Pittsburgh Press; 2004. Heller MA. The psychology of touch. Psychology Press; 2013. Vallbo AB, Johansson RS. Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Hum Neurobiol 1984;3:3
14. Johansson RS, Flanagan JR. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat Rev Neurosci 2009;10:345
59. Available from: https://doi.org/10.1038/nrn2621. Grunwald M. Human haptic perception: basics and applications. Springer Science & Business Media; 2008. Proske U, Gandevia SC. The kinaesthetic senses. J Physiol 2009;587:4139
46. Available from: https://doi.org/10.1113/jphysiol.2009.175372. Fulkerson M. The unity of haptic touch. Philos Psychol 2011;24:493
516. Available from: https://doi.org/10.1080/09515089.2011.556610. Gallace A, Spence C. In touch with the future: the sense of touch from cognitive neuroscience to virtual reality. Oxford: Press, Oxford University; 2014. Jones LA, Lederman SJ. Human Hand Function. New York: Oxford University Press; 2006. Sutter C, Drewing K, Mu¨sseler J. Multisensory integration in action control. Front Psychol 2014;5:544. Soo-Chul L, Hyung-Kew L, Joonah P. Role of combined tactile and kinesthetic feedback in minimally invasive surgery. Int J Med Robot 2014;11:360
74. Available from: https://doi.org/10.1002/rcs.1625. Culbertson H, Schorr SB, Okamura AM. Haptics: the present and future of artificial touch sensations. Annu Rev Control Robot Auton Syst 2018;11225:1
12. Available from: https://doi.org/10.1146/annurev-control-060117. Culmer P, Barrie J, Hewson R, Levesley M, Mon-Williams M, Jayne D, et al. Reviewing the technological challenges associated with the development of a laparoscopic palpation device. Int J Med Robot 2012;8:146
59. Available from: https://doi.org/10.1002/rcs.1421. Mushtaq F, O’Driscoll C, Smith F, Wilkins D, Kapur N, Lawton R. Contributory factors in surgical incidents as delineated by a confidential reporting system. Ann R Coll Surg Engl 2018;100:401
5. Available from: https://doi.org/10.1308/rcsann.2018.0025. Jamieson ES, Chandler JH, Culmer PR, Manogue M, Mon-Williams M, Wilkie RM. Can virtual reality trainers improve the compliance discrimination abilities of trainee surgeons? IEEE Eng Med Biol 2015;466
9. Available from: https://doi.org/10.1109/EMBC.2015.7318400. Nitsch V, Faber B. A meta-analysis of the effects of haptic interfaces on task performance with teleoperation systems. IEEE Trans Haptics 2013;6:387
98. Available from: https://doi.org/10.1109/TOH.2012.62. Weber B, Eichberger C. The benefits of haptic feedback in telesurgery and other teleoperation systems: a meta-analysis. Hum Comput Interact Int 2015;9177. Available from: https://doi.org/10.1007/978-3-319-20684-4. Gwilliam JC, Pezzementi Z, Jantho E, Okamura AM, Hsiao S. Human vs. robotic tactile sensing: detecting lumps in soft tissue. Haptics Symp 2010 IEEE 2010;21
8. Available from: https://doi.org/10.1109/HAPTIC.2010.5444685. Herron DM, Marohn M, Group TS-MRSC. A consensus document on robotic surgery. Surg Endosc 2008;22:313
25. Available from: https:// doi.org/10.1007/s00464-007-9727-5.

Haptics in Surgical Robots Chapter | 15

259

15. Haptics in surgical robots

[21] Westebring-van der Putten EP, Goossens RHM, Jakimowicz JJ, Dankelman J. Haptics in minimally invasive surgery—a review. Minim Invasive Ther Allied Technol 2008;17:3
16. Available from: https://doi.org/10.1080/13645700701820242. [22] Hokayem PF, Spong MW. Bilateral teleoperation: an historical survey. Automatica 2006;42:2035
57. Available from: https://doi.org/10.1016/ j.automatica.2006.06.027. [23] Enayati N, Momi ED, Ferrigno G. Haptics in robot-assisted surgery: challenges and benefits. IEEE Rev Biomed Eng 2016;9:49
65. Available from: https://doi.org/10.1109/RBME.2016.2538080. [24] Okamura AM. Haptic feedback in robot-assisted minimally invasive surgery. Curr Opin Urol 2009;19:102. [25] Maddahi Y, Zareinia K, Sepehri N, Sutherland G. Surgical tool motion during conventional freehand and robot-assisted microsurgery conducted using neuroArm. Adv Robot 2016;30:621
33. Available from: https://doi.org/10.1080/01691864.2016.1142394. [26] Trejos AL, Jayender J, Perri MT, Naish MD, Patel RV, Malthaner RA. Robot-assisted tactile sensing for minimally invasive tumor localization. Int J Robot Res 2009;28:1118
33. Available from: https://doi.org/10.1177/0278364909101136. [27] Koehn JK, Kuchenbecker KJ. Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery. Surg Endosc 2015;29:2970
83. Available from: https://doi.org/10.1007/s00464-014-4030-8. [28] Abbott JJ, Marayong P, Okamura AM. Haptic virtual fixtures for robot-assisted manipulation Robot Res 2007;28:49
64Springer, Berlin, Heidelberg . Available from: https://doi.org/10.1007/978-3-540-48113-3_5. [29] Dı´az CE, Ferna´ndez R, Armada M, Garcı´a F. A research review on clinical needs, technical requirements, and normativity in the design of surgical robots. Int J Med Robot 2017;13. Available from: https://doi.org/10.1002/rcs.1801. [30] Sutherland GR, Maddahi Y, Gan LS, Lama S, Zareinia K. Robotics in the neurosurgical treatment of glioma. Surg Neurol Int 2015;6:S1
8. Available from: https://doi.org/10.4103/2152-7806.151321. [31] Greenberg JA. From open to MIS: robotic surgery enables surgeons to do more with less. Ann Surg 2018;267:220. Available from: https://doi. org/10.1097/SLA.0000000000002335. [32] Amirabdollahian F, Livatino S, Vahedi B, Gudipati R, Sheen P, Gawrie-Mohan S, et al. Prevalence of haptic feedback in robot-mediated surgery: a systematic review of literature. J Robot Surg 2018;12:11
25. Available from: https://doi.org/10.1007/s11701-017-0763-4. [33] Freschi C, Ferrari V, Melfi F, Ferrari M, Mosca F, Cuschieri A. Technical review of the da Vinci surgical telemanipulator. Int J Med Robot 2013;9:396
406. Available from: https://doi.org/10.1002/rcs.1468. [34] Rassweiler JJ, Goezen AS, Klein J, Liatsikos E. New robotic platforms Robot Urol 2018;3
38Springer, Cham . Available from: https://doi.org/ 10.1007/978-3-319-65864-3_1. [35] Peters BS, Armijo PR, Krause C, Choudhury SA, Oleynikov D. Review of emerging surgical robotic technology. Surg Endosc 2018;1
20. Available from: https://doi.org/10.1007/s00464-018-6079-2. [36] Rao PP. Robotic surgery: new robots and finally some real competition!. World J Urol 2018;1
5. Available from: https://doi.org/10.1007/ s00345-018-2213-y. [37] Yang G-Z, Bellingham J, Dupont PE, Fischer P, Floridi L, Full R, et al. The grand challenges of science robotics. Sci Robot 2018;3:eaar7650. Available from: https://doi.org/10.1126/scirobotics.aar7650. [38] Gidaro S, Buscarini M, Ruiz E, Stark M, Labruzzo A. Telelap Alf-X: a novel telesurgical system for the 21st century. Surg Technol Int 2012;22:20
5. [39] Rossitto C, Gueli Alletti S, Fanfani F, Fagotti A, Costantini B, Gallotta V, et al. Learning a new robotic surgical device: Telelap Alf X in gynaecological surgery. Int J Med Robot 2016;12:490
5. Available from: https://doi.org/10.1002/rcs.1672. [40] Alletti SG, Rossitto C, Cianci S, Restaino S, Costantini B, Fanfani F, et al. Telelap ALF-X vs standard laparoscopy for the treatment of earlystage endometrial cancer: a single-institution retrospective cohort study. J Minim Invasive Gynecol 2016;23:378
83. Available from: https:// doi.org/10.1016/j.jmig.2015.11.006. [41] Spinelli A, David G, Gidaro S, Carvello M, Sacchi M, Montorsi M, et al. First experience in colorectal surgery with a new robotic platform with haptic feedback. Colorectal Dis 2017. Available from: https://doi.org/10.1111/codi.13882. [42] Abdel Raheem A, Troya IS, Kim DK, Kim SH, Won PD, Joon PS, et al. Robot-assisted Fallopian tube transection and anastomosis using the new REVO-I robotic surgical system: feasibility in a chronic porcine model. BJU Int 2016;118:604
9. Available from: https://doi.org/10.1111/ bju.13517. [43] Lim JH, Lee WJ, Park DW, Yea HJ, Kim SH, Kang CM. Robotic cholecystectomy using Revo-I Model MSR-5000, the newly developed Korean robotic surgical system: a preclinical study. Surg Endosc 2017;31:3391
7. Available from: https://doi.org/10.1007/s00464-016-5357-0. [44] Kim DK, Park DW, Rha KH. Robot-assisted partial nephrectomy with the REVO-I robot platform in porcine models. Eur Urol 2016;69:541
2. Available from: https://doi.org/10.1016/j.eururo.2015.11.024. [45] Hirzinger G, Hagn U. Flexible heart surgery. Ger Res 2010;32:4
7. Available from: https://doi.org/10.1002/germ.201090020. [46] Sutherland GR, Latour I, Greer AD, Fielding T, Feil G, Newhook P. An image-guided magnetic resonance-compatible surgical robot. Neurosurgery 2008;62:286
93. Available from: https://doi.org/10.1227/01.neu.0000315996.73269.18. [47] Maddahi Y, Zareinia K, Gan LS, Sutherland C, Lama S, Sutherland GR. Treatment of glioma using neuroArm surgical system. BioMed Res Int 2016;2016:9734512. Available from: https://doi.org/10.1155/2016/9734512. [48] Al-Ahmad A, Grossman JD, Wang PJ. Early experience with a computerized robotically controlled catheter system. J Interv Card Electrophysiol 2005;12:199
202. Available from: https://doi.org/10.1007/s10840-005-0325-y. [49] Kesner SB, Howe RD. Robotic catheter cardiac ablation combining ultrasound guidance and force control. Int J Robot Res 2014;33:631
44. Available from: https://doi.org/10.1177/0278364913511350.

260

Handbook of Robotic and Image-Guided Surgery

[50] Russo AD, Fassini G, Conti S, Casella M, Monaco AD, Russo E, et al. Analysis of catheter contact force during atrial fibrillation ablation using the robotic navigation system: results from a randomized study. J Interv Card Electrophysiol 2016;46:97
103. Available from: https://doi.org/ 10.1007/s10840-016-0102-0. [51] Riga CV, Bicknell CD, Wallace D, Hamady M, Cheshire N. Robot-assisted antegrade in-situ fenestrated stent grafting. Cardiovasc Intervent Radiol 2009;32:522
4. Available from: https://doi.org/10.1007/s00270-008-9459-5. [52] Kanagaratnam P, Koa-Wing M, Wallace DT, Goldenberg AS, Peters NS, Davies DW. Experience of robotic catheter ablation in humans using a novel remotely steerable catheter sheath. J Interv Card Electrophysiol 2008;21:19
26. Available from: https://doi.org/10.1007/s10840-0079184-z. [53] Saliba W, Reddy VY, Wazni O, Cummings JE, Burkhardt JD, Haissaguerre M, et al. Atrial fibrillation ablation using a robotic catheter remote control system. J Am Coll Cardiol 2008;51:2407
11. Available from: https://doi.org/10.1016/j.jacc.2008.03.027. [54] Rillig A, Schmidt B, Di Biase L, Lin T, Scholz L, Heeger CH, et al. Manual versus robotic catheter ablation for the treatment of atrial fibrillation: the man and machine trial. JACC Clin Electrophysiol 2017;3:875
83. Available from: https://doi.org/10.1016/j.jacep.2017.01.024. [55] Spinelli A, David G, Gidaro S, Carvello M, Sacchi M, Montorsi M, et al. First experience in colorectal surgery with a new robotic platform with haptic feedback. Colorectal Dis 2018;20:228
35. Available from: https://doi.org/10.1111/codi.13882. [56] Hagn U, Konietschke R, Tobergte A, Nickl M, Jo¨rg S, Ku¨bler B, et al. DLR MiroSurge: a versatile system for research in endoscopic telesurgery. Int J Comput Assist Radiol Surg 2010;5:183
93. Available from: https://doi.org/10.1007/s11548-009-0372-4. [57] Stark M, Pomati S, D’Ambrosio A, Giraudi F, Gidaro S. A new telesurgical platform—preliminary clinical results. Minim Invasive Ther Allied Technol 2015;24:31
6. Available from: https://doi.org/10.3109/13645706.2014.1003945. [58] Bozzini G, Gidaro S, Taverna G. Robot-assisted laparoscopic partial nephrectomy with the ALF
X robot on pig models. Eur Urol 2016;69:376
7. Available from: https://doi.org/10.1016/j.eururo.2015.08.031. [59] Konietschke R, Zerbato D, Richa R, Tobergte A, Poignet P, Fro¨hlich FA, et al. Integration of new features for telerobotic surgery into the MiroSurge system. Appl Bionics Biomech 2011;8:253
65. Available from: https://doi.org/10.3233/ABB-2011-0037. [60] Tobergte A, Helmer P, Hagn U, Rouiller P, Thielmann S, Grange S, et al. The sigma.7 haptic interface for MiroSurge: a new bi-manual surgical console. In: 2011 IEEERSJ international conference on intelligent robots and systems; 2011. p. 3023
30. https://doi.org/10.1109/ IROS.2011.6094433. [61] Sutherland GR, Wolfsberger S, Lama S, Zarei-nia K. The evolution of neuroArm. Neurosurgery 2013;72(Suppl. 1):27
32. Available from: https://doi.org/10.1227/NEU.0b013e318270da19. [62] Maddahi Y, Gan LS, Zareinia K, Lama S, Sepehri N, Sutherland GR. Quantifying workspace and forces of surgical dissection during robotassisted neurosurgery. Int J Med Robot 2016;12:528
37. Available from: https://doi.org/10.1002/rcs.1679. [63] Lin T, Ouyang F, Kuck K-H, Tilz R. THERMOCOOLs SMARTTOUCHs CATHETER—the evidence so far for contact force technology and the role of VISITAGt Module. Arrhythmia Electrophysiol Rev 2014;3:44
7. Available from: https://doi.org/10.15420/ aer.2011.3.1.44. [64] Fanfani F, Monterossi G, Fagotti A, Rossitto C, Gueli Alletti S, Costantini B, et al. The new robotic TELELAP ALF-X in gynecological surgery: single-center experience. Surg Endosc 2016;30:215
21. Available from: https://doi.org/10.1007/s00464-015-4187-9. [65] Rijanto E, Sugiharto A, Utomo S, Rahmayanti R, Afrisal H, Nanayakkara T. Trends in robot assisted endovascular catheterization technology: a review. In: 2017 International conference on robotics, biomimetics, & intelligent computational systems (Robionetics); 2017. p. 34
41. https:// doi.org/10.1109/ROBIONETICS.2017.8203433. [66] Mattei TA, Rodriguez AH, Sambhara D, Mendel E. Current state-of-the-art and future perspectives of robotic technology in neurosurgery. Neurosurg Rev 2014;37:357
66. Available from: https://doi.org/10.1007/s10143-014-0540-z. [67] Goto T, Hongo K, Kakizawa Y, Muraoka H, Miyairi Y, Tanaka Y, et al. Clinical application of robotic telemanipulation system in neurosurgery. Case report. J Neurosurg 2003;99:1082
4. Available from: https://doi.org/10.3171/jns.2003.99.6.1082. [68] Hongo K, Kobayashi S, Kakizawa Y, Koyama J-I, Goto T, Okudera H, et al. NeuRobot: telecontrolled micromanipulator system for minimally invasive microneurosurgery-preliminary results. Neurosurgery 2002;51:985
8 [discussion 988]. [69] Hongo K, Goto T, Miyahara T, Kakizawa Y, Koyama J, Tanaka Y. Telecontrolled micromanipulator system (NeuRobot) for minimally invasive neurosurgery. Acta Neurochir Suppl 2006;98:63
6. [70] Lang MJ, Greer AD, Sutherland GR. Intra-operative robotics: NeuroArm. Acta Neurochir Suppl 2011;109:231
6. Available from: https://doi. org/10.1007/978-3-211-99651-5_36. [71] Vivekananda U, Henderson A, Murphy DG, Althoefer K, Seneviratne L, Dasgupta P. The science behind haptics in robotic urological surgery. BJU Int 2009;104:433
4. Available from: https://doi.org/10.1111/j.1464-410X.2009.08521.x. [72] Dobbs TD, Cundy O, Samarendra H, Khan K, Whitaker IS. A systematic review of the role of robotics in plastic and reconstructive surgery— from inception to the future. Front Surg 2017;4. Available from: https://doi.org/10.3389/fsurg.2017.00066. [73] L’Orsa R, Macnab CJB, Tavakoli M. Introduction to haptics for neurosurgeons. Neurosurgery 2013;72(Suppl. 1):139
53. Available from: https://doi.org/10.1227/NEU.0b013e318273a1a3. [74] Karthik K, Colegate-Stone T, Dasgupta P, Tavakkolizadeh A, Sinha J. Robotic surgery in trauma and orthopaedics: a systematic review. Bone Jt J 2015;97-B:292
9. Available from: https://doi.org/10.1302/0301-620X.97B3.35107. [75] Griffin JA, Zhu W, Nam CS. The role of haptic feedback in robotic-assisted retinal microsurgery systems: a systematic review. IEEE Trans Haptics 2017;10:94
105. Available from: https://doi.org/10.1109/TOH.2016.2598341.

Haptics in Surgical Robots Chapter | 15

261

15. Haptics in surgical robots

[76] Van der Meijden OA, Schijven MP. The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review. Surg Endosc 2009 Jun 1;23(6):1180
90. [77] Kappassov Z, Corrales J-A, Perdereau V. Tactile sensing in dexterous robot hands—review. Robot Auton Syst 2015;74:195
220. Available from: https://doi.org/10.1016/j.robot.2015.07.015. [78] Howe RD, Peine WJ, Kantarinis DA, Son JS. Remote palpation technology. IEEE Eng Med Biol Mag 1995;14:318
23. Available from: https://doi.org/10.1109/51.391770. [79] Lee M, Nicholls H. Review Article Tactile sensing for mechatronics—a state of the art survey. Mechatronics 1999;9:1
31. Available from: https://doi.org/10.1016/S0957-4158(98)00045-2. [80] Eltaib MEH, Hewit JR. Tactile sensing technology for minimal access surgery

a review. Mechatronics 2003;13:1163
77. Available from: https://doi.org/10.1016/S0957-4158(03)00048-5. [81] Schostek S, Schurr MO, Buess GF. Review on aspects of artificial tactile feedback in laparoscopic surgery. Med Eng Phys 2009;31:887
98. Available from: https://doi.org/10.1016/j.medengphy.2009.06.003. [82] Trejos AL, Patel RV, Naish MD. Force sensing and its application in minimally invasive surgery and therapy: a survey. Proc Inst Mech Eng Part C J Mech Eng Sci 2010;224:1435
54. Available from: https://doi.org/10.1243/09544062JMES1917. [83] Tiwana MI, Redmond SJ, Lovell NH. A review of tactile sensing technologies with applications in biomedical engineering. Sens Actuators Phys 2012;179:17
31. Available from: https://doi.org/10.1016/j.sna.2012.02.051. [84] Shademan A, Decker RS, Opfermann JD, Leonard S, Krieger A, Kim PCW. Supervised autonomous robotic soft tissue surgery. Sci Transl Med 2016;8:337ra64. Available from: https://doi.org/10.1126/scitranslmed.aad9398. [85] Peirs J, Clijnen J, Reynaerts D, Brussel HV, Herijgers P, Corteville B, et al. A micro optical force sensor for force feedback during minimally invasive robotic surgery. Sens Actuators Phys 2004;115:447
55. Available from: https://doi.org/10.1016/j.sna.2004.04.057. [86] Gonenc B, Chamani A, Handa J, Gehlbach P, Taylor RH, Iordachita I. 3-DOF force-sensing motorized micro-forceps for robot-assisted vitreoretinal surgery. IEEE Sens J 2017;17:3526
41. Available from: https://doi.org/10.1109/JSEN.2017.2694965. [87] Spiers AJ, Thompson HJ, Pipe AG. Investigating remote sensor placement for practical haptic sensing with EndoWrist surgical tools. In: 2015 IEEE world haptics conference on WHC; 2015. p. 152
7. https://doi.org/10.1109/WHC.2015.7177706. [88] Li Y, Hannaford B. Gaussian process regression for sensorless grip force estimation of cable-driven elongated surgical instruments. IEEE Robot Autom Lett 2017;2:1312
19. Available from: https://doi.org/10.1109/LRA.2017.2666420. [89] Yu L, Wang W, Zhang F. External force sensing based on cable tension changes in minimally invasive surgical micromanipulators. IEEE Access 2018;6:5362
73. Available from: https://doi.org/10.1109/ACCESS.2017.2788498. [90] Cianchetti M, Ranzani T, Gerboni G, Nanayakkara T, Althoefer K, Dasgupta P, et al. Soft robotics technologies to address shortcomings in today’s minimally invasive surgery: the STIFF-FLOP approach. Soft Robot 2014;1:122
31. Available from: https://doi.org/10.1089/soro.2014.0001. [91] Arezzo A, Mintz Y, Allaix ME, Arolfo S, Bonino M, Gerboni G, et al. Total mesorectal excision using a soft and flexible robotic arm: a feasibility study in cadaver models. Surg Endosc 2017;31:264
73. Available from: https://doi.org/10.1007/s00464-016-4967-x. [92] Abushagur AAG, Arsad N, Reaz MI, Bakar AAA. Advances in bio-tactile sensors for minimally invasive surgery using the fibre Bragg grating force sensor technique: a survey. Sensors 2014;14:6633
65. Available from: https://doi.org/10.3390/s140406633. [93] Su H, Shang W, Li G, Patel N, Fischer GS. An MRI-guided telesurgery system using a Fabry-Perot interferometry force sensor and a pneumatic haptic device. Ann Biomed Eng 2017;45:1917
28. Available from: https://doi.org/10.1007/s10439-017-1839-z. [94] Gafford JB, Wood RJ, Walsh CJ. Self-assembling, low-cost, and modular mm-scale force sensor. IEEE Sens J 2016;16:69
76. Available from: https://doi.org/10.1109/JSEN.2015.2476368. [95] Puangmali P, Liu H, Seneviratne LD, Dasgupta P, Althoefer K. Miniature 3-axis distal force sensor for minimally invasive surgical palpation. IEEEASME Trans Mechatron 2012;17:646
56. Available from: https://doi.org/10.1109/TMECH.2011.2116033. [96] McKinley S, Garg A, Sen S, Kapadia R, Murali A, Nichols K, et al. A single-use haptic palpation probe for locating subcutaneous blood vessels in robot-assisted minimally invasive surgery. In: 2015 IEEE international conference on automation science and engineering (CASE); 2015. p. 1151
8. https://doi.org/10.1109/CoASE.2015.7294253. [97] Alleblas CCJ, Vleugels MPH, Coppus SFPJ, Nieboer TE. The effects of laparoscopic graspers with enhanced haptic feedback on applied forces: a randomized comparison with conventional graspers. Surg Endosc 2017;31:5411
17. Available from: https://doi.org/10.1007/s00464-017-5623-9. [98] Kim U, Lee DH, Yoon WJ, Hannaford B, Choi HR. Force sensor integrated surgical forceps for minimally invasive robotic surgery. IEEE Trans Robot 2015;31:1214
24. Available from: https://doi.org/10.1109/TRO.2015.2473515. [99] Kim U, Kim YB, So J, Seok DY, Choi HR. Sensorized surgical forceps for robotic-assisted minimally invasive surgery. IEEE Trans Ind Electron 2018;1. Available from: https://doi.org/10.1109/TIE.2018.2821626. [100] Qasaimeh MA, Sokhanvar S, Dargahi J, Kahrizi M. PVDF-based microfabricated tactile sensor for minimally invasive surgery. J Microelectromechanical Syst 2009;18:195
207. Available from: https://doi.org/10.1109/JMEMS.2008.2008559. [101] Ahmadi R, Packirisamy M, Dargahi J, Cecere R. Discretely loaded beam-type optical fiber tactile sensor for tissue manipulation and palpation in minimally invasive robotic surgery. IEEE Sens J 2012;12:22
32. Available from: https://doi.org/10.1109/JSEN.2011.2113394. [102] Hammond FL, Kramer RK, Wan Q, Howe RD, Wood RJ. Soft tactile sensor arrays for force feedback in micromanipulation. IEEE Sens J 2014;14:1443
52. Available from: https://doi.org/10.1109/JSEN.2013.2297380. [103] Naidu AS, Patel RV, Naish MD. Low-cost disposable tactile sensors for palpation in minimally invasive surgery. IEEEASME Trans Mechatron 2017;22:127
37. Available from: https://doi.org/10.1109/TMECH.2016.2623743.

262

Handbook of Robotic and Image-Guided Surgery

[104] Naidu AS, Naish MD, Patel RV. A breakthrough in tumor localization: combining tactile sensing and ultrasound to improve tumor localization in robotics-assisted minimally invasive surgery. IEEE Robot Autom Mag 2017;24:54
62. Available from: https://doi.org/10.1109/ MRA.2017.2680544. [105] Bandari NM, Ahmadi R, Hooshiar A, Dargahi J, Packirisamy M. Hybrid piezoresistive-optical tactile sensor for simultaneous measurement of tissue stiffness and detection of tissue discontinuity in robot-assisted minimally invasive surgery. J Biomed Opt 2017;22:077002. Available from: https://doi.org/10.1117/1.JBO.22.7.077002. [106] van Beurden MHPH, IJsselsteijn WA, Juola JF. Effectiveness of stereoscopic displays in medicine: a review. 3D Res 2012;3:3. Available from: https://doi.org/10.1007/3DRes.01(2012)3. [107] Culbertson H, Schorr SB, Okamura AM. Haptics: the present and future of artificial touch sensation. Annu Rev Control Robot Auton Syst 2018;1:385
409. Available from: https://doi.org/10.1146/annurev-control-060117-105043. [108] Salisbury JK, Srinivasan MA. Phantom-based haptic interaction with virtual objects. IEEE Comput Graph Appl 1997;17:6
10. Available from: https://doi.org/10.1109/MCG.1997.1626171. [109] Escobar-Castillejos D, Noguez J, Neri L, Magana A, Benes B. A review of simulators with haptic devices for medical training. J Med Syst 2016;40:104. Available from: https://doi.org/10.1007/s10916-016-0459-8. [110] Zareinia K, Maddahi Y, Ng C, Sepehri N, Sutherland GR. Performance evaluation of haptic hand-controllers in a robot-assisted surgical system: Evaluation of haptic devices in a robot-assisted surgical system. Int J Med Robot 2015;11:486
501. Available from: https://doi.org/ 10.1002/rcs.1637. [111] Lambert P, Herder J. A novel parallel haptic device with 7 degrees of freedom. In: 2015 IEEE world haptics conference on WHC; 2015. p. 183
8. https://doi.org/10.1109/WHC.2015.7177711. [112] Vulliez M, Zeghloul S, Khatib O. Design strategy and issues of the Delthaptic, a new 6-DOF parallel haptic device. Mech Mach Theory 2018;128:395
411. Available from: https://doi.org/10.1016/j.mechmachtheory.2018.06.015. [113] Najmaei N, Asadian A, Kermani MR, Patel RV. Design and performance evaluation of a prototype MRF-based haptic interface for medical applications. IEEEASME Trans Mechatron 2016;21:110
21. Available from: https://doi.org/10.1109/TMECH.2015.2429140. [114] Rizzo R, Sgambelluri N, Scilingo EP, Raugi M, Bicchi A. Electromagnetic modeling and design of haptic interface prototypes based on magnetorheological fluids. IEEE Trans Magn 2007;43:3586
600. Available from: https://doi.org/10.1109/TMAG.2007.901351. [115] Song Y, Guo S, Yin X, Zhang L, Wang Y, Hirata H, et al. Design and performance evaluation of a haptic interface based on MR fluids for endovascular tele-surgery. Microsyst Technol 2018;24:909
18. Available from: https://doi.org/10.1007/s00542-017-3404-y. [116] Schorr SB, Quek ZF, Romano RY, Nisky I, Provancher WR, Okamura AM. Sensory substitution via cutaneous skin stretch feedback. In: 2013 IEEE international conference on robotics and automation; 2013. p. 2341
6. https://doi.org/10.1109/ICRA.2013.6630894. [117] Guinan AL, Hornbaker NC, Montandon MN, Doxon AJ, Provancher WR. Back-to-back skin stretch feedback for communicating five degree-of-freedom direction cues. In: 2013 World haptics conference on WHC; 2013. p. 13
8. https://doi.org/10.1109/ WHC.2013.6548377. [118] Prattichizzo D, Pacchierotti C, Rosati G. Cutaneous force feedback as a sensory subtraction technique in haptics. IEEE Trans Haptics 2012;5:289
300. Available from: https://doi.org/10.1109/TOH.2012.15. [119] Meli L, Pacchierotti C, Prattichizzo D. Sensory subtraction in robot-assisted surgery: fingertip skin deformation feedback to ensure safety and improve transparency in bimanual haptic interaction. IEEE Trans Biomed Eng 2014;61:1318
27. Available from: https://doi.org/10.1109/ TBME.2014.2303052. [120] Quek ZF, Schorr SB, Nisky I, Provancher WR, Okamura AM. Sensory substitution of force and torque using 6-DoF tangential and normal skin deformation feedback. In: 2015 IEEE international conference on robotics and automation (ICRA); 2015. p. 264
71. https://doi.org/ 10.1109/ICRA.2015.7139010. [121] Culjat MO, King C-H, Franco ML, Lewis CE, Bisley JW, Dutson EP, et al. A tactile feedback system for robotic surgery. In: Conference proceedings of the annual international conference on IEEE engineering in medicine and biology society; 2008. p. 1930
4. https://doi.org/ 10.1109/IEMBS.2008.4649565. [122] McMahan W, Gewirtz J, Standish D, Martin P, Kunkel JA, Lilavois M, et al. Tool Contact acceleration feedback for telerobotic surgery. IEEE Trans Haptics 2011;4:210
20. Available from: https://doi.org/10.1109/TOH.2011.31. [123] Hannaford B, Rosen J, Friedman DW, King H, Roan P, Cheng L, et al. Raven-II: an open platform for surgical robotics research. IEEE Trans Biomed Eng 2013;60:954
9. Available from: https://doi.org/10.1109/TBME.2012.2228858. [124] Adee S. Eye, robot. New Sci 2016;232:24. Available from: https://doi.org/10.1016/S0262-4079(16)32079-6. [125] de Smet MD, Meenink TCM, Janssens T, Vanheukelom V, Naus GJL, Beelen MJ, et al. Robotic assisted cannulation of occluded retinal veins. PLoS One 2016;11:e0162037. Available from: https://doi.org/10.1371/journal.pone.0162037. [126] Iordachita I, Sun Z, Balicki M, Kang JU, Phee SJ, Handa J, et al. A sub-millimetric, 0.25 mN resolution fully integrated fiber-optic force-sensing tool for retinal microsurgery. Int J Comput Assist Radiol Surg 2009;4:383
90. Available from: https://doi.org/10.1007/s11548-009-0301-6. [127] Hagen ME, Meehan JJ, Inan I, Morel P. Visual clues act as a substitute for haptic feedback in robotic surgery. Surg Endosc 2008;22:1505
8. Available from: https://doi.org/10.1007/s00464-007-9683-0. [128] Friston K. Learning and inference in the brain. Neural Netw 2003;16:1325
52. Available from: https://doi.org/10.1016/j.neunet.2003.06.005. [129] Friston K, Mattout J, Kilner J. Action understanding and active inference. Biol Cybern 2011;104:137
60. Available from: https://doi.org/ 10.1007/s00422-011-0424-z.

Haptics in Surgical Robots Chapter | 15

263

15. Haptics in surgical robots

[130] Ernst MO, Banks MS. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 2002;415:429
33. Available from: https://doi.org/10.1038/415429a. [131] Stein BE, Meredith MA. Multisensory integration. Ann NY Acad Sci 1990;608:51
70. Available from: https://doi.org/10.1111/j.17496632.1990.tb48891.x. [132] Paraskevopoulos E, Herholz SC. Multisensory integration and neuroplasticity in the human cerebral cortex. Transl Neurosci 2013;4:337
48. Available from: https://doi.org/10.2478/s13380-013-0134-1. [133] Tresilian JR, Mon-Williams M, Kelly BM. Increasing confidence in vergence as a cue to distance. Proc Biol Sci 1999;266:39
44. Available from: https://doi.org/10.1098/rspb.1999.0601. [134] Mushtaq F, Bland AR, Schaefer A. Uncertainty and cognitive control. Front Psychol 2011;2. Available from: https://doi.org/10.3389/ fpsyg.2011.00249. [135] Al-Saud L, Mushtaq F, Mirghani I, Balkhoyor A, Keeling A, Manogue M, et al. Drilling into the functional significance of stereopsis: the impact of stereoscopic information on surgical performance. Ophthalmic Physiol Opt 2017;37:498
506. [136] Mon-Williams M, Murray AH. The size of the visual size cue used for programming manipulative forces during precision grip. Exp Brain Res 2000;135:405
10. [137] Flanagan JR, Beltzner MA. Independence of perceptual and sensorimotor predictions in the size
weight illusion. Nat Neurosci 2000;3:737. [138] Rassweiler JJ, Autorino R, Klein J, Mottrie A, Goezen AS, Stolzenburg J-U, et al. Future of robotic surgery in urology. BJU Int 2017;120:822
41. Available from: https://doi.org/10.1111/bju.13851. [139] Whealon M, Vinci A, Pigazzi A. Future of minimally invasive colorectal surgery. Clin Colon Rectal Surg 2016;29:221
31. Available from: https://doi.org/10.1055/s-0036-1584499. [140] Yeung BPM, Chiu PWY. Application of robotics in gastrointestinal endoscopy: a review. World J Gastroenterol 2016;22:1811
25. Available from: https://doi.org/10.3748/wjg.v22.i5.1811. [141] Yang G-Z, Cambias J, Cleary K, Daimler E, Drake J, Dupont PE, et al. Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci Robot 2017;2:eaam8638. Available from: https://doi.org/10.1126/scirobotics.aam8638. [142] Higgins RM, Frelich MJ, Bosler ME, Gould JC. Cost analysis of robotic versus laparoscopic general surgery procedures. Surg Endosc 2017;31:185
92. Available from: https://doi.org/10.1007/s00464-016-4954-2.