Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 304
Virtual Reality: Scientific and Technological Challenges 9 Telerobotics This chapter reviews issues and needs in telerobotics. A telerobot is defined for our purposes as a robot controlled at a distance by a human operator, regardless of the degree of robot autonomy. Sheridan (1992c) makes a finer distinction, which depends on whether all robot movements are continuously controlled by the operator (manually controlled teleoperator), or whether the robot has partial autonomy (telerobot and supervisory control). By this definition, the human interface to a telerobot is distinct and not part of the telerobot. Haptic interfaces that mechanically link a human to a telerobot nevertheless share similar issues in mechanical design and control, and the technology survey presented here includes haptic interface development. INTRODUCTION Telerobotic devices are typically developed for situations or environments that are too dangerous, uncomfortable, limiting, repetitive, or costly for humans to perform. Some applications are listed below: Underwater: inspection, maintenance, construction, mining, exploration, search and recovery, science, surveying. Space: assembly, maintenance, exploration, manufacturing, science. Resource industry: forestry, farming, mining, power line maintenance. Process control plants: nuclear, chemical, etc., involving operation, maintenance, decommissioning, emergency.
OCR for page 305
Virtual Reality: Scientific and Technological Challenges Military: operations in the air, undersea, and on land. Medical: patient transport, disability aids, surgery, monitoring, remote treatment. Construction: earth moving, building construction, building and structure inspection, cleaning and maintenance. Civil security: protection and security, firefighting, police work, bomb disposal. This chapter is divided into five sections, which represent one way of categorizing past developments in telerobotics: Remote manipulators Remote vehicles Low-level control Supervisory control Real-time computing A recent survey including these and other topics is provided by Sheridan (1992a). Relation to Robotics Telerobots may be remotely controlled manipulators or vehicles. The distinction between robots and telerobots is fuzzy and a matter of degree. Although the hardware is the same or is similar, robots require less human involvement for instruction and guidance than do telerobots. There is a continuum of human involvement, from direct control of every aspect of motion, to shared or traded control, to nearly complete robot autonomy. Any robot manipulator can be hooked up to a haptic interface and hence become a telerobot. Similarly, any vehicle can be turned into a teleoperated mobile robot. There are many examples in the literature of different industrial robots that have been used as telerobots, even though that was not the original intended use. For example, a common laboratory robot, the PUMA 560, has frequently been teleoperated (Funda et al., 1992; Hayati et al., 1990; Kan et al., 1990; Lee et al., 1990; Salcudean et al., 1992). There have also been a number of telerobots specifically designed as such, often with a preferred haptic interface. The design issues for robots, telerobots, and haptic interfaces are essentially the same (although Pennington, 1986, seeks to identify differences). Often telerobots have to be designed for hazardous environments, which require special characteristics in the design. Industrial robots have most often been designed for benign indoor environments. Why don't we do everything with robots, rather than involve humans in telerobotic control? We can't, because robots are not that capable. Often there is no substitute for human cognitive capabilities for planning
OCR for page 306
Virtual Reality: Scientific and Technological Challenges and human sensorimotor capabilities for control, especially for unstructured environments. In telerobotics, these human capabilities are imposed on the robot device. The field of robotics is not that old (35 years), and the task of duplicating (let alone improving upon) human abilities has proven to be an extremely difficult endeavor; it would be disturbing if it were not so. There is a tendency to overextrapolate from the few superior robot abilities, such as precise positioning and repetitive operation. Yet robots fare poorly when adaptation and intelligence are required. They do not match the human sensory abilities of vision, audition, and touch, human motor abilities in manipulation and locomotion, or even the human physical body in terms of compact and powerful musculature that adapts and self-repairs, and especially in terms of a compact and portable energy source. Hence in recent years many robotics researchers have turned to telerobotics, partly out of frustration. Nevertheless, the long-term goal of robotics is to produce highly autonomous systems that overcome hard problems in design, control, and planning. As advances are made in robotics, they will feed through to better and more independent telerobots. For example, much of the recent work in low-level teleoperator control is influenced by developments in robot control. Often, the control ideas developed for autonomous robots have been used as the starting points for slave, and to a lesser extent, master controllers. Advances in high-level robot control will help in raising the level of supervisory control. Yet the flow of advances can go both ways. By observing what is required for successful human control of a telerobot, we may infer some of what is needed for autonomous control. There are also unique problems in telerobotic control, having to do with the combination of master, slave, and human operator. Even if each individual component is stable in isolation, when hooked together they may be unstable. Furthermore, the human represents a complex mechanical and dynamic system that must be considered. Relation to Virtual Environments Telerobotics encompasses a highly diversified set of fundamental issues and supporting technologies (Vertut and Coiffet, 1985a, 1985b; Todd, 1986; Engelberger, 1989; Sheridan, 1992b). More generally, telerobots are representative of human-machine systems that must have sufficient sensory and reactive capability to successfully translate and interact within their environment. The fundamental design issues encountered in the field of telerobotics, therefore, have significant overlap with those that are and will be encountered in the development of veridical virtual environments (VEs). Within the virtual environment, the human-machine system
OCR for page 307
Virtual Reality: Scientific and Technological Challenges must allow translation of viewpoint, interaction with the environment, and interaction with autonomous agents. All this must occur through mediating technologies that provide sensory feedback and control. The human-machine interface aspects of telerobotic systems are, therefore, highly relevant to VE research and development from a device, configuration, and human performance perspective. Yet the real-environment aspect of telerobotics distinguishes it from virtual environments to some extent. Telerobots must: interact in complex, unstructured, physics-constrained environments, deal with incomplete, distorted, and noisy sensor data, including limited views, and expend energy which may limit action. The corresponding features of virtual environments are more benign: Form, complexity, and physics of environments are completely controllable. Interactions based on physical models must be computed. Virtual sensors can have an omniscient view and need not deal with noise and distortions. The ability to move within an environment and perform tasks is not energy-limited. Despite such simplifications, virtual environments play an important role in telerobotic supervisory control. A large part of the supervisor's task is planning, and the use of computer-based models has a potentially critical role. The virtual environment is deemed an obvious and effective way to simulate and render hypothetical environments to pose ''what would happen if" questions, run the experiment, and observe the consequences. Simulations are also an important component of predictive displays, which represent an important method of handling large time delays. VE research and development promises to revolutionize the field of multimodal, spatially oriented, interactive human-machine interface technology and theory to an extent that has not been achievable in the robotics field. The two fields should therefore not be viewed as disparate but rather as complementary endeavors whose goals include the exploration of remote environments and the creation of adaptable human-created entities. REMOTE MANIPULATORS This section reviews remote manipulators from standpoints of kinematics, actuation, end effectors, and sensors. Specific examples of robots
OCR for page 308
Virtual Reality: Scientific and Technological Challenges and telerobots in this review will tend to be drawn from more recent devices; some of the older telerobots are reviewed in Vertut and Coiffet (1985a). A review with similar categories is provided by Sheridan (1992a). Kinematics In this section we describe the number of joints and their geometrical layout. Some of the issues within kinematics are discussed below. General Positioning Capabilities A manipulator requires at least 6 degrees of freedom (DOFs) to achieve arbitrary positions and orientations. When a manipulator has exactly 6 DOFs, it is said to be general purpose. Examples include many industrial robots, such as the PUMA 560, as well as a number of commercial telerobots (Kraft, Shilling, Western Electric, ISE). The space shuttle's Remote Manipulator System (RMS), designed by Spar Aerospace, is another example. If there are less than 6 DOFs, the device is said to be overconstrained. Often a task will require fewer DOFs, such as positioning (x - y) and orientating (a rotation θ about the normal z axis) restricted to a plane (a 3-DOF task). Another popular example is the SCARA robot geometry with 4 DOFs; the motions are planar with an extra translation in the direction normal to the plane. A modified SCARA robot, to which one joint was added, is being used for hip replacement surgery (Paul et al., 1992). Teleoperated heavy machinery usually is overconstrained; excavators have 4 DOFs (Khoshzaban et al., 1992). An important subclass of mechanisms is a spherical joint, for which 3 rotations and no translations are required; this joint is useful in head-neck and head-eye systems. An example is the head-neck system described by Tachi et al. (1989). A 2-DOF pan-tilt system is presented in Hirose et al. (1992) and Hirose and Yokoyama (1992). Other pan-tilt systems are reviewed by Bederson et al. (1992), who also proposed a novel head-eye pan-tilt system employing a spherical motor. A 3-DOF parallel-drive head-neck system (Gosselin and Lavoie, 1993) has the potential for very fast motion, with some limitations in rotations. Redundancies When there are 7 or more DOFs, the mechanism is underconstrained. The extra DOFs may be used to fulfill secondary criteria (to general positioning), such as obstacle avoidance. There has been a lot of research in robotics addressing redundancy resolution. The human arm is a redundant
OCR for page 309
Virtual Reality: Scientific and Technological Challenges 7-DOF mechanism (not counting shoulder shrug). Commercial examples include the Sarcos Dextrous Arm (Jacobsen et al., 1990a, 1990b, 1991), the Robotics Research Arm, and the Omnidirectional Arm (Rosheim, 1990). Laboratory examples include the Langley Laboratory Telerobotic Manipulator, the CESARm (Jansen and Kress, 1991), and the Anthropomorphic Tele-existence Slave Robot (Tachi et al., 1989, 1990a, 1990b). The Special Purpose Dextrous + Manipulator (SPDM) designed by Spar Aerospace will have 8 DOFs. For direct control by the human arm, an exoskeleton master with 7 DOFs can be used to guide a slave 7-DOF manipulator. Hand-controllers (ground-based systems) are usually 6-DOF devices. To control a redundant arm, either the resolution is left to the discretion of the computer or an auxiliary control (such as a knob) must be manipulated. Workspace The term workspace describes the extent of the volume within which the manipulator may position the end-point, relative to the size of the manipulator. Certain manipulator geometries are known to offer superior workspaces (Hollerbach, 1985; Paden and Sastry, 1988; Vijaykumar et al., 1985); interestingly, these geometries are similar to the human arm geometry. Serial Versus Parallel Mechanism In a serial mechanism, joints are cascaded. The workspace is the union of motions of the joints, and hence is large. Because a proximal link must carry the weight of distal links, these arms may be slower, heavier, and less strong. Most manipulators (whether in robotics or telerobotics) are serial mechanisms, because a large workspace is often important. In a parallel mechanism, several independent linkages meet at a common terminal link (the end effector). The workspace is the intersection of the independent linkages, and hence is small. One independent linkage does not carry the weight of the others, so these devices can be lightweight, strong, and fast. A prominent example of a parallel mechanism is the Stewart platform used in flight simulators. The human hand can also be viewed as a parallel mechanism; there are 5 independent 4-DOF linkages that can contact an object. A lot of recent research in robotics has focused on parallel mechanisms, to exploit their intrinsic advantages for specific tasks suited to their restricted workspaces. Examples include a 6-DOF parallel manipulator to be teleoperated for excavation (Arai et al., 1992), a 3-DOF microrobot based on beam bending (Hunter et al., 1991), and 6-DOF parallel hand controllers (Hayward et al., 1993; Long and
OCR for page 310
Virtual Reality: Scientific and Technological Challenges Collins, 1992). Landsberger and Sheridan (1985) have designed a cable-driven parallel mechanism. The parallel-drive hydraulic shoulder joint in Hayward et al. (1993) uses actuator redundancy to increase the workspace. Kinematic Solvability For serial mechanisms, the forward kinematics (find the end-point position given the joint angles) is easy, but the inverse kinematics (find the joint angles given the end-point position) is hard. The inverse kinematics is complicated unless the mechanism has a special structure: either a spherical joint or a planar pair (Tsai and Morgan, 1985). Almost all industrial robots have these special structures, but some for design convenience do not, such as the Robotics Research Arm, which has been used in teleoperation. Because of fast computers, iterative techniques to solve the nonlinear kinematics can work in real time. For parallel mechanisms, the reverse is true: inverse kinematics is easy, but forward kinematics is hard (Waldron and Hunt, 1991). Actuation Actuation comprises the force or torque source (henceforth called the actuator) and any transmission element to connect to a joint or link. The actuation is the primary determinant of performance (speed, accuracy, strength). A survey of actuators for robotics is presented by Hollerbach et al. (1992). Macrorobots For macro motion control, standard actuators are electric, hydraulic, or pneumatic. For smaller robots (human size and less), electric actuators dominate. For larger robots (e.g., cranes), hydraulic actuators dominate. Electric Motors and Drives Electric actuators are the most convenient, because the power source is an electric plug. For pneumatic or hydraulic systems, air supplies and hydraulic supplies often make them much less easy to install and maintain. Electric actuators, however, are weak relative to their mass; hence payloads are not great. To amplify motor torque and couple high-rev motors to low-rev joints, nearly all electrical motor drives employ some form of transmission element, primarily gears. Thus nearly all commercial electric robots employ gears of some form. For space applications, the space shuttle RMS employs special high-ratio hollow planetary gears (Wu et al., 1993). These same gears will be employed in the two-armed SPDM (Mack et al., 1991).
OCR for page 311
Virtual Reality: Scientific and Technological Challenges The Flight Telerobotic Servicer (FTS) system produced by Martin Marietta employed harmonic drives (Andary and Spidaliere, 1993; Ollendorf, 1990), which are also commonly employed in industrial robots (e.g., the Robotics Research Arm, ASEA robots). Advanced spherical joint designs employing gears have been produced by Rosheim (1990). Yet gears bring serious drawbacks: substantial friction, backlash, and flexibility. The performance consequences are poor joint torque control, poor end-point force control, reduced accuracy, and slower response. To overcome these drawbacks, some attempts are made to model the gear dynamics so that they may be compensated for in a controller (Armstrong-Helouvry, 1991). Other attempts include better gear designs that reduce friction and losses; examples include the Artisan arm (Vischer and Khatib, 1990) and the ROTEX manipulator (Hirzinger, 1987; Hirzinger et al., 1993), which is meant for space laboratory teleoperation. Cable or tendon drives (including belts) are another common way to reduce arm weight, by remote location of the actuators. A number of master-slave systems have been designed using cable drives. More recent examples include the FRHC from JPL (Hayati et al., 1990; Kan et al., 1990) and the Whole Arm Manipulator (Salisbury et al., 1990) (both Salisbury's design). Nearly all multifingered robot hands employ tendon or cable routing; space constraints preclude direct mounting of actuators at joints (Jacobsen et al., 1986). Another recent development is the integration of gears and actuators. For example, Rosheim (1990) has used miniature integrated lead screw mechanisms for finger joint-mounted actuators. Similar systems, developed originally for ROTEX (Hirzinger et al., 1993), are now commercially available (Wittenstein Motion Control GmbH). A related concept is the harmonic motor, in which the rotor rolls along the inside of the stator (Jacobsen et al., 1989). Analogous to harmonic drives, with harmonic motors, high effective gear ratios can be obtained. To avoid problems with transmission elements, direct drive actuators and robots have become popular in the past decade to provide smooth and controllable motion (An et al., 1988; Asada and Youcef-Toumi, 1987). Examples of direct drive telerobots include the CMU DDArm II (Papanikolopoulos and Khosla, 1992) and MEISTER (Sato and Hirai, 1988); MEISTER is also an example of a direct drive master. Advances in electric motor technology continue, and particularization to robotics is a key to enhanced performance. One example of a new electric motor specifically designed for direct-drive robotics is the McGill/MIT Direct Drive Motor (Hollerbach et al., 1993). An important new area of development in mechanism design is magnetic bearings and levitation, which seek to avoid problems of transmission elements, including bearings as structural members. In principle,
OCR for page 312
Virtual Reality: Scientific and Technological Challenges devices with magnetic bearings should produce the smoothest motions. Hollis (Hollis et al., 1990) has designed a 6-axis magnetically levitated wrist, which can be used either as a hand controller or as a robot end effector. Salcudean (Salcudean et al., 1992) has developed a teleoperated robot, in which the master is a magnetically levitated wrist and the slave is an industrial 6-axis robot (coarse positioner) and the end effector is a magnetically levitated wrist. Because the wrist's range of motion is small, the hand controller is used in rate mode for large excursions and in proportional mode for fine motions. Although not magnetically suspended, a 2-axis force-reflecting mouse was developed by Salcudean using the same actuator elements (Kelley and Salcudean, 1993). Another area under development, related to microelectromechanical systems (MEMS), is electrostatic actuators. All the electric motors mentioned above work by magnetic attraction. At small scales, the electrostatic effect is more favorable than the magnetic effect (Trimmer, 1989). By using small air gaps and many poles, powerful muscle-like actuators are conceivable. In terms of what has been realized on the macro scale, Higuchi has demonstrated lightweight but very strong linear electrostatic actuators (Niino et al., 1993). Hydraulic Actuators Hydraulic actuators offer the most strength for the size. There are a number of commercial telerobot systems that are hydraulic, such as the Kraft arm, the Shilling arm, and the International Submarine Engineering (ISE) arm. Teleoperation of excavators (which are hydraulic) with hand controllers has also been pursued (e.g., Khoshzaban et al., 1992). To some extent, hydraulics have received a bad reputation due to concerns about leakage and controllability. Advances such as the Sarcos Dextrous Arm are a counterpoint to these concerns. Pneumatic Actuators Pneumatic actuators are intermediate between electric and hydraulic drives, in terms of force produced for a given size and mass. There are very few high-performance robots powered pneumatically, because of control problems associated with the compressibility of air. Perhaps the most advanced example is the Utah/MIT Dextrous Hand Master (Jacobsen et al., 1986). Micromotion Actuators One of the more exciting new areas under development is micromotion robotics, in which (macro) robots are able to position precisely down to 1 nanometer (Dario, 1992). As a counterpoint to simulated molecular docking (Ouh-young et al., 1988), these robots would actually be able to manipulate molecules. The development of true microsize robots is still somewhere off in the future, but the new area of microelectromechanical
OCR for page 313
Virtual Reality: Scientific and Technological Challenges systems (MEMS) holds promise for developing the proper components: structures, actuators, and sensors. For micromotion control, piezoelectric actuators dominate. They are used in scanning tunneling microscopes (STMs) and atomic force microscopes (AFMs). Hollis has used a magnetically levitated hand controller to control an STM (Hollis et al., 1990). A stacked actuator consisting of a linear voice coil motor in series with a piezoelectric element was the basis for Hunter's telemicrorobot (Hunter et al., 1991). Hatamura (Hatamura and Morishita, 1990) has teleoperated a 6-axis force-reflecting nanorobot whose individual axes are flexure elements activated by piezoelectrics; the masters are two bilateral joystick mechanisms, and vision is provided by a stereo scanning electron miscroscope (SEM). Shape memory alloy (SMA) actuators hold considerable promise as a compact but powerful actuator source. Various robotic mechanisms have been proposed that incorporate SMA actuators, including active endoscopes (Dario et al., 1991; Ikuta et al., 1988). A tactile stimulator employing cantilever arrays activated by SMA wires has been developed commercially (TiNi Alloy Company). At the moment, two major drawbacks of SMA are highly nonlinear dynamics and slow response speed. Recent developments by Hunter (Hunter et al., 1991) have sped up the response by 100 times and hold promise for the future. End Effectors Most end effectors on robots or telerobots are unremarkable, usually two-jaw grippers or special purpose tooling. Multifingered robot hands have been developed to provide robots with the same dexterity as the human hand. The major commercial examples are the Stanford/JPL hand (Salisbury, 1985) and the Utah/MIT Dextrous Hand (Jacobsen et al., 1986). Master gloves have been used to teleoperate particularly the Utah/MIT Dextrous Hand (Hong and Tan, 1989; Pao and Speeter, 1989; Rohling and Hollerbach, 1993; Speeter, 1992). Force-reflecting multifingered master-slave systems have been developed by Jacobsen et al. (1990a) and Jau (1992). A 3-DOF hand partly inspired by prosthetics is the end effector for the Sarcos Dextrous Arm. Sensors Sensor technologies for telemanipulators include sensors required to monitor the internal mechanical state of the arm (joint angle sensors and joint torque sensors), the external contact state (wrist force/torque sensors and tactile sensors), and proximity sensors. We do not cover visual
OCR for page 314
Virtual Reality: Scientific and Technological Challenges sensors (cameras) and image processing. Position trackers and inertial sensors are reviewed in Chapter 5. Joint Motion Sensors A review of traditional joint motion sensors is provided by deSilva (1989). For rotary motion, common sensors are potentiometers, optical encoders, resolvers, tachometers, Hall-effect sensors, and rotary variable differential transformers (RVDTs). For linear motion, common sensors are linear variable differential transformers (LVDTs), linear velocity transducers (LVTs), and position sensitive detectors (PSDs). Most of the rotary sensors listed above are analog sensors. Potentiometers are not that favored because of noise and sensitivity problems, and they are hard to make small. For use in robot fingers and hand masters, compact Hall-effect sensors are used in the Utah/MIT Dextrous Hand and in the EXOS Dextrous Hand Master. The resolution is not high (0.2 deg), but is adequate for these devices. The VPL DataGlove employs fiber optic sensing, but this effect is too coarse to be really useful and there have been many complaints about the accuracy of this system. Resolvers and tachometers are suitable for larger actuators and joints, such as robot shoulders and elbows. RVDTs are moderate in size but also have a moderate resolution. The trend is increasingly toward digital sensors. Optical encoders offer the highest resolution; for example, Canon produces an incremental laser rotary encoder with 24 bits of resolution. The Sarcos Dextrous Arm has 18 1/2 bit incremental encoders. The trend is for rotary encoders to become less expensive while maintaining high resolution, to become more compact in size, and to provide high-resolution absolute joint angle readings (Steve Jacobsen, personal communication). For linear transduction, LVDTs and LVTs are common. Resolutions in the range of 10-100 nm are possible for LVDTs. Linear PSDs have been reported to have resolutions in the range of 1-5 nm. Digital linear sensors are being developed with a resolution of 2.5 nm (Steve Jacobsen, personal communication). The ultimate in high-resolution linear measurement is of course interferometry, for which resolutions of 0.1 nm are possible. An additional consideration is the sampling rate while maintaining high resolution; Charette et al. (1993) reported a 1 MHz rate. Future developments should result in reduced size of such sensors and increased use in micromanipulation. With increased resolution such as that provided by optical encoders, the calculation of joint velocity and acceleration from positional data will become more accurate. This calculation is required for precise control
OCR for page 351
Virtual Reality: Scientific and Technological Challenges the huge market forces driving these developments, costs of such systems would run at several hundred dollars and severely threaten PC- or work-station-based VE systems. Parallel Processing and Communications To run processes faster and meet real-time constraints, parallel processing can be advantageous up to a point. That point is defined by losses in throughput and increases in latency due to communication, and by limitations in the ability to parallelize a computation. When there are 10 or more processors to coordinate, it can become a formidable task to avoid conflict and to ensure timely performance. In such cases, it may be best to upgrade to a faster processor to reduce the number required. The main approaches toward connecting processors include a common bus, point-to-point links, and crossbars. A common bus is the most traditional approach; industry standards with a roughly equivalent performance of 40 to 80 Mbytes/s include VME Bus, MultiBus II, FutureBus, EISA, NuBus, SBUS, and PCI Bus. Each computing unit is equipped with fast local memory, and a global memory bank is made available to all computing units (Bejczy, 1987; Chen et al., 1986; Clark, 1989; Narasimham et al., 1988). The advantages of such an architecture include modularity, easy expansion for more performance, and the possibility to mix hardware from different sources (Backes et al., 1989). The disadvantages obviously lie in the bottleneck created by the two shared resources: the global memory and the system bus. Point-to-point links circumvent the bottleneck problem of bus-based architectures to some extent: communication between connected points will be fast, but that between distant nodes in a graph will be slow. Such built-in links are used in most engineering workstations for graphic processors, sound processors, memory access, disk controllers, etc. The advantages include design for minimum cost, but then the system, once configured, offers few possibilities for expansion. Some developers have proposed high-performance central processing units (CPUs) equipped with built-in high-speed point-to-point links (1-20 Mbytes/s). Following this philosophy, Inmos from Europe (now a division of SGS-Thomson) introduced the Transputers series (T800, T9000), which can loosely be classified as RISC computers. These computers are each fitted with four communication channels, which permit the user to design a variety of coarsely parallel architectures (Buehler et al., 1989; Zai et al., 1992). Recently Texas Instruments combined the Transputer concept with its experience in DSP design leading to the C40 with six high-speed links, thus competing head-on with the latest Inmos Transputer T9000. The principal disadvantage of point-to-point communication
OCR for page 352
Virtual Reality: Scientific and Technological Challenges systems, such as those made available with Transputers or C40s, is the lack of industry standards in terms of signal conversion and data acquisition hardware. Whereas such hardware is abundantly available for most bus standards, it is only beginning to become so for point-to-point communication systems, requiring for system implementers much custom electronic design. The recent availability of these powerful processors, which lend themselves to parallel processing, together with commercially available development and run-time software, has virtually eliminated the need to build custom computing platforms and develop custom multiprocessing environments. In the past, this has been necessary and has consumed substantial financial investment and personnel. Examples are the CHIMERA II (Stewart et al., 1989) and Condor (Narasimham et al., 1988) systems. Even though most of these custom systems relied increasingly on standard commercial boards, the efforts in upgrading, to stay compatible with the host operating system as well as the latest run-time boards, constituted a major ongoing investment that fewer and fewer labs are willing to undertake. The same situation of the availability of powerful processors applies to VE and teleoperation systems. This is very fortunate, since it means that all the custom development efforts can be focused on the critical I/O interface, a current bottleneck that may be better served by commercial suppliers in the future. The most generic architecture consists of a full N × M crossbar switching network connecting a set of CPUs and register files totalling N inputs and M outputs. This architecture has the advantage of being capable of reaching the theoretical optimum performance, which minimizes latency and maximizes throughput. An example is a system for control of a complete microrobot system (Hunter et al., 1990; Nielson et al., 1989). The disadvantage is the massive complexity and high cost of the system. Although considerably slower, it is also possible for processors to communicate across networks. Researchers have become interested in running robotic experiments or virtual environment setups across a computer network with nodes located in different cities or even in different continents. In one example, robotic sessions are run among a network of sites across the United States (Graves et al., 1994); in another, a telerobotic experiment has been performed between Japan and the United States (Mitsuishi et al., 1992). This type of work is bound to be increasingly investigated given the high potential for applications. In effect, with such systems, the services of specialists in any area could be requested and put to use without asking the specialists to travel where they are needed. Clearly, the demand on communication networks will follow the same route that is being taken by low-level real-time control systems: high volumes of data transactions, low latency and time precision. It must be
OCR for page 353
Virtual Reality: Scientific and Technological Challenges expected that the performance levels required at the scale of a laboratory site or an application site will need to become available across entire networks. Operating Systems and Development Environments A large number of real-time operating systems are currently in competition to provide the services needed to implement high-performance real-time applications (Gopinath and Thakkar, 1990). These operating systems (OS) may be classified as embedded, full-featured real-time, or hybrid. An embedded operating system is targeted at original equipment manufacturer (OEM) applications, and as such provides only essential facilities, like a real-time executive (real-time multitasking, memory management, and device I/O). It is usually a simplified system offering no support for a file system and must be used as a cross-development tool. Yet most offer hard real-time features essential to VE and teleoperation systems. In this category, SPOX (Spectron Microsystems) has emerged as one of the industry standards. A full-featured real-time OS resembles the more standard UNIX but with additional real-time features. An example of this category is HELIOS by Distributed Software Ltd., which offers an eight-level priority real-time scheduler, virtual message routing, X and Microsoft Windows graphic support, as well as SUN and PC host interfaces. Clearly, processing nodes with microsecond interrupt latency requirements can—and must—be stripped of the inherently slow high-level functionality. In between embedded and full-featured real-time operating systems are the hybrids, the most popular of which is probably VxWorks by Wind River Systems. These typically are real-time operating systems running on dedicated shared-memory VME processor boards and provide a transparent interface to the host UNIX operating system. Despite its high cost ($10,000-20,000) VxWorks has been attractive for its convenient interface to existing UNIX hosts, debugging and development facilities, and the availability of a large number of supported processor and I/O VME cards. Nevertheless, many researchers are quite dissatisfied with the services provided by commercial operating systems, usually for reasons of cost and inefficiency. Because these operating systems are designed with general purpose requirements in mind, the few requirements essential to VE and teleoperation systems are not well addressed, and the resulting systems are cumbersome to use and clumsy in their design. For these reasons, countless custom operating and development systems have been designed and implemented, with their scope limited to one or to a handful of laboratories.
OCR for page 354
Virtual Reality: Scientific and Technological Challenges Experience shows that no single operating system (and no single CPU technology) will satisfy the varied needs of VE and teleoperation systems. This is why real-time extensions to the UNIX systems will most likely remain limited to the higher levels of the hierarchy (soft real time). For low-level control and data acquisition, no single run-time environment has been shown to be satisfactory to the community. HELIOS is probably one of the most suitable operating systems for VE and teleoperation systems to date. It provides all the UNIX-like services on a distributed real-time network, allowing one at the same time to ''peel away" the higher levels of the operating system on part of the network dedicated to time-critical code. In practice, the response is mixed: some users report sufficient control at the device level, some others don't (Poussart, personal communication). Systems like HELIOS are clearly going in the right direction and might be the approach of choice for VE and teleoperation systems in the future. To obtain full processor control on a network of only a few processors, enhanced C compilers, like 3L C by 3L Ltd. are minimal systems, adequate for providing standard host I/O, memory management functions, and multiprocessing facilities (network loading, communication management, load balancing, message routing, micro-kernel for multitasking). RESEARCH NEEDS A major trend in telerobotics is toward higher-level supervisory control, that is to say, toward autonomous robotics (robotics for short). Major drivers in this direction include: (1) communication problems (e.g., delays, low bandwidth, low resolution), especially in space and undersea, (2) burden lifting from the operator (e.g., partially automating repetitive tasks), and (3) performance enhancements (e.g., using local sensing when vision is obscured, assisting in obstacle avoidance). Hence the needs in telerobotics are often the same as the needs in robotics. The line between supervisory control and robotics is quite blurred. At the same time, manually controlled telerobots remain important. One reason is safety, particularly in space and medicine applications, due to uneasiness about the loss of control. For example, developers of the planned space station are concerned that the telerobot not damage the space station structure. Another reason is the inability of robots to handle unstructured environments, which obviates the possibility of even partially automating a task—the reason for having telerobots in the first place. In the control of telerobots, the issues of haptic interfaces, computer-generated environments, and real-time systems are important. Although predominantly covered in other sections of the report, these issues are addressed here to some extent in discussing various needs in telerobotics.
OCR for page 355
Virtual Reality: Scientific and Technological Challenges Handling Communication Delays In space applications, the sentiment for ground-based teleoperation is increasing, for a number of reasons. First, for the planned space station, the actual amount of time spent by astronauts in orbit will be relatively short, and of that time the fraction that could be devoted to teleoperation is shorter yet. Because internal vehicular robots (IVRs) and external vehicular robots (EVRs) will be present the whole time, they would be used more efficiently if operated from the ground when astronauts are not attending. Second, humans are much less effective in space than on the ground; weightlessness seriously affects concentration and attention. Third, sending humans into space is expensive and dangerous. To communicate from the ground, several satellite and computer systems must be traversed. The result is variable delays on the order of 5 s. There is also a need to funnel commands through a central computer at the Johnson Space Flight Center, not only to share the communication channel but also to verify the commands. Again, safety is an overriding concern. The trend is to devolve ground control from Houston to other sites; for example, the recent ROTEX experiment (Brunner et al., 1993; Hirzinger et al., 1993) involved control of this German-built telerobot on the space shuttle from Karlsruhe. The Canadian Space Agency is interested in controlling Canadian-built telerobots from its main facility in St. Hubert, Quebec. Other groups will wish to perform scientific experiments in space from their home locations on the ground; the word telescience has been coined to describe this activity. This devolution will accentuate the delays. Time delays are also important in underseas teleoperation. To avoid power and data tethers, communication with submersibles must rely on relatively slow sonar signals. For example, at a distance of 1,700 m, sound transmission imposes a round-trip delay of 2 s (Sheridan, 1992a). There are two main approaches to handling teleoperation with time delays: (1) control theory approaches that incorporate time delays and (2) predictive displays. In general, the greater the delays, the lower the system gains may be (e.g., stiffness and viscosity) in order to avoid instability. Thus the system response slows down and performance suffers, to an extent that depends on the controller. Recently developed controllers based on passivity approaches guarantee stability but seem to suffer in performance because their gains are too low (Lawn and Hannaford, 1993). Exactly how to formulate a controller that addresses both stability and performance, under various time delays, needs further work. Force reflection under delays is more problematic than position control. For delays approaching a second or more, position control becomes superior to force control (Lawn and Hannaford, 1993). In fact, humans
OCR for page 356
Virtual Reality: Scientific and Technological Challenges are sensitive to extremely small delays in terms of fidelity of force reflection (Biggers et al., 1989). One solution is to implement a force controller, such as impedance control, on the slave side and a position controller on the master side. Thus there is no force feedback to the operator, and the slave force controller is presumed capable enough to handle interaction forces. This requires that the slave force controller be appropriately tuned from a knowledge of the environment and the task. This knowledge can either exist from a priori information about the task or from measurements made on the environment. It is therefore important to get several kinds of information: (1) determine whether to use position control or force reflection as a function of delay and tasks; (2) determine how to configure the slave manipulator's force controller as a function of a priori knowledge of the task; and (3) identify the environmental characteristics through slave robot sensing, if a priori knowledge is not available. The last two needs are essentially robotics problems. For delays greater than 1-2 s, direct manual control becomes ineffective without the aid of predictive displays (Sheridan, 1992a). The main issue then becomes how well the remote manipulator and environment are simulated. For known structured environments, good simulations of the geometrical aspects are quite feasible. Force control can also be simulated, but difficulties are present due to closed-chain dynamics and arbitrary contact conditions. In ROTEX, the predictive displays for force were found to be quite close to actual experimental forces. Even in the case of ROTEX, there are small misalignments and inaccuracies, which are accommodated by local sensing. Force, proximity, and tactile sensing are fused to correct such deviations. For less structured problems, a model of the environment must be built using vision and other sensors. This generic problem in robot vision and mobile robots is far from solved. One envisions that a robot would spend some time mapping its immediate environment, from which a simulation model is extracted; the operator then controls the robot through this simulation model. Accurate, Real-Time Simulations Besides predictive control, other uses for telerobotic simulations are training and mission development. Training simulators prevent damage from being done in an operator's learning phase (e.g., learning a surgical procedure), free expensive telerobotic equipment for actual use (e.g., teleoperation of forestry harvesting equipment), and prepare trainees for equipment that is available only at the remote site (e.g., underwater robots) or that cannot function under normal conditions (e.g., space robots that cannot lift themselves against gravity). Mission development involves
OCR for page 357
Virtual Reality: Scientific and Technological Challenges extensive simulation to work out operational scenarios; this is particularly important in space. Three needs, which are similar for virtual environments, are reemphasized here for telerobotics. Construction of a simulated environment. This involves the software tools for representing real environments, creating a particular virtual environment, and sharing VE modules. As mentioned above, it would also be desirable to "reverse engineer" a real environment into a simulation, by using visual and other sensory recognition and representation methods (see, e.g., Oyama et al., 1993). This is a hard problem, especially if object attributes other than geometry (space occupancy) are to be addressed, such as mass and surface properties. Accurate representation of task dynamics. Although the laws of physics have been around for a long time, often we don't understand detailed mechanical interactions between objects in sufficient detail for simulation. A very basic example is motion of three-dimensional objects under friction, such as sliding, and in collisions (Mason, 1989). The field of robotics is just beginning to enumerate how such objects are expected to behave and to develop appropriate concepts. Another difficulty is representing closed-chain systems under arbitrary contact conditions. Examples are locomotion on different surfaces, the behavior of the operator's hand tissues in the interaction with the haptic interface, and the behavior of the remote manipulator with such compliant environmental substances as human tissues (in surgical applications) and rock or soils (in mining applications). Furthermore, there is an issue of experimentally verifying such task dynamics. The particular robot sensing also needs to be simulated (Brunner et al., 1993). In the continuous domain, accurate finite element models are required, which may be nonlinear. They will require that the constitutive equations for real objects be known or measured. Abrupt changes in boundary conditions, such as simulating the cutting of tissue, are difficult to represent. Real-time computation. In addition to generating graphical images, there are substantial difficulties in real-time computation of a simulated environment. Often classical multibody simulation is more concerned with accurate long-term integration of initial value problems than with computational efficiency. Other intensive computations are simulating and detecting collisions and real-time calculation of finite element models. Needs in real-time computing are discussed in greater detail below. Real-Time Computing The multitude of challenges in real-time computing for VE and
OCR for page 358
Virtual Reality: Scientific and Technological Challenges teleoperation systems can be successfully addressed only with computing solutions offering high performance, ease of use, reconfigurability, expandability, and support for massive and fast I/O. Coarse-grained parallel systems, based on the most powerful computational nodes available, which communicate via high-bandwidth, integrated communication links, are the most promising contenders to satisfy all requirements. This has been recognized by academia and industry alike and explains the early strong support for the Transputer processor family and recently the sweeping popularity of C40-based systems. In this setting, a high-speed intercommunication standard would be highly desirable, in a similar fashion to existing bus standards. This would offer users an immense degree of flexibility, since computational nodes could be mixed and matched across vendors and different processor types. Naturally, this communication standard must go hand in hand with a physical module standard, akin to Texas Instrument's TIM-40 modules. Currently, code development is done on traditional host systems connected to dedicated run-time systems. This results typically in a bottleneck at the interface between the two systems. In addition, communicating between run-time code on the dedicated architecture and host programs is typically problematic, due to the difference in development tools, architecture, and response time. Based on the emerging paradigm of coarse grained parallelism based on standardized communication, the boundary between development and run-time hardware could become transparent. Since the host architecture itself could be based on the same paradigm, communication between host (development system) and controller (run-time system) could be fast and flexible, as there could exist numerous communication links between the two. Furthermore, the boundary could be laid dynamically, depending on the requirements of each system. If the input/output devices for control, teleoperation, and VE hardware adhere to the same communication standard, much effort in system development could be eliminated and progress toward powerful real-time teleoperation and VE systems would substantially accelerate. It is likely that, with the increased interest in telerobotics and VE, there will be sufficient thrust to finally address the I/O standardization issue in addition to reducing cost through volume. It should be reemphasized that, in contrast to I/O system interfacing, custom software developments for real-time multiprocessing systems are no longer necessary, and the effort should rather go toward identifying the most suitable commercial products and focus on industry-academia collaboration to develop one or several compatible standards.
OCR for page 359
Virtual Reality: Scientific and Technological Challenges Better Robot Hardware A goal in teleoperation is that performing a task with a remotely controlled manipulator should feel nearly indistinguishable from directly performing the task with one's own limbs. Shortfalls in this goal are primarily due to the mechanical hardware, both on the master and slave sides. Often, limited-performance masters are hooked up to limited-performance industrial robots. Although comparisons of different master-slave control strategies have been published for these systems, it is unlikely that the results generalize beyond the evaluation of these specific low-performance systems. On the slave side, robot arms and hands are required with sufficient dexterity and responsiveness to match that of the human arm. To achieve such devices requires improvements in actuators, sensors, and structures. Some particular needs are discussed below. Multiaxis, High-Resolution Tactile Sensors The robot must have a comparable sense of touch to a human, even before considering how to transmit the sensation to an operator. Tactile sensors continue to be a problem in robotics: hardly any robots have them, and those that do sense touch only coarsely. Although interesting designs have been proposed, few have yet been realized in a usable form. Some robotics researchers are working with tactile physiologists to look for correspondences between physiological and robotic designs, for example, the use of accelerometers to act like Pacinians and piezoelectric strips to sense shear (Cutkosky and Hyde, 1993). Attention to robot skin mechanics is important, for example, the existence of fingerprint-like ridges as stress amplifiers. Robust Proximity Sensors As a step toward supervisory control, small adjustments in grasping or an approach to a surface should be performed with local sensing. The intelligence required is much less than for the general case of autonomous control. A highly sophisticated gripper on ROTEX (Hirzinger et al., 1993), incorporating proximity, tactile, and force sensing, was the key to the success of the predictive display control. Proximity sensors have typically been ultrasonic, electromagnetic, or optical. All have limitations in range, accuracy, and sensitivity to different surface conditions. Visual time-of-flight systems are not yet practical because of complicated electronics, but they could be a future solution.
OCR for page 360
Virtual Reality: Scientific and Technological Challenges Multiaxis Force Sensors Multiaxis force sensors would typically be mounted on a robot wrist, to measure the net force and torque exerted on the end effector. Miniature force sensors would also be useful on finger segments, to control fingertip force accurately. Key issues at the moment are cost, robustness, and accuracy. For wrist sensors, inaccuracies of a few percentage points due to cross-coupling effects are typical and are problematic. A possible solution is based on magnetic levitation. High-Performance Joints Robot limbs should be strong and fast yet should be able to interact gracefully with the environments. Better actuator and transmission designs are the key. Improvements in electric, hydraulic, and pneumatic drive systems are required. Novel actuators such as shape memory alloy and polymeric actuators look promising in terms of force to weight; the result might be lightweight, responsive limbs that accurately track human motion commands and faithfully reflect back contact conditions. Hardware considerations on the master side are very similar. The requirements of a human wearing or interacting with a haptic interface are even more demanding and involve concerns of safety, convenience, and bulk. If reasonably performing devices cannot be obtained at reasonable costs, the spread of such systems will be limited. Improved Telerobotic Controllers Even without time delays, there are unresolved issues about how to best implement controllers for master-slave systems. One reason is limitations in hardware, as mentioned above. Another reason is a lack of tools with which to quantify and evaluate teleoperator performance. There are deficiencies in taxonomies of tasks: we don't understand well enough at a detailed level what we want these systems to do. At a basic level, we don't completely understand human sensorimotor abilities, for example, the discrimination of arbitrary mechanical impedances (stiffness, viscosity, and inertia) (Jones and Hunter, 1992). Clearly this knowledge is necessary to set design goals for telerobotic systems. We also need to understand how more complicated tasks decompose into more basic tasks, which can then be measured and used as discriminators between telerobotic controllers. As mentioned earlier in this chapter, robotics also has this goal. As an example, it is not fully understood when to apply rate control versus position control, or how to include force feedback into rate control.
OCR for page 361
Virtual Reality: Scientific and Technological Challenges Other examples were mentioned in the section on low-level control. When is force reflection more advantageous than the position control of a locally force-controlled robot? If force reflection is used, what is the exact form of controller that best meets goals of robustness and stability? There is still considerable work to be done in this area. A new set of issues arises from scaling: teleoperation of very small and very large robots. The mechanical behavior of objects in the micro domain is very different than in the macro domain. One problem is that the dynamics will be very fast; somehow movements will have to be slowed down for the operator. As robot autonomy improves, so will the level of supervisory control. A number of functions could be increasingly automated: Path planning and collision avoidance. The main issues here are efficient routines and obtaining geometric descriptions of the environment (Latombe, 1991). Trajectory specification. Any trajectory has to stay within system constraints of joint limits, actuators, and safety concerns. Grasping. The robot should be relied on to obtain a stable grasp and to regrasp as necessary. Intermittent dynamic environments. Trajectories should be modified in real time subject to changes in the environment. For example, a robot may swerve to avoid hitting someone entering its workspace. Some forms of hand-eye coordination, such as catching or hitting, may require a speed of response not possible with teleoperation. Force control. With more sophisticated abilities to interact with the environment and to complete such tasks as the generic peg-in-hole problem, the need for force reflection will diminish. A step toward such autonomous control capabilities would be a higher-level transfer of skills between the operator and the telerobot. The idea is to program by kinesthetic demonstrations: the human makes a movement, this movement is measured, and the telerobot extracts symbolic information about how to accomplish the task (Funda et al., 1992; Ikeuchi, 1993). This differs from direct manual teleoperation, in that an exact trajectory is not being commanded, but rather a strategy for completing a task. Difficulties particularly present themselves in transferring force control skills.
Representative terms from entire chapter: