Telepresence: a matter of degree

Telepresence is a matter of degree. Rarely will a telepresence system provide such comprehensive and convincing stimuli that the user perceives no differences from actual presence. But the user may set aside such differences, depending on the application. Watching television, for example, although it stimulates our primary senses of vision and hearing, rarely gives the impression that the watcher is no longer at home. However, television sometimes engages the senses sufficiently to trigger emotional responses from viewers somewhat like those experienced by people who directly witness or experience events. Televised depictions of sports events, or disasters such as the infamous September 11 terrorist attacks can elicit strong emotions from viewers.

As the screen size increases, so does the sense of immersion, as well as the range of subjective mental experiences available to viewers. Some viewers have reported a sensation of genuine vertigo or motion sickness while watching IMAX movies of flying or outdoor sequences.

Even the fairly simple telephone achieves a limited form of telepresence, in that users consider themselves to be talking to each other on the telephone rather than talking to the telephone itself. To an observer with no knowledge of telephones, watching a person chatting to an inanimate object might seem curious, but the telephone is readily usable by almost everyone who can speak and listen.

Most often, currently feasible telepresence gear leaves something to be desired; the user must suspend disbelief to some degree, and choose to act in a natural way, appropriate to the remote location, perhaps using some skill to operate the equipment. In contrast, a telephone user does not see herself as "operating" the telephone, but merely talking to another person with it. A goal of telepresence developers might be to similarly have their users lose direct awareness of the equipment they are using.

Comparison with virtual reality

Telepresence refers to a user interacting with another live, real place, and is distinct from virtual presence, where the user is given the impression of being in a simulated environment. Telepresence and virtual presence rely on similar user-interface equipment, and they share the common feature that the relevant portions of the user's experience at some point in the process will be transmitted in an abstract (usually digital) representation. The main functional difference is the entity on the other end: a real environment in the case of telepresence, vs. a computer in the case of virtual reality.

Implementation

For a user to be given a convincing telepresence experience, sophisticated technologies are required.

Vision

A minimum system usually includes visual feedback. Ideally, the entire field of view of the user is filled with a view of the remote location, and the viewpoint corresponds to the movement and orientation of the user's head. In this way, it differs from television or cinema, where the viewpoint is out of the control of the viewer.

In order to achieve this, the user may be provided with either a very large (or wraparound) screen, or small displays mounted directly in front of the eyes. The latter provides a particularly convincing 3D sensation. The movements of the user's head must be sensed, and the camera must mimic those movements accurately and in real time. This is important to prevent unintended motion sickness.

Sound

Sound is generally the easiest sensation to implement with high fidelity, with the telephone dating back more than 100 years, and very high-fidelity sound equipment readily available as consumer gear. Stereophonic sound is more convincing than monoaural sound, and surround sound is better still.

Manipulation

The ability to manipulate a remote object or environment is an important aspect of real telepresence systems, and can be implemented in large number of ways depending on the needs of the user. Typically, the movements of the user's hands (position in space, and posture of the fingers) are sensed by wired gloves, inertial sensors, or absolute spacial position sensors. A robot in the remote location then copies those movements as closely as possible. This ability is also known as Teleoperation.

The more closely the robot re-creates the form factor of the human hand, the greater the sense of telepresence. Complexity of robotic effectors varies greatly, from simple one axis grippers, to fully anthropomorphic robot hands.

Haptic teleoperation refers to a system that provides some sort of tactile force feedback to the user, so the user feels some approximation of the weight, firmness, size, and/or texture of the remote objects manipulated by the robot.

Applications

Teleconferencing

Rather than traveling great distances, in order to have a face-face meeting, it is now possible to teleconference instead, using a multiway video phone. Each member of the meeting, or each party, can see every other member on a screen or screens, and can talk to them as if they were in the same room. This brings enormous time and cost benefits, as well as a reduced impact on the environment from air travel. A good telepresence strategy puts the human factors first, focusing on visual collaboration solutions that closely replicate the brain's innate preferences for interpersonal communications, separating from the unnatural "talking heads" experience of traditional videoconferencing. These cues include life–size participants, fluid motion, accurate flesh tones and the appearance of true eye contact. This is already a well-established technology, used by many businesses today. The chief executive officer of Cisco Systems, John Chambers in June 2006 at the Networkers Conference compared telepresence to teleporting from Star Trek, and said that he saw the technology as a potential billion dollar market for Cisco.

Michael Venditte,Vice President of Engineering of Telanetix defines Telepresence as a human experience of being fully present at a live real world location remote from one's own physical location. Someone experiencing video Telepresence would therefore be able to behave, and receive stimuli, as though part of a meeting at the remote site. The fore mentioned would result in interactive participation of group activities that will bring benefits to a wide range of users. Application examples could be sited within emergency management and security services, B&I, entertainment and education industries.

Connecting Communities

Telepresence can be used to establish a sense of shared presence or shared space among geographically separated members of a group.

Subsea work

The cost of deep water diving operations is extremely high due to safety regulations, hyperbaric equipment, time spent in decompression, and support vessel costs. Telepresence systems for inspection and teleoperation for repair and maintenance would realise significant cost benefits and also remove divers from hazardous environments.

Hazardous environments

Many other applications in situations where humans are exposed to hazardous situations are readily recognised as suitable candidates for telepresence. Mining, bomb disposal, military operations, rescue of victims from fire, toxic atmospheres, or even hostage situations, are some examples.

Pipeline inspection

Small diameter pipes, otherwise inaccessible for examination, can now be viewed using pipeline video inspection.

Remote surgery

The possibility of being able to project the knowledge and the physical skill of a surgeon over long distances has many attractions. Thus, again there is considerable research underway in the subject. (Locally controlled robots are currently being used for joint replacement surgery as they are more precise in milling bone to receive the joints.) The armed forces have an obvious interest since the combination of telepresence, teleoperation, and telerobotics can potentially save the lives of battle casualties by allowing them prompt attention in mobile operating theatres by remote surgeons.

Education

The benefits of enabling schoolchildren to take an active part in exploration have been shown by the JASON and the NASA Ames Research Center programs. The ability of a pupil, student, or researcher to explore an otherwise inaccessible location is a very attractive proposition; For example, locations where the passage of too many people is harming the immediate environment or the artifacts themselves, e.g. undersea exploration of coral reefs, ancient Egyptian tombs, and more recent works of art.

Research is also being conducted to investigate the use of telepresence to provide professional development to teachers. Research has shown that one of the most effective forms of teacher professional development is coaching, or cognitive apprenticeship. The application of telepresence shows promise for making this approach to teacher professional development practical.

Advertising and sales

Tour operators and property agents could use telepresence to allow potential customers to sample holiday locations and view properties remotely making commitments.

Entertainment

Telepresence systems could be incorporated into theme or nature parks to allow observers to travel through coral reefs or explore underground caves. In amusement parks, the elderly or infirm could experience the thrill of live roller coaster rides without risk.

In the games, users can interact using telepresence, sharing robots to interact one human with another (paired objects as remote surrogate actors). In other words, if one partner shakes the object, the remote object also shakes.

Telepresence Art

In 1998, Diller and Scofidio created the "Refresh", an Internet-based art installation that juxtaposed a live web camera with recorded videos staged by professional actors. Each image was accompanied with a fictional narrative which made it difficult to distinguish which was the live web camera.

In 1993, Eduardo Kac and Ed Bennett created a telepresence installation "Ornitorrinco on the Moon", for the international telecommunication arts festival "Blurred Boundaries" (Entgrenzte Grenzen II). It was coordinated by Kulturdata, in Graz, Austria, and was connected around the world.

Telepresence and AI

Marvin Minsky was one of the pioneers of intelligence-based mechanical robotics and telepresence. He designed and built some of the first mechanical hands with tactile sensors, visual scanners, and their software and computer interfaces. He also influenced many robotic projects outside of MIT, and designed and built the first LOGO "turtle."

Commercial Telepresence Systems

Telepresence systems aimed at corporate customers are commercialized by BrightCom, Cisco, Hewlett-Packard, Telanetix, Tandberg, Teliris, and Polycom. Prices range from tens to hundreds of thousand dollars . These systems include multiple microphones, speakers, High-Definition monitors, cameras, and often dedicated networks and custom-made studios. They strive to be as transparent to users as possible by providing life-size videos, imperceptible transmission delays, and user-friendly interfaces.

A telepresence system aimed at hospitalized, homebound and special needs children is commercialized by telbotics. Called PEBBLES, it is intended to connect children to their home classroom, allowing for participation in classroom activities and social contact. PEBBLES was first used in 1997 at Toronto's Hospital for Sick Children and was launched in 2001 in the United States at Yale-New Haven Children's Hospital. The technology is now in use in Canada, US, and the Netherlands.

History

The first commercially successful telepresence company, Teleport (which was later changed to TeleSuite), was founded in 1993 by David Allen and Harold Williams. The original intent was to develop a system that could allow families to interact across great distances without the hassle or costliness of flying. The first systems (which they called TeleSuites) looked more like something out of an upper class home rather than a conference room in an office suite (which are what most systems are used for today). Hilton Hotels had originally made a deal with them to begin installing them in their hotels throughout the United States and other countries, but usage was low. The idea lost momentum and Hilton eventually backed out. They later began to focus on business oriented telepresence systems. Shareholders eventually held enough stock to take over the company, which ultimately led to its collapse. David Allen purchased all of the assets of TeleSuite and then called the new company Destiny Conferencing. Although they survived, the idea didn't actually truly catch on until other mega corporations jumped onboard such as HP, Cisco, and Polycom who released similar systems around the mid 2000's. Polycom eventually bought them out (Destiny Conferencing) and now carries the TeleSuite telepresence system that is now known as the RPX (real presence experience) mentioned above.

Telerobotics is the area of robotics concerned with the control of robots from a distance, chiefly using wireless connections (like Wi-Fi, Bluetooth, the Deep Space Network, and similar), "tethered" connections, or the Internet. It is a combination of two major subfields, teleoperation and telepresence.

Teleoperation

Teleoperation means "doing work at a distance", although by "work" we mean almost anything. What we mean by "distance" is also vague: it can refer to a physical distance, where the operator is separated from the robot by a large distance, but it can also refer to a change in scale, where for an example a surgeon may use micro-manipulator technology to conduct surgery on a microscopic level.

A telemanipulator (teleoperator) is a device that is controlled remotely by a human operator. If such a device has the ability to perform autonomous work, it is called a telerobot. If the device is completely autonomous, it is called a robot. In simple cases the controlling operator's command actions correspond directly to actions in the device controlled, as for example in a radio controlled model aircraft or a tethered deep submergence vehicle. Where communications delays make direct control impractical (such as a remote planetary rover), or it is desired to reduce operator workload (as in a remotely controlled spy or attack aircraft) , the device will not be controlled directly, instead being commanded to follow a specified path. At increasing levels of sophistication the device may operate somewhat independently in matters such as obstacle avoidance, also commonly employed in planetary rovers.

Devices designed to allow the operator to control a robot at a distance is sometimes called telecheric robotics.

Two major components of Telerobotics and Telepresence are the visual and control applications. A remote camera provides a visual representation of the view from the robot. Placing the robotic camera in a perspective that allows intuitive control is a recent technique that although based in Science Fiction (Robert Heinleins WALDO AND MAGIC COMPANY 1959) has not been fruitful as the speed, resolution and bandwidth have only recently been adequate to the task of being able to control the robot camera in a meaningful way. Using a head mounted display, the control of the camera can be facilitated by tracking the head as shown in the figure below.

This only works if the user feels comfortable with the latency of the system, the lag in the response to movements, and the visual representation. Any issues such as, inadequate resolution, latency of the video image, lag in the mechanical and computer processing of the movement and response, and optical distortion due to camera lens and head mounted display lenses, can cause the user 'simulator sickness' which is exacerbated by the lack of vestibular stimulation with visual representation of motion.

Mismatch between the users motions such as registration errors, lag in movement response due to overfiltering, inadequate resolution for small movements, and slow speed can contribute to these problems.

The same technology can control the robot, but then the hand eye coordination issues become even more pervasive through the system, and user tension or frustration can make the system difficult to use.

Ironically the tendency to build robots has been to minimize the degrees of freedom because that reduces the control problems. Recent improvements in computers has shifted the emphasis to more degrees of freedom, allowing robotic devices that seem more intelligent and more human in their motions. This also allows more direct teleoperation as the user can control the robot with their own motions.

Telepresence

Telepresence means "feeling like you are somewhere else". Some people have a very technical interpretation of this, where they insist that you must have head-mounted displays in order to have telepresence. Other people have a task-specific meaning, where "presence" requires feeling that you are emotionally and socially connected with the remote world. It's all a little vague at this time.

Interfaces

A telerobotic interface can be as simple as a common MMK (monitor-mouse-keyboard) interface. While this is not immersive, it is inexpensive. Telerobotics driven by interet connections are often of this type. A valuable modification to MMK is a joystick, which provides a more intuitive navigation scheme for planar robot movement.

Dedicated telepresence setups utilize a head mounted display with either single or dual eye display, and an ergonomically matched interface with joystick and related button, slider, trigger controls.

Future interfaces will merge fully immersive virtual reality interfaces and port real-time video instead of computer-generated images. Another example would be to use an omnidirectional treadmill with an immersive display system so that the robot is driven by the person walking or running. Additional modifications may include merged data displays such as infrared thermal imaging, real-time threat assessment, or device schematics.

Applications

With the exception of Project Apollo most space exploration has been conducted with telerobotic space probes. Most space-based astronomy has been conducted with telerobotic telescopes. Recent noteworthy examples include the Mars exploration rovers (MER) and the Hubble Space Telescope. In the case of the MER mission, the spacecraft and the rover were each telerobotically operated.

Marine remotely operated vehicles (ROVs) are widely used to work in water too deep or too dangerous for divers. They repair offshore oil platforms and attach cables to sunken ships to hoist them. They are usually attached by a tether to a control center on a surface ship. The wreck of the Titanic was explored by an ROV, as well as by a crew-operated vessel.

Additionally, a lot of telerobotic research is being done in the field of medical devices, and minimally invasive surgical systems. With a robot system a surgeon can work inside the body through tiny holes just big enough for the manipulator, with no need to open up the chest cavity to allow hands inside.