FREE ESSAY ON USE OF HAPTICS FOR THE ENHANCED MUSUEM WEBSITE-USC

USE OF HAPTICS FOR THE ENHANCED MUSUEM WEBSITE-USC

Use of Haptics for the Enhanced Musuem Website-USC 
Interactive Art Museum
Our mission for the Enhanced Museum project is to explore new technologies for the
exhibition of three-dimensional art objects (Goldberg, Bekey, Akatsuka, and Bressanelli,
1997; McLaughlin, 1998; McLaughlin, Goldberg, Ellison, and Lucas, 1999; McLaughlin and
Osborne, 1997; Schertz, Jaskowiak, and McLaughlin, 1997). Although it is not yet
commonplace, a few museums are exploring methods for 3D digitization of priceless
artifacts and objects from their sculpture and decorative arts collections, making the
images available via CD-ROM or in-house kiosks. For example, the Canadian Museum of
Civilization has collaborated with Ontario-based Hymarc to use the latter's ColorScan 3D
laser camera to create three-dimensional models of more than fifty objects from the
museum's collection (Canarie, Inc., 1998; Shulman, 1998). A similar partnership has been
formed between the Smithsonian Institution and Synthonic Technologies, a Los Angeles-area
company. At Florida State University , the Deparment of Classics is working with a team
to digitize Etruscan artifacts using the RealScan 3D imaging system from Real 3D
(Orlando, Florida), and art historians from Temple University are collaborating with
researchers from the Watson Research Laboratory's visual and geometric computing group to
create a model of Michaelangelo's Pieta with the Virtuoso shape camera from Visual
Interface (Shulman, 1998). 
In collaboration with our colleagues at USC's accredited art museum, the Fisher Gallery,
our IMSC team is developing an application for the Media Immersion Environment that will
not only permit museum visitors to examine and manipulate digitized three-dimensional art
objects visually, but will also allow visitors to interact remotely, in real time, with
museum staff members to engage in joint tactile exploration of the works of art. Our team
believes that the hands-off policies that museums must impose limit appreciation of
three-dimensional objects, where full comprehension and understanding rely on the sense
of touch as well as vision. Haptic interfaces will allow fuller appreciation of
three-dimensional objects without jeopardizing conservation standards. Our goal is to
assist museums, research institutes and other conservators of priceless objects in
providing the public with a vehicle for object exploration, in a modality that could not
otherwise be permitted. 
Our initial application will be to a wing of the virtual museum focusing on examples of
the decorative arts: the Fisher Gallery's collection of teapots. The collection is
comprised of 150 teapots from all over the world. It was a gift to USC in memory of the
late Patricia Daugherty Narramore by her husband Roth Narramore. The Narramores, USC
alumni, collected the pots on their many domestic and international journeys. Some items
are by local artists, others by artists and makers from other countries, including China,
Indonesia, Canada, Japan, Brazil, England, Portugal, Morroco, and Sweden. Materials used
to make the pots range from porcelain and clay to wicker and metal. The teapots are ideal
candidates for haptic exploration, not only for their varied shapes but also for their
unusual textures and surface decoration. 
Figure 1. Teapots from the Fisher Gallery's Narramore Collection
Haptics for the Museum
Haptics refers to the modality of touch and the associated sensory feedback. Haptics
researchers are interested in developing, testing, and refining tactile and force
feedback devices that allow users to manipulate and feel virtual objects with respect to
such features as shape, temperature, weight and surface texture (Basdogan, Ho, Slater,
and Srinavasan, 1998; Bekey, 1996; Burdea, 1996; Brown & Colgate, 1994; Buttolo, Oboe,
Hannaford & McNeely, 1996; Dinsmore, Langrana, Burdea, and Ladeji, 1997; Geiss, Evers, &
Meinzer, 1998; Ikei, Wakamatsu, & Fukuda, 1997; Liu, Iberall, & Bekey, 1989; Howe, 1994;
Howe and Cutkosky, 1993; Mar, Randolph, Finch, van Verth, & Taylor, 1996; Massie, 1996;
Millman, 1995; Mor, 1998; Nakamura & Inoue, 1998; Rao, Medioni, Liu, & Bekey, 1988;
Srinivasan & Basdogan, 1997; Yamamoto, Ishguro, & Uchikawa, 1993).
Haptic acquisition and display devices
Researchers have been interested in the potential of force feedback devices such as pen
or stylus-based masters, like Sensable's PHANToM (Massie, 1996; Salisbury, Brock, Massie,
Swarup, & Zilles, 1995; Salisbury & Massie, 1994), as alternative or supplemental input
devices to the mouse, keyboard, or joystick. The PHANToM is a small, desk-grounded robot
that permits simulation of single fingertip contact with virtual objects through a
thimble or stylus. It tracks the x, y, and z Cartesian coordinates and pitch, roll and
yaw of the virtual probe as it moves about a three-dimensional workspace, and its
actuators communicate forces back to the user's fingertips as it detects collisions with
virtual objects, simulating the sense of touch. The CyberGrasp from Virtual Technologies
is an exoskeletal device which fits over a 22 DOF CyberGlove, providing force feedback
and vibrotactile contact feedback; it is used in conjunction with a position tracker to
measure the position and orientation of the forearm in three-dimensional space. Similar
to the CyberGrasp is the Rutgers Master II (Burdea, 1996; Gomez, 1998; Langrana, Burdea,
Ladeiji, and Dinsmore, 1997) which has an actuator platform mounted on the palm that
gives force feedback to four fingers. Position tracking is done by the Polhmeus Fastrak.

Alternative approaches to haptic sensing and discrimination have employed the
vibrotactile display, which applies multiple small force vectors to the fingertip. For
example, Ikei, Wakamatsu, and Fukuda (1997) used photographs of objects and a contact pin
array to transmit tactile sensations of the surface of objects. Each pin in the array
vibrates commensurate with the local intensity (brightness) of the surface area. Image
intensity is roughly correlated with the height of texture protrusions. A data glove
originating at Sandia (Sandia, 1995) uses rod-like plungers to tap the fingertips lightly
to simulate tactile sensations, and a magnetic tracker and strain gauges to follow the
movements of the user's hand and fingers. Howe (1996) notes that vibrations are
particularly helpful in certain kinds of sensing tasks, such as assessing surface
roughness, or detecting system events (for example, contact and slip in manipulation
control). 
Researchers at the Fraunhofer Institute for Computer Graphics in Darmstadt have developed
a glove-like haptic device they call the ThermoPad, a haptic temperature display based on
Peltier elements and simple heat transfer models; they are able to simulate not only the
environmental temperature but also the sensation of heat or cold one experiences when
grasping or colliding with a virtual object. At the University of Tsukuba, Japan, Iwana,
Yano, and Hashimoto (1997) are using the HapticMaster, a 6 DOF device with a ball grip
that can be replaced by various real tools for surgical simulations and other specialized
applications. A novel type of haptic display is the Haptic Screen (Iwana, Yano, and
Hashimoto, 1997), a device with a rubberized elastic surface with actuators, each with
force sensors, underneath. The surface of the Haptic Screen can be deformed with the
naked hand. An electromagnetic interface couples the ISU Force Reflecting Exoskeleton,
developed at Iowa State University, to the operator's two fingers, eliminating the
burdensome heaviness usually associated with exoskeletal devices. Finally, there is
considerable interest in 2D haptic devices. For example, Pai and Reissell at the
University of British Columbia have used the Pantograph 2D haptic interface , a two-DF
force-feedback planar device with a handle the user moves like a mouse, to feel the edges
of shapes in images (Pai & Reissell, 1997). 
At IMSC we are currently working with both the PHANToM and the CyberGrasp, using the
Polhemus Fastrak for tracking the position of the CyberGrasp user's hand. The tracking
problem has been widely studied in the context of mobile robots at USC (Roumeliotis,
Sukhatme, and Beckey, 1999a, 1999b). In the museum application the visitor and the museum
staff member will be able to manipulate haptic data jointly, regardless of display type.
Thus one of our primary concerns is insure proper registration of the disparate devices
with the 3D environment and with each other. Of potential use in this regard is work by
Iwata, Yano, and Hashimoto (1997) on LHX (Library for Haptics), a modular software that
can support a variety of different haptic displays. LHX allows a variety of mechanical
configurations, supports easy construction of haptic user interfaces, allows networked
applications in virtual spaces, and includes a visual display interface. We are
particularly eager to begin work with the CyberGrasp; to date we have been unable to
identify any published work or conference papers reporting research using the device,
which we attribute in part to its expense and relative infancy as a haptic display
device. 
Figure 2. Haptic acquisition and display devices
Representative applications in haptic acquisition and display 
A primary application area for haptics has been in surgical simulation and medical
training. Langrana, Burdea, Ladeiji, and Dinsmore (1997) used the Rutgers Master II
haptic device in a training simulation for palpation of subsurface liver tumors. They
modeled tumors as comparatively harder spheres within larger and softer spheres.
Realistic reaction forces were returned to the user as the virtual hand encountered the
tumors, and the graphical display showed corresponding tissue deformation produced by the
palpation. Finite Element Analysis was used to compute reaction forces corresponding to
deformation from experimentally obtained force/deflection curves. Andrew Mor of the
Robotics Institute at Carnegie Mellon (Mor, 1998) has used the PHANToM in conjunction
with a 2DOF planar device so that the new device would generate a moment measured about
the tip of a surgical tool in an arthroscopic surgery simulation, thus providing a more
realistic training for the kinds of unintentional contacts with ligaments and fibrous
membranes that an inexperienced resident might encounter. At MIT, De and Srinivasan
(1998) have developed models and algorithms for reducing the computational load required
to generate visual rendering of organ motion and deformation and the communication of
forces back to the user resulting from tool-tissue contact. They model soft tissue as
thin-walled membranes filled with fluid. Force-displacement response is comparable to
that obtained in in vivo experiments. Giess, Evers, and Meinzer (1998) integrated haptic
volume rendering with the PHANToM into the pre-surgical process of classifying liver
parenchyma, vessel trees and tumors. Surgeons at the Pennsylvania State University School
of Medicine in collaboration with Cambridge-based Boston Dynamics used two PHANToMs in a
training simulation in which residents passed simulated needles through blood vessels,
allowing them to collect baseline data on the surgical skill of new trainees. Iwata,
Yano, and Hashimoto (1998) report the development of a surgical simulator with a free
form tissue which behaves like real tissue, e.g., can be cut. Gruener (1998), in one of
the few research reports which expresses reservations about the potential of haptics in
medical applications, found that subjects in a telementoring session did not profit from
the addition of force feedback to remote ultrasound diagnosis. 
There have been a few projects in which haptic displays are used as alternative input
devices for painting, sculpting and computer-assisted design. At CERTEC, the Center of
Rehabilitation Engineering in Lund, Sweden, Sjostrom (Sjostrom, 1997) and his colleagues
have created a painting application in which the PHANToM can be used by the visually
impaired; line thickness varies with the user's force on the fingertip thimble and colors
are discriminated by their tactual profile. Marcy, Temkin, Gorman, and Krummel (1998)
have developed the Tactile Max, a PHANToM plug-in for 3D Studio Max. Dynasculpt, a
prototype from Interval Research Corporation (Snibbe, Anderson, and Verplank, 1998)
permits sculpting in three dimensions by attaching a virtual mass to the PHANToM position
and constructing a ribbon through the mass's path through the 3D space. Gutierrez,
Barbero, Aizpitarte, Carrillo, and Eguidazu (1998) have integrated the PHANToM into
DATum, a geometric modeller. Objects can be touched, moved, or grasped (with two
PHANToMs), and the assembly/disassembly of mechanical objects can be simulated. 
Haptics has also been incorporated into scientific visualization. Drubeck, Macias,
Weinstein, Johnson, and Hollerbach (1998) have interfaced SCIrun, a computation software
steering system, to the PHANToM. Both haptics and graphics displays are directed by the
movement of the PHANToM stylus through haptically rendered data volumes. Similar systems
have been developed for geoscientific applications (e.g., the Haptic Workbench, Veldkamp,
Truner, Gunn, and Stevenson, 1998), Green and Salisbury (1998) have produced a convincing
soil simulation (Green and Salisbury, 1998) where they have varied parameters such as
soil properties, plow blade geometry, and angle of attack. At Interactive Simulations, a
San Diego-based company, researchers have succeeded in adding a haptic feedback component
to Sculpt, a program for analyzing chemical and biological molecular structures, which
will permit analysis of molecular conformational flexibility and interactive docking. 
Acquisition of 3D models
There are several commercial 3D digitizing cameras available for applications like the
museum, such as the ColorScan and the Virtuoso shape cameras. The latter uses six digital
cameras, five black and white cameras for capturing the shape information and one color
camera which acquires texture information which is layered onto the triangle mesh. Our
digitization process begins with models acquired from photographs, using a semiautomatic
system to infer complex 3-D shapes from photographs developed at IMSC (Chen, 1998, 1999).
Images are used as the rendering primitives, beginning with six input images of our
teapots at 60 degrees separation; multiple input pictures are allowed, taken from nearby
viewpoints with different position, orientation and camera focal length. Other comparable
approaches to digitizing museum objects (e.g., Synthonics) use an older version of the
shape-from-stereo technology which requires the cameras to be calibrated whenever the
focal length or relative position of the two cameras is changed. The direct output of the
IMSC program is volumetric but is converted to a surface representation for the purpose
of graphic rendering. The reconstructed surfaces are quite large, on the order of 40 MB.
They are decimated with a modified version of a program for surface simplication using
quadric error metrics written by Garland and Heckbert (1997). 
Figure 3. Teapot digitization: 1 of six input views; an image of the reconstructed point
set; an image of the omnidirectional solid model (reconstructed surface)
Pai and Reissell(1997) report on a technique based on wavelets for multiresolution
modeling of 2D shapes. The models rely on a robust edge detector to detect boundary
curves in the image. These curves are then rendered as solid objects using a haptic
interface. The system also incorporates a fast contact detection algorithm based on
collision trees. The paper includes a discussion of a state machine that serves as a
simple model for contact transition and hence, force computation. 
Volumetric data is used extensively in medical imaging and scientific visualization.
Currently the GHOST SDK, which is the development toolkit for the PHANToM, construes the
haptic environment as scenes composed of geometric primitives. Huang, Qu, and Kaufman of
SUNY-Stony Brook have developed a new interface which supports volume rendering, based on
volumetric objects, with haptic interaction. The APSIL library (Huang, Qu, and Kaufman,
1998) is an extension of GHOST. To date the Stony Brook group has developed succesful
demonstrations of volume rendering with haptic interaction from CT data of a lobster, a
human brain, and a human head, simulating stiffness, friction, and texture solely from
the volume voxel density. The development of the new interface may facilitate working
directly with the volumetric representations of the teapots obtained through the view
synthesis methods. 
The surface texture of an object can be displacement mapped (consisting of thousands of
tiny polygons) (Srinivasan and Basdogan, 1997), although the computation demand is such
that force discontinuities can occur, or more commonly, a texture field can be
constructed from 2-D image data. For example, Ikei, Wakamatsu, and Fukuda (1997 created
textures from images converted to greyscale, then enhanced to heighten brightness and
contrast, such that the level and distribution of intensity corresponds to variation in
the height of texture protrusions and retractions (Ikei et al., 202). They then employed
an array of vibrating pins to communicate tactile sensations to the user's fingertip,
with the amplitude of the vibration of each pin driven at the intensity level of the
underlying portion of the image. Surface texture may also be rendered haptically, through
techniques such as force perturbation, where the direction and magnitude of the force
vector is altered using the local gradient of the texture field to simulate effects such
as coarseness (Srinivasan and Basdogan, 1997). Synthetic textures such as wood,
sandpaper, cobblestone, rubber, and plastic may also be created using mathematical
functions for the height field (Anderson, 1996; Basogan, Ho, and Srinivasan, 1997). The
ENCHANTER environment (Jansson, Faenger , Konig and Billberger, 1998) has a texture
mapper which can render sinus, triangular, and rectangular textures as well as textures
provided by other programs, for any haptic object provided by the Ghost SDK.
Issues in haptic rendering
Researchers working with force feedback devices for object sensing have been concerned
with issues of presence, or the fidelity (realism) of the haptic experience. For
instance, Brown and Colgate (1994), in their physics-based approach to haptic display,
address the issue of stability guarantees in virtual environments. In particular they
note the threat to presence created when the virtual environment becomes computationally
unstable, as for example when a normally passive tool, such as a chisel, begins to move
independently of the control of the user who is wielding it. Similarly, a virtual wall
must unilaterally constrain the user's forward movement. Brown and Colgate develop a
model for improving the passivity of the haptic display through inherent physical damping
and the impedence of virtual walls through increased sampling (update rates). 
The many potential applications in industry, the military, and entertainment for force
feedback in multi-user environments, where two or more users orient to and manipulate
objects in a shared environment, have led to work such as that of Buttolo and his
colleagues (Buttolo, Hewitt, Oboe, & Hannaford, 1997; Buttolo, Oboe, Hannaford, &
McNally, 1996), who note that the addition of force feedback to multi-user environments
demands low latency and high collision detection sampling rates. LANs, because of their
low communication delay, may be conducive to applications in which users can touch each
other, but for wide area networks, or any environment where the demands above cannot be
met, Buttolo et al propose their one-user-at-a-time architecture. Mark and his colleagues
(Mark, Randolph, Finch, van Verth, and Taylor, 1996) have proposed a number of solutions
to recurring problems in haptics, such as improving the update rate for forces
communicated back to the user. They propose the use of intermediate representation of
force through a plane and probe method: a local planar approximation to the user's hand
location is computed when the probe or haptic tool penetrates the plane, and the force is
updated at approximately 1 kHz by the force server, while the application recomputes the
position of the plane and updates it at approximately 20 kHz. Mark et al. also propose
solutions to add surface texture and friction to what otherwise would be the slick
surface produced under their model, using a parameterized snag distribution on the object
surface. They also present a method for specifying torques as well as force, and a
recovery-time algorithm for preventing force discontinuity artifacts, such as occur when
the haptic probe's sideways movement is too fast relative to the computation of the new
intermediate representation. Mark et al. have developed a device-independent library of
routines for haptic interfaces, Armlib, which supports multi-user and multi-hand
applications. Armlib works with a number of different haptic display devices, including
the PHANToM. 
Psychophysical studies: perceptions of shape and texture in multimodal virtual
environments
The behavior of the human haptic system has been the subject of far more systematic study
than has touching with robotic masters. Texture, apprehended by most subjects through
lateral, side-to-side hand movement or exploratory procedure, is only one of several
haptically important dimensions of object recognition, including hardness, shape, and
thermal conductivity (Klatzky, Lederman, & Reed, 1987). Most researchers report that
subjects are able to discriminate textures and to a lesser extent shapes using the haptic
sense only. For example, Ballesteros, Manga, and Reales (1997) reported a moderate level
of accuracy for single-finger haptic detection of raised-line shapes, with asymmetric
shapes being more readily discriminated. Hatwell (1995) found that recall of texture
information coded haptically was successful when memorization was intentional, but not
when it was incidental, indicating that haptic information processing may be effortful
for subjects. 
Hughes and Jansson (1994) lament the inadequacy of embossed maps and other devices
intended to communicate information to the visually handicapped through the sense of
touch, a puzzling state of affairs insomuch as texture perception by active touch
(purposeful motion of the skin surface relative to the surface of some distal object)
appears to be comparatively accurate, and even more accurate than vision in apprehending
certain properties, such as smoothness (Hughes & Jansson, 302). The authors note in their
critical review of the literature on active-passive equivalence that active and passive
touch (as when a texture is presented to the surface of the fingers, see Hollins et al.,
1993, below) have repeatedly been demonstrated by Lederman and her colleagues (Lederman,
1985; Lederman, Thorne, & Jones, 1986; Loomis & Lederman, 1986) to be functionally
equivalent with respect to texture perception, in that touch modality does not seem to
account for a significant proportion of the variation in judgments of such basic
dimensions as roughness, even though the two types of touch may lead to different sorts
of attributions (respectively, about the texture object and about the cutaneous sensing
surface) and motor information should clearly be useful in assessing the size and
distribution of surface protrusions and retractions. Active-passive touch is more likely
to be equivalent in certain types of perceptual tasks; active touch should be less
relevant to judgments of hardness than it is to assessments of springiness.
Bibliography
NONE