Computer Graphic and PHANToM Haptic Displays: Powerful Tools to Understand How Humans Perceive Heaviness

on lifting movements or heaviness study the effects of cues regarding pictorial depth and volumetric size cues on the size-weight integration in perceived heaviness. Weight displays were created using the PHANToM haptic stylus, synchronized and superimposed onto corresponding 2D graphic objects displayed on a 2D monitor, in accordance with manipulation of the stylus (virtual objects) by subjects. hypothesized the degree of the SWI weakened, point of when the dimensions of pictorial depth cues were reduced from being a 3D a 2D


Introduction
The development of human/computer interfaces related to human haptics is subject to greater degrees of difficulty and complexity than those related to human visual and/or auditory senses. Such interfaces require direct contact and/or interaction with humans as opposed to visual and/or auditory devices that do not require such direct contact, manipulation and other force-related interactions. Therefore, without a deeper understanding of the mechanisms involved in haptics, such interfaces may be far from being user-centered or easy-to-use.
The focus of our research has been to understand the perceptual system of heaviness in humans. Heaviness perception, categorized as one aspect of haptic perception, is considered to be a vital ability in everyday life not only to recognize objects, but also to lift and manipulate them. Research in the field of experimental psychology, in particular psychophysics, has focused on identifying the properties and mechanisms of heaviness ever since Weber (1834, as translated by H. E. Ross & Murray, 1978) undertook his inquiries. Such properties and mechanisms have not yet been fully identified. Rather, the more experimental techniques and/or experimental environments have evolved, the more complex human perception of heaviness appears. This is because heaviness: (1) involves both perceptual systems and sensorimotor systems, such as the force programming system for lifting or holding objects, (2) is affected not only by object weight, but also by physical, functional and other properties of objects and (3) is affected by bottom-up processing by lower-order senses and by top-down processing by higher-order cognitive processes such as expectation and rationalization.
The purpose of this chapter is to overview human perception of heaviness to decipher its complexity. In addition, we introduce the usefulness of virtual reality systems to isolate and understand constraints on heaviness perception. One such system adopted for our research is the Virtual Hand Laboratory, creating virtual or augmented environments, in which humans interact with computer displayed objects or real physical objects. We illuminate mechanisms of heaviness perception with fundamental findings that might not have been www.intechopen.com obtained from the real world (i.e., Real Haptic + Real Vision). Also, we posit that superimposing computer-generated graphical objects precisely onto real, physical objects (i.e., Real Haptic + Virtual Vision) allows manipulation of visual properties of objects independently from physical properties. Finally, we introduce Virtual Haptic + Virtual Vision conditions, using a unique experimental augmented environment using both computer haptics AND computer graphics displays. This allows for selective experimental manipulations in which the haptic device affects haptic perception while computer generated graphics have an effect on visual perception, by superimposing computergenerated object forces precisely onto the graphical objects existing within the computer display. Presented are fresh findings regarding heaviness perception that enhance the knowledge base for the human perception of heaviness. These basic findings are expected to facilitate future research and development of haptic and graphic computer systems relating to human recognition, lifting, transport or manipulation of physical objects. Figure 1 shows a schematic that provides an overview of three categories of factors influencing human perceived heaviness: (1) factors related to the sensorimotor system responsible for lifting and holding an object (Object Lifting Phase), (2) factors or physical properties relating to an object itself (Object), and (3) factors relating to perceiving or judging heaviness (Perceiving Heaviness Phase). These factors act not only individually but also interactively. It should be emphasized here that the human psychological unit of heaviness differs from the physical unit of measuring weight. Fig. 1. Factors influencing human perceived heaviness. Question marks indicate whether or how humans use perceived heaviness for subsequent object lifting. This is controversial and requires further investigation (See Sec. 5.1).

Factors related to lifting and holding movements
Whenever a person attempts to perceive the heaviness of an object or to compare the difference in weight between two or more objects, grasping, lifting and holding movements www.intechopen.com must precede these behaviors (MacKenzie & Iberall, 1994). When an object is lifted by an adult, the sensorimotor system functions automatically, without any particular attention, to achieve a safe and effective grip to lift and hold, transport or manipulate the object, maintaining stability. This system is composed of two subsystems: a feedforward system and a feedback system.
A feedforward system works before the initiation of the grasping movement to predict object properties including weight and estimate the required motor commands, based on long-term memories (Gordon et al., 1993) and/or short-term memories if the lifts are repeated during a short period of time (Johansson, 1996). Finally, these programmed motor commands are sent to the related muscle groups to achieve stable lift of the target object. Interestingly, the motor commands are hypothesized to be sent not only to the effectors, i.e., related muscle groups for object lifts, but also for "efferent copies" to be sent to the CNS structures related to sensation, feedback processing or perception (Sperry, 1950;Holst, 1954;McCloskey, 1978).
Once the lift of an object has been initiated, the feedback system acts to optimize the output forces in the muscles through a local reflex feedback via Ia afferents in a muscle spindle (for muscle stretch) and Ib in a Golgi tendon organ (for muscle tension) (Crago et al., 1982;Rothwell, et al., 1982). These peripheral-origin signals ascend via the dorsal column through transcortical loops in the central nervous system (CNS) relating to motor control (S1, M1, basal ganglia, and cerebellum), and ongoing motor commands are probably modified via cortico-cerebellar connections with the CNS-origin signals "copied" to optimize subsequent discharge based on detected errors between the efferent copies and afferent information. The optimized signals, then, may contribute to achieving safer and more stable lift (Brodie & H.E. Ross, 1985). Grip forces applied at the object/digit interfaces are also automatically adjusted, based on the information from mechanoreceptors on glabrous skin, from the initiation of lifts according to frictional forces, object slipperiness (Johansson, 1996;Rinkenauer, et al., 1999) and object torque (Kinoshita et al., 1997).
The "copies" have been termed in such various ways as "sense of effort" (McCloskey, 1978), "collorary discharge" (Sperry, 1950) or "efferent copy" (Holst, 1954). Furthermore, in psychological research such as that for the size-weight illusion (SWI), (Charpentier, 1891 as cited in Murray, et al., 1999;See Sec. 3), efferent copy is replaced by the term "expectation" (H.E. Ross, 1969) or "ease with which could be lifted" (Müller & Schumann, 1889, as cited in Davis, 1973). These copies are thought to play an important role in object perception as well as for motor control. A well-known example is the ability to perceive an object and/or its surroundings as being at rest and clear without blurring when the eyes are moved. This is due to the copied signals relating to self-generated movement being compared with the signals obtained from vision (Holst, 1954). Without this system, we could not perceive an object accurately.
Interestingly, the forces generated when lifting an object correlate with object weight (Johansson, 1996) and that of heaviness perception (Harper & Stevens, 1948;Stevens, 1958). However, as to the correlation between the forces generated and perceived heaviness, researchers differ. Some researchers support their correlation (H.E. Ross, 1969;Davis & Roberts, 1976;Gordon et al., 1991) and others report their dissociation (Flanagan & Beltzner, 2000;Grandy & Westwood, 2006;Chang et al., 2008). What factors give rise to these opposing views and which view will eventually prevail remain matters to be resolved, noted by question marks in Fig. 1.
Yet, the motor-related commands for force generation and adaptation, at least partly or indirectly, relate to perceived heaviness. Evidence has shown involvement of the sensorimotor system in human perceptual system of heaviness (McCloskey, 1978). That is, the degree of perceived heaviness is reported to increase due to the effects of fatigue on related muscles (Jones & Hunter, 1983;Buckingham et al., 2009), to partial curarization or a peripheral anaesthesia effect on cutaneous or joint sensation (Gandevia & McCloskey, 1977a), and to muscle vibration on related muscle spindles (Brodie & H.E. Ross, 1984). Neural disorders relating to the sensorimotor system are also reported to affect perceived heaviness. In comparison to normal subjects, for example, it is reliable to be overestimated in patients with paresis (Gandevia & McCloskey, 1977b), in deafferented patients for muscle spindles and Golgi tendon organ (Rothwell et al., 1982), in those with cerebellum disorders (Holms, 1917;Cf. Rabe et al., 2009), and in those with Parkinson's disease (Maschke et al., 2006). Furthermore, attention should be focused on the lifting conditions, i.e., how to discern the heaviness of an object as a higher order or more cognitive and strategic matter (Fig.1). The manner of lifting, for example, affects perceiving heaviness in various conditions: active lifting, such as jiggling an object (Brodie & H.E. Ross, 1985), tends to make more accurate weight discrimination than passive pressure (Weber, 1834, translated by H. E. Ross & Murray, 1978). In addition, the perceived heaviness when an object is lifted depends on which parts of the hand are in contact, with an increase of perceived heaviness being reported when an object is lifted distally, by the fingertips, compared to when lifted proximally, by the base of fingers or near the palm (Davis, 1974). Holway et al., (1938) reported that the same object is perceived as heavier in the second trial than that in the first. After repeated lifts of sets of heavier objects, the discriminative thresholds decreased in the sets of lighter objects compared to those without such preceding lifts of sets of heavier objects (Holway & Hurvich, 1937). Further, the degree of perceived heaviness changed when lifting two objects simultaneously using both hands compared to that when lifting two objects alternately using only one hand (Jones & Hunter, 1982).

Bottom-up and top-down influences on heaviness perception
As peripheral or bottom-up processing issues related to perceptual system of heaviness, individual learning and sensitivity in weight discrimination are important factors. The Weber fraction, i.e., weight sensitivity, widely differs among individuals (0.02~0.16) Raj et al., 1985). Age, especially, is a crucial factor leading to a decrease of sensitivity to weight or heaviness (Gandevia, 1996;Dijker, 2008). Serious deterioration is also reported for neural disorders including leprous neuropathy (Raj, et al., 1985), lesions to inferior-frontal cortex including PMv (Halstead, 1945), and left parietal and temporal lesions (Li et al., 2007).
As for higher level, cognitive-based or top-down processing, individual learning or experience is also an important factor to influence perception of heaviness (Fig. 1). An expectation that a larger object should be heavier than a smaller object, for example, is thought to affect perceived heaviness. Accordingly, when lifted, the larger of two objects of equal weight tends to be perceived as lighter than the smaller (H.E. Ross, 1969;Davis & Roberts, 1976;Gordon et al., 1991;Rabe et al., 2009;Buckingham & Goodale, 2010). This expectation factor is also experienced in relation to object colour (Payne, 1958), object material (Ellis & Lederman, 1999;Buckingham et al., 2009), and even the human conditions of gender and age (Dijker, 2008). For example, when viewed, darker or metallic objects are judged to be heavier than brighter or wooden ones, but are then perceived as lighter when actually lifted under same-weight conditions. When confronting conflicting issues, such as with the SWI, when two objects have different size but are of identical weight, subjects tend to rationalize that if the weight is the same, the larger object should be lighter, rather than depending on the current sensations of heaviness (Mon-Williams & Murray, 2000). This tendency seems to be more pronounced when subjects are subject to the forced-choice condition in which they must choose either Heavier or Lighter (Mon-Williams & Murray, 2000;Buckingham & Goodale, 2010). This raises the question, what is the best way to obtain accurate and natural responses from subjects: using the two category method "Heavier and Lighter", the three category method "Heavier, Lighter, and Similar", the bimanual or unilateral matching methods, or the magnitude estimation method? Other questions posed by experimenters such as, "Which is heavier?" or "Which is the heaviest" might cognitively bias perception of heaviness.

Effects of object "size" on perceived heaviness
Object size is the oldest, most-studied factor since Müller and Schuman (1889, as cited in Davis, 1973) and Charpentier (1891, as cited in Murray et al. 1999) reported the phenomenon of the size-weight illusion (SWI): the larger of two objects of equal weight is perceived as lighter than the smaller. The mechanism underlying the SWI has been continuously discussed with various interpretations: Expectation Theory (H.E. Ross, 1969), Information Integration Theory (Anderson, 1970;Masion & Crestoni, 1988), Density Theory (J. Ross & Di Lollo, 1970), Gain Adjustment Theory (Burgess & Jones, 1997), Inertia Tensor Theory (Amazeen, 1999), Bayesian Approach (Brayanov & Smith, 2010), and Throwing Affordance Theory (Zhu & Bingham, 2011). The reasons for these ongoing differences of opinion are that heaviness is affected by various factors (as noted in Sec. 2) and the difficulty of strictly manipulating object size as a single independent variable, uncontaminated by other object properties. As a result, object size remains a wide-open topic regarding perception of heaviness.
The next sections present our research on the effect of object size on perceived heaviness. The experimental environment must be set up in accordance with the objective of the experiment: objective first and methodology second. Therefore, note that totally different experimental environments were set up, based upon the questions asked and sensory modalities examined.

Effects of haptically perceived object size on perceived heaviness: Use of the Real Haptics -Real vision environment: First experiments
It is a common tendency to interpret object size to mean that which has been visually perceived rather than that which has been haptically acquired during grasping and lifting (MacKenzie & Iberall, 1994). Similarly, the majority of researchers on heaviness perception have focused specifically on the effects of visually perceived object size (Ellis & Lederman, 1993) despite the fact that object size affects heaviness perception both visually and haptically (Charpentier 1891, as cited in Murray et al. 1999). As a consequence, experimental environments have kept haptic size constant through the use of grip apparatus, a handle, or strings (Gordon et al., 1991;Mon-Williams & Murray 2000;Buckingham & Goodale, 2010). Surprisingly few studies specifically focused on the effects of haptic size on heaviness perception, in spite of the evidence that effects on heaviness were considerably stronger for haptic size than visual size (Ellis & Lederman, 1993).
Selecting an experimental environment to focus solely on the haptically perceived object size was quite simple. Where we normally interact with an object in the real world through both haptics and vision (left of Fig. 2), we had only to enclose the subject's working space with screens (right of Fig. 2), to remove the real world visual input. The next step was to determine the type of objects to be used. Since heaviness is affected by an object's physical properties as described in Fig. 1, it was decided to use cubes given the properties of distribution of mass, center of gravity, density, and inertia tensor. When grasping cubes it is possible to constantly maintain the center of gravity between the thumb and index finger. This keeps the points of action on the object surface where normal forces are applied along an opposition vector between the thumb and index finger pads, with the center of gravity on the same line. Spherical objects were also considered (Charpentier 1891, as cited in Murray et al. 1999;Zhu & Bingham, 2011), but it proved difficult to consistently grasp the exact center of gravity between the thumb and index finger in a precision grip, especially when visual input was not available.
The next experimental design decision was whether or not haptically perceived size should be treated as a single independent factor. Objects of equal size can vary in weight depending on their specific density (Harper & Stevens, 1948;Zhu & Bingham, 2011). On the other hand, objects of equal density can vary in weight depending on their specific size (J. Ross & Di Lollo, 1970;Zhu & Bingham, 2011). Logically then, objects of equal weight can vary in both size and density. Thus, all three factors of weight, size and density were considered. This led to the decision to use three sets of cubes with different densities, one set each of: copper (CP; 8.93 g/cm 3 ), aluminum (AL; 2.69 g/cm 3 ), and plastic (PL; 1.18 g/cm 3 ), with 10 cubes in each set having progressively varying weights ranging from 0.05 N to 0.98 N. The surfaces of all cubes were covered with smooth, black vinyl to standardize input related to texture, thermal conductivity, friction, compliance, and colour. As a result, subjects received no cues as to the underlying materials. Each subject performed weight discrimination trials between sets of two cubes of identical density, e.g., CP vs. CP. In addition, equal numbers of trials for each weight difference with identical combination of weights were presented for all density conditions. Kawai, 2002a. Copyright © 2002 Springer-Verlag.) Figure 3A indicates, as an example, the differences between the 0.10 N-cube and the 0.20 Ncube among three different materials. By evaluating the accuracy of weight discrimination among three material conditions, it was possible to investigate whether or not haptic object size affected heaviness perception (Kawai, 2002a). As indicated in Fig. 3B, it was possible to investigate the accuracy of weight discrimination for all possible combinations by manipulating both weight (dotted arrows) and density (solid arrows) using these cubes. Surprisingly, this was the first published research attempt to study the psychological scales or discriminative abilities for weight or heaviness that covered all weight-density conditions (Kawai, 2003a). The probable reason for this is that researchers focusing on weight have concentrated on constant density conditions (D, E, and F in Fig. 3B) in which the psychological scale complemented physical weight. On the other hand, researchers focusing on the SWI have been interested primarily in equal-weight with different density conditions (B and H) believing that only in such conditions can object size affect heaviness perception. We believe that humans judge heaviness in the same manner for any combination of weight and density; our goal is to establish a heaviness model that explains heaviness perception for all weight-density conditions.

Fig. 2. A Real Haptics + Real Vision environment (Left) and a Real Haptics -Real Vision environment (Right). (Reproduced by permission from
With these manipulations, new findings were obtained regarding the effect of haptic size on perceived heaviness: (1) Whenever we lift objects to compare heaviness, haptically perceived object size is significantly involved in heaviness perception (Kawai, 2002a), (2) The effect of haptic size is not limited to a specific situation such as Charpentier's SWI (Kawai, 2002a), (3) The effect of haptically perceived size was strong for some subjects but less for others (Kawai, 2002a), (4) Object density also contributes to perception of heaviness, with significant interaction with weight (Kawai, 2002b), (5) As no other forms of input were presented to subjects to detect object density, it is concluded that approximate, but not exact, information about density is derived possibly from the integration of haptic size information www.intechopen.com with weight (Kawai, 2002b), (6) The Weight/Aperture, the finger span formed during thumb-index finger grasp (the opposition vector), or the width of cube itself, can be derived as a heaviness model as it reflects subjective responses in any weight-density conditions (Kawai, 2003a(Kawai, , 2003b. While it is clear that haptic object size is systematically involved in heaviness perception, the results of this study offer no neurological evidence about how, where and when in the nervous system this occurs Chouinard et al., 2009).

Effects of visually perceived object size on heaviness perception: Use of the Real Haptics + Virtual Vision environment: Second experiments
In Sec. 3.1, investigation of the effect of haptic size was made possible by simply removing the Real Vision from the Real Haptics + Real Vision environment. Such a simple and inexpensive experimental set-up has a great advantage since it is easily reproduced and replicated by others. Now we turn to visually perceived size. In experimental paradigms for the role of visual size in heaviness perception, it has been conventional that two or more objects of equal weight but unequal sizes are alternatively or in parallel lifted by subjects using wires, handles, or grip apparatus, and then rated for heaviness (See left in Fig. 4A and 4B) (H.E. Ross, 1969;David & Roberts, 1976;Masin & Crestoni, 1988;Gordon et al., 1991;Mon-Williams & Murray, 2000;Chouinard, et al., 2009;Rabe et al., 2009). Thus the haptically perceived size is kept constant. All studies unanimously reported that visually perceived size affected perceived heaviness: the smaller object felt heavier than the larger object although they were identical in weight.
However, these experimental set-ups clearly involved the factor of inertia tensor (Amazeen, 1999) which has significant impact on perceived heaviness. When lifting objects of different length, width, volume, shape, and orientation, it is possible to haptically perceive the differences in heaviness, even without visual information (Turvey & Carello, 1995). Thus, strictly speaking, we cannot determine conclusively whether the SWI is derived from differences in visual size or inertia tensor, when we adhere to the conventional methodology of the Real Haptic + Real Vision environment (left in Fig.4A and Fig.4B). This is the limitation of the real physical world for experimental control.
We decided, therefore, that the only way to separate the factor of visual size cues completely from other factors such as inertial tensor was to use a combination of 3D motion analysis and 3D computer graphics techniques to create virtual objects for the SWI paradigm. This type of augmented environment was developed as the Virtual Hand Laboratory at Simon Fraser University in Burnaby, Canada, in which 3D graphics (and other displays) were driven by 3D motion and forces. Computer graphics were developed to ensure completely independent manipulation of visual size information by superimposing computer-created graphics of different sizes on a single object (Fig. 4A, top) or two physically identical objects (Fig.4B, bottom); thus, haptic information was kept constant. As seen in Fig. 4, two different experimental set-ups were designed: a single physical grip apparatus was used specifically to record grip and load forces applied on the grip handle (top in Fig 4A), while two physical cubes with identical physical properties were used to investigate the effects of visual size on heaviness (bottom in Fig. 4B). In both environments, visual size varied as a single independent parameter without changing the inertia tensor. As shown in the right of Fig.  4B, each subject wore Crystal Eyes goggles and viewed through a semi-silvered mirror the stereo images (dotted line) on a monitor.  Kawai et al., 2007. Copyright © 2007 www.intechopen.com An OPTOTRAK 3D Motion Analysis system tracked infrared emitting markers attached to the physical cubes and the goggles and these 3D position data were used to create the stereographic images of the cubes in real time and subsequently, to analyze the lifting motion of the cubes. The physical cubes of identical size (3.0 x 3.0 x 3.0 cm) and mass (30.0 g) were invisible to subjects. The graphical size of the standard cube placed on the lefthand side of each subject was constant (5.0 x 5.0 x 5.0 cm; indicated as triangles in Fig. 5), while the comparison cube presented on the right-hand side varied from 1.0 x 1.0 x 1.0 to 9.0 x 9.0 x 9.0 cm (See details in Fig. 5). After lifting each pair of cubes, subjects were asked to report whether the comparison cube presented was perceived as Heavier, Lighter, or Similar in heaviness as compared to the standard cube.  Kawai et al., 2007. Copyright © 2007 Prior to undertaking trials, care was taken to accurately determine whether or not the augmented objects could produce for subjects a sense of reality, existence, or presence similar to real objects. Every participant had a strong sense of presence of the graphical objects and felt as if they were interacting with a physical object both in terms of force programming  and perceived heaviness (Kawai et al., 2007). It was concluded to be due to the characteristics of this augmented environment that the graphical objects moved without any noticeable delay (timing), in exactly the direction intended (direction), and synchronized with the subject's lifting movement (speed), developing a kind of personal relationship or ownership (Ehrsson, et al.,2004).
Findings indicated that: (1) visual size cues systematically affected heaviness; when the comparison cube was smaller in size than the standard cube, it was perceived to be heavier and vice versa (Fig. 5), (2) visual size cues influenced heaviness for all subjects under conditions with sufficient size differences between standard and comparison cubes of equal mass; that is, (3) when the comparison cube was smaller than 4.0 cm all the subjects perceived it to be heavier than the 5.0 cm standard cube (upper in Fig. 5), and when the comparison cube was larger than 7.0 cm all the subjects perceived it to be lighter (bottom in Fig. 5). (4) Interestingly, whether or not test subjects experienced the SWI was significantly correlated with their sensitivity to weight discrimination, but not their sensitivity to discriminate small differences in visual size, (5) Erroneously programmed motor commands were not systematically correlated to perceived heaviness or experience of the SWI (Kawai et al. 2007).
We emphasize that usage of the Real Haptic + Virtual Vision environment was the way to verify effects of only visual size cues on perception of heaviness, while presenting both visual and haptic cues "synchronously" to participants; it is impossible to obtain this evidence from any experimental set-ups in the real world (Real Haptic + Real Vision).

Pictorial depth cues of an object on perceived heaviness: Virtual Vision + Virtual Haptics environment: Third experiment
In Sec. 3.2, the Real Haptics + Virtual Vision environment ensured a strict manipulation of visual information independently from haptic information and suggested possibilities for presenting any type of visual/graphical stimulus with a range of intensities, unlike the Real Vision environment. Concerning vision, this suggested further study of visual components such as pictorial depth cues or stereopsis that may contribute to perceived object size and heaviness. For haptics, it suggested the development of a haptic display making possible the presentation of any type of haptic stimulus with a range of intensities. This led to the challenging experiment using a Virtual Haptic + Virtual Vision environment. Thus we demonstrate here an experiment using Virtual Vision + Virtual Haptics environment enabling us to address pictorial depth cues.

Volumetric information and pictorial depth cues in the size-weight illusion
Since volumetric information of objects has long been thought to be critical, numerous studies have focused on the volume of the object as the essential parameter for size in the size-weight illusion (SWI) (Scripture, 1897;H.E. Ross, 1969;Anderson, 1970;Ellis & Lederman, 1993). However, there is no direct evidence as to whether or not volumetric information is encoded in the process of size-weight integration in perceived heaviness. Further, although the dimensions of objects have been examined on reach-to-grasp movements (Westwood et al., 2002;Kwok & Braddick, 2003), no such research has been done on lifting movements or heaviness perception. This study investigated the effects of cues regarding pictorial depth and volumetric size cues on the size-weight integration in perceived heaviness. Weight displays were created using the PHANToM haptic stylus, synchronized and superimposed onto corresponding 2D graphic objects displayed on a 2D monitor, in accordance with manipulation of the stylus (virtual objects) by subjects. It was hypothesized that the degree of the SWI would be weakened, even to the point of disappearing, when the dimensions of pictorial depth cues were reduced from being a 3D cube to a 2D square.
www.intechopen.com  Figure 6A is a schematic of the programmed augmented environment, (see Kuang, et al., 2004 for details). Software on a Silicon Graphics Inc. computer was developed to display on the monitor (a), a graphic hook (b), suspending a graphic object (c) by a graphic wire. A PHANToM haptic device (e) by SensAble Technologies Inc., integrated into the Virtual Hand Laboratory system, was programmed with forces (at 1000 Hz) to reflect the subject's manipulation of the PHANToM stylus (d), as the object was lifted with the haptic stylus/graphic hook. Either 0.5 or 0.75 N was programmed for virtual object weight. This weight was constant for the visual conditions in Fig. 6B. Thus, subjects grasped and lifted virtual objects with programmed haptic and graphic properties. They perceived the heaviness of the virtual objects based on the lift of the virtual object with the stylus/graphic hook. 3D position and acceleration of lifting were processed by the system at 200 Hz. Temporal lag for the system to sample human-generated forces acting on the system, calculate reflected force directions for the haptic device, and display the motion of the graphic object was 2-3 frames at 60 Hz, i.e., 50 ms max. Subjects did not notice any delay between their initial movement with the PHANToM stylus and initiation of movement of the graphic object. Figure 6B shows three graphic stimuli with different sizes and pictorial depth cues (solid, wire-frame and square). Solid cubes (Fig. 6B-a) had 3D pictorial cues, with different colors on each of the three visible sides so that participants readily perceived them as solid or three-dimensional. Three different sizes of solid graphic cubes were used (large: 6.9 x 6.9 x 6.9 cm, medium: 5.0 x 5.0 x 5.0 cm and small: 3.0 x 3.0 x 3.0 cm, respectively). The large and small cubes were used to assess the frequency of the SWI, while the medium cubes were used for controls. The size ratios between the large and small solid cubes were approximately 12 : 1 in volume and 5.4 : 1 in planar area. Wire-frame graphic cubes (Fig. 6Bb) provided weaker 3D cues compared to solid cubes since colors were not added to any sides. As a result, the perception of depth or dimension was unstable like the Necker's cube; sometimes the graphic object was perceived with a cube shape and sometimes a plane hexagon. The ratios between the large and small frame cubes were identical to those for solid cubes. Graphic squares (Fig. 6B-c) had no 3D pictorial cues so that subjects perceived them as a two-dimensional plane. Three different sizes of squares (large: 10.4 x 10.4 cm, medium: 6.5 x 6.5 cm and small: 3.0 x 3.0 cm, respectively) were used. The ratio in size between the large and small squares was about 12 : 1 in planar area. For all graphic stimuli, the medium size was used for control purposes. All stimuli were experienced by all subjects. Subjects performed 4 trials in which the second object was larger in size than the first object (the Larger condition), 4 trials in which the second object was smaller in size than the first object (the Smaller condition), and 4 trials in which the size of the second object was identical to that of the first object (the Identical condition). Thus each subject performed 12 trials for each visual condition, and 36 trials in total. The trials were presented in random order across conditions and subjects.

Augmented environment created by computer graphics and computer haptics
Sixteen right-handed adults participated; none had any visual, muscular, or cutaneous problems. Further, none had previous experience with the experimental tasks or were familiar with the hypothesis being tested. Basically, subjects grasped and lifted virtual objects with programmed haptic and graphic properties, using the PHANToM stylus. They perceived weight of the virtual objects based on the reflected haptic 3D forces, and the graphic object motion viewed during the lift. Instructions were to grasp the stylus with the thumb and index finger of the dominant hand and to confirm that the stylus and graphic object motion were synchronously related to the lifting movement. At the outset, subjects were instructed to maintain the speed of the lift as constant as possible throughout the trials. To facilitate this, there were five to ten practice lifts to become accustomed to the Virtual Haptic and Virtual Vision environment, lifting each medium size stimulus. In addition, subjects were requested to view the object without the eyes being intentionally closed during the lifting process. Next for the test trials, each subject was instructed to grasp the stylus when the first of two graphic objects appeared on the monitor, to lift it vertically only once with a single smooth movement to a height of approximately 5 cm, and then to replace it after attempting to memorize their perception of its heaviness. Immediately after replacing the first graphic object, it disappeared. Following that, the second graphical object immediately appeared, to be maneuvered in exactly the same manner as the first. After completing the two lifts, each subject was requested to state whether the second object was perceived to be Heavier, Lighter, or Similar in comparison to the first object. Because cues related to visual size were very weak on perceived heaviness (Ellis & Lederman, 1993) and any cognitive bias is liable to affect judgment of heaviness in the final decision phase (Mon-Williams and Murray, 2000) or in the initial lifting phase in the form of expectation (H.E. Ross, 1969), the subject was requested to report perception of heaviness exactly and without hesitation, once the two lifts were completed in a given trial. These instructions, requests, and requirements were deemed essential to minimize such cognitive biases on perceived heaviness.
Similar to previous studies (Kawai, 2002a(Kawai, , 2002b, the percentage of subject responses were calculated as a function of the visual size of the second object and response categories (Heavier, Similar, and Lighter) for each individual subject. For example, if a subject's response was Heavier in two trials, Lighter in one trial, and Similar in one trial from among four trials for a particular visual condition, the percentage of responses for each category in this condition is 50 % for Heavier, 25 % for Lighter, and 25 % for Similar. The individual percentages were then averaged across subjects and visual conditions. To assess the effect of pictorial cues (3 levels), size (2 levels), and directions of size variation (2 levels) on perceived heaviness, a three-way within-subject ANOVA was performed on the individual means of www.intechopen.com frequency of subject responses. We summarize below the effects on the frequency of occurrence of the SWI. Figure 7 indicates the mean values and standard errors of the percentages for the occurrence of the SWI obtained from each pictorial depth cue, i.e., solid cube (Cube), wire-frame cube (Frame) and square (Square). The white bars (A) indicate the Smaller condition in which the second object was smaller in size than the first. The smaller object was reported as heavier than the larger one. The gray bars indicate the Identical Condition in which sizes were identical between the first and second object. Although the graphical sizes remained constant in the Identical condition, for some reason, subjects accidentally reported a difference in heaviness between the first and the second lifts (See Sec. 2-1). The black bars (B) are those obtained from the Larger condition in which the second object was larger in size than the first. For these conditions, the larger object was reported as lighter than the smaller one. Again, the grey bars are those obtained from the Identical condition in which they reported the second object to be perceived as lighter. Fig. 7. Mean percentages (%) of the size-weight illusion (SWI) for each pictorial depth cue for each condition through which subjects compared the heaviness of the small size with that of the large (white bars), the large with the small (black bars), and the median with the equivalent median (gray bars).

Results: Effects of depth cues and size on perceived heaviness, and the sizeweight illusion
There was a significant effect of change in size (F (1,15) =28.15, p< .001), indicating that the SWI occurred significantly for each pictorial depth cue when the size was changed compared to the conditions when the size was not changed. There were, however, no effects of pictorial depth cues (F(2, 30) =0.26) nor for the direction of size change (F(1, 15 = 0.46), suggesting pictorial depth cues do not affect the frequency or strength of the SWI. A significant interaction was observed only between pictorial depth cue and direction of size change (F(2, 30) = 31.53, p < .001). This is due to the fact that the SWI occurred more frequently in the Smaller condition (57.8% in Cube,67.2% in Frame and 59.4 % in Square) than in the Larger condition (51.6% in Cube, 34.4 % in Frame and 45.3% in Square). There were no significant differences in probability in the Identical condition: the probability when www.intechopen.com the reports of test subjects regarding the small object being Heavier was similar to that of the large object being Lighter. This phenomenon has been described in detail and discussed previously, according to Weber's Law (Kawai, 2002a).

Discussion
All subjects experienced the SWI for all three visual stimuli in Fig. 6B without any significant differences, suggesting that, contrary to our hypothesis, the SWI occurs to the same degree with any pictorial depth cues! Volumetric information of an object has long been thought to be critical in bringing about the SWI (Jones, 1986). As a result, numerous studies have investigated the phenomenon of the SWI and discussed based on the volume of an object as the essential parameter for size (Scripture, 1897;H.E. Ross, 1969;Ellis & Lederman, 1993). Furthermore, in studies related to motor programming for the lifting of an object, information of volume has also been thought to be a critical factor in estimating weight of an object as part of the process for producing the required lift forces (H.E. Ross, 1969;Gordon et al., 1991). Thus, Westwood and his colleagues (2002) proposed the necessity of a volumetric object description in the process of specifying the grip force and the load force necessary for lifting and manipulation because such forces must be scaled to the anticipated mass of the object either by estimating its volume and density or by accessing stored knowledge related to the mass of the other object. It was, therefore, hypothesized at the beginning of this study that the SWI would decrease in frequency or even disappear when dimensional cues were reduced from 3D structure such as a cube to a 2D structure such as a square. The results, however, indicated that all the subjects equally experienced the SWI for all three visual stimuli with any cues of a 2D graphical object. This suggests that whatever the pictorial depth cues are, these factors are not critical for the occurrence of the SWI. The present results, therefore, conclude, contrary to commonly accepted theory, that pictorial depth cues responsible for 3D perception are not crucially necessary in the process of producing the SWI. In short, the perceptual system of heaviness may not be responsive to the dimension provided by pictorial depth cues, at least under the VIRTUAL VISION + VIRTUAL HAPTIC environment of this experiment.

What is heaviness?
Various physical attributes are involved in the perception of heaviness of objects, leading to an understanding that psychological heaviness is not equal to physical weight. Heaviness perception is not simply a function for sensing object weight. Rather it functions as a form of recognition of the object. That is, all information obtained synchronously when humans grasp and lift an object may be gathered across modalities, integrated with or subtracted from each other, interpreted by object knowledge, and then formed as perceived heaviness.
Among physical properties influencing heaviness, rotational inertia can be considered vital for future investigation, given evidence rotational inertia firmly affects heaviness (Turvey & Carello, 1995;Amazeen, 1999;Streit et al., 2007;Cf. Zhu & Bingham, 2011). However, such evidence has not explained neurologically how it is obtained from lifting, holding and manipulating an object and how it contributes to forming its heaviness. Gaining an understanding of how humans process the information about the rotational inertia of objects www.intechopen.com will contribute to understanding how humans recognize an object as well as how humans dexterously manipulate an object or tool (MacKenzie & Iberall, 1994).
Another important issue for the future investigation is the connection between perceiving heaviness phase and the object lifting phase indicated in Fig. 1. How are heaviness perception and object manipulation interrelated? That is, what needs to be investigated is where heaviness ends and lifting begins (cf. Goodale, 1998) as well as whether or not humans make use of heaviness for object lifting in the sensorimotor system. The motor system for programming lifting forces is posited to operate independently of the perceptual system of heaviness (Goodale, 1998;Flanagan & Beltzner, 2000;Grandy & Westwood, 2006;Brayanov et al., 2010). This is based on the evidence that the load forces generated in the motor system quickly adapt to object weight while the heaviness perceptual system did not (Flanagan & Belzner, 2000;Grandy & Westwood, 2006;Flanagan et al., 2008). It seems, nevertheless, natural and reasonable to think that there should be some linkage enabling the exchange of some information, especially weight-related information, between the two systems (Maschke, et al., 2006), considering evidence related to the role of the sensorimotor system in heaviness perception (Sec. 2.1). It is believed that the solution of this actionperception matter will lead to the development of an optimal internal model in force programming to lift objects. These considerations, however, are not limited to heaviness, but probably hold true also for how human perceive movement and spatial orientation.

Computer graphics and haptic displays: Powerful tools to understand how humans perceive heaviness
Computer graphic and haptic displays have recently become more popular and widely used as experimental tools -or environments -in research related to heaviness (Heineken & Schulte, 2007;Haggard & Jundi, 2009;Mawase & Karmiel, 2010). The reason for this may be that researchers have noticed practical advantages of using such devices for experimental control even if reality or physical presence is slightly sacrificed. The spatial resolutions and temporal lags for these devices are also important; with high resolution, the objects and environments created can affect the human sensorimotor and perceptual systems in the same manner as real, physical objects. However, such superior technologies should not be depended upon haphazardly since they exhibit, in some cases, difficulties in experimental replication. Therefore, whether or not to adopt and/or adapt evolving technology is dependent on research goals (Sec. 3.2). The experimental set-up should always be objectivecentered rather than technology-centered. Within this restriction, computer graphics and haptic displays have been incorporated into the experimental set-ups for this study. The resulting progress reveals the usefulness of such equipment and suggests many fundamental findings that can be expected from future experiments.
The perceived usefulness of these powerful tools includes: 1. The reality and presence of the environments and objects created by these computer displays depend on temporal and spatial precision, and on consistency of the timing, amplitude, and direction for forces/motions as humans interact with objects in the environments. 2. The created stimuli can functionally act on human perceptual and motor systems in a similar manner as real, physical stimuli.
3. These devices can accurately isolate or focus on only one single factor from all factors possible. Such a result is often impossible or exceptionally hard to accomplish with only physical objects. 4. These devices can present stimuli with a considerably wider range of intensity or even in smaller increments than physical stimuli. Thus, they can act on the sensorimotor and perceptual systems of individuals for whom sensitivities or discriminative abilities are quite different. In the physical world, individual variability in sensitivity sometimes prevents the uncovering of mechanisms underlying the perceptual system. That is, it is often the case that the intensities determined by the experimenters are sufficient for some humans to detect and discriminate, but not others. However, it is almost impossible to cover all the range of intensities of stimuli using hundreds of physical objects. In this sense, these devices are powerful tools for obtaining detailed assessments of the discriminative ability of human perception and for users of human/computer systems whose sensitivities are widely different. 5. These devices can quickly, in a timely, constant and invariant manner, vary or exchange a part or the whole of multimodal stimuli, for presentation to humans. This is also advantageous, compared to the physical environment, to the tasks in which subjects are required to compare two or more bits of sensory information among all the stimuli presented. This may elucidate the mechanisms of multimodal integration and unitary perception, e.g., object recognition. 6. These computer devices could possibly generate and present a stimulus possessing a weight-size relationship or an incredible density that humans are physically unable to experience on earth. 7. These devices enable accurate, precise and high resolution temporal and spatial perturbations to human vision, haptics, and goal-directed movement.
In the history of the development of haptic interfaces, little or no concern seems to have been paid to the human unit of heaviness. Instead, haptic devices -including robot hands and arms -have been developed based on the machine-centered unit of weight and reflected forces. This seems to hold true also for psychologists who have termed all psychological phenomena as illusions and have avoided interpreting them based on biological or physiological views. Of course, such weight-based devices have assisted humans to lift or manipulate a variety of objects in a smooth and safe manner. This success seems to depend, in large part, on efforts of users via their adaptation and/or learning systems, rather than via the engineered haptic systems. In the future, however, users of haptic devices will certainly include those lacking the abilities of adaptation and/or learning such as the elderly, the physically challenged, and medical patients who cannot easily adjust to such computer-centered systems. Thus, such systems developed in the future should be humancentered. Ideally such haptic devices should be suited for adaptation to the human haptic systems, to perceive and act in a manner similar to that of humans. We hope this chapter contributes to future development of human/computer interfaces, based on human haptics, that can contribute to quality of human life and human experience.

Acknowledgments
This research was supported by grants from the Japan Society for the Promotion of Science There has been significant progress in haptic technologies but the incorporation of haptics into virtual environments is still in its infancy. A wide range of the new society's human activities including communication, education, art, entertainment, commerce and science would forever change if we learned how to capture, manipulate and reproduce haptic sensory stimuli that are nearly indistinguishable from reality. For the field to move forward, many commercial and technological barriers need to be overcome. By rendering how objects feel through haptic technology, we communicate information that might reflect a desire to speak a physicallybased language that has never been explored before. Due to constant improvement in haptics technology and increasing levels of research into and development of haptics-related algorithms, protocols and devices, there is a belief that haptics technology has a promising future.

How to reference
In order to correctly reference this scholarly work, feel free to copy and paste the following: http://www.intechopen.com/books/haptics-rendering-and-applications/computer-graphic-and-phantom-hapticdisplays-powerful-tools-to-understand-how-humans-perceive-heavin