Abstract
Many handheld force-feedback devices have been proposed to provide a rich experience with mobile devices. However, previously reported devices have been unable to generate both constant and translational force. They can only generate transient rotational force since they use a change in angular momentum. Here, we exploit the nonlinearity of human perception to generate both constant and translational force. Specifically, a strong acceleration is generated for a very brief period in the desired direction, while a weaker acceleration is generated over a longer period in the opposite direction. The internal human haptic sensors do not detect the weaker acceleration, so the original position of the mass is \\\"washed out\\\". The result is that the user is tricked into perceiving a unidirectional force. This force can be made continuous by repeating the motions. This chapter describes the pseudoattraction force technique, which is a new force feedback technique that enables mobile devices to create a the sensation of two-dimensional force. A prototype was fabricated in which four slider-crank mechanism pairs were arranged in a cross shape and embedded in a force feedback display. Each slider-crank mechanism generates a force vector. By using the sum of the generated vectors, which are linearly independent, the force feedback display can create a force sensation in any arbitrary direction on a two-dimensional plane. We also introduce an interactive application with the force feedback display, an interactive robot, and a vision-based positioning system.
1. Introduction
Haptic interfaces in virtual environments allow users to touch and feel virtual objects. Significant research activities over 20 years have led to the commercialization of a large number of sophisticated haptic interfaces including PHANToM and SPIDAR. However, most of these interfaces have to use some type of mechanical linkage to establish a fulcrum relative the ground (Massie & Salisbury, 1994; Sato, 2002), use huge air compressors (Suzuki et al., 2002; Gurocak et al., 2003), or require that a heavy device be worn (Hirose et al., 2001), thus preventing these mobile devices from employing haptic feedback.
Although haptic feedback provides many potential benefits as regards the use of small portable hand-held devices (Ullmer & Ishii 2000; Luk et al., 2006), the haptic feedback in mobile devices consists exclusively of vibrotactile stimuli generated by vibrators (MacLean et al., 2002). This is because it is difficult for mobile devices to produce a kinesthetic sensation. Moreover, the application of low-frequency forces to a user requires a fixed mechanical ground that mobile haptic devices lack. To make force-feedback devices available in mobile devices, ungrounded haptic feedback devices have been developed since they are more mobile and can operate over larger workspaces than grounded devices (Burdea, 1996). The performance of ungrounded haptic feedback devices is less accurate than that of grounded devices in contact tasks. However, ungrounded haptic feedback devices can provide comparable results in boundary detection tests (Richard & Cutkosky, 1997). Unfortunately, typical ungrounded devices based on the gyro effect (Yano et al., 2003) or angular momentum change (Tanaka et al., 2001) are incapable of generating both constant and directional forces; they can generate only a transient rotational force (torque) sensation. In addition, Kunzler and Runde pointed out that gyro moment displays are proportional to the mass, diameter, and angular velocity of the flywheel (Kunzler & Runde, 2005).
There are methods for generating sustained translational force without grounding, such as propulsive force or electromagnetic force. Recently, there have been a number of proposals for generating both constant and directional forces without an external fulcrum. These includeusing two oblique motors whose velocity and phase are controlled (Nakamura & Fukui, 2007), simulating kinesthetic inertia by shifting the center-of-mass of a device dynamically when the device is held with both hands (Swindells et al., 2003), and producing a pressure field with airborne ultrasound (Iwamoto et al., 2008).
2. Pseudo-Attraction Force Technique
2.1. Haptic interface using sensory illusions
To generate a sustained translational force without grounding, we focused on the characteristic of human perception, which until now has been neglected or inadequately implemented in haptic devices. Although human beings always interact with the world through human sensors and effectors, the perceived world is not identical to the physical world (Figure 1). For instance, when we watch television, the images (a combination of RGB colors) we see are different from physical images (a composition of all wavelengths of light), and TV animation actually consists of a series of still pictures. Such phenomena are usually interpreted by converting them to subjectively equivalent phenomena. These distortions of human perception, including systematic errors or illusions, have been exploited when designing human interfaces. Moreover, some illusions have the potential to enable the development of new haptic interfaces (Hayward 2008). Therefore, the study of haptic illusions can provide valuable insights into not only human perceptual mechanisms but also the design of new haptic interfaces.
2.2. Principle
The method, which is called the pseudo-attraction force technique, exploits the characteristics of human perception to generate a force sensation, using different acceleration patterns for two directions to create a perceived force imbalance, and thereby produce the sensation of directional pushing or pulling. Specifically, a strong acceleration is generated for a very brief time in the desired direction, while a weaker acceleration is generated over a longer period in the opposite direction. The weaker acceleration is not detected by the internal human haptic sensors, so the original position of the mass is "washed out". The result is that the user is tricked into perceiving a unidirectional force. This force can be made continuous by repeating the motions. Figure 2 shows a model of the nonlinear relationship between physical and psychophysical quantities. If the acceleration patterns are well designed, a kinesthetic illusion of being pulled can be created because of this nonlinearity.
2.3. Implementation
To generate the asymmetric back-and-forth motion of a small, constrained mass, we have adopted a swinging slider-crank mechanism as a quick motion mechanism (Figure 3). In the mechanism, the rotation of a crank (OB) makes the weight slide backwards and forwards with asymmetric acceleration. The force display is composed of a single degree of freedom (DOF) mechanism. The force vector of the asymmetric oscillation is
where
where
We fabricated a prototype of the force display. In the prototype,
3. Requirements for perceiving pseudo-attraction force
There are still many aspects of the perception of the pseudo-attraction force that are not well understood, but knowledge has been accumulating. In this section, we introduce various parameters for eliciting the pseudo-attraction force through experimental results.
3.1. Acceleration profile
First, we determined whether oscillations with asymmetric acceleration elicit the perception of a pseudo-attraction force. Two oscillations with different acceleration profiles were compared as haptic stimuli: asymmetric acceleration (shown in Figure 4) and symmetric acceleration (control). For the asymmetric acceleration stimuli, the average percentage correct scores (i.e., how often the perceived force direction matched the crank-to-slider direction in Figure 3) for all subjects were approximately 100% at frequencies below 10 cycles per second when we used a binary judgment task (forward or backward). For the symmetric acceleration stimuli, the scores were between 25% and 75%, which is the chance level. These results show that the symmetric acceleration could not generate a directed force sensation. We performed a binomial test for the average percent correct scores, which showed no significant effect of the control stimuli for any of the frequencies. This means that symmetric acceleration does not elicit the perception of a pseudo-attraction force. Again, no directional force was felt if the mass were merely moved back and forth, but different acceleration patterns for the two directions to create a perceived force imbalance produced the perception of a pseudo-attraction force (Amemiya & Maeda, 2009).
3.2. Frequency of acceleration
Frequency of acceleration plays an important role in eliciting the perception of a pseudo-attraction force. Oscillations with high frequency might create a continuous force sensation, but previous experimental results showed that the performance decreased steadily at frequencies over ten cycles per second (Amemiya et al., 2008). However, low frequency oscillation tends to be perceived as a knocking sensation. If we wish to create a sustained rather than a transient force sensation such as the sensation of being pulled continuously, the frequency should be in the 5 to 10 cycles per second range. In addition, those who experienced the stimuli strongly perceived the sensation at 5 cycles per second independent of other parameters (Amemiya & Maeda, 2009).
3.3. Gross weight of force display
Changes in the gross weight and the weight of the reciprocating mass affects the perceived force sensation. Experimental results have shown that lighter gross weights and a heavier reciprocating mass yield higher percent-correct scores in binary judgment tasks for all frequencies (Amemiya & Maeda, 2009). Considering the Weber fraction of force perception, the differential threshold of force perception is thought to increase as the gross weight increases. In addition, the increase in the gross weight may work as a mass damper, which would reduce the gain of the effective pulse acceleration. The threshold of the ratio of the gross weight and the weight of the reciprocating mass was 16 %, which is a rough standard for effective force perception in the developed prototype.
3.4. Angular resolution
The azimuth accuracy of the perceived force direction versus the stimulated direction generated by an asymmetric acceleration has been examined (Amemiya et al., 2006). The orientation of the force vector was altered from 0 to 360 on the horizontal plane in 15 steps (24 vectors). The subjects were required to reply with one of 360 degrees in a static posture. The results showed that the root mean square of the angular errors between response and stimulus was approximately 20 degrees. When users move or rotate their bodies, i.e., dynamically explore the force vector, their angular resolution would be higher than that in a static posture.
3.5. Cancellation of orthogonal oscillation
If asymmetric oscillation was generated by rotational mechanism such as the slider-crank mechanism, a force perpendicular to the directional one were created because of the motion of the linkages. The side-to-side force prevents the user from sensing the desired direction. The side-to-side force should be cancelled out completely, for instance, by using two identical but mirror-reversed mechanisms (Amemiya et al., 2008).
4. Application
4.1. Overview
For broad social use, we designed an interactive application of haptic navigation based on a pseudo-attraction force display. The scenario was as follows.
4.2. System configuration
The system consists of a tray held by the user (waiter), a small bag containing a battery and a control device, and a position and direction identification system based on infrared LEDs and a wide-angle camera (Figure 5). The force display and infrared LEDs are embedded in the tray. The user's position and posture are detected by placing three super-high luminance infrared LEDs (OD-100, OPTO Diode Corp., peak wavelength 880 nm, beam angle 120 degrees), at the corners of a right-angled isosceles triangle (side length = 100 mm) on the tray. The infrared rays are captured by a ceiling-mounted IEEE1394 black and white CMOS camera (Firefly MV, FFMV-03MTM; Point Grey Research Inc.) with a wide-angle lens (field angle 175 degrees). The positions and orientations of each IR-LED are obtained by binarizing the brightness value from the acquired camera image with OpenCV library, and calculating the position and orientation from the relationship with a right-angled isosceles triangle formed by three dots (Figure 6).
The user must hold the tray horizontally because of the drink being carried on it. Therefore, the user’s posture can be presumed by detecting three IR-LEDs. The image capture rate is about 60 fps. The camera height is about 3.0 m and the camera faces the ground. When three points can be acquired, the position measurement error does not exceed 100 mm. This is sufficient for our demonstration since the distance to the targets is about 1,000 mm.
There are two ways to generate a two-dimensional force vector (Figure 7), and we fabricate each prototype. A turntable-based force display is one module based on a slider-crank mechanism with a turntable (Figure 8). The direction of the force display module is controlled with a stepper motor (bipolar, step angle 1.8 degrees, 1/4 micro step drive; KH42HM2-851; Japanese Servo Ltd.). engaged by a belt with a belt pulley installed in the turntable (Amemiya et al., 2009).
A vector-summation-based force display is designed to generate a force sensation in at least eight cardinal directions by the summation of linearly independent force vectors. Four slider-crank mechanism pairs are embedded in the force display in the shape of a crosshair. By combining force vectors generated by each slider-crank mechanism, the force display can create a virtual force in eight cardinal directions on a two-dimensional plane.
The target is several bear-shaped robots (RobotPhone; Iwaya Inc.). As the customer speaks, he also moves his head and hands to communicate with gestures.
4. 3. Demonstration procedure
The user moved towards the target following the direction indicated by the perceived force sensation. The force direction was controlled so that it faced the target (customer) based on the posture detection system. Control instructions were sent from the computer to the microcomputer via a wireless link (XBee-PRO (60 mW) ZigBee module; MaxStream) when required. The user chose one customer by stopping in front of the target. If this choice was correct, the customer (bear-shaped robot) said, ‘‘thank you’’; otherwise, the customer said, ‘‘I did not order this’’ while moving his head and hands to communicate with gestures.
4. 4. Discussion
We demonstrated the above application at an international conference and exhibition. The average rate of correct delivery to the target exceeded 75 % (note that none of the participants received any initial training), indicating that the navigation support provided is effective and appropriate. The results show the usefulness of the proposed technique. The few delivery failures appear to be due to tracking errors in the camera system or a delay between the rotation of the stepper motor and the user’s perception of the change. Moreover, we believe the force’s amplitude to be attenuated by the connection of the device to the tray, and this attenuation also influenced delivery failure. We sometimes observed that not all the LEDs could be detected since some were occasionally obscured by the participant. System robustness could be improved by adopting a different position and posture identification system. This haptic navigation could be also applied to a navigation system for the visually impaired (Amemiya & Sugiyama 2009).
5. Conclusion and future potential
The developed haptic display based on a pseudo-attraction force technique conveyed a kinesthetic illusion of being pulled or pushed. The ability of the haptic display to realize a wide-area social support system was discussed. Future work will include extending the reach by using a global positioning system to allow outdoor use.
AcknowledgementsWe thank Dr. Ichiro Kawabuchi for his assistance. This study was supported by Nippon Telegraph and Telephone Corporation and was also partially supported by the sponsorship of CREST, Japan Science and Technology Agency.
References
- 1.
Amemiya T. Ando H. Maeda T. T. 2005 Virtual force display: Direction guidance using asymmetric acceleration via periodic translational motion, ,619 622 , IEEE Computer Society. - 2.
Amemiya T. Ando H. Maeda T. T. 2006 Directed force perception when holding a nongrounding force display in the air. . Paris, France.317 324 . - 3.
Amemiya T. Ando H. Maeda T. T. 2007 Hand-held force display with spring-cam mechanism for generating asymmetric acceleration, ,572 573 , March 2007, IEEE Computer Society. - 4.
Amemiya T. Ando H. Maeda T. 2008 Lead-Me interface for pulling sensation in hand-held devices, ,5 3 1 17 . - 5.
Amemiya T. Maeda T. 2008 Asymmetric oscillation distorts the perceived heaviness of handheld objects, ,1 1 9 18 . - 6.
Amemiya T. Maeda T. 2009 Directional force sensation by asymmetric oscillation from a doublelayer slider-crank mechanism, ,9 1 011001, ASME. - 7.
Amemiya T. Maeda T. Ando H. 2009 Location-free Haptic Interaction for Large-Area Social Applications, ,13 5 379 386 , Springer. - 8.
Amemiya T. Sugiyama H. 2009 Haptic Handheld Wayfinder with Pseudo-Attraction Force for Pedestrians with Visual Impairments, ,107 114 , ACM Press. - 9.
Burdea G. C. 1996 Force & Touch Feedback for Virtual Reality, Wiley, New York. - 10.
Chang A. O’Sullivan C. 2005 Audio-haptic feedback in mobile phones. ,1264 1267 , ACM Press. - 11.
Gurocak H. Jayaram S. Parrish B. Jayaram U. 2003 Weight sensation in virtual environments using a haptic device with air jets, ,3 2 ASME,130 135 . - 12.
Hirose M. Hirota K. Ogi T. Yano H. Kakehi N. Saito M. Nakashige M. 2001 HapticGEAR: The Development of a Wearable Force Display System for Immersive Projection Displays, ,123 130 . - 13.
Iwamoto T. Tatezono M. Hoshi T. Shinoda H. 2008 Non-Contact Method for Producing Tactile Sensation Using Airborne Ultrasound, ,504 513 . - 14.
Kunzler U. Runde C. 2005 Kinesthetic Haptics Integration into Large-Scale Virtual Environments,,551 556 . - 15.
Luk J. Pasquero J. Little S. Mac Lean. K. Levesque V. Hayward V. 2006 A role for haptics in mobile interaction: Initial design using a handheld tactile display prototype. , ACM Press,171 180 . - 16.
Mac Lean. K. E. Shaver M. J. Pai D. K. 2002 Handheld Haptics: A USB Media Controller with Force Sensing, ,311 318 . - 17.
Massie T. Salisbury J. K. 1994 The phantom haptic interface: A device for probing virtual objects, ,55-1 , Chicago, IL., 1994,295 300 . - 18.
Nakamura N. Fukui Y. 2007 Development of Fingertip Type Non-grounding Force Feedback Display, ,582 583 . - 19.
Richard C. Cutkosky M. 1997 Contact Force Perception with an Ungrounded Haptic Interface, ,181 187 . - 20.
Sato M. 2002 Spidar and virtual reality, ,13 17 23 . - 21.
Suzuki Y. Kobayashi M. Ishibashi S. 2002 Design of force feedback utilizing air pressure toward untethered human interface, . ACM Press, 2002,808 809 . - 22.
Swindells C. Unden A. Sang T. 2003 TorqueBAR: an ngrounded haptic feedback device. . ACM Press,52 59 . - 23.
Tanaka Y. Masataka S. Yuka K. Fukui Y. Yamashita J. Nakamura N. 2001 Mobile torque display and haptic characteristics of human palm. ,115 120 . - 24.
Ullmer B. Ishii H. 2000 Emerging frameworks for tangible user interfaces.39 Nos. 3-4,915 931 . - 25.
Yano H. Yoshie M. Iwata H. 2003 Development of a nongrounded haptic interface using the gyro effect, . IEEE Computer Society,32 39 .