Tactile Displays with Parallel Mechanism

1. Introduction

Since more intuitive and realistic interaction between human and computer/robot has been requested, haptics has emerged as a promising element in the field of user interfaces. Particularly for tasks like real manipulation and exploration, the demand for interaction enhanced by haptic information is on the rise.

Researchers have proposed a diverse range of haptic devices. Force feedback type haptic devices with robotic link mechanisms have been applied to teleoperation system, game interfaces, medical simulators, training simulators, and interactive design software, among other domains. However, compared to force feedback interfaces, tactile displays, haptic devices providing skin sense, have not been deeply studied. This is at least partly due to the fact that the miniaturization and the arrangement necessary to construct such systems require more advanced mechanical and electronic components.

A number of researchers have proposed tactile display systems. In order to provide tactile sensation to the skin, work has looked at mechanical, electrical and thermal stimulation. Most mechanical methods involve an array of pins driven by linear actuation mechanisms with plural number of solenoids, piezoelectric actuators, or pneumatic actuators. In order to realize such compact arrangement of stimulators, parallel mechanisms have been commonly adopted.

This chapter deals with parallel mechanisms for tactile displays and their specialized designs for miniaturization and feasibility. In addition, the chapter also covers application of tactile displays for human-computer/robot interfaces.

2. Tactile display research review

Researchers have proposed a diverse range of haptic interfaces for more realistic communication methods with computers. Force feedback devices, which have attracted the most attention with their capacity to physically push and pull a user’s body, have been applied to game interfaces, medical simulators, training simulators, and interactive design software, among other domains (Burdea, 1996). However, compared to force feedback interfaces, tactile displays have not been deeply studied. It is clear that haptic applications for mobile devices such as PDAs, mobile computers and mobile phones will have to rely on tactile devices. Such a handheld haptic system will only be achieved through the development of a fast, strong, small, silent, safe tactile display module, with low heat dissipation and power consumption. Furthermore, stimulation methods reflecting human tactile perception characteristics should be suggested together with a device.

A number of researchers have proposed tactile display systems. In order to provide tactile sensation to the skin, work has looked at mechanical, electrical and thermal stimulation. Most mechanical methods involve an array of pins driven by linear actuation mechanisms such as a solenoids, piezoelectric actuators, or pneumatic actuators. Particularly, their mechanisms are focused on miniaturized parallel arrangement of actuators. In 1995, a tactile display composed of solenoids has been investigated and it was applied to an endoscopic surgery simulator (Fisher et al., 1997). One of well known tactile displays is composed of RC servomotors. The servomotor occur linear motion of tactor and the parallel arrangement of tactors form a tactor array of the tactile display (Wagner et al., 2002). Another example is the “Texture Explorer”, developed by Ikei’s group (Ikei & Shiratory, 2002). This 2×5 flat pin array is composed of piezoelectric actuators and operates at a fixed frequency (~250Hz) with maximum amplitude of 22μm. Summers et al. developed a broadband tactile array using piezoelectric bimorphs, and reported empirical results for stimulation frequencies of 40Hz and 320Hz, with the maximum displacement of 50μm (Summers & Chanter, 2002). Since the tactile displays mentioned above may not result in sufficiently deep skin indentation, Kyung et al. (2006a) developed a 5x6 pin-array tactile display which has a small size, long travel and high bandwidth. However, this system requires a high input voltage and a high power controller. As an alternative to providing normal indentation, Hayward et al. have focused on the tactile sensation of lateral skin stretch and designed a tactile display device which operates by displaying distributed lateral skin stretch at frequencies of up to several kilohertz (Hayward & Cruz-hernandez, 2000; Luk et al., 2006). However, it is arguable that the device remains too large (and high voltage) to be realistically integrated into a mobile device. Furthermore, despite work investigating user performance on cues delivered by lateral skin stretch, it remains unclear whether this method is capable of displaying the full range of stimuli achievable by presenting an array of normal forces. More recently, a miniaturized tactile display adopting parallel and woven arrangement of ultrasonic linear actuators have been proposed (Kyung & Lee, 2008). The display was embedded into a pen-like case and the assembly realized haptic stylus applicable to a touchscreen of mobile communication device.

Konyo et al. (2000) used an electro-active polymer as an actuator for mechanical stimulation. Poletto and Doren (1997) developed a high voltage electro-cutaneous stimulator with small electrodes. Kajimoto et al. (1999) developed a nerve axon model based on the properties of human skin and proposed an electro-cutaneous display using anodic and cathodic current stimulation. Unfortunately, these tactile display devices sometimes involve user discomfort and even pain.

We can imagine a haptic device providing both force and tactile feedback simultaneously. Since Kontarinis et al. applied vibration feedback to a teleoperation (Kontrarinis & Howe, 1995), some research works have had interests in combination of force and tactile feedback. Akamatsu and MacKenzie (1996) suggested a computer mouse with tactile and force feedback increased usability. However, the work dealt with haptic effects rather than precisely controlled force and tactile stimuli. Kammermeier et al. (2004) combined a tactile actuator array providing spatially distributed tactile shape display on a single fingertip with a single-fingered kinesthetic display and verified its usability. However, the size of the tactile display was not small enough to practically use the suggested mechanism. As more practical design, Okamura and her colleagues design a 2D tactile slip display and installed it into the handle of a force feedback device (Webster et al., 2005). Recently, in order to provide texture sensation with precisely controlled force feedback, a mouse fixed on 2DOF mechanism was suggested (Kyung et al., 2006b). A small pin-array tactile display was embedded into a mouse body and it realized texture display with force feedback. More recently, Allerkamp et al. (2007) developed a compact pin-array and they tried to realize the combination of force feedback and tactile display based on the display and vibrations. However, in previous works, the tactile display itself is quite small but its power controller is too big to be used practically.

This chapter focuses on design and evaluation of two tactile displays developed by authors. The tactile displays are based on miniaturized parallel arrangement of actuators. In the section 3, 5x6 pin array based on piezoelectric bimorphs are introduced. The performance of tactile display has been verified by pattern display and the tactile unit is installed in a conventional mouse to provide tactile feedback while using the mouse. In the section 4, a compact tactile display with 3x3 pin array is described. The tactile display unit is embedded into a stylus-like body and the performance of the haptic stylus is introduced.

3. Texture display mouse

3.1. Planar distributed tactile display

Fig. 1 shows the side view of the tactile display assembly (Kyung et al. 2006a). Each step of the stair-like bimorph support holds six bimorphs arranged in two rows. The lower and upper rows are laterally offset by 1.8 mm. Each step is longitudinally offset 1.8mm from the next. 10 tiers of 3 piezoelectric bimorphs are interwoven to address 5 rows and 6 columns of pins (tactors) on 1.8 mm centers. The maximum deflection is greater than 700μm and the bandwidth is about 350Hz. The blocking force is 0.06N. The specifications of the tactile stimulator with piezoelectric bimorphs were verified to ensure that it deforms the user’s skin within 32dBSL (sensation level in decibels above threshold). Each bimorph is 35 mm × 2.5 mm with a thickness of 0.6 mm. The size of the cover case is 40 mm × 20 mm × 23 mm. Efforts to minimize the weight of the materials and wiring produced a finished design with a weight of only ~11 grams. The contact area is 9.7mm×7.9mm—a previous study showed this area is sufficient to discern difference in textures.

Figure 1.

Profile of the tactile display.

Fig. 2 shows the contact interface of our tactile display. The frame is 40mm × 20mm × 23mm. The 30 stacked actuators are piezoelectric bimorphs driven by 150 VDC bias. Since the tactile display unit, which is described in Section 3.1, is small enough to be embedded into a computer mouse, we developed a new texture display mouse that has a tactile display function as well as all functions of a conventional mouse. Fig. 3 shows a prototype of the tactile display mouse. The pin array part of the tactile display is located between two click buttons of the mouse and it does not provide any interference during mouse movement (Kyung et al., 2007).

Figure 2.

The texture display unit.

Figure 3.

A prototype of the texture display mouse.

3.2. Static pattern display

In order to use the proposed haptic mouse as a computer interface, the system should provide some kinds of symbols, icons, or texts in a haptic manner. Therefore, in this set of experiments, the performance of the tactile display was evaluated by asking subjects to discriminate between plain and textured polygons, round figures, and gratings. In these experiments, the actuator voltages were adjusted to set the desired shape, which was then held constant. Subjects were allowed to actively stroke the tactile array with their finger pad. Thus, the experiments were conducted under the condition of active touch with static display.

Figure 4.

Planar polygon samples.

Figure 5.

Rounded shaped samples.

Figure 6.

Grating samples.

Experiment I. Polygon discernment: In the first experiment, subjects were asked to ascertain the performance of a tactile display that presented 6 polygons created by the static normal deflections of the pin array. Fig. 4 shows the 6 test samples consisting of blank and filled polygonal outlines. After the presentation of the stimulus, subjects were free to explore it with their finger and were required to make a determination within ten seconds. Each sample was displayed 5 times randomly. Twenty-two naïve subjects (13 men and 9 women), all in their twenties, performed the task (Table 1). The proportion of correct answers (90-99%, depending on the stimulus) far exceeded chance (10%), indicating that the display provides a satisfactory representation of polygons, and that fine features such as fill type and polygon orientation are readily perceived.

Experiment II. Rounded shapes: The purpose of this experiment was to verify that the system could simulate the differences between shapes that were similar and those that had identical boundaries. Four round shapes with distinctive features were presented to the same subjects who participated in Experiment 1. The other conditions, such as response time and active touch, were the same. Three of the samples in this experiment (Fig. 5, the three leftmost shapes) were simple planar outlines. The fourth was a three dimensional half ellipsoid. It is reasonable to suppose that the conspicuous difference of the fourth sample caused the 100% correct answer rate (Table 1). Results for the other shapes are comparable to those found in the polygon discrimination task, indicating that the display does a satisfactory job of rendering round shapes.

Experiment III. Gratings: The same experiment as above was performed using the four grating samples shown in Fig. 6. The interval between each convex line was 3.6 mm. The purpose of this experiment was to verify that the developed system can present gratings and their directions. Table 1 shows the proportion of correct answers for the different gratings.

Sample No.		1	2	3	4	5	6
Percentage of Correct Answers	Experiment I	90.8	98.7	93.3	93.2	97.3	95.9
Experiment II	97.3	100	91.5	100
Experiment III	93.3	95.9	100	95.9

Table 1.

Experimental results.

3.3. Vibrotactile pattern display

In this section, we investigate how vibrotaction, particularly at low frequencies with identical thresholds, affects the identification of forms in which only passive touch, and not rubbing, is used. Craig (2002) has already compared the sensitivity of the scanned mode and static mode in discerning tactile patterns, but here we compare the sensitivity of the static mode and synchronized vibrating mode. In these experiments, subjects were not allowed to rub the surface of the tactile display. In order to set the other conditions identical to those in the experiment of section 3.2, except for the vibrotaction, the same texture groups used in section 3.2 were deployed with three different low frequencies: static, 1Hz, and 3Hz. The frequencies were selected based on identical sensation levels, since the magnitudes of the threshold value in the frequency band of 0~5Hz are almost the same.

Table 2 shows that the proportion of correct answers generally increases as the frequency rises from static to 1 Hz to 3Hz. The proportion of correct answers is similar for stimuli presented at 3 Hz and for active touching (Table 2). This suggests that passive touch with low frequency vibration may be a viable alternative to active touch. From a psychophysical and physiological point of view, it seems likely that a 3Hz vibration can effectively stimulate the Merkel cells and that the associated SA I afferent provides the fine spatial resolution necessary for the subject to make the required discriminations. From these results, we expect that the haptic mouse is capable of displaying virtual patterns and characters in real time while the user simply grasps and translates the mouse while exploring the virtual environment.

Sample No.			1	2	3	4	5	6
Percentage of Correct Answers	Polygonal Samples	0Hz	51.4	72.9	55.7	82.9	60.0	45.7
1 Hz	55.4	90.8	67.1	94.7	90.5	94.7
3 Hz	70.7	90.5	81.3	86.5	86.8	93.3
Rounded Samples	0Hz	71.4	72.9	73.2	100
1 Hz	89.2	73.0	63.3	94.7
3 Hz	81.6	80.3	88.5	94.7
Grated Samples	0Hz	56.6	74.3	66.7	59.2
1 Hz	93.3	90.8	81.3	81.6
3 Hz	83.8	93.2	94.7	85.9

Table 2.

Experimental results.

4. Tactile feedback stylus

4.1. Compact tactile display module

This section describes another type of tactile display composed of 3x3 pin array for embedding into a portable device. In order to make a tactile display module, actuator selection is the first and dominant step. The actuator should be small, light, safe, silent, fast, powerful, consume modest amounts of power and emit little heat. Recently, we developed a small tactile display using a small ultrasonic linear motor. We here briefly describe its operation principle and mechanism.

Figure 7.

Operation principle of an actuator.

The basic structure and driving principle of the actuator are described in Fig. 7. The actuator is composed of a transducer, shaft and a moving element. The transducer is composed of two piezoelectric ceramic disks and elastic material membranes. The convex motion of the membranes causes lift in the shaft of the motor. The fast restoring concave motion overcomes the static frictional force between the moving element and the shaft and makes the moving element maintain its position. The displacement ‘A’ of one cycle is sub-micrometer scale and rapid vibration of the membrane at a frequency of 45 kHz (ultrasonic range) causes rapid movement of the moving element. The diameter of the transducer is 4mm and its thickness is 0.5mm. The thrusting force of the actuator is greater than 0.2N and the maximum speed of the moving element is around 30mm/sec. In order to minimize the size of the tactile display module, the actuators were arranged as shown in Fig. 8. Essentially, this figure shows the arrangement of two variations on the actuators - each with different shaft lengths. This design minimizes the gap between actuators. Another feature is that the elements previously described as "moving" are now stationary and fixed together, causing the shafts to become the elements which move when the actuators are turned on. This minimizes the size of the contact point with a user’s skin (to the 1mm diameter of the shaft), while maintaining the mechanical simplicity of the system. Fig. 9 shows the implemented tactile display.

Figure 8.

Design drawing of a tactile display module.

Figure 9.

Implemented tactile display.

From the design specification described above, the prototype of the tactile display module has been implemented as shown in Fig. 9. In order to embed the module in a pen, we constructed only a 3x3 pin array. However, it should be noted that the basic design concept is fully extensible; additional columns and rows can be added without electrical interference or changes in pin density. The shaft itself plays the role of tactor and has a travel of 1mm. The distance between two tactors is 3.0mm. Since the actuators operate in the ultrasonic range, they produce little audible noise. The average thrusting force of each actuator exceeds 0.2N, sufficient to deform the skin with an indentation of 1 mm. The total size of the module is 12x12x12 mm and its weight is 2.5grams. Since the maximum speed of a pin is around 30mm/sec the bandwidth of the tactile display is approximately 20Hz when used with a maximum normal displacement of 1mm. If the normal displacement is lower than 1mm, the bandwidth could be increased.

Figure 10.

The prototype of the Ubi-Pen.

4.2. Implementation of pen-like tactile display

The pen is a familiar device and interface. Since they are small, portable and easy to handle, styli have become common tools for interacting with mobile communication devices. In order to support richer stylus based tactile cues, we embedded our tactile display module into a pen-like prototype. In addition, as shown in Fig. 10, we installed a pancake-type(coin-type) vibrating motor in the tip of the pen to provide a sense of contact (Kyung & Lee, 2008). The housing of the pen was manufactured by rapid prototyping, and it has a length of 12cm and a weight of 15 grams. Currently, its controller is not embedded. We named this device the Ubi-Pen and intend it for use as an interface to VR, for the blind, to represent textures, and as a symbolic secure communication device. We also suggest it could be used generally as the stylus of a mobile communication device.

4.3. Pattern display of the tactile display module

A common method to evaluate the performance of tactile displays is to test user’s performance at recognizing specific patterns. We use Braille as a stimulus set to conduct such a test. Specifically, we conducted a study involving the presentation of the Braille numbers 0~9 on the Ubi-Pen.

Figure 11.

Braille Patterns for the Experiment.

Fig. 11 shows the experimental Braille patterns. Subjects were required to hold the pen such that the tip of their index finger rested over the pin-array part of tactile display module. In this experiment, the Braille display test bas been conducted for the normal and the blind. After setup stage, we conducted a study on recognition rate of the 10 numeric digits in the Braille character set. As these can be displayed on only four pins, we mapped them to the corner pins on our tactile display module. We chose to do this as our user-base was composed of sighted Braille novices. We used three different stimulation frequencies: 0, 2 and 5Hz. (Pins move up and maintain static position at the 0Hz). Pins movement was synchronized. We presented 60 trials in total, each number at each frequency, twice. All presentations were in a random order, and subjects were not advised about the correctness of their responses. 10 subjects participated in the experiment. The Braille stimuli were generated continuously and changed as soon as the subject respond using the graphic user interface. There were 2 minutes breaks after every 20 trials.

Two blind people have participated in the same experiment and the visual guidance in the experiment has been replaced by the speech guidance of experimenter. For all stimuli, they responded exactly and quickly. The Braille expert usually read more than 100 characters, and the blind subjects respond they don’t feel any difficulties to read the Braille numbers. Since the duration of each trial was shorter than 1~2 seconds and they answer in the form of speech, we could not measure the duration exactly. Moreover, 4 neighborhood pins have been presented again with identical procedure for the blind people. And they responded more quickly since the gap of pins was more familiar with them. Duration of each trial was always shorter than 1 second.

	Normal subjects	Blind Subjects
Average Percentage of Correct Answers	80.83	100
Average Duration of Each Trial (sec)	5.24	1~2

Table 3.

Experimental Results.

Table 3 shows the summary of experimental results. Although normal subjects were novice in using the tactile display, the average percentage of correct answers exceeded 80 percent. The confusions come from the relatively low tactile sensitivity of the novices compared with the sensitivity of the blind. Since the various analysis of the tactile display for the blind is another interesting topic, this will be investigated in our future work

4.4. Image display on touch screen

The Ubi-pen mouse enables tactile pattern display. This program provides a symbolic pointer in the shape of a square, with a size of 15x15 pixels. A user can load any grayscale image. As shown in Fig. 12, when the user touches an image on the touch screen with the Ubi-Pen, the area of the cursor is divided into 9(=3x3) sub-cells and the average gray value of each cell is calculated. Then, this averaged gray value is converted to the intensity of the stimuli displayed on each pin of the tactile display. Figure 13 shows the methodology of the pattern display.

In order to verify texture display performance of the Ubi-Pen, 3 kinds of texture sample groups have been chosen. As described above, every sample is gray images. And we prepared three image groups classified by their feature characteristics. This experiment is to test user’s performance at recognizing specific patterns. One of five images in a group is displayed on the screen, but a participant is not able to see the image. He/she sees only a blank square covering the image. The size of the box is same as the image’s one and the actual gray values of the image is obtained although the users rubs the blank square. While the user contacts a touch screen, he/she is required discriminating surfaces from scratch-like feeling. The experimental results show in Table 4 and the data verify that the Ubi-Pen and image display scheme works well.

Fig. 12.(a) shows 5 image samples of group I, those are characterized by directions of gratings. The size of each image is 300x270 pixels. The percentage of correct answers in Table 4 clearly shows that the pen type tactile display works very well in discriminating gratings. Average duration of a trial is about only 10 seconds. Fig. 12.(b) shows 5 image samples of group II, those are characterized by groove width. A user feels horizontal gratings during rubbing surfaces, in this experiment however, he/she should detect the variation of gap distance. In order to discriminate these patterns, the stimuli in accordance with movement on the plane should be detected. As shown in Table 4, sample 1, 2 and 4 are easily recognized, and the results for sample 3 and 5 are also acceptable. Users feel a bit more difficult than group I, but the performance of the device is still acceptable. Figure 12.(c) shows 5 image samples of group III, those are characterized by shapes. Since average percentage of correct answers in this group is 77.5, we can accept that we can recognize various patterns by rubbing surface using the proposed device. However, as shown in Table 4, participants have been a bit confused among the image samples except sample 5 whose geometric connection is different. And it takes twice time to give an answer compared to group I. In case of complex pattern, it is reasonable that it takes a long time and error increases. However, improvement of the device is necessary since device itself can cause confusion such as low reality, inconveniency or low density.

	Percentage of Correct Answers					Duration of a Trial (sec)
S1	S2	S3	S4	S5	Ave.	Std.
Group1	97.5	92.5	85.0	95.0	92.5	10.7	2.9
Group2	92.5	100	77.5	97.5	75	13.4	4.0
Group3	62.5	77.5	80.0	72.5	95.0	20.6	10.7

Table 4.

Experimental Results.

Figure 12.

Braille Patterns for the Experiment.

Figure 13.

Methodology of pattern display.

5. Summary

This chapter deals with tactile displays and their mechanisms. We briefly reviewed research history of mechanical type tactile displays and their parallel arrangement. And this chapter mainly describes two systems including tactile displays.

The 5x6 pin arrayed tactile display with parallel arrangement of piezoelectric bimorphs has been described in the section 3. The tactile display has been embedded into a mouse device and the performance of the device has been verified from pattern display experiment.

Another focus of this chapter is describing a compact tactile display module and verifying its performance in a pen-like form factor. As described in section 4, a small, safe, low power consuming, silent and light tactile display module with parallel and woven arrangement of ultrasonic linear motors has been built. Using the tactile display, we propose the Ubi-Pen which can provide texture and vibration stimuli. This system shows satisfactory preliminary performance in representing tactile patterns. We also evaluate its capacity to support GUI operations by providing scratching sensation when a user rubs surface displayed on a touch screen.

There have been various trials to develop tactile displays for simulating surface gratings, patterns, roughness and etc. However, so far, the best candidate in designing tactile display has been a pin-array. In order to provide enough indenting stimulation in a pin-array, parallel arrangement of linear mechanism has been necessarily required. In the future, invention of new materials will suggest compacter and more effective design. In this chapter, we have focused on two technologies suggesting examples of miniaturized design concepts of tactile displays adopting parallel mechanisms.