Pen-type Sensor for Surface Texture Perception

1. Introduction

The measurements of surface properties by contact sensing have been investigated for a long time. Recent advances in tactile related applications that previously rely only on visual feedback, e.g. telepresence, interactions with objects in virtual environments, and minimally invasive surgery, have raised the requirement for the feedback of haptic information to get realistic sensations of direct contact. Surface texture is one of the most important surface properties that affect the feeling of touch. However, it is difficult to describe and measure the property of tactile texture of a surface. Unlike the measurements of texture to characterize the mechanical performance of a surface, research of measuring tactile texture is still in its initial stage, and most efforts have been spent in the developments of texture sensors.

The common methods of measurements of surface properties are based on the measurements of contact forces/pressure. For example, geometric parameters of surfaces can be estimated based on the stress map of the contact area which is measured by arrays of force sensing units. Sophisticated force-based applications can measure the contact locations, surface curvatures, edges and shapes of objects (Fearing & Binford, 1991; Heidemann & Schopfer, 2004; Murakami & Hasegawa, 2005). By measuring the dynamics of contact forces in dexterous manipulations, the incipient slip between the object and robot hands can also be detected (Tremblay & Cutkosky, 1993). However, in contrast to force-based measurements of geometry, the goal of tactile sensing is to obtain local contact parameters, such as surface roughness/ texture, the hardness/softness of object, heat transfer properties, frictional properties, the material of the object, etc.

The development of force-based texture sensors for the perception of texture is difficult because the required modalities for the characterization and measurement of surface texture are still ambiguous. Psychophysical researches showed that contact vibrations provide the most useful information for the perceptions of surface fine textures with inter-element spacing less than about 1 mm (Johnson & Hsiao, 1994; Hollins et al., 1998). Therefore, there are many sensors imitate the structures of the human finger by designing components of nails, bones, ridges on the sensor surfaces, multilayered sensing skin, and use different transducers embedded in the sensors to measure the stimuli of contact vibrations (Mayol-Cuevas et al., 1998; Yamada et al., 2001; Tada et al., 2003).

Based on the responses of haptic explorations, several sensors showed the ability of discriminations of different types of fine-textured surfaces. Mayol-Cuevas et al. developed a fingertip-like sensor using an electret piezoelectric microphone embedded in a rugged material as the transducer (Mayol-Cuevas et al., 1998). Contact sounds generated during sliding motions against a surface were picked up by a microphone and analyzed with FFT and learning vector quantization technique (LVQ) for texture recognition. Baglio et al. presented a tactile sensor that used bimorph piezo-ceramic actuators and sensors for stimulation and sensing of response signals (Baglio et al., 2002). By combining signal power spectral density analysis with fuzzy recognition, this system can recognize different types of materials. Fend et al. developed an active multi-whisker array modeled on the rat whisker system (Fend et al., 2003). This whisker array can discriminate different textures based on the frequency response elicited by the whiskers. Mukaibo et al. developed a finger-like multilayered texture sensor (Mukaibo et al., 2005). The sensor is able to identify the differences in roughness, softness and frictional properties of different materials and quantitatively detect the texture information of a surface. Hosoda et al. developed a soft fingertip with randomly distributed strain gauges and PVDF films (Hosoda et al., 2006). With force signals from strain gauges and the variances of the signals from the PVDF films produced by pushing and rubbing movements, this anthropomorphic fingertip can discriminate five different types of materials. In our previous research (Ye et al., 2006), a texture sensor with embedded PVDF films was developed, which can discriminate the fine textures on different types of sandpapers based on the patterns of responses in the frequency domain.

Vibration-based texture sensors have showed the ability to perceive and discriminate textures. Meaningful results having been obtained individually; however, the comparison of experimental results in terms of precision, efficiency, or even the validity, seems to be difficult. Based on previous results, it would be also difficult to define any essential requirement for texture sensing. Therefore, fundamental researches are required to understand the mechanisms of fine texture perception to which the texture perceptions to which the developments of texture sensors can be referred.

In this report we propose a sensor for the perception of surface roughness. Rather than targeting to the discrimination of perceived roughness from the point of view of sensation, efforts are made for the measurements of surface roughness profiles, with which the analysis and reconstruction of surface textures can be performed. The proposed sensor has a rigid contact probe and is able to measure the 3D contact forces applied to the probe’s tip and the dynamic contact signal along the probe’s axis direction. It has been used in our previous research (Ye et al., 2007). It can be discriminated from the hand-held device developed by Pai and Rizun which measures 3D accelerations and 1D contact force in the normal direction (Pai et al., 2003). The measurement of 3D contact forces offer more useful information for estimations of surface profiles and perceptions of textures.

2. Requirements for texture sensing

Because psychophysical and neurophysiological studies have showed that contact vibrations are necessary and sufficient for the perceptions of fine textures by human fingers, the dynamic sensing has been adopted for texture perceptions and techniques for analysis of time-varying signals have also been applied. Tactile texture sensors are developed by emulating the structure of the human finger; sensing processes also start with contact explorations on objects. However, the dynamic response signals are affected by the details of sensor design and the exploratory parameters (contact force, speed, etc.) as well as the surface texture itself, making it difficult to quantitatively describe the obtained texture. On the other hand, recent psychophysical studies with metal gratings showed that the groove widths between ridges have the strongest effect on the estimation of perceived roughness, and the estimated magnitudes of roughness can be described accurately as a function of groove widths and ridge widths (Yoshioka et al., 2001). This finding suggests that the perceived texture can be represented more accurately by parameters of surface profile instead of the frequency components of contact response signals.

One of the difficulties in texture sensing comes from the unquantifiable characteristics of tactile sensors. Most of tactile sensors are composed of a soft layer between the textured surface and the transducers. This soft layer is subject to all kinds of contact texture stimuli in explorations. For example, there are tangential forces and normal height variations in the contact area in the case of surface height detection. When the sensor slides onto a bump, tangential and normal stimuli are both significant. However, it is easy to draw a conclusion that the sensor characteristic is inhomogeneous if only the normal stimuli are taken into consideration. The existence of such intermediate layer makes it difficult to interpret sensor outputs with respect to surface textures. It can simplify the system design and the development of algorithm for analysis if the intermediate uncertainties of measurements can be minimized.

Another factor that makes texture sensing difficult is that the sizes of texture elements are generally smaller than the size of the contact area. Measuring a fine-textured surface with a comparatively large-sized transducer will filter out the details of texture, as illustrated in Figure 1. The existence of an intermediate soft layer worsens this problem as the contact stimuli propagate within the layer. For the perception of fine texture, the developments of tactile sensors have to reduce the size of contact interface during explorations to preserve surface details.

Figure 1.

The effect of large contact area on tactile sensing. The local texture details are filtered out because of the relatively large contact area.

In this report, we present a handheld pen-type texture sensor (see Figure 2) that can satisfy the requirements described above. By using a rigid contact probe for texture sensing, the contact stimuli (static and dynamic forces) are measured with minimum distortions. The contact between the probe’s hemispherical tip and the surface can be considered as a single point contact. The profile of surface in the path of exploration can be estimated based on the measurements of contact forces and the motion of the sensor.

3. Development of pen-type texture sensor

This section describes the development of a miniaturized handheld pen-type texture sensor for the perception of surface texture. The proposed pen-type sensor is capable of measuring

Figure 2.

The proposed handheld texture sensor for texture sensing.

3D contact forces applied at the tip and the dynamic force in the axial direction. It consists of one aluminium probe with a hemispherical tip and three kinds of sensing elements: four strain gauges, a force sensor, and a PVDF module. The structure of the sensor is shown in Figure 3(a). Two half-bridge strain gauge circuits are used to measure the tangential forces in x- and y- directions, respectively. The rigid probe works as a cantilever in the measurements of tangential forces. The applied axial force component is measured by a pair of strain gauges which are connected to a half-bridge circuit and attached close to the supported end at opposite sides of the probe. The normal contact force in z-direction is measured by a force sensor (LPM 562 500G by Cooper Instruments & Systems) installed coaxially at the end of the probe. The force sensor also functions as an axial bearing and prevents the axial movement of the probe under the load of normal contact force.

A PVDF module (H×W×L=2.5×12×5 mm) is installed between the force sensor and the contact probe to measure the dynamic of force in z-direction. There is one thin PVDF film (H×W×L=0.1×1×10 mm) embedded in the center of a silicon layer. While the silicon layer delivers the contact force from the probe to the force sensor, the PVDF film embedded inside measures the change rate of force in its thickness direction (d₃₃). As a transducer, the PVDF film has very wide frequency range and dynamic range. Combined with the force sensor, they function similarly to the different types of receptors distributed in the fingertip skin. As a ferroelectric polymer, the PVDF film exhibits piezoelectric and pyroelectric properties. To minimize the pyroelectric effect, the temperature of the PVDF module is maintained approximately at the room temperature.

As a compact handheld device, the circuit (see Figure 3(c)) is also included in the pen-type sensor. Common circuit modules are used in the circuit design, as shown in Figure 3(b). A differential amplifier is used for the amplification of the output signals of the force sensor. Two half-bridge circuits are used to readout the signals from the four strain gauges, followed by proper amplifications.

Figure 3.

The mechanical structure of pen-type texture sensor and the layout of transducers.

A charge amplifier and a second amplifier are used to read the signal of the PVDF module. These four signal channels are digitalized using the built-in A/D converter of the micro-controller and transmitted to a host computer via a Bluetooth wireless communication link. A data processing program written with MATLAB is running on the host computer to perform further signal analysis.

One more problem that should be considered is how to support the contact probe. Because the load on the probe is small, the simple and widely used bush bearing was firstly adopted. However, the outputs of the force sensor were distorted and the stimuli to the PVDF film were strongly suppressed. This is due to the static contact friction of the bush bearing. When tangential contact forces are applied at the tip of the probe, a torque about the point of bearing is introduced. To counterbalance this torque, a pair of counteractive forces is generated at the both ends of the bush. Although the sliding frictional coefficient of metal is small, such counteracting forces and the consequent sticking effects are severe enough to filter out the dynamic force.

Although the rigidity of the sensor in the probe’s axial direction is large and there is no relative motion between the contact probe and the base of bearing, frictionless bearing is required in the design. Therefore, the flexure bearing is built by using a pair of disk springs for the support of the sensing probe, as shown in Figure 4. The disk-shaped springs is made by carving involute-shaped grooves through thin copper disks. For the radial support, the equivalent model of the spring is a simple beam with force and torque loads at its ends. The probe can move freely in the disk’s axial direction but be restricted in the radial direction. A pair of separated disk springs is used to counterbalance the torque introduced by the tangential contact forces. When the probe moves from its equilibrium position, a spring force proportional to the displacement is generated. This force can be calculated according to Hooke’s law. Because the contact probe does not move during tactile sensing, the probe can be set to stay at its equilibrium position, and no static spring force is applied to the force sensor. However, a small preloaded spring force is intentionally applied to keep a tight contact in the axial direction between the probe, the PVDF module, and the force sensor. One disadvantage of the disk spring mechanism is its low radial stiffness, which results in the coupling effect between the measured tangential forces and the normal contact force, and therefore, a decoupling calculation is required.

Figure 4.

The disk spring mechanism for the supporting of the contact probe. (a) The structure of the disk spring. The probe can move (b) forward and (c) backward in the probe’s axis direction.

Unlike the sensor used in our previous research (Ye et al., 2006), there is no foam layer in this design, and therefore, the contact probe exhibits higher rigidity in the normal direction. A wider signal bandwidth is expected and a higher sampling rate of A/D conversion is required. The micro-controller is able to perform A/D conversions up to 25 KHz for each of the four signal channels. However, a 3 KHz sampling rate limitation is introduced by the relatively low bitrate of the wireless communication. Explorations with reduced speeds can alleviate the effect of sampling rate limitation.

4. Experimental evaluations

This section presents the evaluation of the proposed texture sensor. For all experiments, the sensor is held by hand and perpendicular to the target surface. The contact point locates at the hemispheric end of the aluminum contact probe. Four kinds of handy materials are used for experiments: the hard surface of a desk, a mouse pad, a transparent duct tape with smooth surface and a towel. Contact forces are controlled approximately at a constant value in the same exploration. Sliding speeds are between 40 ~60mm/s.

4.1. Measurement of 3D contact force

The purpose of 3D contact force measurements is to evaluate the Coulomb frictional property of the surface which is calculated based on the normal forces and tangential forces. The contact force can be measured with a 3DOF force sensor, and there are many commercial force/torque sensors can be used for this purpose. However, for a small hand held device, the size constraint is critical. It is difficult to find a cheap force sensor that can satisfy all of these requirements.

The proposed texture sensor provides good results of 3D contact force measurements while maintaining a relatively small size. According to the beam theory, there are transverse coupling effects between the forces in x-, y- and z-directions. These coupling effects mean that the measured values of f_x, f_y and f_z are affected each other by the Poisson’s ration of aluminium.

Ideally, there is no coupling between the normal force and the tangential forces. However, the experiments showed that such coupling effect exists and becomes stronger with the increase of the applied tangential forces. This coupling effect can be explained as follows. When the torque introduced by tangential forces is totally counteracted by the pair of disk springs, the force sensor is only sensitive to the normal force f_z. In this case, however, the radial bearing rigidity provided by the disk springs is not sufficiently strong. The contact probe rotates in a small scale under the combination of bearing forces and tangential forces. Because the assembly of the sensor is tight in the axial direction, the rotations generate small displacement with respect to the force sensor, and therefore, additional axial contact force is introduced.

A correction for decoupling is required to obtain the precise values of applied contact forces. Assuming that the coupling factors are linear and can be represented by a constant value, every factor is determined by a calibration. Firstly, the amplifiers for every force components are tuned to unify three measurement channels with the 2 N contact force corresponding to the maximum voltage outputs of amplifiers. After the gain unification, forces in the full range are applied in one axial direction; the outputs of other two force components are recoded simultaneously. The relation between the coupled outputs and the applied forces is approximately linear. The coupling factors from the given component to the others are determined by the ratio of corresponding coupled outputs to the applied force. With all determined coupling factors, the contact forces are calculated as follows.

$$\left[\begin{array}{c}{F}_X\\ F_Y\\ F_Z\end{array}\right]=\left[\begin{array}{ccc}1& -0.09& -0.2\\ -0.10& 1& -0.25\\ -0.15& -0.18& 1\end{array}\right]\left[\begin{array}{c}{f}_x\\ f_y\\ f_z\end{array}\right],$$E1

where F_x, F_y and F_z are the applied forces, f_x, f_y and f_z are outputs of amplifiers. It is noticed that the decoupling matrix is not symmetric because of the sensitivities of strain gauges, which are manually attached, are not identical.

4.2. Measurement of frictional coefficient

With the normal and tangential force measurements, the Coulomb frictional coefficient can be evaluated using the equation μ = f /N, where the normal contact force N = F_z, and the friction is the tangential force$$f=\sqrt{F_x^2+}{F}_y^2$$.

A mouse pad and a towel were used in the experiments. Three different normal contact forces were applied for explorations on each surface. The experiments results are shown in Figs. 5(c) and 5(d). Each column in the figure is the result for one kind of normal contact force. The first row is the tangential force (frictional force) f; the second row is the normal force applied N; and the third row is the frictional coefficient calculated with equation μ = f /N.

The results showed that the calculated frictional coefficient changes dynamically. This is due to the changing of applied normal forces and the uneven surface textures. Especially in the responses of scanning on the towel, an approximately constant frequency response is observed with increased normal contact forces. This is due to the regular textile texture of the towel which can be treated as a coarse texture. The calculated coefficients are similar under different normal contact forces. Furthermore, the difference between the coefficients of friction of these two materials is small, which is consistent with the feeling of exploring on both surfaces with bare fingers.

4.3. Dynamic sensing with PVDF film

The PVDF module is located between the contact probe and force sensor. Normal contact is transmitted to the force sensor via the silicon layer while the dynamic signals are picked up by the PVDF film. In this tactile sensor configuration, PVDF film is adopted because of its high sensitivity and wide bandwidth response to dynamic stimuli.

One of the known effects of measuring contact dynamic via a contact probe is the consequent low sensitivity of the system. The gains of amplifiers need to be increased several hundreds times higher than those used in our previous research (Ye et al., 2006) to generate equal amplitudes of outputs from similar inputs. This is due to two factors. Firstly, the reduced contact area indicates the reduction of the input intensity; and secondly, the use of the contact probe and the bearing disk springs attenuates the tactile stimuli, because they act as a spring-mass system with high spring constant.

4.3.1. Dynamic response of the PVDF film

To find out the respond characteristics of PVDF film, two kinds of primitive inputs were used: step change of the normal contact force F_z and tapping inputs. The results are shown in Figure 6. The PVDF film is sensitive to stress rate, its output magnitude is proportional to the changing rates of F_z, which is qualitatively verified from the results.

Figure 5.

The measurements of coefficient of frictional of a mouse pad and a towel with different applied normal contact forces.

Figure 6.

The dynamic output of PVDF film vs. force sensor. (a) Response to step input of normal contact force. (b) Response to tapping against a mouse pad. (c) Response to tapping against a hard desktop. (d) Response of sliding over three bands of duct tape.

From the response of desk tapping (see Figure 6(b) and 6(c)), both the force sensor and the PVDF film generate a high peak output at each moment of contact. For the PVDF film, the most important signal is the peak response, so it should be detected by the micro-controller. However, as mentioned in the section of sensor development, limited by the architecture of the micro-controller, A/D conversion for each channel is performed one after another at a limited sampling rate, which means that there is a conversion time delay for each signal channel.

4.3.2. Sliding response of the PVDF film

Figure 6(d) shows the response of sliding over of three parallel bands of duct tape stuck on the desktop with intervals equal to the tape’s width. Although the desktop surface is felt smooth to the human fingers, the pen-type sensor picks up the high frequency response signals because of the high rigidities of the desktop and the high axial stiffness of the sensor. Therefore, even small height variations at the contact area can trigger strong dynamic tactile stimuli.

When the sensor is sliding on the smooth tape, its outputs indicate the change rate of the normal force and the white noise of circuit. Therefore, the outputs in this case can be considered the response of the sensor to the explorations of surface with “zero texture.” When sliding on and off the edges of duct tape, the PVDF module generates much stronger responses than the force sensor, which can be used for edge detections.

5. Conclusions

In this report, a compact handheld pen-type texture sensor for the measurement of fine texture was presented. Because surface roughness and friction properties are most critical parameters in tactile texture sensing, the proposed texture sensor was designed with a metal contact probe and was able to measure the roughness and frictional properties of a surface. Using a rigid contact probe for contact explorations, the sensor can reduce the size of contact area and separate the normal stimuli from tangential ones, which facilitates the interpretation of the relation between dynamic responses and the surface texture.

As for the measured 3D contact forces, they can be used to estimate the surface profile in the path of exploration. Based in the profiles of surface, sophisticated algorithm can be applied for the analysis and restriction of textured surface. In our latest research, the proposed pen-type texture sensor was applied in the reconstruction of periodic texture with limited number of scans on the surface. The dynamic force sensing can be used for the estimation of surface texture in various ways. For example, we can use the rate of strike to represent the rate of appearance of micro peaks of surface texture that come in contact with the sensor’s probe. When the amplitudes of the response signal exceed a given threshold, contact strikes are considered happened and a surface with higher rate of contact strikes during explorations can be considered as a rougher surface.