Shape Recognition and Position Measurement of an Object Using an Ultrasonic Sensor Array

Shape recognition of a transparent object is usually difficult to perform by image processing techniques, because the major portion of the light projected onto an object passes through the object. As a result, the object cannot gain the light intensity required for image processing. Ultrasonic sensors are often utilised in situations where such optical sensors cannot be used. Moreover, systems using ultrasonic sensors are simpler and cheaper than systems using other types of sensors. Although the ultrasonic method has such advantages, results from the conventional ultrasonic method do not always have high measuring resolution, due to the wide directional pattern of the sensing. Up to now, the application of the ultrasonic method has been limited compared to the optical method.


Introduction
Shape recognition of a transparent object is usually difficult to perform by image processing techniques, because the major portion of the light projected onto an object passes through the object.As a result, the object cannot gain the light intensity required for image processing.Ultrasonic sensors are often utilised in situations where such optical sensors cannot be used.Moreover, systems using ultrasonic sensors are simpler and cheaper than systems using other types of sensors.Although the ultrasonic method has such advantages, results from the conventional ultrasonic method do not always have high measuring resolution, due to the wide directional pattern of the sensing.Up to now, the application of the ultrasonic method has been limited compared to the optical method.
In an ultrasonic recognition system, ultrasonic sensors are combined with neural networks (Yoneyama et al., 1988), (Farhat, 1989), (Watanabe & Yoneyama, 1990), (Masumoto et al., 1993), (Serrano et al., 1997).In these systems, the capability of the neural networks compensates for the low resolution of the ultrasonic method.However, in the conventional ultrasonic methods using neural networks (Holland, 1992), data used for the neural networks are given by acoustic holography, tomography or time-of-flight measurement.As a result, it is difficult for the conventional ultrasonic methods to improve the low resolution, because shape recognition using these methods has been limited to objects with simple shape.In addition, it has been difficult to measure the position of an object.
The time-of-flight method involves measuring the time arriving at an object by ultrasonic wave pulses or amplitude modulations, in which the distance information obtained is input into a neural network for measurement and recognition, as shown in Fig. 1 (b).Although this method yields high resolution in the depth direction, the width direction resolution is limited by the arrangement interval of the ultrasonic sensor array.As a result, there are measurement and recognition constraints in principle, such as the facts that it is difficult to measure the position in the width direction with high resolution and objects differing slightly in size cannot be differentiated.In contrast, the acoustic holographic method reproduces an image of an object by interfering with the scattering or penetrating waves produced by irradiating ultrasound on a measured object with reference waves, as shown in Fig. 1 (c).This method yields good resolution in the width direction and is superior in two-Sensor Array 54 dimensional measurements and recognition.However, since the distance information in the depth direction in the holographic image that is obtained, it has the drawbacks that a holographic image is needed for each distance if the depth-direction distance to an object varies, and it is difficult to directly detect changes in the pose of a target object.Thus, there are a number of problems that must be resolved to achieve further progress in measurement and recognition systems based on these methods and neural networks.The authors have studied an ultrasonic object position and shape recognition system that can improve the distance resolution in both depth and width directions simultaneously by directly using sound pressure signals of an ultrasonic sensor array based on viewpoints different from those of the past methods (Ohtani, 2002), (Ohtani & Baba, 2007).From the directional pattern of the ultrasonic sensor and the attenuation characteristics of ultrasonic signals, the sound pressure signals of the sensor are generated that contain information required for identifying the position in the form of sound pressure intensity and information required for object shape recognition based on the sound pressure distribution.In this chapter, the construction of a new ultrasonic recognition system for transparent objects with complex shapes is introduced, for which a commercial polyethylene terephthalate bottle (PET bottle) is employed as a model.The sensor system consists of an ultrasonic transmitter, an ultrasonic receiver consisting of an ultrasonic sensor array, and a recognition unit with neural networks.The system locates the object between the ultrasonic transmitter and the ultrasonic receiver.It can identifies the object and measures the position of the object simultaneously.In the following section, the basic principle, system configuration, and experimental results of the ultrasonic identification system is described.
Fig. 2 shows the construction of the proposed system.The system primarily consists of an ultrasonic sensor array unit, a signal processing unit, and an identification unit.In this study, the x coordinate position on the xz plane is defined as the width direction position of the measured object, and the z coordinate position on the x − z plane is defined as the depth direction position of the measured object in the coordinate system as shown in Fig. 2 The slope of the base of the object with respect to the x − y plane is defined as the pose of the object.The ultrasonic sensory array unit has two-dimensional ultrasonic receivers and one transmitter.The transmitter positions the centre of the receiver's array and irradiates an ultrasonic signal to the measured object.The signal processing unit extracts the features of the measured object from the sensor array outputs using the signal processing circuit.The extracted data is inputted into the identification unit.The identification unit consists of two types of neural networks that perform shape identification and material identification, respectively.
Fig. 2. System configuration of the ultrasonic sensor system

Ultrasonic wave propagation characteristics
The reflected ultrasonic pressure from a measured object is required in relation to three elements: the reflection ratio of the measured object, the distance between the object and the sensor array unit, and the directivity of the ultrasonic signal (Kocis and Figura, 1996).First, the reflection ratio relates to the acoustic impedance.The acoustic impedance, z, is defined as the product of the density and the acoustic velocity, c, of the object, as expressed by equation ( 1).When an ultrasonic signal propagated in Medium 1 reaches Medium 2, as shown in Fig. 3, the reflected ratio, r, and the permeation ratio, τ, at the changing point of the mediums is expressed as equations ( 2) and (3).In this way, the reflection ratio depends on the acoustic impedance of the object.The acoustic impedance of a measured object is correlated with the material.Therefore, the magnitude of the reflected ultrasonic pressure gives the information required for material identification.Third, because of the directivity of an ultrasonic wave, the ultrasonic pressure detected by a sensor array has a relation to the shape and the orientation of the measured object.As a result, even if the positions of two objects are the same, but the pose of the objects are different from each other, the reflected ultrasonic pressure from these objects are different.Therefore, in this stage, the proposed method has a little limitation to the pose of measured objects to simplify the identification process.In these circumstances, the reflected ultrasonic pressure is regarded as the key to identifying the shape and position of measured objects.  of the ultrasonic waves.Thus, the positional information of the object is contained in the peak position and the peak value.

Fig. 6. Position measurement principle
In the case of PET bottles, Fig. 7 shows the received ultrasonic wave distribution of the receiver for PET bottle 1, PET bottle 2, and PET bottle 3 among 7 PET bottles, as shown in Fig. 12.These PET bottles have almost the cross section of a square on the bottom side and the cross section of a circle in the top side.In these figures, because the size of the receiver array is 8 x 2, the output distributions are plotted by two kinds of lines.Every peak value of the ultrasonic distribution of the bottom side in these three PET bottles is lower than that of the top side, because the indentation of the surface of the bottom side causes a scattering of ultrasonic waves.In this way, the proposed method identifies the shape of a PET bottle using information of the distorted ultrasonic waves.As the figure shows, the shape of the PET bottle under study can be identified by analyzing the ultrasonic wave distributions.PET bottle 5 is a hexagon, and PET bottle 6 is a cylinder.

Recognition and position measurement by a neural network
Neural networks perform recognition and position measurement of a target object.Fig. 9 shows the neural network used for the experiment, which is a five-layered feed-forward neural network consisting of one input layer, three hidden layers, and one output layer.The input layer is composed of 28 input units.The data inputted to the input units include all outputs of the sensor array of the upper side and bottom side, each of which has 14 ultrasonic sensors.The hidden layer has 3 x 15 hidden units.The output units are given corresponding to the number of objects of shape recognition.In the following experiment, the output layer has 8 units, which includes 7 output units for recognition of the target object and 1 output unit for position measurement.
The teaching pattern data for output units is given as follows.In the teaching mode of the neural network, let the unit for the target object be 1 and the others be 0 for shape recognition units.For the position measurement unit, let the unit be from 0 to 1 according to the normalised range to be measured.Therefore, in the recognition mode and the measurement mode, a recognition result is evaluated using the ratio of the output for the measured object to the amount of all the outputs.Successful recognition is regarded as a ratio of over 50%.

Measured objects
Fig. 12 shows the objects used for the experiment.These are 7 kinds of transparent plastic commercial PET bottles, which are filled with coloured liquid so that their shape can be easily discerned.The PET bottles used in the experiment have the right rectangular prism form (PET bottle 1 -PET bottle 4), hexagonal column form (PET bottle 5 and PET bottle 6), and cylinder form (PET bottle 7).PET bottle 4 is shorter than the other bottles.Although the cross sections of these PET bottles have almost the same shape, the indentation of the surface is somewhat different.PET bottle 5 and PET bottle 6 have the same cross section at the bottom side of the bottle.However, they are different in the cross section at the top side of the bottle, that is, PET bottle 5 is a hexagon and PET bottle 6 is a cylinder.

Experimental procedure
The experimental procedure was as follows: 1.One PET bottle was placed in front of the transmitter and the receiver.The distance between the transmitter (the receiver) and the PET bottle was about 130 mm, as shown in Fig. 13. 2. The object moved in the x-direction in 2 mm increments to acquire the data to be used in the neural networks in the teaching mode.In this way, teaching data were acquired for 21 patterns.3.In this experiment, there are 7 output units of the neural networks for the object identification, because the recognition system needs to identify 7 kinds of PET bottle.If the measured PET bottle is PET bottle 1, the system is taught in such a manner that the output unit 1 is nearly 1, while the outputs of the other units are nearly 0. Similarly, if the measured PET bottle is PET bottle 2, the output unit 2 is nearly 1, and those of the other units are nearly 0. 4. The number of the output unit of the neural network for the position measurement is one, and the neural network outputs the analogue values of the positions of the measured PET bottle in the x-direction or the z-direction.5.These steps were repeated to teach the neural network until the squared error became smaller than 0.00001 for every pattern in the teaching mode.6. (6) Data were acquired for the 20 patterns used in the experimental mode for the PET bottle in question.7. (7) These steps were repeated for all 7 PET bottles.

Results
Table 1 shows the recognition results in the case of free position of an object.The recognition was perfect for all PET bottles, indicating that the method has the ability to recognize a PET bottle.
Table 1.Shape recognition results Fig. 14 shows the linearity of three typical PET bottles in the x-direction and the z-direction, respectively.The actual positions are plotted along the vertical axis, and the values outputted by the neural network are plotted along the vertical axis.The maximum position error in the x-direction was about 3.21 mm and in the z-direction was about 5.46 mm.Table 2 lists the maximum position error of all measured PET bottles in the x-direction and the zdirection.From Table 2, it can be seen that the measurement accuracy of the position in the z-direction became worse compared to the measurement accuracy of the position in the xdirection.The reason is that although the positions in the z-direction were measured based on peak values of the ultrasonic pressure distribution, the change of the peak value was small in the present experiment.The experimental results indicate that the sensor system is effective for industrial applications that identify and measure the position of a transparent object such as a PET bottle or a glass bottle.

Other experimental results
Another experiment was performed with the prototype sensor system, in which a sphere, rectangular prism, cylinder, triangular prism, regular pyramid, triangular pyramid, and spherical pyramid, as shown in Fig. 15 were measured.The measurement procedure was as follows.First, the teaching pattern data for the neural networks were obtained in 3 mm intervals in the z direction.The distance between the sensor array and the object was 200-290 mm.Next, learning was performed by the neural network over the appropriate distance.In this experiment, the neural network required the teaching patterns obtained in the range of 6 mm in the z direction in consideration of the error of the distance measurement.Finally, an object was measured as a test pattern.31 measurement patterns were used for shape identification.3 shows the results of shape identification.In this experiment, the pose of the object was fixed, and the number of test patterns was 31.As the table indicates, all the identifications were successful.Table 3. Shape recognition results

Discussion
The results of the experiments verify that high-resolution position measurements and shape recognition are possible by the method proposed in this study with respect to PET bottles.That is, the proposed scheme is a method that can obtain high-resolution distance information in both the depth and width directions, which is the original goal of the scheme, and that this scheme is a promising method for ultrasonic position and shape recognition.The method in this study is limited to the illustrated object orientation in its current stage.To relax the limitations on the measurements and recognition, it is necessary to discuss the arrangement of the sensor array and the irradiation direction of the ultrasonic waves with respect to the measured objects.

Conclusions
In this chapter, the construction of a new ultrasonic recognition system for a transparent object using both ultrasonic sensors and a neural network is described.The proposed system consists of an ultrasonic transmitter, an ultrasonic receiver, and a recognition unit.It simultaneously can identify a PET bottle and measure the position of the PET bottle with a neural network.A prototype sensor system has been used to recognise and measure 7 kinds of PET bottle.Experimental results demonstrate that the sensor system achieved perfect recognition, and position measurement with accuracy to within 3.21 mm in the x-direction and within 5.46 mm in the z-direction.In conclusion, the proposed system is effective for applications that identify and measure the position of transparent objects such as a PET bottle or a glass bottle.
Although problems remain with the proposed scheme, such as the fact that it is limited to the current orientation of the measured objects, it is possible to use it for identifying and differentiating products on plant production lines, which indicates the practical significance of the method.To relax the constraints on the measurements and recognition and to recognise more complicated objects, the extraction of new characteristic quantities and the arrangement of the sensor array are planned for future study.
Fig. 1.Acoustic image obtained by conventional ultrasonic methods

Fig. 3 .
Fig. 3. Reflection and penetrationSecond, the ultrasonic pressure, p, is reduced by the propagation distance, d, as expressed by equation (4).Here, a stands for the radius of a vibration board that produces an ultrasonic wave, and v 0 stands for the velocity of the vibration, as shown in Fig.4.In this way, the reflected ultrasonic pressure from the measured object has a relation to the flight distance.
Fig.5shows how the reflected waves from an object vary according to the shape of the side of the object.Fig.5(a) presents the output distribution of the reflected waves when the side of a rectangular prism and the ultrasonic transmitter are placed in parallel.Fig.5 (b) and (c) present the case of a cob of corn and a triangular prism as the measured object, respectively.The edge of the triangular prism faces the transmitter.Fig.5 (d)  presents the positional relationship between the object and the sensor array.In these figures, because the size of the receiver array is 8 x 5, the output distributions are plotted by five kinds of marks.As seen in the figures, in the case of a rectangular prism, the peak value of the output distribution is larger than that of others.There are two peaks in the output distribution of a triangular prism because of the edge of the object.The area of the output distribution of a cob of corn is comparatively wide.Therefore, the peak values and the area of the amplitude output distribution of the reflected signals received by the ultrasonic sensor array are different, corresponding to the edge forms of the sides of the objects.

Fig. 5 .
Fig. 5. Shape recognition principle Fig. 6 shows the output distributions that differ in accordance with the position of the object.Fig. 6 (d) presents the arrangement of the object with respect to the sensor array.As shown in Fig. 6 (b), when the measured object is leaning to the right, one peak occurs in the sensor output distribution, and the peak position moves to the right.In the same way, when the measured object is leaning to the left, the peak position of the output distribution moves to the left.Moreover, as shown in Fig. 6 (c), when the measured object is leaning to the depthdirection, the peak value of the output distributions gets smaller because of the attenuation Fig. 7. Ultrasonic pressure distribution of different shapes of PET bottle Figure 8 shows the mechanism for measuring the object's position.The positions in the xdirection and in the z-direction of a PET bottle have been measured by moving in parallel with and vertical with respect to the sensor array, respectively, as shown in Fig. 8 (a).Fig. 8 (b) shows the ultrasonic pressure distribution when the PET bottle is shifted in the xdirection.Fig. 8 (c) shows the distribution when the PET bottle is moved in the z-direction.From these figures, it is shown that the positions and values of the peaks of the ultrasonic pressure distribution change as the PET bottle moves.In this way, the proposed method uses the values and locations of the wave peaks of the ultrasonic pressure distribution to perform position measurement of the PET bottles.

Fig. 8 .
Fig. 8. Mechanism of position measurement for PET bottles

Fig
Fig. 9. Construction of the neural network

Fig
Fig. 15.Measured objects Table3shows the results of shape identification.In this experiment, the pose of the object was fixed, and the number of test patterns was 31.As the table indicates, all the identifications were successful.