In this chapter a particular image processing problem existing in quality assurance for steel industry is addressed. Since the quality standards in steel industry are ever increasing, even steel industry starts to use machine vision systems to monitor their product quality. In case of steel coils a particular figure of merit in quality is the roundness (ellipticity) or lack thereof of the coil. The mathematical correct measure of the roundness is the eccentricity, with the major semi axis a and the minor semi axis b. But the fastest way for the quality inspector is just to measure a and b of the coil and calculate their ratio. If ( exceeds a set limit (≈1.03 measured at the innermost coil layer), the coil is considered as cull. As discussed by Schleicher & Zagar (2008; 2009a) the extraction of elliptic arc segments for the ellipticity estimation and the basic concentric ellipse fitting algorithm are the main parts of the algorithm estimating the coils ellipticity. Comparing to ordinary ellipse fitting tasks that require input data distributed over the whole circumference, the estimation of the ellipse parameters using only data forming low length arcs is a much more challenging problem. To achieve a precise estimation of the ellipticity the fact of the common center of the elliptic arcs is used by the concentric ellipse fitting algorithm to estimate the center of the coil precisely. The functionality of the algorithm is show based upon of real coil images.
Indeed some error sources reduce the accuracy, which are discussed at the end of the chapter. The chapter is organized as follows: First the image acquisition system is described. Then the algorithm is explained consecutively in Sec. 3. Further sources of different errors are investigated in Sec. 4. Finally major results are presented (Sec. 5) and a conclusion (Sec. 6) is given.
2. Image acquisition system
The image acquisition setup shown in Fig. 1a consists of a camera placed on a tripod, a laser trigger unit to initiate the acquisition and a laptop to store, process and view the images. A single image frame of the coil is acquired, when the coil transported by a lifting ramp reaches its end position triggering a light barrier. The camera is a 1/2” monochrome firewire camera (AVT Guppy 146–B) with 1392×1040 pixels. Its optical zoom is adjusted so as to image the biggest coil full frame, which has an outer diameter of 2 m. The sheet metal layers are 0.5 to 2.5 mm thick and the inner diameter of each coil is known to be 0.6 m. To ensure a circular image of a truly circular coil, the camera chip plane must be oriented parallel to the coils front side (Scheimpflug condition (Luhmann, 2003, p.147)). Figure 2 shows the camera coordinate system in relation to the world coordinate system, where the–plane (coil front side) is kept parallel to the–plane (camera chip). To assure the correct alignment of the camera, a spirit level was used to do the horizontal alignment (,–plane). Furthermore the horizontal edges of the coils carrier were used to align the rotation about the–axis that the coils front side is parallel to the camera chip plane. With this setup the image is acquired and processed by the algorithm in the next section.
The block diagram of the algorithm, shown in Fig. 3, reflects the structure of this section. The preprocessing, circular hough transform and extraction of all curve segments are applied on the pixel level and then a reduced dataset of selected curve segments is used for the ellipticity estimation.
The first preprocessing step is to crop to the region of interest, which fits the biggest coil to capture. Further the unsharp masking sharpening algorithm Eqn 1 (Jain, 1989, pp.249– 251) is used to boost the edges of the coil.
where G(x,y) is the discrete Laplacian
of the raw image. The parameter λ determines the sharpening power of the algorithm. Setting λ too large leads to a very noisy image result, since the Laplacian is very sensitive to noise. Since the sharpening process is similar to a high pass filter, any influence of the inhomogeneous natural lighting is reduced.
The next important preprocessing step is the segmentation of the image into background and foreground pixels (front side highlighted in Fig. 1a). Usually this is done by comparing the actual image to a background image only (Zivkovic, 2004). Since only a single image of the coil is available this procedure can not be applied and the results of the circular hough transform are used to first find the coils center and then to segment the coils front side, which is described in the next section.
3.2. Circular Hough Transform (CHT)
The circular Hough transform (Illingworth & Kittler, 1987; Peng, 2007; Peng et al., 2007) is a transform mapping points in the x,y–plane belonging to circular slopes into parameter space. Thus filling a so called accumulator. A circle in x,y–space can be described by
In parameter space it is then. This 3D parameter space can be reduced to a 2D with, we incorporate the constraint that the vectors, which are normal to the circle boundary, must all intersect at the circle center. Figure 4 illustrates this relation between the circle and its center. These normal directions can be obtained by local gray–level detection operators. The normal direction can also be expressed by
Rearranging Eqn 4 yields
and transfers the problem in a form similar to the well known (line) Hough transform (HT). The accumulator of the HT, where the circles centers is determined by the peak, is shown in Fig. 5a. Since the determined center is only a rough estimate (denoted by ˆ), the center. After incorporating the circle center in Eqn 3, the radius can be found by histogramming,. This histogram, also called signature curve, shown in Fig. 5b, is used to obtain an estimate for the inner radius ri aswell as the outer:
radius ro (marked by a red asterisk). Local maxima in the signature curve are marked with a blue asterisk and signals the presence of a circular shape detected in the front side image.
To reduce the execution time of the CHT algorithm the preprocessed image is resized (down sampled) to 150×170 pixels.
Due to the reduced image size the center estimation is only very rough, though fast. Furthermore the two stage concept causes the error of the center estimate to propagate further to the radius estimation in the signature curve. That’s why the signature curve is not very reliable to return the proper real inner and outer radius, respectively. When no proper radii are found, the maximum coil size is chosen to segment the coil front side, since subsequent selection criteria will be applied to the curves before they are used for the fitting process.
3.3. Curve extraction
To extract arc curve segments at least of some of the sheet metal layers the Canny edge detector (Canny, 1986) is applied to. As a result this edge detector creates a binarized image with thinned out edges. Due to the hysteresis threshold technique (Nixon & Aguado, 2002) implemented within the Canny edge detector, even weak edges are reliably extracted. As a drawback however, some edges, especially those resulting from rather weak gradients, have ramifications and squiggles that need to be taken care of.
After the erosion, the resultant image contains only disrupted curve segments, which are remapped from the image matrix onto a curve set, which contains all extracted curve segments denoted by (See Fig. 7). The curve segment cj is a vector containing the curve pixels coordinates.
3.4. Curve selection
Since the extracted curve set also contains curves derived from the edges of the coils tapes or packaging labels, which must not be processed in the ellipticity estimation algorithm, these curves must also be remove to increase the quality of the estimators input data. Therefore figures of merits are calculated to select only those that are kept for further processing and reject straight lines, rather short and very squiggled curves. Useful figures of merit are the curve length and the extend, which is the ratio between curve bounding box and curve length. Furthermore closed curve segments, with an eccentricity of the ellipse that has the same second-moments as the region (is actually a circle; is an ellipse degenerated to a line segment) over a specific threshold (in this application 0.1 turned out to be useful), are also rejected (Rafael C. Gonzalez, 2003).
To select only elliptic arc segments which are part of the coil the scalar product of, the normalized radius vector, and t, the tangential vector, is calculated for each point on each arc curve segment cj. Figure 8 shows two curves with typical cases indicated. Concerning an ideal round coil, the scalar product for each arc segment. Since nr and t are normalized, the range of the result of the scalar product is [0,...,1]. The subsequently applied hysteresis thresholding technique (Nixon & Aguado, 2002) returns only reasonable curve segments for the final ellipticity estimation algorithm.
3.5. Ellipticity estimation
To estimate the ellipticity an ellipse fit is applied to the dataset. Curve fitting is an extensive topic and even ellipse fitting is discussed in literature very detailed (Gander et al., 1994; Matei & Meer, 2000b; Leedan & Meer, 2000; Halir, 1998; Ahn et al., 2001; Nair & Saunders, 1996; Liu et al., 2006; Lei &Wong, 1999; Fitzgibbon et al., 1999; Halir, 1998).
The paper written by Gander et al. (1994), was one of the first that covered circle and ellipse fitting using least squares and iterative methods.
Figure 10 shows typical fitting results when applying the general ellipse fit algorithm to the previously extracted arc segment dataset. Obviously, the common center of the coil can’t be reliably determined, since the general ellipse fit algorithm is executed only for one arc segment at a time. This renders it necessary to consider additional constraints for the fitting algorithm and adopt it to use the whole dataset with all arc segments simultaneously.
In the following sections first the chosen curve fitting algorithm is described on a general way. Then the general, the simple and the new concentric ellipse fit, which incorporates the fact of the common coil center, is described in more detail.
3.5.1. Curve fitting
Before doing the ellipse fit, a short introduction into curve fitting will be given. Therefore the same notation as proposed by Chernov (2007) will be used.
The principal equation for algebraic curve fitting is
To estimate Θ the approximate maximum likelihood (AML) method (Chernov, 2007) is used.
The square matrices and depend only on xi and have the size.
Using these matrices for the representation of the input data many popular schemes can be used to solve the minimization problem:
All of the above algorithms were tested for the concentric ellipse fitting problem and only REN showed unstable behavior. The other algorithms only vary in execution time and achieved accuracy, which is also dependent on the maximum number of iterations and the set tolerances. For additional information about these schemes we refer to Chernov (2007). In the following sections the way to obtain the and matrices, and, respectively, will be described.
3.5.2. General ellipse fitting
A general ellipse can be described by the equation for the algebraic distance
To obtain the unknown parameters A,..., F of Eqn 13 the AML method is used. Therefore the equation is written in terms u(x) and Θ.
Further we obtain
3.5.3. Simple ellipse fitting
The denotation simple ellipse fitting (SEF) is used when the simplest form of an ellipse, with the center at the origin (0,0) and no rotation, is to be fitted. Equation 12 is then simplified to
Here a and b are the major and minor semi axes and u and v are the coordinates related to the semi axis of the ellipse. To obtain u and v the ellipse must be centered in the origin thus:
to calculate and.
3.5.4. Concentric ellipses fitting
The concentric ellipse fit takes the whole dataset of the coils elliptic arc segments into account, in contrast to the previously discussed algorithms, which use only one curve segment at a time. Thereby a highly precise center estimation is possible. To achieve this, some simplifications needs to be made:
For a typical coil the first assumption is justifiable since mainly the acting force of gravity determines the orientation of the semi major axis to be horizontal. Further the requirement of constant ellipticity for all elliptic curve segments doesn’t hold here exactly and will cause a small error, but the center estimation is not affected by this fact (Schleicher & Zagar, 2009b).
First Eqn 12 is rewritten using the center coordinate and the semi axes a and b.
Introducing and factorizing leads to
Concerning Eqn 27 the newly introduced parameters are independent on b, while contains the curve dependent parameter b. Therefore must now be written as
using all curve segments in the dataset s, which is indexed as follows: The dataset contains J curves indexed by j, where curve j has Mj points. A point on curve j is denoted by, where m is the index of the point (m = 1,...,M).
When concatenating the curves to the input data vector x a point is addressed by index i.
Furthermore u(x) must also be extended to
In the general sense we can write
For the gradient we obtain
Having given the matrices and are calculated and is estimated using one of the estimators described in Sec. 3.5.1.
Out of all parameters in jointly estimated the center coordinate (xc,yc) and parameter and b are calculated.
First the parameter vector is normalized so that is set 1. By comparison of coefficients the center coordinate are obtained:
Since the other parameters and b
are biased due to the (invalid) assumption of equal ellipticity of all segments the simple ellipse fit (Sec.3.5.3) is done for each curve separately, using the previously estimated center coordinate.
The result of the concentric ellipse fit followed by the simple ellipse fit, where the ellipticity for each curve segment is calculated (shown in Fig.13) and discussed after the error analysis in Sec. 5.
4. Error analysis
In the following sections the main error sources which influence the accuracy of the estimated ellipticity will be discussed.
When sampling in spatial coordinates the aliasing effect (Jähne, 2002; Burger & Burge, 2006) is similar to consider as the case in the time domain. To be able to reconstruct a sampled signal the sampling frequency must be larger than twice the largest occurring frequency component. In case of the coil image the thinnest sheet metal layer is 0.5 mm, which means. Since the Nyquist criterion is also valid for any orientation the spatial frequencies must be multiplied by
For a coil with a diameter of 2 m a minimum camera chip size of
in horizontal and vertical direction, respectively, is needed. This corresponds to a 32 Mpixel camera chip. Even if we neglect –factor for the diagonal layers, we still would require a camera chip with 4000 × 4000 = 16 Mpixel.
Luckily the small axial displacement of the sheet metal layers with respect to each other causes some shadings which can be captured very good avoiding aliasing. Also the roughly–textured cut surface of most thick sheet metal layers improves the possibility to recognize a single layer. In the worst case when a coil has thin layers, these layers appear blurred in the image.
Further the goal is still to estimate the ellipticity and not to capture each layer. Assuming that the arc segment is recognized only by its shadow which extends from one layer to the next (one pixel = 2.5 mm in this setup), the relative error for the ellipticity is bounded by
with the major semi axis and the arc displacement. Considering this error is quite large for high precise ellipticity estimation.
4.2. Camera orientation
One possible error source is the miss–orientation of the camera. As described previously the camera chip plane should be oriented strictly parallel to the coils front side. Even when the setup is build up very carefully using a spirit level and using edges of the coils carrier for the adjustment, an error free orientation can not be obtained. For a circular coil, with radius r and a misalignment angle, a rough error estimate can easily be obtained:
The angle can have any direction, since the semi major axis of the resulting ellipse is always and the semi minor axis is. Assuming leads to an error of 0.0152% and even a larger angle of 5 , which should be possible to underbid when adjusting the camera, leads only to 0.55%.
4.3. Ellipticity–radius dependency
The assumption of equal ellipticity for all arc segments made in Sec. 3.5.4 leads to a erroneous . Knowing the dimensions of the coil a simulation of the ellipticity progression can be done using the following model.
Assume an ideal circular coil with the inner radius, outer radius, metal sheet thickness Δs and initial oil gap between the layers (see Fig.11). Approximating the layers of the coil, which in reality are forming an so–called Euler’s–spiral, by concentric circular layers, the radius of the layer is
The assumed maximum number of turns on the coil is
When applying a virtual vertically acting force (e.g. gravity) on this coil, it is compressed somewhat flat, the layers get elliptic and the oil gap decreases in vertical direction while increases in horizontal direction. Optionally, the oil gap can be modeled to be smaller at the middle layers, since there is a higher inner pressure caused by the winding action in production, by following equation:
Or, since the oil gap is very small in ratio to the metal sheet thickness, to be a constant setting. For the vertical minor semi axis of the coil we obtain:
where is the compression ratio of the inner radius.
Since the mass of the circular coil mc is equal to the mass of the deformed coil, also the cut surface of the metal sheet layers must be the same (... density of steel,... width of the coil).
The areas are calculated by
Resolving using Eqns 48, 49 and 50 we obtain
Figure 12 shows two simulations of the ellipticity dependency over the turn number for various coil parameter. The layer dependent oil gap has the most effect on coils with thin metal sheets (see Fig. 12c vs. Fig. 12d). Generally the ellipticity decreases strongest within the first third of the radius. Setting and smaller gives similar results like shown in Fig. 12a. Since the exact conditions of the pressure distribution in the oil gap is unknown and no proper literature could be found considering this problem, this simulation can only give a qualitative evidence for the ellipticity progression.
4.4. Estimation errors
A comprehensive analysis of the estimation errors was done by Schleicher & Zagar (2009b), whose results are summarized in this section shortly. Using synthetic curve datasets, similar to the different estimators where evaluated. First the CEF algorithm was applied onto 1500, 5000 and 20000 synthetic datasets. Then the Euclidean distance between d the true
center and estimated center was calculated, which corresponds to the error of the center estimation.
With increasing number of datasets decreases, justifying and proofing that the CEF algorithm has no bias. Using the best performing HEIV algorithm an error of 0.00251±0.251 pixel distance to the was obtained. The variance was very similar for all algorithms and is probably limited by the quantisation of the dataset itself.
Furthermore the SEF algorithm was examined, concerning an erroneous center coordinate as input and arc segments with various center angles and lengths. The result was, that for a center displacement of (1,1) (which is very large in relation to) the relative ellipticity error was smaller than 0.2 ± 0.3% for all algorithms (except Taubins) when curve segments with a length longer than 1/25 of their circumference where used. Furthermore arcs symmetric around the ellipses apex lead to larger errors than others.
Figure 13 shows the results of the algorithm described in Sec. 3. The estimated parameters of the extracted curve segments are used to overlay the ellipses onto the image of the coils front side. Furthermore the estimated ellipticities are plotted over their major semi axis in Fig. 14. Moreover some possible figures of merits, which could characterize the coils quality, are plotted. Depending on the extracted curve segments, the model of ellipticity progression derived in Sec. 4.3 fits more or less perfect. The main error source results from the low camera resolution, which might be changed in future when the cost of high resolution cameras decrease.
This chapter introduced the problem of ellipticity estimation on hand of a practical problem. To obtain a good ellipticity estimation proper datasets must be obtained. These datasets are generally not very easy to obtain, for which reason also the preprocessing to extract the datasets was discussed in detail. Since only arc segments could be used for the ellipticity estimation, normal ellipticy fits fail. Hence the estimation was split into two parts, first to estimate the center of the coil with the concentric ellipticity estimator and then to estimate the ellipticity of the arc segments. Using the general approach for the ellipse fit, enables also to use different solver implementations for fitting problems. When discussing the errors, it was accented that a lot of error sources affect the final result. The correct understanding of the partial errors helps to optimize the corresponding section of the algorithm.