Open access peer-reviewed chapter - ONLINE FIRST

SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial Printing

By Azhar Hussain and Kamal Bhattarai

Submitted: October 24th 2018Reviewed: March 6th 2019Published: June 14th 2019

DOI: 10.5772/intechopen.85680

Downloaded: 87

Abstract

Recently, the advancement in industrial automation and high-speed printing has raised numerous challenges related to the printing quality inspection of final products. This chapter proposes a machine vision-based technique to assess the printing quality of text on industrial objects. The assessment is based on three quality defects such as text misalignment, varying printing shades, and misprinted text. The proposed scheme performs the quality inspection through stochastic assessment technique based on the second-order statistics of printing. First, the text-containing area on a printed product is identified through image processing techniques. Second, the alignment testing of the identified text-containing area is performed. Third, optical character recognition is performed to divide the text into different small boxes, and only the intensity value of each text-containing box is taken as a random variable, and second-order statistics are estimated to determine the different printing defects in the text under one, two, and three sigma thresholds. Fourth, the k-nearest neighbors (k-NN)-based supervised machine learning is performed to provide the stochastic process for misprinted text detection. Finally, the technique is deployed on an industrial image for the printing quality assessment with varying values of n and m. The results have shown that the proposed SAML-QC technique can perform real-time automated inspection for industrial printing.

Keywords

  • stochastic assessment
  • machine learning
  • k-NN
  • printing quality
  • automation

1. Introduction

The quality control and inspection of the high-speed printing process is a critical topic in various industries ranging from electronics to pharmaceutical products. The advent of higher quality requirements from end users and higher costs of raw materials has limited the profit margin of labeling producers. The label producers need to improve the quality of their products and improve the production efficiency by keeping the cost and management thresholds to a minimum level. Typical industrial label printing mechanisms include offset-printing, letterpress-printing, combination-printing, and flexography printing. Various challenges faced by the labels are the limited sized of text-containing region, relatively small size of labels, clear text, ink-flow control for same shades, and text alignment. Usually, the common defects encountered in the printing process are misprinting, text fading, and various shades in printed text. According to the report [1], in 2010 the Chinese pharmaceutical label market demand was more than 165 million m2, and the increase is predicted to be 10–12% in the next 3–5 years. Europe and America had a market demand of 300 million m2 and 250 million m2, respectively, and both are expected to increase by about 3% annually. It is evident that the market demand of pharmaceutical labels in western countries is higher than in China.

The Chinese market is expected to witness an increase in demand in the coming years. It is also reported that in the past 3 years, the labor costs have increased dramatically and QC management has become more difficult. Manual QC inspection can be easily affected by factors such as lack of experience, emotions, human eye perception, and environmental conditions. These factors lead to an inconsistent and poor standard of inspection. Although a review process is usually added in the inspection process in order to recheck the misprinted labels and quality defects, it cannot guarantee 100% quality inspection and also leads to extra labor costs and time consumption. In order to win the customer satisfaction, the inspection QC should be improved through automation. Even tiny mistakes in the label printing can put a bad impression in the overall product quality and can reduce a good customer relationship. Therefore, automatic inspection system is an obvious choice for the industries, and it will have room to grow in the near future.

The proposed scheme performs quality inspection in three steps. The first step is related to the inspection of printed text alignment with respect to the industrial object. Although the requirement for printing alignment is company specific, the proposed mechanism is adaptable and adjustable according to the specific requirement. The second step performs quality inspection based on detection of varying printing shades in the detected text. The third step is aided by supervised machine learning, and it performs the detection of misprinted text. The rest of the paper is composed as follows. Section 2 addresses the related work for computer-aided quality inspection of printing. Section 3 explains the SAML-QC algorithm. Section 4 shows the results of performing the proposed inspection on a given industrial object. Finally, Section 5 presents conclusion and future work.

2. Related works

Recently, a few researches have investigated computer-aided detection and image quality assessment. In [2] a technique based on the comparison of an inspected document with its referential version is discussed. In [3] an image quality assessment algorithm is proposed that does not rely on reference images, and its general framework emulates human quality assessment by first detecting visual components and then assessing quality against an empirical model for face detection. In the algorithm of Rowley et al. [4], a neural network is trained to detect face patterns in a region of 20-by-20 pixels. The determination of an arbitrary image such that if the square is a face region, the square is down-sampled to the size of 20 by 20 and equalized, resulting in a normalized signal.

A similar proposition holds for the face detector of Viola and Jones [5]. In their algorithm a feature-based classifier is trained over a square box of 24-by-24 pixels, with each feature defined by a feature template composed of a group of rectangular sub-windows. In [6], two machine learning algorithms are used for feature selection: mixture of Gaussian and radial basis function (RBF). The first one is a statistical distribution estimation-based algorithm, while the second is a function approximation-based algorithm. However, the preliminary experiment based only on face detection and face-based quality modeling yielded encouraging results. However, more work needs to be done in the areas such as object detection, feature selection, and machine learning to better establish this method.

As an efficient alternative, the machine vision systems can filter out the physical limitations and subjective judgmental decisions of humans. In [7] an image processing technique for the development of a low-cost machine vision system is explored for the inspection of the pharmaceutical capsule. This work discusses the two-part gelatin capsule inspection system by using image processing techniques for border tracing and approximation of the capsule to a circle. A quality control feedback performs pass/reject decision and puts capsules to the appropriate bin. In [8] a new approach for detecting the printing accuracy based on the technology of machine vision is presented.

The process consists of image acquisition, filtering, segmentation, and image matching. In this chapter, JSEG [9] algorithm is used to segment the textile printing images with obvious texture and then make a match for the segmented edge information, to obtain deviation area of the image and calculate the deviation position. However, in their experiments, several limitations are found for the system. One case is when the similar colors between the two neighbor regions cannot be segmented. The work in [10] introduced the algorithm based on wavelet packet and the regional analysis. It analyzes the significance of the threshold of binarization in defect detection combined with morphology area analysis method to extract specific features of defects and determine the type of defect. In [11] a study was conducted to evaluate how accurately a machine learning method can predict human judgment about printed pattern quality. Similarly [12] also proposed machine learning-based quality assessment scheme.

3. Proposed SAML-QC scheme

The block diagram of the proposed scheme is shown in Figure 1. It shows that the algorithm receives the input image and performs pre-filtering. The pre-filtering process starts by RGB to gray scale conversion and resizing the input image to a width of 500 pixels without losing the aspect ratio. A Gaussian blur (of kernel size 9 × 11) filters the salt and pepper noise in the image. Then the histogram equalization process enhances the contrast. The next step is the initialization of structuring kernels. These kernels play a major role in the morphological closing operation. This operation combines the text-containing region and supports the detection of text-containing region for the alignment testing. The next step is the initialization of TopHat morphology [13] to find the white regions against the darker ones.

Figure 1.

Block diagram of SAML-QC algorithm.

It follows the computation of Scharr gradient [14] of the TopHat image for edge detection. The proposed scheme used the Sobel operator in horizontal axis to calculate the absolute value element-wise. The minimum and maximum values of Scharr gradient are obtained followed by scaling to the range 0–255 per pixel value. The next step performs the morphological closing operation to fill the gaps. It is followed by Otsu’s auto-thresholding [15] to binaries the image. The morphological closing and dilation process is performed to convert the text-containing region as a combined blob. It is intuitive that this combined block contains the actual area of the text and makes the image ready for the detection of contours. The purpose of finding contours in the image is to detect the text-containing region inside the given object. After that, the text-containing region is cropped and stored in a buffer for later use in stochastic assessment. The text-containing region is identified by selecting the bounding box with optimum aspect ratio in the process of iterating over the found contours. The red rectangle is displayed on the image of the selected text. The coordinates of the text-containing region are used for the testing of text alignment.

3.1 Text alignment assessment

In printing industry the common text alignment errors are related to the horizontal and vertical alignments. In the current paper, it is assumed that the best position of the text is right at the center of the object. Therefore, any text region that is printed too much horizontally or vertically should be identified. However, the height and width of the industrial objects and their respective printed texts are different due to the varying nature of performance of the printing labelers as discussed in [10]. Therefore, the proposed algorithm presents a mathematical formulation to detect alignment error of the text inside an industrial object. Figure 2 shows an abstract image of an industrial object (red portion) with markers showing the dimensions. The width and height of the industrial object are represented by Wo and Ho, respectively. The width and height of text-containing region are represented by w and h. Let (xo,yo) and (xt,yt) represent the top left corners for the object and text boxes, respectively, as shown in Figure 2. The absolute difference in terms of pixels between the top edge of industrial object and top of the bounding box of the text region is given by Eq. (1), and similarly, the absolute differences for down and left are given by Eqs. (2) and (3):

Figure 2.

Parameters for the alignment test of text-containing region.

U=absyoytE1
D=absyo+Hoyt+hE2
L=absxoxtE3

The proposed criterion for text alignment is based on threshold value for vertical and horizontal alignments. Let UDThresh represent threshold for up-down or vertical alignment testing. The value of UDThresh is set as h. Let LThresh represent the left-right or horizontal alignment.

The value of LThresh is set as Wo/4. It means that it contains the width of one-fourth of the full width of object. The decision boundaries for horizontal and vertical alignment passed or failed are given as follows. If abs{(U-D)} ≤ UDThresh, then the vertical alignment is considered as passed else it is failed. If L ≤ LThresh, then the horizontal alignment is considered as passed, else it is considered as failed.

3.2 Optical character recognition

After the assessment of alignment, the next step is to perform the optical character recognition. For this purpose the proposed scheme crops the text-containing region (xt,yt, x + h,w + h) of the input image. The median filter is applied to remove any salt paper noise in the image. The next step is to perform the Gaussian blur with emphasis in the vertical direction by selecting the kernel size of 1 × 5. The reason behind this step is to exploit the nature of text printing. For example, in Figure 2 the vertical difference between the TEXT CONTAINING and REGION IS HERE is more than the horizontal difference between consecutive individual letters. The next step in the process of obtaining optimum image ready to perform optical character recognition (OCR) is to perform the image contrast enhancement.

The proposed scheme used bilateral filtering [16] to improve the text regions while suppressing the background. The bilateral filtering alters the intensity value of each pixel with the weighted average of its neighbors that is Gaussian distributed. The next step is to perform the gray scale conversion and perform Otsu’s autothresholding and bit-wise logical NOT operation to obtain binary image. This image is saved as a PNG format. This PNG image is provided as input image to the state-of-the-art Tesseract open-source OCR engine v3.02 [17]. The purpose of performing OCR is to get bounding boxes and location of anything that looks like a character. As a result an html file containing the position and sizes of each detected letter is generated. Figure 3(a) and (b) shows an image that is cropped and processed by the stated procedure to get the detected boxes of letters in the text and marked with green rectangles.

Figure 3.

The detection of bounding boxes of letters: (a) text area selected; and (b) detection of letters.

3.3 Stochastic assessment of printing quality

The next block performs the stochastic assessment of the printing quality in terms of detection of overly faded printed text due to the imperfections in printing process as discussed in [1].

3.3.1 Detection of printing in varying shades

The intensity value of each printed pixel can be considered as a random variable H sampled at time t, and since its distribution is also unknown, therefore, it follows Gaussian distribution. From [18] it is clear that a sequence of random variables is independent and identically distributed (IID), if every random variable has the identical probability distribution and all are mutually independent. Thus it is also assumed that H is IID Gaussian random variable. The noise related to the intensity variation is Gaussian in nature; hence the mean and the acquired image also have additive Gaussian noise ni with H. In order to estimate the parameter H, from parametric space to the estimation space H′ via observation space y(H,t) at any time t, the maximum a posteriori probability (MAP) estimation [19] is used. MAP maximizes the a posteriori probability, which means most likely the value of H is given by Eq. (4):

maxHfHy=maxHfyHfHfyE4

It is quite clear that the probability density function (p.d.f) of parameter H needs to be determined in order to maximize the expression Eq. (4). It is assumed that ni is IID with N (0, σn2) and H is independent of ni with N (0, σH2). Let K be numbers of pixels which are available for a given box to estimate H. The conditional p.d.f of y given H is given by Eq. (5):

fyH=i=1K12πσn2eyiH22σn2E5

And the p.d.f of H is given by Eq. (6):

fH=12πσH2eH22σH2E6

It is known that the conditional p.d.f of H given y is in Eq. (7):

fHy=fyHfHfyE7

Inserting the values from Eqs. (5) and (6) in Eq. (7) provides Eq. (8):

fHy=i=1K12πσn2ei=1K12σn2yiH2eH2/2σH21fy2πσHE8

The expression in Eq. (8) can be defined in terms of q(y) as shown in Eq. (9):

fHy=qye1/2σ2Hσ2/σn2i=1Kyi2E9

The notation σ is given by Eq. (10):

σ2=1KσH2+σn2σH2σn2E10

It should be noted that q(y) is only the function of y; therefore, the best estimate HMAP of H is the value where f(H|y) gets the highest peak, which is obtained from Eq. (11) when H = HMAP:

HMAP=σ2/σn2i=1KyiE11

Eq. (10) can be solved to get Eq. (12):

σ2=σH2σH2+σn2/K1Ki=1KyiE12

If σS2σn2/K, then the best estimate of H is given as Eq. (13):

HMAP1Ki=1KyiE13

HMAP is the best estimate of intensity levels of the character-containing region of a detected box, yet its value varies for every other detected box, and its probability distribution is also unknown, so it can also be assumed as a random variable that follows Gaussian distribution. If B represents the number of detected character boxes in a given image, then the set of real values hMAP(u) assigned to HMAP for all detected boxes is shown in Eq. (14), where u represents the index of each box.

HMAP=hMAP1hMAP2hMAP3.hMAPBE14

Let E [HMAP] be the expectation of HMAP, and it is expressed as Eq. (15). The variance σ 2HMAP can be found in Eq. (16):

EHMAP=u=1BhMAPuPHMAP=hMAPuE15
σ2HMAP=EHMAPEHMAP2E16

Definition 1: (Set of bad boxes) The set of all those boxes such that members do not satisfy the conditions in Eq. (17) is called set of bad boxes.

Here n represents the quality index, and smaller value of n corresponds to higher demand of quality of label printing,

EHMAPnσHMAPhMAPuEHMAP+nσHMAPE17

The selection of bad box is performed by Eq. (17), as one of the main concerns in the quality inspection of label printing is to find the overly printed or faded printed characters. All the character boxes that fall under the stated condition Eq. (17) are considered as members of the set of good boxes. It is quite intuitive that high-quality printing demands larger set of good boxes. Therefore, the proposed scheme used two separated counters called GBc and BBc representing the numbers of good boxes and bad boxes, respectively. The % quality success for similar intensity levels QSI is given by Eq. (18):

QSI=GBcBBc+GBc×100E18

It should also be noted that the value of QSI has a direct relationship with quality index n. The smaller value of quality index means strict requirement of quality and for the same image QSI decreases with the decrease in value of n and vice versa.

In order to plot the probability density estimate of HMAP, the procedure for kernel density estimation [20] is used. It returns a probability density estimate f for the sampled data in the vector or two-column matrix. It estimates the density at 100 points for univariate data. Figure 4 shows the plot of probability density estimate for HMAP of the image in Figure 3(a). It is clear that p.d.f approximately follows Gaussian curve with a certain value of mean and variances.

Figure 4.

Kernel density estimate plot of HMAP for the image in Figure 3(a).

3.3.2 Detection of misprinted boxes

The detection of misprinted box is challenging in terms of its probability of occurrences and detection. The OCR engine can detect it as a text although being probably wrong detected and classified. The proposed scheme achieves the detection of misprinted box by deploying supervised machine learning using k-NN. The k-NN algorithm is a nonparametric method used for the classification and regression [21] in the pattern recognition. The supervised training aggregates the human responses to the appearance of a letter inside the detected box. The purpose of this step is to provide supervised learning samples and responses for the k-NN algorithm.

The supervised machine learning procedure of the proposed scheme takes input sample image of industrial object which is printed with no defects. All the characters in this sample image are alphabets from A to Z, a to z, -, and numeric digits from 0 to 9. Let us call the set of these input images as no-misprinted-images (NMI). The training process starts by iterating the NMI for all detected boxes, and for each detected box, a human response is provided as a label and aggregated to a human responses database (HRD) file, because the human needs to press the related key from the keyboard. As for each detected letter box, its ratio width/height = 2/3; therefore each box is aggregated as a matrix of 20 × 30 pixels in separate matrices received database (MRD) file.

Therefore, at the end of training process, two (HRD and MRD) files are ready for the k-NN algorithm. The k-NN algorithm takes these two files as input and finds the nearest neighbor in terms of its output as hamming distance D as stated above. It is quite intuitive that D will have higher value for the misprinted or unknown letters, because the training process did not consider those letters. The value of D for each printed letter can be considered as a random variable sampled from a random process at time t, and since its distribution is also unknown, therefore, it is assumed that it follows Gaussian distribution. Based on the fact stated by [22], the proposed scheme also assumes that D is IID Gaussian random variable. The next step is finding the expectation of D for each letter box. It should be noted that D is normalized by the size of box, i. e., 20 × 30. Let us represent E[D] and σD2 as the mean and variance of D. We can compute these values using the similar way as described in Eqs. (15) and (16), which provide the second order statistics for the distance. Let the set of real values d(u) assigned to the random variable D be given by Eq. (19):

D=d1d2d3.dBE19

Definition 2: (Misprinted box detection) The set of all those boxes such that members do not satisfy the conditions Eq. (20) is called set of misprinted bad boxes.

EDmσDduED+mσDE20

Again the quality index m represents the strict boundary for the selection of misprinted boxes. All the character boxes that fall under the stated condition Eq. (20) are considered as members of the set of good printed boxes. It is quite intuitive that high-quality printing demands larger set of good boxes. Therefore, two separated counters called GPBcount and MPBcount represent the numbers of good printed boxes and misprinted boxes, respectively. The % quality success for good printed boxes QSGPB is given by Eq. (21):

QSGPB=GPBcountMPBcount+GPBcount×100E21

The final step is the printing of the acquired results related to the text alignment, QSI and QSGPB. These results can not only draw the outcome of printing inspection process but can also provide feedback to the printing labelers to automatically adjust the process according to the type of printing errors.

4. Results

Table 1 summarizes the values of horizontal and vertical alignments for sample image in Figure 5. The results support the human observation of central text alignment for this kind of particular object. It should also be noted that text alignment specification and parameters are user specific.

ImagesULDUDThreshLThreshAlignment testing
HorizontalVertical
1425950170119PassedPassed

Table 1.

Text alignment assessment results for the given image.

Figure 5.

Text alignment detection.

Figure 6 shows the bar graph of the estimated values for the random variable HMAP for each detected text in Figure 3(b).

Figure 6.

Estimated intensity values of HMAP.

Figure 7 shows the histogram of HMAP in terms of numbers of boxes and the value of random variable HMAP. It also shows the corresponding normal distribution curve fitted based on the values of HMAP.

Figure 7.

Plot shows fitting a normal distribution to the values of HMAP.

Table 2 summarizes the results of printing quality assessment in order to test the intensity variations in the printed labels. It shows mean, variance, quality (Q) factor nσHMAP, the sum of good and bad boxes, and finally the QSI. It is obvious from the results that Q factor decides the value of QSI.

Q Index nMean E [HMAP]Variance σ 2HMAPQ factor nσHMAPBox counts GBc + BBcQSI (%)
16662.4413 + 1056.52
26664.8922 + 195.65
36667.3523 + 0100

Table 2.

Printing quality assessment results for HMAP.

The value of quality index is varied as n = 1, 2, and 3 that increases the value of GBc as 13, 22, and 23, respectively. Additionally, the values for QSI for n = 1, 2, and 3 are 56.52, 95.65, and 100%, respectively.

Figure 8 shows the output of SAML-QC scheme for the detection of bad printed labels. The higher quality control requires lower value of n.

Figure 8.

Detection of bad printed boxes for different n: (a) n = 1, (b) n = 2, and (c) n = 3.

Figure 8(a) shows that for the restriction n = 1, all good boxes are detected and marked with green color, whereas, all the boxes in red color are the ones with value of hMAP(u) ≥ E[HMAP] + nσ2HMAP, whereas, boxes with hMAP(u) ≤ E[HMAP]—nσ2HMAP are represented by yellow boxes because they belong to the faded printing category. Similarly, Figure 8(b) and (c) shows the output images for n = 2 and 3, respectively.

The next assessment is related to the detection of misprinted characters, and Figure 9 shows result of k-NN classification performed through supervised machine learning and the distance D for each detected character.

Figure 9.

Bar graph of k-NN classification distance D.

It is clear from the results in Figure 9 that the two higher spikes for the detected character S and 3 are located at the position of misprinted characters.

Table 3 shows results of the process for the detection of misprinted boxes as discussed in Section 3. It is observed that the proposed scheme put a stable restriction on the misprinted labels, and it is shown in Table 3 that the value of QSGBP remains 91.30% for all three values of m.

Quality index mMean E [D]Variance σD2Q bound nσDBox counts GPBc+MPBcQSGBP %
13397.881.38 × 1073727.6821 + 291.30
23397.881.38 × 1077455.3621 + 291.30
33397.881.38 × 10711183.0421 + 291.30

Table 3.

Misprinted labels’ assessment results.

Figure 10 shows the detection of misprinted labels, and they are marked with the red boxes for m = 1, 2, and 3.

Figure 10.

Detection of misprinted boxes: (a) m = 1, (b) m = 2, and (c) m = 3.

5. Conclusions

This chapter proposes a stochastic assessment and machine vision-based technique to assess the printing quality of text on industrial objects. The assessment is based on three quality defects such as text misalignment, varying printing shades, and misprinted text. It is concluded from the results that second-order statistics related to the intensity values of pixels of text located under the threshold region of the original image can provide enough information to perform quality inspection. On the other hand, the hamming distance acquired by the k-NN supervised machine learning can also be taken as a random variable, and again the second-order statistics are helpful in order to detect the misprinted letter. In the future SAML-QC technique needs to be evaluated on various other images such as electronics parts and medical parts.

Acknowledgments

My deepest acknowledgment is to Prof. Tao JIANG of the College of Information and Communication Engineering, Harbin Engineering University, for guidance, kind support, and wisdom.

Download

chapter PDF

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution 3.0 License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

How to cite and reference

Link to this chapter Copy to clipboard

Cite this chapter Copy to clipboard

Azhar Hussain and Kamal Bhattarai (June 14th 2019). SAML-QC: A Stochastic Assessment and Machine Learning-Based QC Technique for Industrial Printing [Online First], IntechOpen, DOI: 10.5772/intechopen.85680. Available from:

chapter statistics

87total chapter downloads

More statistics for editors and authors

Login to your personal dashboard for more detailed statistics on your publications.

Access personal reporting

We are IntechOpen, the world's leading publisher of Open Access books. Built by scientists, for scientists. Our readership spans scientists, professors, researchers, librarians, and students, as well as business professionals. We share our knowledge and peer-reveiwed research papers with libraries, scientific and engineering societies, and also work with corporate R&D departments and government entities.

More About Us