Real-Time DSP-Based License Plate Character Segmentation Algorithm Using 2D Haar Wavelet Transform

The potential applications of Wavelet Transform (WT) are limitless including image processing, audio compression and communication systems. In image processing, WT is used in applications such as image compression, denoising, speckle removal, feature analysis, edge detection and object detection. The use of WT algorithms in image processing for real-time custom applications may require dedicated processors such as Digital Signal Processor (DSPs), Field Programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs) as reported in (Ma et al., 2000), (Benkrid et al., 2001) and (Wong et al., 2007) respectively.


Introduction
The potential applications of Wavelet Transform (WT) are limitless including image processing, audio compression and communication systems.In image processing, WT is used in applications such as image compression, denoising, speckle removal, feature analysis, edge detection and object detection.The use of WT algorithms in image processing for real-time custom applications may require dedicated processors such as Digital Signal Processor (DSPs), Field Programmable Gate Arrays (FPGAs) and Graphics Processing Units (GPUs) as reported in (Ma et al., 2000), (Benkrid et al., 2001) and (Wong et al., 2007) respectively.
The interest in this chapter is the use of WT in image objects segmentation, in particular, in the area of Automatic Number Plate Recognition (ANPR) also known as License Plate Recognition (LPR).ANPR algorithm is normally divided into three sections namely LP candidate detection, character segmentation and recognition.The focus of this chapter is on the use of Haar WT algorithms for License Plate (LP) character segmentation on a DSP using Standard Definition (SD) and High Definition (HD) images.This is an extension of the work reported in (Musoromy et al., 2010) by the authors, where Daubechies and Haar WT are used to detect image edges and to enhance features of an image to detect a LP region that contain characters.The work in (Musoromy et al., 2010) demonstrated that 2D Haar WT is favourable in ANPR using DSP due to its ability to operate in real-time.The drive here is the consumer interest in real-time standalone embedded ANPR systems.The next section describes the proposed LP character segmentation algorithm.
The chapter organisation is as follows: Section (2) reviews dedicated hardware for WTbased image processing algorithms.Section (3) gives a review of image processing techniques using WT and in ANPR application.Section (4) presents the proposed LP character segmentation algorithm based on 2D Haar WT edge detector.Section (5) presents experimental setup.Section (6) presents results and analysis.Section (7) gives conclusion and Section (8) gives references.www.intechopen.comAdvances in Wavelet Theory and Their Applications in Engineering, Physics and Technology 4

Dedicated hardware for WT review
The objective of this work is to investigate a suitable hardware that is able to perform image processing algorithms using WT in real time.Processing an image with the WT filter is faster in terms of computational cost in applications such as edge detection where a single filter is capable of producing three types of edges in comparison to standard methods where more than one filter masks are required to achieve the same results.In this section we review the special hardware dedicated for WT including DSPs, FPGAs and GPUs.
GPUs provide programmable vertex and pixel engines that accelerates algorithm mapping such as image processing.An example of a cost effective SIMD algorithm that performs the convolution-based DWT completely on a GPU using a normal PC (baseline processor) is reported by Wong (Wong et al., 2007).It is reported, the algorithm unifies forward and inverse WT to an almost identical process for efficient implementation on the GPU through parallel processing (Wong et al., 2007).This demonstrate that GPUs are capable of processing WT algorithms cost effectively, however it is not suitable for our application, which is PC independent.
An example of a scalable FPGA-based architecture for the separable 2-D Biorthogonal Discrete Wavelet Transform (DWT) decomposition is presented by (Benkrid et al., 2001).The architecture is based on the Pyramid Algorithm Analysis, which handles computation along the border efficiently by using the method of symmetric extension using Xilinx Virtex-E (Benkrid et al., 2001).FPGA's are suitable for real-time embedded applications due to their parallel processing abilities.
DSPs are also reported to be powerful and portable for embedded systems.An example system by Desneux and Legat (Desneux & Legat, 2000) show a DSP with an architecture designed specifically for DWT.Their DSP design stops any wait cycles during algorithm execution by using a bi-processor organization.It is able to perform a 3-stage multiresolution transform in real time.Their DSP is fully programmable in terms of filters and picture format as well as being capable of image edge processing.
Using a floating-point DSP, Patil and Abel (Patil & Abel, 2006) used redundant wavelet transform as a tool for the analysis of non-stationary signals as well as the localization and characterization of singularities.Their work focused on producing an optimized method for the implementation of a B-spline based redundant wavelet transform (RWT) using a (DSP) for integer scales leads to an improvement in the execution speed over the standard method.
A DSP-based edge detection comparison is explained in (Abdel-Qader & Maddix, 2005) where three edge detection algorithms performance on DSP are compared using Canny, Prewitt and Haar wavelet-based.The reported outcome is that the Haar wavelet-based edge detector performed best in terms of SNR in noisy images.The authors recommended postprocessing of the output edges to make them more optimal.
The review favours DSPs as a suitable choice for our ANPR application.In addition, following successful results in LP detection using a DSP as reported in (Musoromy et al., 2010) using WT, this work extends the use of WT in the LP character segmentation investigation of SD and HD images using a Texas Instrument's C64plus DSP with minimum of 600MHZ clock speed and 1MB of RAM (TI, 2006).

Image processing and ANPR using WT
This section gives a review of interesting ANPR algorithms using WT.The use of discrete wavelet transform (DWT) (described in Section 4.2) in ANPR is reported by Wu (Wu et al., 2009) in LP detection process.The methodology works by applying the "high-low" subband feature of 2D Haar DWT twice to increase the recognition of vertical edges while decreasing background noise in real world applications.The authors noted an increase in the ease of location and extraction of the license plate by orthogonal projection histogram analysis from the scene image in comparison with the vertical Sobel operator (a single level 2D Haar DWT) used in most License Plate Detection Algorithms.However, due to the downsampling used in this technique, it is only suitable for use with high-resolution images or cameras in close proximity to the plate (Wu et al., 2009).
An interesting algorithm is proposed by Roomi (Roomi et al., 20011) that consists of two main modules, one for the rough detection of the region of interest (ROI) using vertical gradients and another for the accurate localisation of vertical edges using the vertical subband feature of 2D discrete wavelet transform (DWT).This is followed by the identification of the orthogonal projection histogram for the extraction of the license plate.This method combines the advantage of relatively short runtimes whilst still maintaining accuracy, across a range of vehicle types.The authors reported that the number plates recognition accuracy was reduced where the plates were tilted (Roomi et al., 20011).
WT is also used in the simplification of skew correction in order to reduce computational demands to make the process suitable for real time applications (Paunwala et al., 2010).The method uses two levels WT to extract a skewed feature image of the original LP image, which is then transformed into a binary image from which the feature points can be identified by applying a threshold.These feature points help identify the angle at which the plate is tilted using principal component analysis, from which the correction to the whole plate image can be applied (Paunwala et al., 2010).
To conclude, the use of WT and the advantages are widely reported in the ANPR algorithms and therefore the focus of this chapter is the suitability of WT in HD images and DSPs for real time performance in LP character segmentation but firstly, LP detection process used in this work is summarized in the following section.

LP detection algorithm
The LP detection is the first part of an ANPR algorithm, which gives the rectangle region that contains characters.The plate detection algorithm used here is divided into four parts.These are input image normalization, edges enhancement using filters, edges finding and linking to rectangles using connected component analysis (CCA) and plate candidate finding (Musoromy et al., 2010).We have used the edge finding method in (Musoromy et al., 2010) to verify the presence of an edge.The edge finding method works by scanning the image and a list of edges is found using contrast comparison between pixel intensities on the edges' boundaries using the original gray scale image.The WT methodologies described by the authors in the literature above are mainly applied to LP detection process and benchmarked on baseline processors.In this chapter, we have expanded the use of Haar based edges in LP character segmentation algorithm.In addition, we have applied these edges in HD images and benchmarked their DSP and baseline processor performance to meet real-time requirement.

LP character segmentation algorithm based on 2D Haar WT edge detector
In image processing, edge detection is the key pre-processing step for identifying the presence of objects in images.This is achieved by identifying the boundary regions of an object.There are several robust edge detection techniques widely reported in the literature from early works by Canny (Canny, 1986) and some of the most recent, such as Palacios (Palacios et al., 2011).However, in custom applications, such as embedded ANPR system where both real-time performance and LP recognition success is demanded, a choice of good edge detector that balances these two factors is important.
The proposed algorithm is based on 2D Haar WT edge detector, which is shown to enhance image edges and improve LP region detection in Musoromy (Musoromy et al., 2010).The algorithm used for LP region detection and extraction explained in Section 3.1 is adapted to perform LP character segmentation.The main reasons for adapting the Haar WT for character segmentation are:


The ability of Haar WT to detect three types of edges using a single filter while traditional methods such as Sobel would require more than one mask for the operation  Simplicity of the algorithm and its suitability in real-time application The following sections describe the LP character segmentation algorithm based on a 2D Haar WT edge detector starting with the WT definition.

Wavelet Transform
In image processing, we can define a function f(x,y) as an image signal and Ψ(x,y) as a wavelet.A wavelet is a function of Ψ Є L 2 (R) used to localise a given function such as f(x,y) in both translation (u) and scaling (s).The family of wavelet is obtained by translation and scaling in time (t) using individual wavelet as given in equation ( 1) and ( 2) by (Mallat, 1999): Wavelets are useful in transforming signals from one domain to another, giving useful information for easier analysis hence the term Wavelet Transform which can be defined as: This represents a Continuous WT (CWT) of a function f at scales s>0 and translated by u Є R, which can also be explained as a 1D.When processing an image, we can apply this wavelet in the x direction where Ψ Є L 2 (R) as follows: The x and y directions can represent rows and columns of an image f(x,y) Є L 2 (R 2 ) and therefore we can also apply the CWT in 2D using wavelet Ψ Є L 2 (R 2 ) as (Palacios et al., 2011): We can rewrite equation ( 4) with dilation factor s as and The large number of coefficients produced by CWT makes it necessary to discretely sample signals in order to simplify signal analysis process and also for the use in real-time applications such as image processing.This process is technically known as discrete wavelet transform (DWT).

Discrete Wavelet Transform
Discrete wavelet transform (DWT) or fast wavelet transform (FWT) is a specialised case of sub-band filtering, where DWT is a sampled signal of size N using scale at 2 j s  for j < 0 and time (for scale 1) (Mallat, 1999).Using the wavelet equation: DWT is also a circular convolution where: The convolution of signal f and the wavelet is written as follows: Calculations of DWT is done using filter bank which can be a series of cascading digital filter.Implementing the DWT using filter banks entails the signal sampled being passed through high-pass and low-pass filters simultaneously to produce detailed and approximated confidents respectively (Qureshi, 2005).The high frequencies DWT are contained similar to equation (9) as follows: The low frequencies are contained in equation ( 12), in the computation of periodic scaling filter where the scaling function in equation ( 11) is sampled with scale z and integer k (Mallat, 1999).
The high-pass filter

 
HP hn is formed from the low pass filter   LP hn using the following equation (Qureshi, 2005): where h is the filter and N is the number of taps in the low-pass filter.If the length N of analysis low-pass filter is 4, and Applying equation ( 13), we obtain: To analyse DWT the input signal f (x,y) [n] is passed through both filters explained in equations ( 10) and ( 12) to give filtered output y[n].The output is then decimated or down sampled by a factor of two (Qureshi, 2005).Decimation means every other sample is taken from an input to form an output such that: The analysis of DWT with the resulting coefficients is shown in figure 1.The 2D DWT of an image function f(x,y) of the size M x N can be written using wavelet functions in equation ( 17) and ( 18) (Mallat, 1999) as follows: where i= {1, 2, 3}.
At the end of analysis stage, the transformed image can be reconstructed back to an original image or to a new image using the inverse of DWT (IDWT).The reconstruction is a process of upsampling the wavelet coefficients by a factor of two and passed through reversed lowpass ( LP g ) and high-pass ( HP g ) filters simultaneously (Qureshi, 2005).The reconstruction to an original image is demonstrated in figure 2.

2D Haar WT
There is a countless number of wavelets available in the wavelet family with more being reported in the literature of wavelets (Mallat, 1999).For this application, we are interested in the simplest but efficient DWT.The Haar is the first and simplest WT in the family of wavelets (Haar, 1911).Haar WT is derived starting with Haar wavelet function defined as: Its scaling function φ(x) can be defined as: The Haar matrix can be obtained using the wavelets defined in equations ( 17) to ( 20) and applying the formula in (10) to form high-pass filter from the low pass filter.The simplest Haar 2x2 matrix when N is 2 is as follows: and when N is 4 to give Haar 4x4 matrix as follows: The Haar WT filter can be derived by transformation, for example transforming 2 H to: The 2D Haar WT is computed similarly as shown in equations ( 14) to ( 17).The result of applying single level 2D Haar WT in an image is a decomposition of an image into four bands including a low-pass filtered approximation "low-low" (LL) sub image, which is the smaller version of the input image and three high-pass filtered detail subimages, "low-high" (LH), "high-low" (HL) and "high-high" (HH).The subbands and shown in figure 3 and the corresponding resulting images are shown in figure 4. In addition the images can also be discomposed using different levels with a series of cascading filter bank to produce a multiresolution (Mallat, 1989).

2D Haar WT based edge detector
The main advantage of applying 2D DWT such as Haar to an image is that it decomposes it to four sub images as seen in figure 4, which is mathematically less intensive operation and more suitable for our application.The suitable edges for our application are obtained by applying a 2D Haar WT (2x2) on an image f(x,y) to obtain high and low frequency subimages as shown by the following equation At this stage, the edges can be computed using reconstruction through the use of wavelet transform modulus of

 
LH HV dx , y and d( x , y) and then followed by the calculations of www.intechopen.comedge angles (Mallat, 1999).Alternatively, an estimate of the wavelet transform modulus of the horizontal and vertical components without taking into account the angle of the DWT as reported in (Qureshi, 2005).In this case, the wavelet modulus is compared to the local average.This is the approximation to the wavelet modulus maxima which is then compared to a global threshold dynamically calculated from the coefficients of the estimated modulus of the detail coefficients.
In our application, we choose to perform reconstruction on HV d( x , y) using inverse DWT (IDWT) using 2D Haar WT to obtain horizontal and vertical edges HV (E (x,y)) .This is computationally efficient on a DSP and it also provides enough edge details for our application.This process is shown in figure 5. Ex , y E( x , y)  and then post processing is applied to the edges to make them more prominent and inversion for optimal display is performed using an 8-bit dynamic range.Our application demands more edges and less noise therefore, an automatic thresholding method called autonomous percentile (P-tile) thresholding followed by histogram analysis (Qureshi, 2005).P-tile histogram thresholding is used here due to the fact that the texts inside the license plate region covers a known region 1/p of the total image.The threshold is automatically detected such that 1/p of the image area has pixel intensities less than some threshold T knowing that the text is dark and the background is white or the other way around, which is easily determined through inspection.Starting with the normalized histogram is a probability distribution: That is, the number of pixels n g having intensity g as a fraction of the total number of pixels n.The intensity level (c) of g is given as,  0 () Finally the threshold T is set such that  The results from reconstruction of the vertical and horizontal edges, absolute edges and prominent edges using single level decomposition and reconstruction are shown in figure 6 and figure 7 respectively.

LP character segmentation algorithm
The LP character segmentation process follows LP region detection as explained in Section 3.1.In this algorithm shown in figure 10, we segment the characters inside LP rectangle.The procedurals steps following LP detection include:  The Haar edges are used as a reference without further processing of the Haar edges like thinning; we apply the edges comparison algorithm explained in Section 3.1 and compare location where an edge is verified if a match is found.The flow chart is shown in figure 10.
The LP candidate has unique properties where the typical number of edges is between 100 to 2000 edges per plate.There are seven characters in UK LPs, a single character in a LP candidate contains between 30 to 150 edges, the gap of the character is between 2 to 4 pixels, the height of the character is about 20 pixels and width is about 16 pixels.This knowledge is applied to Connected Component Analysis (CCA) (Llorens, 2005) and a window (box) is drawn when a character is found.Finally, histogram analysis is applied to verify the presence of characters in a LP candidate.

Experimental setup
The proposed algorithms are optimized using similar experimental setup as reported in (Musoromy etal., 2010) and tested on Standard Definition (SD) and High Definition (HD) images that are a mixture of colour (day) and IR (night) with varying complexity levels such as over exposure, very dark and noisy.The proposed algorithm described in Section 4 forms a unified approach to resolve problems related to the above.The algorithm is implemented in DSP using the following tools: The implementation of Haar WT based edge detector is performed using a TI's DSP.TI provides an image library which has a unique implementation of the DWT through a highly optimised image columns transformation, which provides horizontal and vertical wavelet transform functions (TI, 2006).We apply reconstruction to the vertical and horizontal wavelet transform functions to obtain the edges.

Results
The main performance evaluation criteria for the proposed algorithm are average execution time and LP character segmentation rate as shown in Table 1.The results clearly show an improvement when 2D Haar WT is used especially in terms of the character segmentation rate, which is tested on 6000 images combining both image sets of SD and HD.It is also noted that the execution time for character segmentation is close for both SD and HD images due to similar LP candidate size but higher character segmentation rate is observed at higher resolution.It is observed that when using high resolution images and reduced number of wavelet decomposition (small scale single level in our case) the result is noisier and more discontinuous edges while at lower resolution and high number of wavelet decomposition have an opposite effect.This was also reported by Qureshi (Qureshi, 2005).In our application, the former effect leads to failed character segmentation due to "bad edges" while the latter improve character segmentation rate at an expense of losing speed for real time application as shown in our results in Table 1.In this case, a good balance between image resolution and wavelet decomposition levels is required.
In conclusion, in Table 1, two levels provide better character segmentation rate compared to a single level.However, the slower times is the downfall, therefore we choose decomposition at a single level that meet real-time requirement, which also gives a good character segmentation rate.
The difference between lower and higher decomposition levels around the LP region are demonstrated in figure 15 for a lower resolution image and similarly, in figure 16 decomposition levels for higher resolution image are shown using similar post processing edge threshold.The results clearly shows images at higher resolution performs better at lower decomposition levels.It is also noted in Table 2 that there is a small character segmentation success advantage in images taken at night compared to images taken in the day time.This can be explained due to the fact that at night, an Infra-Red (IR) camera is used to capture license plate which provides good images due to license plate's reflectivity to IR camera where the other objects in the background are not captured.
As well as the "bad edges", there are a number of factors that cause license plate character segmentation failure including;  Dirty due to mud or rain drops  Broken due to accidents  Non reflective to IR camera  Over exposure or uneven lit  Illegal against known rules such as seven characters per LP in the UK

Conclusion
It is demonstrated from the results that Haar based edges can be used not only to enhance image features but also to give an idea on where the objects of interest are located.The major advantages of Haar edges in LP character segmentation application are: ability to detect most edges in image, higher character segmentation rate on HD images, fewer noises (unwanted edges) when using the appropriate decomposition and threshold levels, and speed.
A licence plate algorithm under 40ms is capable of delivering 25 fps, which is in real-time and able to deal with vehicles moving at 70 miles per hour.Therefore, the results suggest that the proposed algorithm will work in real time with SD and HD images in both PC and DSP for embedded systems.
In conclusion, the methodology provides a unified character segmentation process that caters to number plates captured at any time of the day (both day and night), and also different types of noises existing in real World applications, low and high resolution images.
It is observed that higher character segmentation rate is at higher decomposition levels; therefore the future work will focus on further DSP optimisation methods for implementing higher level decompositions on both HD and SD images.

Fig
Fig. 5.A reconstruction of HV d( x , y) into

Fig. 6 .Fig. 7 .Fig. 8 .Fig. 9 .
Fig. 6.The original image is shown in (a) and the resulting image from reconstruction using single level IDWT is shown in (b)

Fig. 11 .
Fig. 10.The LP character segmentation algorithm based on Haar edges


A Windows host PC (2.4 GHz clock speed) with Code Composer Studio and a monitor acting as baseline processor  A Texas Instrument's C64plus DSP (fixed-point DSP based on an enhanced version of the second generation high-performance, advanced Very-Long-Instruction-Word (VLIW)) with minimum of 600MHZ clock speed and 1MB of RAM (TI, 2006)  DSP host board with a JTAG interface debugger to provide interface between the DSP and the host PC during debugging DSP algorithm  Testing database of 5000 images of 768X288 resolutions (SD) and 1000 images of 1394X1040 resolutions (HD) provided by CitySync Ltd (CitySync, 2011) Fig. 12.(a) The input LP Candidate (f(x,y)) and (b) the detected edges using 2D Haar WT (E(x,y))

Fig. 15 .Fig. 16 .
Fig. 15.The original license plate candidate of a lower resolution image 384x144 (a), one level (b), two levels (c), three levels (d), four levels (e) and five levels (f) decomposition of the original image Edge detection within the original LP region using 2d Haar WT  Edge detection through grayscale variation analysis using original image  Compare Haar edges with the grayscale variation analysis edges to validate the presence of edges as explained in Section 3.1

Table 2 .
Segmentation success rate for day and night images