Adaptive Segmentation Method for 2-d Barcode Image Base on Mathematic Morphological

Segmentation is a key process of 2-D barcode identification. In this study we propose a fast adaptive segmentation method that is based on morphological method which is suitable for kinds of 2-D barcode images with different scale, angle and sort. The algorithm is based on mathematical morphology, the basic idea of the algorithm is to use Multi-scale open reconstruction of mathematical morphology to transform the image continuously, then choose whether to terminate by the results of the adjacent image transformation and finally get the final segmentation results by further processing of the images obtain from termination.The proposed approach is applied in experiments on 2-D barcodes with complicated background. The results indicated that the proposed method is very effective in adaptively 2-D barcode image segmentation.


INTRODUCTION
Among the wide range of automatic identifications such as RFID, magnetic card, barcode is becoming increasingly popular in recent years.There are 2 basic types of barcode: 1-D (1-Dimension) and 2-D.1-D barcode stores information only in horizontal direction.While 2-D barcode stores information in both the horizontal and vertical directions using organized blanks and bars, so 2-D barcode can effectively solve defects such as low information capability, low information density, poor stability and safety that exist in 1-D barcode (Kato and Tan, 2007).Therefore, 2-D barcode is attracting more and more attention from research to industry community, such as electronic tickets, product labels and so on (Hu et al., 2010).
The key point to 2-D barcode identification lies in the processing of barcode image.Images acquired by camera often contain a lot of complex backgrounds.These complex backgrounds bring great challenges to 2-D barcode identification.In order to get a high barcode identification rate, we must extract code image from complex backgrounds.
2-D barcode image segmentation technique has always being receiving great attention in recently decades and a lot of segmentation algorithms have been proposed.Sun demonstrates a progressive-scan segmentation method that could find the feature patterns which conform to corresponding relationship of scales through scanning the whole object (Hahn and Joung, 2002;Sun et al., 2009).That main weakness of this method is what it is sensitive to noises and the segmentation accuracy is low.Muniz et al. (1999) makes use of the Hough transformation to complete the image segmentation.However, this global-based method takes too much computation and the accuracy is not high (Muniz et al., 1999).Bai et al. (2008) uses the texture of 2-D barcode and a group of Gabor filters to acquire images' Gabor features and the result is used to complete discrete Fourier transformation.At last selforganizing map is employed to fuse the images.This method can get a good segmentation result, but it takes too much computation (Bai et al., 2008).Parikh and Jancke (2008) divides image into four equal areas, then segments the image based on gray balance.This method is suitable for 2-D barcodes with simple background.When dealing with images with complex background this method seems unpractical.Xie et al. (2008) takes use of opening and closing operation in MM (Mathematical Morphology) to filter and detect barcode image.Liu et al. (2010) takes use of MM to segment gradient transformed images.This kind of method is proven to be effective most of the time, but it needs different SEs choices according to different 2-D barcodes, so it is not suitable for auto image recognition.
MM is a well-known technique used in medical imaging, material sciences and computer vision (O'Callaghan and Bull, 2005;Plissiti et al., 2011).The essence of MM processing is to take use of SE.
(Structure Element) with certain scale and shape to detect image and extract features relative to SE.The shape and the size of the SE play important role in detecting or extracting features.As we cannot acquire any prior knowledge about image under processing in unsupervised image processing domain, choosing suitable SE becomes a problem.In recent years, researchers have tried various ways to solve this problem.Santos et al. (2005) proposed a system for automatic luminal contour segmentation, it simply use Morphological filtering to get solution points obtained by linear scanner as the edge of the image, so this method cannot be applied in complex background systems.Bosworth and Acton (1999) proposed a general multi-scale mathematical morphological segmentation method but it requires some prior knowledge of the image.Mukhopadhyay and Chanda (2003) proposed a method of segmenting gray level images using multiscale morphology, this method gets a good segmentation image but the CPU time and memory space requirement are very high.
In this study we have proposed a novel method for segmenting gray-level 2-D barcodes on complex background using multi-scale MM reconstruction.Also, some operators to segment image are introduced.At the heart of our method is motivated by the study of Mukhopadhyay and Chanda (2003).The main difference between our approach and his is that the Segmentation result obtained by his is the integration of segmentation results of various scale and the results of our segmentation is only simple processing of the last results without Retention of variety of intermediate results, which greatly reduce the system's storage space and improve the efficiency of processing.Our method is composed of 2 passes preceded by a preprocessing step for filtering what can cause over-or undersegmentation.In the 1 st pass, it Use multi-scale morphological opening reconstruction in the pretreatment images and obtain rough segmentation results.In the 2 nd pass, it gets the final segmentation results by simple MM transform of the results of the first process.

THE BASIS OF ALGORITHM
A multi-scale MM: MM which is an extension of Murkowski's set theory was first systematically examined by Matheron and Serra in the 1960s (Bosworth and Acton, 1999).It is a well-known technique used in image processing and computer vision.Morphological operators were consisted of dilation, erosion, opening, closing and other derived transformer.
It is well known that the erosion and dilation is a pair of dual operators.The result of the erosion operation to an image shows where the SE fits the objects in the image.In gray scale, eroding an image f by SE B is defined as: The result of the dilation operation to an image shows where the SE hits the objects in the image.The dilation is denoted and defined as: The opening operation performs erosion 1 st , followed by dilation; while the closing operation performs dilation 1 st and followed by erosion (Vincent, 1993).The idea behind opening is to dilate an eroded image in order to recover the eroded image as much as possible.In contrast, the closing is to recover the dilated image possibly: Though the SE takes care of the shape of the features while processing the image, it cannot, however, treat objects of same shape but of different size equally.
A SE along with its higher order homothetic can process the image features based on not only shape but also size.This operation is termed as multi-scale morphology (Pastore et al., 2007).Multi-scale opening and closing are defined, respectively, as ( 5) and ( 6): where, n is an integer representing the scale of the SE, The n-th homothetic of a SE is obtained by dilating recursively times with itself as: In MM, once an image was eroded, there is no perfect reveres transformation to recover original image.Opening operation is to some certain degree recover original image using dilation.After opening operation, image edge would be blurred.Compared with MM opening operation, opening reconstruction can recover graphics which were not completely erased by erosion.Opening reconstruction is to reconstruct the dilation of eroded image, whereas dilation reconstruction is to repeatedly dilate bounded image until stable morphology transformation is obtained.

Geodesic dilation involves two images:
Mark image ƒ and mask image g, ƒ≤g and the domain D ƒ = D g , geodesic dilation of mark image ƒ relative to mask image g can be expressed as δ (1)  g (ƒ) when the scale value is 1.Thus geodesic dilation can be defined as the point by point minimal value of the basic dilation operation δ (1) (ƒ) between mark image and mask image: When the scale value is n, geodesic dilation of mark image ƒ relative to mask image g can be realized by continuously performing n times geodesic dilations on ƒ relative to g: Dilation reconstruction can be expressed as R δ g (ƒ), it is defined as geodesic dilation of ƒ relative to g until stable.When In ( 10), i represents cycle number.When scale is n, opening reconstruction filtering are defined as Cowan and Mavor (1980): Because every geodesic dilation result under each scale should get minimal value with mask image, however, for condition dilation, we only need to use dilation result of mark image under n scale to get minimal value with mask image.Thus condition dilation can reduce computation while keeping a relative good accuracy.Condition dilation δ g (n) (f) can be expressed as:

A the basis of corrosion image restoration of MM:
A digital grid non-directed graph ζ is the combination of vertex v and lines between vertex: is the set of neighborhood gray values of vertex v which are equal or greater than neighborhood points of v: If every route in the sub region ∏ of graph ζ can be connected using points in ∏ , we can say that ∏ is connected.Based on the relationship between neighbor points, digital graph can be divided as connected and not connected.
In gray image processing, image could be segmented into a series of image combinations based on specific gray values and the difference between connected subset areas: the differe region areas (h     In real-time 2-D barcode identification, the number of barcodes which can be identified per sec is mainly decided by the image processing time.For the split of the 2-D bar code processing, we divided Multi-scale open reconstruction into rough extraction and fine extraction, preprocessed image is taken as the input.The flow of algorithm is shown in Fig. 6.The central algorithms in Fig. 6 described in detail and the analysis results of it can list as follows: 1. Image preprocessing: To reduce the effect of undesired perturbations which might cause over-or under-segmentation, we use algorithm for denoising (Toet, 1990) and then use the multistructure elements morphology to light balanced.Similar approach may be found in Xu et al. (2008): .( ) 2. Simple binary for image: In the transformation of MM, grayscale image processing takes more time than binary image processing.In order to further reduce timing cost, we perform a simple binaryzation in rough extraction process.That means we firstly implement binaryzation on eroded image and then perform condition dilation.We take 1/2 of the sum of maxim and minimal gray value as the threshold, all pixels which are bigger than threshold will be set to 255 and pixels which are less than threshold will be set to 0: .( ) If skipping binaryzation and performing gray computation, the time complexity of each pixel would be O (n 2 ) using typical bubbling method.Even using fast sorting algorithm the time complexity would also be as large as O (nlog 2 n ) and the result is unstable.Implementing binaryzation 1st would cut time complexity down to O (n), thus reducing timing cost.This simple binaryzation would make image details fuzzy, however, at this step we just roughly mark the region 2-D barcode located and don't implement fine extraction, so this binaryzation will not affect the final result.At the same time, comparing with gray image, binaryzation can increase contrast of image and facilitate image extraction.

Corrosion of the image: Using disc SE B to erode
image and get eroded gray image: 4. Expanding the scale of disc SE and implementing dilation on eroded image: As opening is non-expand operation, when i≤j, f°iB≥f°jB, that means as the increase of the scale of SE image tends to shrink.If n→ ∞, then f°nB→0.In practical application, value of n is decided by adjacent multi-scale reconstruction.As long as the SE scale under value of n is bigger than the value of the biggest connected region in image, erosion in opening operation would erode image to an empty set.To avoid image shrink in opening operation, scale of SE should be bigger than erosion scale in condition dilation so that noise could be erased while regions where original image located remain as much as possible.When the scale of dilation SE is one pixel larger than the scale of erosion SE, flat region in object area would not shrink, but the computation is still intensive and could not make sure that regions with sudden change would be recovered.To improve speed, we can increase the size of the structural elements, but certainly dilation SE should not be too big because the dilation result might take over the whole image and all image regions would be marked out.Through experimenting on various 2-D barcode like QR, PDF417, we get that the scale of dilation SE should be less than 10 times of the scale of erosion SE. in this study, we choose 7 times: 5. Taking original image as mask and limiting dilated image under original image, in this way we can roughly mark out the region of 2-D barcode: In original multi-scale opening reconstruction, if the maxim scale of multi-scale opening reconstruction loop is m, then condition dilation need to process only once.For an image with N×N resolution, computation time complexity is much less than the first algorithm: We can appropriately expand the scale of iB then use it to do an opening operation: By now, object has already been extracted.Because we choose disc SE in typical opening operation and it will rounding the vertex, so we still need to do some job to realize fine extraction.Taking image f n as mark image and preprocessed image f b as mask image, we perform geodesic dilation until stable.As image f n slightly smaller than 2-D barcode object, image would be quick to stable.Finally we obtain the final extraction 2-D barcode result.

ALGORITHM VERIFICATION
According to different kinds of 2-D barcode, this study employs the most popular PDF417, QR code to evaluate the proposed approach.Experiments mainly focus on verifying proposed approach's adaptive capacity on different barcode's scales, tile angles and textures.
Verifying proposed approach's adaptive capacity on different barcodes' scales: In MM, SE plays a key role.The processing result would be degenerated with the decrease of match degree between SE and concerned features.In order to obtain a good processing result, SE is usually chosen according to the priority knowledge of  From the results, we can see that proposed algorithm can adaptively process different kinds of 2-D barcodes.This indicates that proposed algorithm has a good adaptive capacity on different kinds of 2-D barcodes.

CONCLUSION
In this study, a robust and efficient segmentation for 2-D barcode images of different shape and size is presented.The methodology builds on existing work, but extends it to achieve efficiency and robustness.This method consists of three parts.Firstly, it preprocesses image to filter noises and normalize illumination.Secondly, by using of the multi-scale open reconstruction we define the algorithm roughly marks out the located region of barcode image.In order to improve the processing efficiency of the algorithm, we also use a number of optimization methods for the 2-D bar code.Thirdly, it implements opening operation and geodesic dilation on the region obtained in the 2 nd part to get precise region of 2-D barcode.The presented techniques have been evaluated by using these techniques to process several 2-D barcode images.Results show that this method has a good adaptability and can automatically and accurately segment different kinds of 2-D barcode with different scales and tilt angles under complex background.The next research will mainly focus on optimizing and reducing storage space, changing the termination condition of the multi-scale reconstruction and improving the computational efficiency.
12) Features of 2-D barcode: There are various kinds of 2-D barcodes today.The 2-D barcode image consists of both the bright and dark features at varying scales.According to different shapes of image, 2-D barcodes can be classified as stack code and matrix code.Most commonly used stack code includes PDF417, code49; most commonly used matrix code includes QR, Data Matrix.In Fig. 1, 1a is a PDF417 barcode; Fig. 1b is a QR barcode.As is shown in Fig.1, barcode image mainly consists of feature codes and information region.In some high versions, barcode images might have calibration and position information.Feature code is mainly used to identify different kinds of 2-D barcode and feature code takes the largest connected region in the barcode image; information region consists of blanks

Fig. 1 :
Fig. 1: PDF417 barcode and QR code and spaces, which represents information included in the 2-D barcode.In order to facilitate image identification, every barcode specifications specify that there should be a circle of white blank outside barcode information region.Proposed algorithm: In this section, we first describe the two basic methods and principles of corrosion image restoration of MM and then export our propose Multiscale open reconstruction by the related image processing method.Finally, Taking Image processing efficiency into consideration, we improve the steps of Multi-scale open reconstruction and obtain our corresponding algorithm.

Fig
Fig. 4: T n In image shown noise w We images bar cod size of bar cod connec connec pretreat connec distribu In marked number code im noise a As is sh n and connec n+200.reconst (n R 

Fig. 5 :
Fig. 5: Flow of multi-scale opening reconstruction algorithm 29) 6. Comparing rough extraction result with the last extraction result, if they were not the same, then select the SE (n + 1) B→B and jump to step (3), otherwise terminate the rough extraction algorithm.7. Fine extraction of barcode region: After rough extraction of barcode region, noises may still exist at edge of the barcode image.Using typical opening operation can erase those noises but it is hard to decide the scale of SE.Multi-scale method could be used to deal with different scales and different kinds of image, but it takes a lot of time.Multi-scale opening reconstruction in rough extraction already provides us with the SE iB which image gets stable.

Fig. 7 :
Fig. 7: Segmentation results of barcode images with different scales Figure 8a and d are images with different tile angles.Figure 8b and e are images processed by proposed algorithm.Figure 8c and f are images which were extracted from original images with corresponding coordinates of segmented images.From the experimental results we can see that proposed algorithm can adaptively process different tile angles.This indicates that proposed algorithm has a good adaptive capacity on different barcode's tile angles.Verifying proposed approach's adaptive capacity on different types of 2-D barcodes: In MM, processing image needs to choose different SEs according to different textures.Textures in 2-D barcode include image signature, size of connected areas of data code and difference between shapes and scales.According to shape, 2-D barcodes can be classified into stacked code and matrix code.Stacked 2-D barcode is stacked by certain kind of 1-D barcode and different kinds of 1-D barcode can form different kinds of 2-D barcode; matrix 2-D barcode consists of matrix.Different types of 2-D barcodes own different textures.Figure9ais PDF417 barcode image, Fig.9dis matrix QR barcode image.Figure9band e are the processing results of original images, Fig.9cand f are images which were extracted from original images with corresponding coordinates of segmented images.