Dual Tracking Method for Real Time Object Tracking Using Moving Camera

This study presents dual tracking method for real time object tracking using a moving camera. A real time object tracking using self aligning servo mechanism with webcam, dual tracking and effective localization of object is presented. The proposed dual tracking method works in two phases: In first phase tracking is done by joint color texture histogram with mean shift and in second phase tracking is done by servo setup. The proposed dual tracking method enjoys the benefit of double tracking feature, not only tracking but also to find out the coordinates of the tracking object which is of particular interest. The coordinates of a moving object enable us to estimates the real time location of the object which is helpful in surveillance and shooting purposes of suspected person in security area. The tracking of some specific objects in real life is of particular interest. Due to its enhanced automation the proposed dual tracking method can be applied in public security, surveillance, robotics and traffic control etc. The experimental results demonstrate that the proposed dual tracking method improves greatly the tracking area with accuracy and efficiency and also successfully find the coordinates of moving object.


INTRODUCTION
Object tracking is rapidly growing field in present scenario, there are two main reasons, first one is increasing number of suspicious persons, objects and complexity and second one is to make the things automatic in real life.It is aroused great interest in security areas, surveillance and many more.Real time object tracking system with moving camera is a system having servo setup mounted a CMOS camera on it with intelligent embedded circuit board (i.e., controller) and a personal computer; the system is capable to track human face or non human objects effectively.Today there is a large interest worldwide in development of object tracking systems for a number of security and monitoring tasks such as borderline surveillance, screening of specific object, track a terrorist activity and many others.
During last decade, many tracking algorithms have been proposed by various researchers in order to overcome the problems arising from noise, clutter, occlusion and changes in the foreground object or in the background environment (Yilmaz et al., 2006;Bradski, 1998;Comaniciu et al., 2003;Haritaoglu and Flickner, 2001;Nguyen et al., 2006;Yang et al., 2005;Ning et al., 2009).Among these different tracking algorithms (Yilmaz et al., 2006), mean shift tracking algorithms have recently become very popular because of their simplicity and efficiency.Originally, the mean shift algorithm is proposed for data clustering (Fukunaga and Hostetler, 1975).Later on it is introduced into the image processing community by modified mean shift algorithm and developed the Continuously Adaptive Mean Shift (CAMSHIFT) algorithm to track a moving face (Bradski, 1998;Cheng, 1995).Further, the mean shift algorithm is successfully applied to object tracking and image segmentation (Comaniciu et al., 2003;Comaniciu and Meer, 2002).The mean shift tracking algorithm is an iterative kernel-based deterministic procedure which converges to a local maximum of the measurement function with certain assumptions based on the kernel behaviors.Furthermore, mean shift tracking algorithm is a low complexity algorithm, which provides a general and reliable solution to object tracking.
Recently, a widely used form of target representation is the color histogram (Comaniciu et al., 2003;Nummiaro et al., 2003), which could be viewed as the discrete Probability Density Function (PDF) of the target region.The colour histogram is an estimating mode of point sample distribution and is very robust in representing the object appearance.However, using only colour histograms in mean shift tracking has two main difficulties (Yang et al., 2005) first one is the spatial information of the target is lost and second one is when the target has similar appearance to the background, then the colour histogram will become invalid to distinguish them.Therefore, for a better target representation, the gradient or edge features is used in combination with color histogram (Comaniciu et al., 2003;Haritaoglu and Flickner, 2001).In past years various object representations that exploit the spatial information have been proposed and developed by partitioning the tracking region into fixed size fragments (Adam et al., 2006), meaningful patches (Jain et al., 2006) or the articulations of human objects (Ramanan et al., 2007).For each subregion, a color or edge feature based target model is presented.The texture patterns (Gotlieb and Kreyszig, 1990;Sonka et al., 2007;Wouwer et al., 1999;Wu and Hsieh, 1992;Zhao and Pietikainen, 2007), which reflect the spatial structure of the object are effective features to represent and recognize targets very easily.Therefore, the texture features introduce new information that the colour histogram does not convey.Therefore, using the joint colour-texture histogram for target representation is more reliable than using only colour histogram in tracking complex scenes.The main idea behind combining colour and edge for target representation has been exploited by various researchers (Comaniciu et al., 2003;Haritaoglu and Flickner, 2001).However, how to utilize effectively both the color intensity and texture features is still a difficult problem because though many texture analysis methods, such as gray concurrence matrices and Gabor filtering have been proposed (Gotlieb and Kreyszig, 1990;Wu and Hsieh, 1992).These texture analysis methods have high computational complexity and cannot be directly used together with color histogram.Therefore, the local binary pattern (LBP) technique is very effective to describe the image texture features (Ojala et al., 2002(Ojala et al., , 2007)).The local binary pattern has advantages such as fast computation and rotation invariance.It has wide applications in the field of texture analysis, image retrieval, face recognition, image segmentation and etc. (Ojala and Pietikainen, 2000;Ahonen et al., 2006;Pietikainen and Zhao, 2000a;Pietikainen et al., 2000b;Savelonasa et al., 2008;Zhang et al., 2006;Heikkia and Pietikainen, 2005).
During past years, local binary pattern is successfully applied to the detection of moving objects via background subtraction (Heikkiaand and Pietikainen, 2005).In local binary pattern, each pixel is assigned a texture value, which can be naturally combined with the colour value of the pixel to represent targets.In Nguyen et al. (2006), it is employed the image intensity and the LBP feature to construct a twodimensional histogram representation of the target for tracking thermo graphic and monochromatic video.Recently, robust object tracking using joint colourtexture histogram is proposed by Ning et al. (2009).In this algorithm the main tracking methodology is a single tracking.This tracking methodology restricts us to track objects in entire region, it can be shown that the current technologies are adequate and fast for real time object tracking in limited area, but it hard to find out coordinates of the moving object (i.e., target) with this single tracking.Therefore in order to overcome these drawbacks a dual tracking method is proposed for an optimizing object tracking and coordinates finding.The proposed tracking system comprises of a servo system having two dc motors with shaft encoders and control circuits for each motor; servo controller is connected to microcontroller circuit board which is further connected to the computer.The functionality of servos is decided by the computer after process the current frame from CMOS camera.The controller continuously sensing the coordinates of moving object and sending it to computer via RS-232 port.At the computer the processing is done in two phases in first phase joint color texture histogram with means shift algorithm for object representation and in second phase proposed algorithm is used.Therefore, the proposed dual tracking method provides better object tracking as compared to other tracking methods.

REVIEW OF MEAN SHIFT TRACKING ALGORITHM WITH JOINT COLOR-TEXTURE HISTOGRAM
Target representation: A target is usually defined by a rectangle or an ellipsoidal region in the image.Most existing target tracking schemes are used for color histogram to represent the rectangle or ellipsoidal target.First let us review the target representation in the mean shift tracking algorithm (Comaniciu et al., 2003).
Let us denote {  * } =1…. the normalized pixel positions in the target region, which is supposed to be centered at the origin point (Ning et al., 2009).The target model  � corresponding to the target region is computed as:  (Ning et al., 2009) and constant C = A normalization function defined by: (2) Similarly, the target candidate model ˆp(y) corresponding to the candidate region is given by: where, ̂ () represents the probability of feature u in the candidate model (), {  } i=1……..nh denote the pixel positions in the target candidate region centered at y, h is the bandwidth and constant C h is a normalization function which is defined by: In order to calculate the likelihood of the target model and the candidate model, a metric based on the Bhattacharyya coefficient is defined between the two normalized histograms () and  � as follows (Ning et al., 2009): (5) The distance between () and  � is then defined as: Mean shift tracking: Minimizing the distance between () and  � defined in Eq. ( 6) is equivalent to maximizing the Bhattacharyya coefficient in Eq.
(5).The iterative optimization process is initialized with the target location  0 in the previous frame (Ning et al., 2009).By using Taylor expansion around ̂ ( 0 ) the linear approximation of the Bhattacharyya coefficient defined in Eq. ( 5) is obtained as: where, Since the first term in Eq. ( 7) is independent of y, to minimize the distance in Eq. ( 6) is to maximize the second term in Eq. ( 7).In the iterative process, the estimated target moves from y to a new position y 1 , which is defined as: when we choose kernel g with the Epanechnikov profile (Bradski, 1998) in Eq. ( 9) is reduced to: By using in Eq. ( 10), the mean shift tracking algorithm finds in the new frame the most similar region to the object.For more information about color histograms based target representation and mean shift tracking (Comaniciu et al., 2003).

Target tracking with joint color-texture histogram: Local Binary Pattern (LBP):
The Local binary pattern operator labels the pixel in an image by thresholding its neighborhood with the center value and considering the result as a binary number (i.e., binary pattern) (Ojala et al., 2002(Ojala et al., , 2007)).The general version of the LBP operator is defined as follows: where,   corresponds to the gray value of the center pixel (  ,   ) of a local neighborhood and   to the gray values of P equally spaced pixels on a circle with radius R (Comaniciu et al., 2003) .The function s (x) is defined as follows: The texture model derived by Eq. ( 11) has only gray-scale invariance.The grayscale and rotation invariant LBP texture model is obtained by Ojala et al. (2007): where, The superscript "riu2" means that the rotation invariant "uniform" patterns have a U value of at most 2. By definition, the P+1 "uniform" binary patterns occur in a circularly symmetric neighbor set of P pixels.Equation ( 13) assigns a unique label to each of them corresponding to the number of "1" bits in the pattern (0 to P), while the "non-uniform" patterns are grouped under the "miscellaneous" label (P + 1).

Target representation with joint color-texture histogram:
A limitation of LBP is that it does not work robustly on flat regions where the gray values have small fluctuations.In order to make LBP more robust against these subtle changes in pixel values, Heikkia and Pietikainen (2005) modified the thresholding strategy in the LBP operator by replacing the term �  −   � in Eq. (11, 13 and 14) with �  −   + .The greater the value |a|, the higher fluctuations in pixel values are allowed without affecting much the thresholding result.In this study, we adopt this modified thresholding method and employ LBPriu2 (Fukunaga and Hostetler, 1975;Adam et al., 2006) to describe the target texture features because of its low computational complexity.With the above analysis, we calculate the LBP feature of each point in the image region, whose value is between 0 and 9. Thus an appearance model combining the color and texture is constructed and it consists of color channel and LBP texture pattern.However, compared with the usual color based target representation, this direct combination does not enhance much mean shift tracking.Especially, when the target is very similar to background, representing the target by using color and LBP texture features on the whole tracking region is hard to distinguish them due to the strong interference from the background.Therefore, a demand exists to find a new way to combine the color and LBP texture features more effectively.The LBPriu2 (Fukunaga and Hostetler, 1975;Adam et al., 2006) model has nine uniform texture patterns (Ojala et al., 2002).Each of the LBPriu2 (Fukunaga and Hostetler, 1975;Adam et al., 2006) uniform patterns is regarded as a microtexton (Chen and Wang, 2005).The local primitives detected by the LBPriu2 (Fukunaga and Hostetler, 1975;Adam et al., 2006) model include spots, flat areas, edges, line ends, corners and etc.
In target representation, the micro-textons (Chen and Wang, 2005) such as edges, line ends and corners, by name of "major uniform patterns", represent the main features of target, while spots and flat areas, called "minor uniform patterns", are minor textures.Thus, we extract the main uniform patterns of the target by the following equation: ,  � , � ≤ 2 and In LBPriu2 (Fukunaga and Hostetler, 1975;Adam et al., 2006) the labels corresponding to minor uniform patterns are 0, 1, 7 and 8 respectively and the label of non-uniform patterns is 9.The labels corresponding to main uniform patterns are 2-6, which have five patterns.Equation ( 15) groups the minor uniform patterns as non-uniform patterns.Generally, the main LBP features of a target are more important than its minor features to represent the target.Therefore, by Eq. ( 15) we extract only the pixels corresponding to the main LBP features and then use the color and texture features of these pixels to model the target.That is to say, we first use Eq. ( 15) to form a mask and then use the color and LBP features within this mask to model the target appearance model.

The tracking algorithm with the joint color-texture histogram:
We use the RGB channels and the LBP patterns extracted by Eq. ( 15) to jointly represent the target and embed it into the mean shift tracking framework.To obtain the color and texture distribution of the target region, we use Eq. ( 1) to calculate the color and texture distribution of the target model q, in which u = 8×8×8×5.The first three dimensions (i.e., 8×8×8) represent the quantized bins of color channels and the fourth dimension (i.e., 5) is the bin of the modified LBP texture patterns by Eq. ( 15).Similarly, the target candidate model (y) is calculated with Eq.

Algorithm of mean shift tracking algorithm with joint color-texture histogram:
Input: the target model q and its location y0 in the previous frame (Ning et al., 2009).
Step 2 : In the current frame, calculate the distribution of the target candidate model (y0).
Step 4 : Calculate the new location y1 of the target candidate using (10).
Step 6: Load the next frame as current frame with initial location y0 and go to step (1).

PROPOSED DUAL TRACKING METHOD
The proposed dual tracking method is divided into three main sections; image acquisition and tracking section, controller section and processing section.Acquisition and tracking section includes the CMOS Camera, shaft encoders, motors and its control system.Controller section is to decode the shaft encoder to calculate coordinates and establish communication with computer via RS-232 port.And finally the processing section of the system which process the image and find center of mass of tracking object.It generates signal to move motors for frame adjustment, decodes the signals from controller board and also verify integrity.
The functionality of this system is based on crisp rules and mean shift algorithm.The computer acts as Fig. 3: Frame planning the master and computer terminal used for data processing and target acquisition.The servo setups are controlled by the controlling signal generated by computer after process the current frame.The system is equipped with two dc motors, shaft encoders, motor controllers, main controller board, power unit, USB to serial converter and a CMOS camera.The whole controller section is controlled by an Atmega 162 microcontroller through automation code which is written in embedded C and commands from the computer section.The coordinates of tracking object is calculated by controller board.It is done by continuously sensing the shaft encoders.The system operational block diagram of the object tracking system (Fig. 1) the coordinate's measurement of the tracking object is shown in Fig. 2, frame planning is shown in Fig. 3 and object tracking flow chart is shown in Fig. 4.  Camera is able to capture an image or video at a frame rate of 30fps.Images acquired from Camera can be access by the PC through USB port.The moving target system consists of the moving target and a workspace.
Controller section: An Atmega162 based controller section is used and is interfaced with servo setup and computer.This control section can process data at 16MIPS; it has a flash of 162KB, 512 bytes EEPROM and 1K SRAM.USART is used to establish the communication with computer via MAX232, PWM channels are used for servo setup controlling and ADC channels 0,1 are used for position sensing from shaft potentiometer.
Servo setup: It consists of a RF-020TH DC motor with 9580rpm and 3.93g-cm torque at maximum efficiency, gear system is used to increase torque, reduce rpm and to provide feedback to a position encoder and control electronics adequate pulse width to voltage converter and error amplifier.

Power unit:
The power section has battery, voltage level sensing circuitry; auto cut system and over load protection.Power unit is capable to provide 12, 5 and 3.3 voltage; it could bear load up to 2A.

Coordinate finding:
The coordinate calculation is done after decoding the shaft encoder θ(theta) could range from 0 to 360 degree and Ф(phi) could range from 0 to 180 degree, this range is tested in many real time scenarios and show very good results.

Dual tracking method:
The dual tracking technique with frame adjustment is being implemented to maximize the target workspace area.In the given system, a current frame is divided in eleven sections; upper U, center square, lower U, first quadrant, second quadrant, third quadrant, fourth quadrant, left extreme line, right extreme line, upper extreme line and lower extreme line as shown in Fig. 3.
The complete tracking of object is done in two phases, in first phase tracking is done by mean shift algorithm and then on the bases of center of mass (i.e., location) of tracking window the algorithm comes in picture and adjust the tracking window to center of the frame which is shown in Fig. 4.
The controller board connected to computer via rs232 port.The controller passes the coordinates of the current location of the target to the computer via serial RS232 port.These coordinates are calculated after sensing the shaft encoders of two motors.

EXPERIMENTAL RESULTS AND DISCUSSION
The experimental results demonstrate that the tracking system presented in this study can be used for surveillance, localization and increases the area of target workspace.The experimental results illustrated that proposed dual tracking method provided better object tracking as compared to single tracking method and coordinates of tracking object is calculate successfully.Experimental results of single tracking method: The experimental results of single tracking method are shown from Fig. 5 to 10 with a non frame adjustment single tracking.In this phase joint color-texture model based mean shift tracking algorithm M3 (Ning et al., 2009) is used to extract face tracking results.
Experimental results of proposed dual tracking method: Figure 11 to 20 shows the tracking results with dual tracking method, it is easy to notice in all the frames the tracking window is adjusted to center position of the frame.This technique helps enormously to increase the tracking area of the target.The coordinates of tracking object is tabulated in Table 1.

CONCLUSION
This study proposed new tracking method for real time object tracking with moving camera.The proposed method adopted the local binary pattern scheme to represent the target texture feature and then a joint colour texture histogram method for a more distinctive and effective target representation.The proposed dual tracking method eliminated smooth background and reduces noise during tracking process.The experimental results of proposed dual tracking method was evaluated and compared with single tracking method.The experimental results clearly revealed that proposed dual tracking method was effective for real time moving object and also suitable for findings the coordinates of object in terms of θ and Φ. Future research will be devoted to find the distance of moving object from the camera as well as further improving image segmentation and edge-based feature extraction methods.
probabilities of feature u = in target model  � m = The number of feature spaces δ = The Kronecker delta function (  * ) = Associates the pixel   * = The histogram bin k(x) = An isotropic kernel profile

Fig. 1 :
Fig. 1: System operation block diagram of proposed dual tracking method

Table 1 :
Coordinates of tracked image