Video Serial Images Registration Based on FBM Algorithm

: In this study, we have a research of the video serial images registration based on fbm algorithm. In order to overcome low contrast and complex distortion and limited field of view, computer image processing technology must be used to video serial images for obtaining the image. It is hard to find and evaluate the change in the scene which appeared between the aeronautic consecutive image acquisitions to mosaic. So registration is important to successful image mosaic. A matching method based on the improved FBM algorithm can get good image fusion and image registration which is introduced for attaining more precise aggregate of matching points. In order to estimate the fundamental matrix which encapsulated the whole geometry accurately and robustly, an improved SVD decomposition with weighted normalized fundamental matrix calculating method is proposed. By using geometric invariant, cross ratio to rectify the matching points, the rectification is realized automatically and coarse error is reduced effectively.


INTRODUCTION
The panoramic mosaic of video serial images is an active area of research in the fields of photo disposing, computer vision and image processing and computer graphics.Mosaic is one of the techniques of image processing and image analysis which is useful for tiling video serial images to get a big field view.Registration failure between two contiguous frames of sequence always results in the defeat of the whole work.The main reason of failure lies in the mismatching and erroneous accumulation of registration errors.
These images represented in this research were obtained from robot flying flat in advance.Because of the air resistance, air buoyancy, gravity and Magnus force, the camera lens will change its places in the 3-D space.Moreover, along with the swing of lens, its change space will be more complex.Rotation, scale and translation correspondingly exist in the obtained serial images.And some geometric distortions will exceed the scope of the image deformation adaptation.There are no sufficient overlapping features to registrant.Therefore geometric correction must be implemented to reduce the distortion effect.Congli (2006) have a research of the elimination of pseudo matching in image stitching.Brabara and Jan (2003) study the image registration method survey.Wang (2002) shows the image moment functions-theory, arithmetic and application.Pratt (1991) studies the digital image processing.Leng (2005) have a research of the design and implementation of image stitching application system.Chen (2002) study the linear algorithm with high accuracy for estimating fundamental matrix.
In order to consummate the mosaic work, a novel strategy is presented to increase the registration precision.Based on the region feature and gray correlation matching, we carry on a counterpart point adjustment by the projection geometry invariants.The epipolar geometry is independent to the scene and it is unique, continuous and order consistent.So it can be used to remove the abnormal matching points and prevent the wrong pairs from participating in the calculation of transformation parameters for getting more precise relative position of two frames and improving the mosaic accuracy ulterior.
This study proposes a series of corresponding algorithm aiming at the low precision of registration which results in mosaic defeat of serial image, realizes multi-frame high precision mosaic and develops the mosaic technology in application.An exceptional point removing strategy based on optimal algorithm for estimating the fundamental matrix is presented in this research.At first the exceptional points are eliminated and the initial value of the fundamental matrix is obtained, Secondly, a criterion of minimizing the initial fundamental matrix.It can be seen from the experimental results that this algorithm eliminates the unfavorable effect of exceptional points on the fundamental matrix.Therefore it is more precise and has an advantage over other conventional algorithms in terms of both average residual error and average epipolar distance.A new robust linear method weighted normalization algorithm is developed by inducing a cost function related to residual errors.Experiments on simulated and real image data are conduced and the results show that this algorithm can enhance the accuracy and it is robust and easy to perform.

INIIAL REGISTRATION
Area based method: The area-based method (ABM) introduced in this research, called correlation-like methods or template matching merge the feature detection step with the matching part.Windows of predefined size or even entire images are used for the correspondence estimation in the second registration step.
ABM registers these images preprocessed by 2-D Fourier transform, edge sharpening and image enhancement.This method generally needs not to be implemented with complex pretreatment, but it measures one image using the gradation value of the image itself and other statistical information.Compared the window statistical features of matching image with the reference one, the gray correlation similarity measure of region can be employed.Classical ABM like cross-correlation exploit for intensities of matching images directly, without any structural analysis.Consequently, they are sensitive to the intensity changes, introduced for instance by noise, varying illumination, or by using different sensor types.

Feature based method:
In contrast to the area-based method, the feature-based one (FBM) does not work directly with image intensity values.The features represent information on higher level.This property makes FBM suitable for situations when illumination changes are expected or multi-sensor analysis is demanded.Features from the reference images and matching images with the most similar invariant description are paired as the corresponding ones, FBM is typically applied when the local structural information is significant and it is allowed to register images of complete different nature and can handle complex between-image distortions.

Mutual method:
• The first layer: feature based matching: At first, matching features should be chosen.The image shot by robot flying flat has low quality and the edges are indistinct, so it does not adapt for registration with line and edge features.
Region features are better than point, line and edge ones for scene adaptability.Region has geometric shape and pixel gray features.The disadvantage of region features is inaccurate of boundary definition, so it causes the low precision of registration.The first registration is to get an initial matching position.The next registration step can cover the shortage of region registration.
The literature (Wang, 2002) proposes many image feature functions which can be used as registration method based on the region feature convergent to center using area, Legendre moment, Zermike moment, Fourier-Mellin invariant, perimeter, semi-major axis, semi-minor axis, moment, shape parameters, eccentricity ratio, ellipticity, adjacent region relationship.
The classical representative of the ABM is the normalized cross-correlation: where, a il and b jl (l = 1, 2…k) are feature properties of region A i and B j .The cognominal matrix of feature regions is defined as follows: (2) The mapping relationship of matching image and reference one can be got by using the cancroids of a pair of cognominal regions as the control points.
• The second layer: area based matching: The second layer matching can be divided into three steps: gray correlation computation, local extremer searching and threshold judgment.
By taking the center of feature regions as the control points, the cognominal pairs in the first layer registration are reduced.If the point a(x a , x a ) of the reference image f 1 (x, y) and the point b(x b , x b ) of the matching one are cognominal pairs, they will be the control points.The correlation coefficient is measured by: While, m is the ration of local matching window, µ 1 is the local mean of f 1 (x, y) which is neighboring a(x a , x a ) µ 2 has the similar expression to µ 1 .
After the ABM and FBM, the precision of matching will be improved, but error still exists.

Final registration:
The projection geometry has geometric invariance and invariants.And among them, the cross-ratio has rotation, translation, scale and projective invariabilities.
The projective transformation has two important features: The projective transformation maps point to point, straight line to straight line, track group to track group.
Under the projective transformation, cross-ratio is unchangeable.It shows the projective correspondence of points in two planes.
Because the shape of the 2-D image will change along with the alteration of internal parameters and external positions of camera, some geometric arithmetic operators, which are the projection geometric invariants of investigated object, will be in demand.
Suppose the composition groups of lines are four base straight lines AB, AC, AD and AE.These four groups of lines and their five vertexes constitute a fourbase-line polygon P, its cross-ratio is: The polygon ABCD in plane I is projected on the plane I 1 , I 2 as polygon A' B' C' D' E' and A" B" C" D" E" (Fig. 1).Their cross-ratios are identical as follows: They can be calculated by transforming the value of sine to slope.
In order to guarantee the cross-ratio of lines invariable, these control points must be in the same plane, but not on the same line.The pretreatments are as follows: • Select every five points of the preferential queue of gradation correlation coefficient to build one group.For every group, fix on one pair of matching points as the control ones and calculate the slope of beelines connected this point with other four points(that is AB, AC, AD In fact, after these steps mentioned above, this research has achieved the automatic adjustment of matching points, but not manual adjustment as former Abnormity elimination: The final geometric constrains matching based on FBM and ABM can nod avoid the bad effect of mismatching of abnormal points.So we use robust estimation of fundamental matrix of matching image and reference one to recover the epipolar geometry and use the epipolar constraints to eliminate the abnormal points.Then more precise matching points can be gained.
The distance minimization criterion be used to implement nonlinear calculation of F-matrix and used d i , the sum of relative departure of two matching points from epipolar line to express the feature measure.d i is called as epipolar distance, it is composed of two parts: The linear distance between epipolar line and control points is given as follows: P 1 , P 2 is corresponding control points of matching image and reference one, the epipolar line of reference image is l 2 = FP 1 , because of noise, P 2 will not always on the line l 2 exactly.
(FP 1 ) i Expresses the i th item.As the same reason, the distance d(P 1 , l 1 ) between P 1 and l 1 has the similar form.Then the calculation of F-matrix can be changed into the minimize distance calculation as follows: The iteration method can be used to solve the nonlinear equation and the specific steps are listed as follows: • Suppose w i = 1, calculate the initial value of Fmatrix using eight-point algorithm.• According to the initial value of F estimated by least-squares procedure, calculate w i of every pair of matching points to be the new weight and substitute the new weight for the old one in the aforementioned equation.• Then calculate the new F-matrix by eight-point method.
• Compare the deviation of twice calculations ‫ܨ‖‬ − ‫ܨ‬ ି ‖, if it is less than δ (predefined threshold), then end the whole task, else jump to the second step and go on iterating.
The projection on the view plane of the discrete points is correctly matched and the wrong matching points will be removed as follows: • Put the rectified matching points into the queue after cross ratio correction, then extract the points with maximal correlative value, normalize them and estimate the F-matrix.Calculate the matching parameters of reference image A, matching image B for homonymy areas: rotate angle r (A, B), scale ration s (A, B), then make assistant verify reused by last matching parameters.
According to obtained matching parameters, accumulative rotating angle and scaling ration can be calculated as follows: ‫ݎ‬ ሺ‫,ݔ‬ ‫ݕ‬ሻ = ∑ ‫,ܣ‪ሺ‬ݎ‬ ‫ܤ‬ሻ , sሺx, yሻ = ΠsሺA, Bሻ Rotate and scale the matching image B by ratio r(x, y) and s(x, y) respectively.At the same time, coordinate origin is remarked to get translation deviation of image A and B and full transform deviation can be obtained by: Then trim concatenation frame image A and B. Fusion image B with the mosaic display area transparently at poison t(x, y) of full translation deviation based on the method of gradation forced correction of stitching joint, then refresh the display area dynamically.Reset the matching image B as new reference image A, then free all display area pointers and matrices spaces.Figure 2 shows the result of image

CONCLUSION
This research proposes a series of corresponding algorithm aiming at the low precision of registration which results in mosaic defeat of serial image, realizes multi-frame high precision mosaic and develops the mosaic technology in application.
An exceptional point removing strategy based on optimal algorithm for estimating the fundamental matrix is presented in this research.At first the exceptional points are eliminated and the initial value of the fundamental matrix is obtained, Secondly, a criterion of minimizing the initial fundamental matrix.It can be seen from the experimental results that this algorithm eliminates the unfavorable effect of exceptional points on the fundamental matrix.
Therefore it is more precise and has an advantage over other conventional algorithms in terms of both average residual error and average epipolar distance.A new robust linear method weighted normalization algorithm is developed by inducing a cost function related to residual errors.Experiments on simulated and real image data are conduced and the results show that this algorithm can enhance the accuracy and it is robust and easy to perform.

Fig
Fig. 1: Four-base-line polygon projection transforms and AE)of the fivepoint-group.Because the base point affects crossratio extraordinarily, we ought to select the matching points with minimal errors as the base points.At first, use proved 8-point algorithm to calculate F-matrix, then utilize M T FM, in which Fmatrix is the fundamental matrix between two correlation images, M' is the cognominal matrix of feature region of matching image b and M is of reference image a. Then select the point with minimal ‫ܯ|‬ ் ‫|ܯܨ‬ as the base point and sort other points based on this value • Calculate the cross ratio R' of the beelines in the first image I 1 (I 1 the local window of reference image a), adjust the other window of matching image b), calculate R''.If adjust the four points of I 2 simultaneously, it will result in adjusting the 8 points, so we choose the point pairs with the maximal ‫ܯ|‬ ் ‫|ܯܨ‬ to adjust firstly, then adjust the other points.If the error is small, the corresponding point should change in the circle with radius r which set the original position of the point as the centre.Utilize the complex-type method to calculate the extremism, if a point which satisfied this condition is found, then this point will replace the former, but do not change the original centre of this circle.If the iterative times exceed the maximal predefined value or cannot find the minimal point, the present point must be kept unchanged.Then deal with the other points which do not change • Repeat the second step to other groups from the preferential queue of gradation correlation coefficient until all groups finished the iterative calculation

Fig. 2 :
Fig. 2: Panoramic image mosaic of aerial serial images mosaic after aforementioned adjustment where the visible artifacts are no longer apparent, the joint are smoother and then more frames can be tiled together accurately.
If error exceeded 3ߪ (σ is variance) and the differentia value of the foreand-after F-matrices less than initialized threshold, go to read data of image A, allocate more bigger mosaic display space S[1, n], set p 1 to the centre of S[1, n] and mark original place; calculate F-matrix if error exceeded 3σ and the differentia value of the fore-ad aft F-matrix exceeded initialized threshold.Return the value of F-matrix.•Calculate the weighted value of every pair of points and adjust F-matrix.• Use the value of the last F-matrix to calculate the average epipolar geometric distance as the differentia value and use this value to appraise the feature points right or wrong.Ultimately, build the right similar pairs of points by removing the wrong points.
Image tiling: At first, read the reference image A and apply for bigger mosaic display space S[1…n].And markdown the initial position t 01 (x, y).