Global Convergence of a New Nonmonotone Algorithm

: In this study, we study the application of a kind of nonmonotone line search in BFGS algorithm for solving unconstrained optimization problems. This nonmonotone line search is belongs to Armijo-type line searches and when the step size is being computed at each iteration, the initial test step size can be adjusted according to the characteristics of objective functions. The global convergence of the algorithm is proved. Experiments on some well-known optimization test problems are presented to show the robustness and efficiency of the proposed algorithms.


INTRODUCTION
Unconstrained optimization problems: The quasi-Newton algorithm BFGS method because of its stable numerical results and fast convergence is recognized as one of the most effective methods to solve the unconstrained problem (1).Iterative formula of this method is as follows: where, α k is step length, g k = ∇(  ),   is the search direction: where, s k = x k+1 -x k , y k = g k+1 -g k .Inexact line search, especially in the monotone line search method global convergence of many results.Byrd et al. (1987) proved in addition to the DFP method of Broyden family Wolfe line search, global convergence for solving convex minimization problem.Byrd and Nocedal (1989) proved the global convergence of the BFGS method Armijo line search for solving convex minimization problem.Sun and Yuan (2006) constructed a counter-example to show that the BFGS method under the Wolfe line search for non-convex minimization problem does not have global convergence.Since 1986, Grippo et al. (1986) first proposed a non-monotonic linear search technology has broader means non-exact line search.One benefit of the technology of non-monotonic is does not require the function value decreases, so that the step the selection of a more flexible, even with step as large as possible.Panier and Tits (1991) proved that a nonmonotonic search technology to avoid Maratos effect.A large number of numerical results show that non-monotonic search is better than the monotonous search numerical performance; in particular, it helps to overcome along the bottom of narrow winding produces slow convergence of iterative sequence (Dai, 2002a;Hüther, 2002).Quasi-Newton method to introduce the nonmonotonic technology also has its practical significance, but not more discussion on global convergence.Han and Liu (1997) prove that the nonmonotone Wolfe modified linear search, BFGS method global convergence of convex objective function.Since the beginning of this century, new nonmonotone line search methods continue to put forward, such as the Zhang and Hager (2004) and Zhen-Jun and Jie (2006) proposed a new non-monotone line search method.

NONMONOTONIC LINE SEARCH
This study a class of non-monotonic linear search is belongs to the Armijo type of linear search the ideological sources.Dai (2002c) proposed a class of monotone line search him and conjugate gradient method combined study.Dai (2002b) monotonous line search is: find α k , so that the following two formulas: σ At the same time set up.This study ‖.‖ refers to the Euclidean norm.In this study, the two equations improvements for weaker conditions, further transformed into a linear search of non-monotonic and the BFGS algorithm combined into a class of quasi-Newton algorithm.Nonmonotone linear search of the text of the study also draws (Zhen-Jun and Jie, 2006), the line search in each step to calculate the step length factor α k .When to introduce timely changes in the initial test step r k , instead follows the Grippo-Lampariello-Lucidi search in the fixed initial test step.If the initial test step is fixed, the non-monotonic class of linear search is essentially a class of linear search without derivative.Its monotonous situation, initially by Leone et al. (1984) study, but there in the form of relatively complex; such line search form research this study are concise and full of operability.
Given σ>0, β ∈ (0, 1),  ∈ (0, 1), M is a nonnegative integer and to let   = −      ‖  ‖ 2 .Take   =   ()   , () = 0, 1, 2, … ()makes holds the smallest non-negative integer: 2 where, l(0) = 0, 0≤l(k)≤min{l(k-1)+1, M}, k≥1.Obviously, if d k is a descent direction, that      < 0, then, when the m(k) sufficiently large, the inequality (4) is always true, thus satisfying the conditions α k exist.The line search in each iteration, the initial test step length is no longer maintained constant, but can be automatically adjusted to r k .Global convergence proof which will see the reasonableness of this proposal.In magnitude, change the initial test step length of practice better results can be obtained to calculate the larger step length factor α k , thereby reducing the number of iterations.

ALGORITHM
Based on this non-monotonous line search technique, we give the nonmonotonic following BFGS algorithm.
2°solution of linear equations calculates the direction of the search d k : 3°step factor αR k R calculated according to the nonmonotonic linear search NLS.

4° let
• In order to step factor calculated in Step 3 to take advantage of the non-monotonic linear search NLS αR k R must make the search direction dR k R descent direction, by ( 5), only to meet    =   = −     −1   < 0, This requires nonmonotonic line search NLS based on research in this chapter, every step BFGS correction formula BR k+1 R is positive definite, it see Theorem 1.
• In order to be able to calculate a larger step length factor αR k R, we can consider a class of mixed non-monotone line search that (4) can be rewritten as: where, δR 1 R, δR 2 ∈ (0, R1) Theorem and proof: Nonmonotonic line search global convergence proof often needs to meet: (i) The Sufficient descent conditions: where CR 1 R and CR 2 R is a positive number.These two conditions are strong, difficult to meet the general quasi-Newton method.This is also the problem of study difficulty.
This study is not BFGS formula to make any changes and non-monotone line search NLS1, does not require the search direction dR k R satisfy the conditions (i)-(ii) of the premise, to prove the global convergence.The general assumption in this section is given below: P }is convex sets and there is cR 1 R>0: where, G(x) = ∇ 2 ().
The above assumptions with the references (Byrd and Nocedal, 1989) the same, paper (Liu et al., 1995) weakened, in particular, is to remove the literature (Grippo et al., 1986) desired search direction d k satisfy the sufficient descent condition and boundedness conditions.
Assumptions (H1) and (H2) conditions, we easily obtain the following results: (Sun and Yuan, 2006)) • The function f(x) in the level set LR 0 R bounded and uniformly continuous where, g(x) = ∇(),  1 > 0 is a constant, such as assuming that (H2) as defined.
Proof: To be a vector-valued function g use of the integral form of the mean value theorem may: Thus, by ( 6), there exists cR 1 R>0, making the: Lemma: 1 Let BR k R is symmetric positive definite matrix,      > 0, Broyden family formula: , when ∅  ≥ 0,  +1 ∅ maintaining positive definite.
Theorem 1: On the assumption that (H1), (H2) Conditions, the step factor αR k R by nonmonotonic line search NLS BR 0 R is symmetric positive definite matrix, then when ∅  ≥ 0, by the correction formula of Broyden family BR k R also maintained positive definite.
Proof: Proved by induction.When k = 0, BR 0 R is symmetric positive definite matrix, the resulting search direction dR 0 R downward direction.By nonmonotonic line search can find αR 0 R, resulting in xR 1 R, it is seen from ( 4): As a result,  1 ∈  0 .Thus, by Lemma 2.3: By Lemma 1, BR 1 R also maintained positive definite: Assume BR k R is positive definite, so the resulting search direction dR k R is down direction.Can be found by the non -monotone line search NLS αR k R, resulting xR k+1 R, it is seen from ( 4): By Lemma 1, BR k+1 R also maintained positive definite.
For the sake of simplicity, we have introduced the notation: Namely h(k) is a non-negative integer and satisfies the following two formulas: (8) Thus, the nonmonotonic line search NLS (4) can be rewritten as: Lemma 2: Under the conditions of assumption (H1), the sequence {f(xR h(k) R)} decreases monotonically.
and then by f(x) in L 0 uniformly continuous, so: i.e., (13) established for i = 1: Now suppose for a given i, ( 12) and ( 13).From ( 9), there are , On both sides so that k→∞, by (13) and: And notes δ>0, so: This indicates that, for any i≥1, (12) was established.The (15) also implies: Due to f(x) in the level set L 0 is uniformly continuous and thus This shows that, for any i≥1, ( 13) is also true.By definition of ℎ � and ( 7) can be obtained: Thus, for any k, do deformation: On where  ℎ � () transposition and notes ( 16), was: On both sides so that k→∞, by (12): Then and then by f(x) consistent continuity: 14), we can see: Nine On both sides of order k→∞, by ( 19) and noted that δ>0, Lemma 4 holds.

Proof:
Use reduction ad absurdum.Assume that the conclusion is not established, there is a constant  > 0, such that for any k≥0, there is: Lemma 5: Shows that the algorithm is in line with the conditions of Lemma 6.Thus, for any  ∈ (0, 1), the presence of a positive constant number β 1 , β 2 , β 3 make any k≥0 , the inequality: Based on the above discussion, we define the index set J t and J are as follows (out of habit , in the above formula , the subscript i replaced by k) : We can put it another way, there is a positive constant number β 1 , β 2 , β 3 and infinite indicators set J, such that for any k∈ , to meet Nonmonotonic line search NLS (4), for any k∈ , there is 2 2 the use of the upper -left of the mean value theorem and finishing, where  =           ∈ (0, 1),  ∈ .
‖ 2 < +∞, in particular, have Lemma 4.Here the weak non-monotonic search, to prove Lemma 4 fee to a lot of twists and turns.}at least [pk] indicators established.[x] is not less than x the smallest integer. ‖