Two Scaling Conjugate Gradient Methods Using the Approximate Optimal Stepsizes for Unconstrained Optimization Problem
DOI:
https://doi.org/10.55578/jdso.2610.010Keywords:
Unconstrained optimization, approximate optimal stepsize, Barzilai-Borwein method, scaling conjugate gradient method, global convergenceAbstract
This paper proposes a new method, which combines the gradient method using the approximately optimal stepsizes with the Barzilai and Borwein scaling conjugate gradient method, and propose two scaling conjugate gradient methods for nonlinear unconstrained optimization problems. And then, the sufficient descent property and the global convergence are proved under some reasonable conditions and the standard Wolfe line search. We give some preliminary numerical results to show the effectiveness of the proposed method.
References
[1] Barzilai, J., & Borwein, J. (1988). Two-point step size gradient methods. IMA Journal of Numerical Analysis, *8*(1), 141–148.
[2] Raydan, M., & Svaiter, B. (2002). Relaxed steepest descent and Cauchy-Barzilai-Borwein method. Computational Optimization and Applications, *21*(2), 155–167.
[3] Dai, Y., & Liao, L. (2002). R-linear convergence of the Barzilai and Borwein gradient method. IMA Journal of Numerical Analysis, *22*(1), 1–10.
[4] Dai, Y., & Yuan, Y. (2003). Alternate minimization gradient method. IMA Journal of Numerical Analysis, *23*, 377–393.
[5] Fletcher, R. (2005). On the Barzilai-Borwein method. Optimization and Control Applications, *56*, 235–256.
[6] Raydan, M. (1997). The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM Journal on Optimization, *7*(1), 26–33.
[7] Sun, W., & Yuan, Y. (2006). Optimization theory and methods: Nonlinear programming. Springer.
[8] Zhang, Y., Sun, W., & Qi, L. (2010). A nonmonotone filter Barzilai-Borwein method for optimization. Asia Pacific Journal of Operations Research, *27*(1), 55–69.
[9] Raydan, M. (1993). On the Barzilai and Borwein choice of steplength for the gradient method. IMA Journal of Numerical Analysis, *13*(3), 321–326.
[10] Grippo, L., Lampariello, F., & Lucidi, S. (1986). A nonmonotone line search technique for Newton's method. SIAM Journal on Numerical Analysis, *23*(4), 707–716.
[11] Dai, Y. H., Hager, W. W., & Schittkowski, K. (2006). The cyclic Barzilai-Borwein method for unconstrained optimization. IMA Journal of Numerical Analysis, *26*(3), 604–627.
[12] Dai, Y. H., Yuan, J. Y., & Yuan, Y. X. (2002). Modified two-point stepsize gradient methods for unconstrained optimization problems. Computational Optimization and Applications, *22*(1), 103–109.
[13] Xiao, Y. H., Wang, Q. Y., & Wang, D. (2010). Notes on the Dai-Yuan-Yuan modified spectral gradient method. Journal of Computational and Applied Mathematics, *234*(10), 2986–2992.
[14] Zhou, B., Gao, L., & Dai, Y. H. (2006). Gradient methods with adaptive stepsizes. Computational Optimization and Applications, *35*(1), 69–86.
[15] Liu, Z., & Liu, H. (2018). An efficient gradient method with approximate optimal stepsize for large-scale unconstrained optimization. Numerical Algorithms, *78*(1), 21–39. https://doi.org/10.1007/s11075-017-0365-2
[16] Liu, Z., & Liu, H. (2019). An efficient gradient method with approximately optimal stepsize based on tensor model for unconstrained optimization. Journal of Optimization Theory and Applications. https://doi.org/10.1007/s10957-019-01475-1
[17] Liu, H., & Liu, Z. (2018). An efficient Barzilai-Borwein conjugate gradient method for unconstrained optimization. Journal of Optimization Theory and Applications. https://doi.org/10.1007/s10957-018-1393-3
[18] Zhang, J. Z., Deng, N. Y., & Chen, L. H. (1999). New quasi-Newton equation and related methods for unconstrained optimization. Journal of Optimization Theory and Applications, *102*(1), 147–167.
[19] Yuan, Y. X. (1991). A modified BFGS algorithm for unconstrained optimization. IMA Journal of Numerical Analysis, *11*(3), 325–332.
[20] Wang, L., Sun, W., de Sampaio, R. J. B., & Yuan, J. (2015). A Barzilai and Borwein scaling conjugate gradient method for unconstrained optimization problems. Applied Mathematics and Computation, *262*, 136–144.
[21] Dai, Y. H., & Kou, C. X. (2016). A Barzilai-Borwein conjugate gradient method. Science China Mathematics, *59*(8), 1511–1524.
[22] Hager, W. W., & Zhang, H. C. (2005). A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM Journal on Optimization, *16*(1), 170–192. https://doi.org/10.1137/030601880
[23] Gilbert, J. C., & Nocedal, J. (1992). Global convergence properties of conjugate gradient methods for optimization. SIAM Journal on Optimization, *2*(1), 21–42.
[24] Zheng, Y., & Zheng, B. (2017). A new modified Barzilai-Borwein gradient method for the quadratic minimization problem. Journal of Optimization Theory and Applications, *172*(1), 179–186.
[25] Liu, M., et al. (2020). Two new conjugate gradient methods for unconstrained optimization. Complexity, *2020*, Article 9720653. https://doi.org/10.1155/2020/9720653
[26] Kim, Y. J., Jong, Y. C., & Kim, Y. (2024). A self-scaling memoryless BFGS based conjugate gradient method using multi-step secant condition for unconstrained minimization. Applied Mathematics, *69*(6), 847–866.
[27] Lv, J., Deng, S., & Wan, Z. (2020). An efficient single-parameter scaling memoryless Broyden-Fletcher-Goldfarb-Shanno algorithm for solving large scale unconstrained optimization problems. IEEE Access, *8*. https://doi.org/10.1109/ACCESS.2020.2992340
[28] Dolan, E. D., & Moré, J. J. (2002). Benchmarking optimization software with performance profiles. Mathematical Programming, *91*(2), 201–213.
[29] Peng, Z., Wu, D., & Zheng, Q. (2013). A level-value estimation method and stochastic implementation for global optimization. Journal of Optimization Theory and Applications, *156*(3), 493–523.
[30] Hu, P., & Liu, X. Q. (2013). A nonmonotone line search slackness technique for unconstrained optimization. Journal of Optimization Theory and Applications, *158*(3), 773–786.
[31] Ali, M. M., Khompatraporn, C., & Zabinsky, Z. B. (2005). A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. Journal of Global Optimization, *31*(4), 635–672.
Downloads
Published
Data Availability Statement
The authors declare that all data in the paper are available.
Issue
Section
License
Copyright (c) 2026 Yunchol Jong, Yongjin Kim, Pyongil Choe (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.