A New Adaptive Weighted Hybrid Conjugate Gradient Method for Unconstrained Optimization Problems
DOI:
https://doi.org/10.55578/jdso.2606.007Keywords:
Unconstrained optimization, Conjugate gradient methods, Global convergence, Backtracking line search, Sufficient descentAbstract
In this paper, a new adaptive weighted hybrid conjugate gradient (AWHCG) method with backtracking line search is proposed to solve unconstrained optimization problems. Our method is based on the weighted hybridization of HS (Hestenes-Stiefel) and FR (Fletcher- Reeves) conjugate gradient methods. In contrast with the existing hybrid conjugate gradient methods, our AWHCG method updates adaptively the coefficient of gradient of objective function to be negative at each iteration so that our current search direction is always a combination of the anti- gradient of the objective function and the preceding direction. Under some reasonable assumptions, the global convergence of the proposed algorithm is established. Some preliminary numerical experiments show that the AWHCG algorithm is competitive.
References
[1] Sun, W., & Yuan, Y. X. (2006). Optimization theory and methods: Nonlinear programming. Springer.
[2] Stanimirović, P. S., Ivanov, B., Djordjević, S., & Brajević, I. (2018). New hybrid conjugate gradient and Broyden-Fletcher-Goldfarb-Shanno conjugate gradient methods. Journal of Optimization Theory and Applications, *178*, 860-884.
[3] Dong, X. L., Dai, Z. F., Ghanbari, R., & Li, X. L. (2019). An adaptive three-term conjugate gradient method with sufficient descent condition and conjugacy condition. Journal of the Operations Research Society of China. https://doi.org/10.1007/s40305-019-00257-w
[4] Zheng, Y., & Zheng, B. (2017). Two new Dai-Liao-type conjugate gradient methods for unconstrained optimization problems. Journal of Optimization Theory and Applications, *175*, 502-509.
[5] Mtagulwa, P., & Kaelo, P. (2019). An efficient modified PRP-FR hybrid conjugate gradient method for solving unconstrained optimization problems. Applied Numerical Mathematics, *145*, 111-120.
[6] Chen, Y., Cao, M., & Yanget, Y. (2019). A new accelerated conjugate gradient method for large-scale unconstrained optimization. Journal of Inequalities and Applications, *2019*, Article 300.
[7] Zheng, X., Dong, X., Shi, J., & Yang, W. (2020). Further comment on another hybrid conjugate gradient algorithm for unconstrained optimization by Andrei. Numerical Algorithms, *84*, 603-608.
[8] Jian, J., Han, L., & Jiang, X. (2015). A hybrid conjugate gradient method with descent property for unconstrained optimization. Applied Mathematical Modelling, *39*, 1281-1290.
[9] Andrei, N. (2013). A simple three-term conjugate gradient algorithm for unconstrained optimization. Journal of Computational and Applied Mathematics, *241*, 19-29.
[10] Fletcher, R., & Reeves, C. M. (1964). Function minimization by conjugate gradients. The Computer Journal, *7*(2), 149-154.
[11] Fletcher, R. (1987). Practical methods of optimization: Unconstrained optimization (Vol. 1). Wiley.
[12] Dai, Y. H., & Yuan, Y. (1999). A nonlinear conjugate gradient method with a strong global convergence property. SIAM Journal on Optimization, *10*(1), 177-182.
[13] Hestenes, M. R., & Stiefel, E. L. (1952). Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, *49*(6), 409-436.
[14] Polak, E., & Ribière, G. (1969). Note sur la convergence des méthodes de directions conjuguées. Revue Française d'Informatique et de Recherche Opérationnelle, *16*, 35-43.
[15] Polyak, B. T. (1969). The conjugate gradient method in extreme problems. U.S.S.R. Computational Mathematics and Mathematical Physics, *9*, 94-112.
[16] Liu, Y., & Storey, C. (1991). Efficient generalized conjugate gradient algorithms, part 1: theory. Journal of Optimization Theory and Applications, *69*(1), 129-137.
[17] Yang, X., Luo, Z., & Dai, X. (2013). A global convergence of LS-CD hybrid conjugate gradient method. Advances in Numerical Analysis, *2013*, Article 517452.
[18] Dai, Y. H., & Liao, L. Z. (2001). New conjugacy conditions and related nonlinear conjugate gradient methods. Applied Mathematics and Optimization, *43*(1), 87-101.
[19] Touati-Ahmed, D., & Storey, C. (1990). Efficient hybrid conjugate gradient techniques. Journal of Optimization Theory and Applications, *64*(2), 379-397.
[20] Zhang, L., & Zhou, W. (2008). Two descent hybrid conjugate gradient methods for optimization. Journal of Computational and Applied Mathematics, *216*, 251-264.
[21] Wei, Z. X., Yao, S. W., & Liu, L. Y. (2006). The convergence properties of some new conjugate gradient methods. Applied Mathematics and Computation, *183*, 1341-1350.
[22] Dai, Y. H., & Yuan, Y. (2001). An efficient hybrid conjugate gradient method for unconstrained optimization. Annals of Operations Research, *103*, 33-47.
[23] Yao, S. W., Wei, Z. X., & Huang, H. (2007). A note about WYL's conjugate gradient method and its application. Applied Mathematics and Computation, *191*, 381-388.
[24] Dolan, E. D., & Moré, J. J. (2002). Benchmarking optimization software with performance profiles. Mathematical Programming, *91*, 201-213.
[25] Alkhazishvili, L., & Gorgadze, L. (2019). A collection of test functions for unconstrained optimization solvers with serial and parallel C++ implementations [Preprint]. ResearchGate. https://www.researchgate.net/publication/334226455
[26] Liu, J., & Ma, C. (2013). A nonmonotone trust region method with new inexact line search for unconstrained optimization. Numerical Algorithms, *64*, 1-20.
[27] Peng, Z., Wu, D., & Zheng, Q. (2013). A level-value estimation method and stochastic implementation for global optimization. Journal of Optimization Theory and Applications, *156*, 493-523.
[28] Wang, C., Liu, K., & Shen, P. (2020). A novel genetic algorithm for global optimization. Acta Mathematicae Applicatae Sinica (English Series), *36*(2), 482-491.
[29] Hu, P., & Liu, X. Q. (2013). A nonmonotone line search slackness technique for unconstrained optimization. Journal of Optimization Theory and Applications, *158*, 773-786.
[30] Ali, M. M., Khompatraporn, C., & Zabinsky, Z. B. (2005). A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. Journal of Global Optimization, *31*, 635-672.
Downloads
Published
Data Availability Statement
The authors declare that all data in the paper are available.
Issue
Section
License
Copyright (c) 2026 Yunchol Jong, Yongjin Kim, Daesong Ri (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.