Abstract
High-dimensional nonlinear programming (NLP) models have been increasingly and frequently appearing in the current age of social networks, bioinformatics, digital communications, and quantum computing. Actually, since real-world phenomena naturally behave nonlinearly, nowadays NLP can be regarded as the core of the practical models in many disciplines such as machine learning, data mining, and artificial intelligence. Hence, it is a matter of great urgency to devise and diversify the limited-memory strategies for solving large-scale NLP models. Since a wide range of NLP algorithms try to benefit from the second-order information of the cost function, developing sparse approximations for the (inverse) Hessian based on the quasi-Newton aspects could be effectively helpful, although some accuracy concerns may arise. Hence, we need to take care of the traditional balance between sparsity (simplicity) and accuracy (precision). Meanwhile, the accuracy can be controlled using the secant equations proposed in the literature, with a reasonable focus on the distribution of the eigenvalues of the approximate matrix, being straightly connected to the well-conditioning of the given sparse approximation of the (inverse) Hessian. As an influential factor, the condition number of the sparse matrix approximations can be assessed by the measure functions as well, traditionally employed to analyze the scaling and sizing of the quasi-Newton updating formulas. It is worth noting that sparse (inverse) Hessian estimations, mostly of the diagonal form, can be directly used in the spectral steepest descent method, spectral (preconditioned) conjugate gradient algorithms, scaled (preconditioned) quasi-Newton methods, and trust-region algorithms. Such approximations also turned out to be practically effective for large-scale NLP models, while being capable of ensuring global convergence as well.