Abstract
It is a matter of routine in the operational research discipline to analyze the relaxed versions of the discrete decision-making models, mainly by converting the integer variables to the real variables in certain intervals. This fundamental scheme makes it possible to benefit from the continuous optimization aspects to design appropriate algorithms to address such discrete models. On the other side, a wide range of practical problems in machine learning and data mining are originally in the framework of nonlinear programming models [5]. Meanwhile, since real-world decision-making models often have high-dimensional forms, we should decrease the computational cost of their solution process to the greatest possible extent. To this aim, devising memoryless versions of the classic algorithms could be helpful [4]. Generally, the optimization algorithms are designed based on successive local searches in the framework of the line search (LS) or the trust-region (TR) approaches. In both the LS and TR algorithms, the current approximation of the optimal solution is updated iteratively [1], by solving a subproblem as a local estimation of the original model. Although there exists a wide range of techniques for approximately solving the LS and TR subproblems, in some cases their solution process may be costly or even unsuccessful. Hence, since the second-order information of the model could play a crucial role in solving both the LS and TR subproblems, it is technically meaningful to develop sparse approximations of the Hessian matrix [2]. With such plans, although we may lose accuracy in some levels, which in large-scale cases is indispensable, the solution process often turns out to be low-cost and more efficient, and we can preserve the convergence. Meanwhile, we can effectively accelerate such memoryless algorithms with heuristic strategies as well [3]. Preliminary numerical experiments show the efficiency of such algorithmic initiatives.