Abstract
In the contemporary world that is intrinsically characterized by huge flows of the data in various disciplines such as commerce, biology and social networks, a great deal of interest has rightly been waged in data compression techniques. Meanwhile, since considerable classes of the data sets are typically nonnegative, it is an important must to maintain their nonnegativity in the data mining processes [3]. On the other hand, for the compact representation of the data sets given in the matrix forms, matrix factorization techniques have been traditionally regarded as helpful tools [5]. Nonnegative matrix factorization (NMF) has been devised as an approach to address such concerns [2]. Generically, NMF is designed with the aim of generating two low dimension matrices from a given input matrix with nonnegative entries, such that multiplying the output matrices leads to a meaningful approximation of the input matrix [6]. Meantime, there exists a class of essential constraints in the sense of nonnegativity of the entries of the factorization elements. So, the NMF problem has been traditionally modelled as a constrained optimization problem, customarily solved by the alternating nonnegative least-squares (ANLS) method which is recognized by alternatingly solving two least-squares convex subproblems of the nonconvex NMF model. Although ANLS subproblems can be effectively solved by unconstrained optimization algorithms [1], the solution trajectory is generally vulnerable to deviation resulted by ill-conditioning of the intermediary successive approximations of the factorization elements. Thus, to maintain the solution path in the right context, we can take advantage of the measure functions that have been principally utilized to analyze the scaling and sizing of the quasi-Newton updates [8]. Among the measure functions suggested in the literature, the Dennis–Wolkowicz (DW) function [4] meaningfully benefits all the eigenvalues of the targeted positive definite matrix, not only taking the boundary eigenvalues of the matrix as done in the spectral condition number [8]. Although the DW measure function can be straightly embedded to the NMF model with the aim of taking control over the condition number of the consecutive approximations, its formulaic structure may make the algebraic manipulation of the model to some extent challenging. So, here we study some simplified versions of the DW measure function to better address the great concern of numerical instability in the factorization process caused by collinearity [9]. At the same time, we note to the fact that doing computations with large-scale dense matrices which frequently appear in the data mining models is not reasonable with respect to the CPU time and the computational error. So, in another guideline of our study, we deal with sparse approximations of such matrices which recently attracted special attention [7]. Finally, we provide some computational evidence for our theoretical assertions.
Keywords: Nonnegative matrix factorization; Alternating nonnegative least-squares method; Well-conditioning; Measure function; Diagonal matrix approximation.
AMS 2010: 15A23, 65K10.
References
[1] S. Babaie–Kafaki, A survey on the Dai–Liao family of nonlinear conjugate gradient methods, RAIROOperations Research, 57 (2023), 43–58.
[2] M.W. Berry and M. Browne and A.N. Langville and V.P. Pauca and R.J. Plemmons, Algorithms and
applications for approximate nonnegative matrix factorization, Computational Statistics and Data Analysis, 52 (2007), no. 1, 155–173.
[3] Y.C. Cho and S. Choi. Nonnegative features of spectro-temporal sounds for classification. Pattern Recognition Letters, 26 (2005), no. 9, 1327–1336.
[4] J.E. Dennis and H. Wolkowicz. Sizing and least-change secant methods. SIAM Journal of Numerical Analysis, 30 (1993), no. 5, 1291–1314.
[5] L. Eld´en, Matrix Methods in Data Mining and Pattern Recognition, SIAM, Philadelphia, 2007.
[6] J. Gan and T. Liu and L. Li and J. Zhang, Nonnegative matrix factorization: A survey, The Computer
Journal 64 (2021), no. 7, 1080–1092.
[7] M. Roozbeh and S. Babaie–Kafaki and Z. Aminifard, Improved high-dimensional regression models with
matrix approximations applied to the comparative case studies with support vector machines. Optimization Methods and Software, 37 (2022), no. 5, 1912–1929.
[8] W. Sun and Y.X. Yuan, Optimization Theory and Methods: Nonlinear Programming, Springer, New York,
2006.
[9] D.S. Watkins, Fundamentals of Matrix Computations, John Wiley and Sons, New York, 2002.