Parallelization of the Hierarchical Alternating Least Squares Algorithm for Nonnegative Matrix Factorization

The Nonnegative Matrix Factorization (NMF) approximates a large nonnegative matrix as a product of two significantly smaller nonnegative matrices. Because of the nonconvexity of the constrained optimization problem of finding the best approximation, all current algorithms are iterative and optimize the two factor matrices alternatingly. The resulting sublinear convergence rates give rise to the demand for parallel implementations on high performance computers. One of the best algorithms for NMF in terms of convergence is the Hierarchical Alternating Least Squares (HALS) algorithm. While other Alternating Nonnegative Least Squares (ANLS) algorithms have been shown to have a rather straight-forward parallelization because of independent matrix rows and columns, the row and column updates in HALS must be strictly consecutive, which is more difficult to parallelize. We show that a parallelization strategy similar to ANLS parallelizations exists and yields good speedups for up to 64 processes and satisfactory beyond. These are competitive in comparison to previous solutions to the problem. To our knowledge, HALS has not been parallelized before.