On The Stability Of Singular Values And Subspaces Under Perturbation

Abstract

In statistics, signal processing, and machine learning, the information we want from a matrix is carried by its singular values and singular vectors, yet the matrix we observe is almost never the one we want: it has been corrupted by measurement error, finite-sample fluctuation, or noise. Matrix perturbation theory makes precise what survives this corruption, and this paper is a self-contained account of that theory for the singular value decomposition, organized around one question: when A is replaced by A+E, how far can its singular values and singular subspaces move? We construct the decomposition from first principles and prove that it exists for every matrix. We then prove the Eckart–Young theorem, which identifies the best low-rank approximation of a matrix, together with Mirsky's extension of it to the spectral norm. The perturbation theory itself divides in two: Weyl's inequality shows that each singular value moves by at most the spectral-norm size of E, while the Davis–Kahan theorem shows that a singular subspace rotates by an angle controlled by that same size divided by the spectral gap isolating it. Gershgorin's circle theorem enters as a localization tool, confining the eigenvalues of a matrix to discs read directly from its entries. Read together, the theorems answer a question at the center of working with real data: when the input is noisy, which features of a matrix are signal that persists, and which are noise that does not.

Domains and classifications

Version history

Expositio keeps public paper versions together so readers can see the current PDF while still understanding how the record has changed.