For a [latex]m\times n[/latex] matrix [latex]A[/latex], we define the largest singular value (or, LSV) norm of [latex]A[/latex] to be the quantity

[latex]\begin{align*} ||A|| := \max\limits_{x} ||Ax||_2: ||x||=1. \end{align*}[/latex]

This quantity satisfies the conditions to be a norm (see here). The reason why this norm is called this way is given here.

The LSV norm can be computed as follows. Let us square the above. We obtain a representation of the squared LSV norm as a Rayleigh quotient of the matrix [latex]A^TA[/latex]:

[latex]\begin{align*} ||A||^2 = \max\limits_{x: ||x||=1} x^TA^TAx. \end{align*}[/latex]

This shows that the squared LSV norm is the largest eigenvalue of the (positive semi-definite) symmetric matrix [latex]A^TA[/latex], which is denoted [latex]\lambda_{\max}[/latex]. That is:

[latex]\begin{align*} ||A|| = \sqrt{\lambda_{\max} (A^TA)}. \end{align*}[/latex]