[latexpage]
Returning to this example, involving a matrix with row size \(m=4\) and column size \(n=5\):
\[ A=\left(\begin{array}{lllll} 1 & 0 & 0 & 0 & 2 \\ 0 & 0 & 3 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 4 & 0 & 0 & 0 \end{array}\right) . \]
As seen here, the SVD is given by \(A=U \tilde{S} V^T\), with
\[ U=\left(\begin{array}{cccc} 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & -1 \\ 1 & 0 & 0 & 0 \end{array}\right), \quad \tilde{S}=\left(\begin{array}{ccccc} 4 & 0 & 0 & 0 & 0 \\ 0 & 3 & 0 & 0 & 0 \\ 0 & 0 & \sqrt{5} & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right), \quad V^T=\left(\begin{array}{ccccc} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ \sqrt{0.2} & 0 & 0 & 0 & \sqrt{0.8} \\ 0 & 0 & 0 & 1 & 0 \\ -\sqrt{0.8} & 0 & 0 & 0 & \sqrt{0.2} \end{array}\right) . \]
The matrix is rank \(r=3\). A rank-two approximation is given by zeroing out the smallest singular value, which produces
\[ \begin{aligned} \hat{A}_2 & =\left(\begin{array}{cccc} 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & -1 \\ 1 & 0 & 0 & 0 \end{array}\right)\left(\begin{array}{ccccc} 4 & 0 & 0 & 0 & 0 \\ 0 & 3 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right)\left(\begin{array}{ccccc} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ \sqrt{0.2} & 0 & 0 & 0 & \sqrt{0.8} \\ 0 & 0 & 0 & 1 & 0 \\ -\sqrt{0.8} & 0 & 0 & 0 & \sqrt{0.2} \end{array}\right) \\ & =\left(\begin{array}{ll} 0 & 0 \\ 0 & 1 \\ 0 & 0 \\ 1 & 0 \end{array}\right)\left(\begin{array}{ll} 4 & 0 \\ 0 & 3 \end{array}\right)\left(\begin{array}{lllll} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \end{array}\right) \\ & =\left(\begin{array}{lllll} 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 3 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 4 & 0 & 0 & 0 \end{array}\right) . \end{aligned} \]
We check that the Frobenius norm of the error \(\|A-\hat{A}_2\|_F\) is the sum of singular values we have zeroed out, which here reduces to \(\sigma_3=\sqrt{5}\):
\[ E:=A-\hat{A}_2=\left(\begin{array}{lllll} 1 & 0 & 0 & 0 & 2 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right), \quad \|E\|_F^2=1^2+2^2=5 . \]
