Low-rank approximation of a 4 x 5 matrix via its SVD

L. El Ghaoui

Low-rank approximation of a 4 x 5 matrix via its SVD

Returning to this example, involving a matrix with row size $m=4$ and column size $n=5$ :

$A=\left(\begin{array}{lllll} 1 & 0 & 0 & 0 & 2 \\ 0 & 0 & 3 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 4 & 0 & 0 & 0 \end{array}\right)$

As seen here, the SVD is given by $A=U \tilde{S} V^T$ , with

$U=\left(\begin{array}{cccc} 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & -1 \\ 1 & 0 & 0 & 0 \end{array}\right), \tilde{S}=\left(\begin{array}{ccccc} 4 & 0 & 0 & 0 & 0 \\ 0 & 3 & 0 & 0 & 0 \\ 0 & 0 & \sqrt{5} & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right), \quad V^T=\left(\begin{array}{ccccc} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ \sqrt{0.2} & 0 & 0 & 0 & \sqrt{0.8} \\ 0 & 0 & 0 & 1 & 0 \\ -\sqrt{0.8} & 0 & 0 & 0 & \sqrt{0.2} \end{array}\right) .$

The matrix is rank $r=3$ . A rank-two approximation is given by zeroing out the smallest singular value, which produces

$\begin{aligned} & \hat{A}_2=\left(\begin{array}{cccc} 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & -1 \\ 1 & 0 & 0 & 0 \end{array}\right)\left(\begin{array}{ccccc} 4 & 0 & 0 & 0 & 0 \\ 0 & 3 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right)\left(\begin{array}{ccccc} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ \sqrt{0.2} & 0 & 0 & 0 & \sqrt{0.8} \\ 0 & 0 & 0 & 1 & 0 \\ -\sqrt{0.8} & 0 & 0 & 0 & \sqrt{0.2} \end{array}\right) \\ & =\left(\begin{array}{ll} 0 & 0 \\ 0 & 1 \\ 0 & 0 \\ 1 & 0 \end{array}\right)\left(\begin{array}{ll} 4 & 0 \\ 0 & 3 \end{array}\right)\left(\begin{array}{lllll} 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \end{array}\right) \\ & =\left(\begin{array}{lllll} 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 3 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 4 & 0 & 0 & 0 \end{array}\right) . \\ & \end{aligned}$

We check that the Frobenius norm of the error $\left\|A-\hat{A}_2\right\|_F$ is the sum of singular values we have zeroed out, which here reduces to $\sigma_3=\sqrt{5}$ :

$E:=A-\hat{A}_2=\left(\begin{array}{lllll} 1 & 0 & 0 & 0 & 2 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array}\right),\|E\|_F^2=1^2+2^2=5 .$

Low-rank approximation of a 4 x 5 matrix via its SVD

License

Share This Book