Fundamental theorem of linear algebra
Fundamental theorem of linear algebra
Let 
. The sets 
and 
form an orthogonal decomposition of 
, in the sense that any vector 
can be written as
![\[x=y+z, y \in \mathbf{N}(A), \quad z \in \mathbf{R}\left(A^T\right), \quad y^T z=0 .\]](https://ecampusontario.pressbooks.pub/app/uploads/quicklatex/quicklatex.com-7ec6b07630e26064178aebb86f53e3d6_l3.png "Rendered by QuickLaTeX.com")
In particular, we obtain that the condition on a vector 
to be orthogonal to any vector in the nullspace of 
implies that it must be in the range of its transpose:
![\[x^T y=0 \text { whenever } A y=0 \Longleftrightarrow \exists \lambda \in \mathbf{R}^m: x=A^T \lambda \text {. }\]](https://ecampusontario.pressbooks.pub/app/uploads/quicklatex/quicklatex.com-bc4b09b3205ebb57a461fb5b42a1884b_l3.png "Rendered by QuickLaTeX.com")
Proof: The theorem relies on the fact that if a SVD of a matrix is
![\[A=U \tilde{S} V^T, \quad \tilde{S}=\operatorname{diag}\left(\sigma_1, \ldots, \sigma_r, 0, \ldots, 0\right)\]](https://ecampusontario.pressbooks.pub/app/uploads/quicklatex/quicklatex.com-e1cfde27723f6c01f2dfc856d0826d29_l3.png "Rendered by QuickLaTeX.com")
then an SVD of its transpose is simply obtained by transposing the three-term matrix product involved:
![\[A^T=\left(U \tilde{S} V^T\right)^T=V \tilde{S} U^T .\]](https://ecampusontario.pressbooks.pub/app/uploads/quicklatex/quicklatex.com-34d921550135f38ca9652d1dcc7573e7_l3.png "Rendered by QuickLaTeX.com")
Thus, the left singular vectors of are the right singular vectors of 
.
From this we conclude in particular that the range of is spanned by the first 
columns of 
. Since the nullspace of is spanned by the last 
columns of , we observe that the nullspace of and the range of are two orthogonal subspaces, whose dimension sum to that of the whole space. Precisely, we can express any given vector in terms of a linear combination of the columns of ; the first columns correspond to the vector 
and the last to the vector 
:
![\[x=V\left(V^T x\right)=\underbrace{\sum_{i=1}^r \tilde{x}_i v_i}_{=z}+\underbrace{\sum_{i=r+1}^n \tilde{x}_i v_i}_{=y}, \quad \tilde{x}:=V^T x .\]](https://ecampusontario.pressbooks.pub/app/uploads/quicklatex/quicklatex.com-82e6c8f38c5d0585c1560a773c2e2e0b_l3.png "Rendered by QuickLaTeX.com")
This proves the first result in the theorem.
The last statement is then an obvious consequence of this first result: if is orthogonal to the nullspace, then the vector 
in the theorem above must be zero, so that 
.