Basics

Basics

  • Definitions
  • Independence
  • Subspaces, span, affine sets
  • Basis, dimension

Definitions

Vectors

Assume we are given a collection of n real numbers, x_1, \cdots, x_n. We can represent them as n locations on a line. Alternatively, we can represent the collection as a single point in a n-dimensional space. This is the vector representation of the collection of numbers; each number x_i is called a component or element of the vector.

Vectors can be arranged in a column or a row; we usually write vectors in column format:

(1)   \begin{align*} x &= \begin{pmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{pmatrix} \end{align*}

We denote by \mathbb{R}^n denotes the set of real vectors with n components. If x_i \in \mathbb{R}^n denotes a vector, we use subscripts to denote components, so that x_i is the i-th component of x. Sometimes the notation x(i) is used to denote the i-th component.

alt text A vector can also represent a point in a multi-dimensional space x_i \in \mathbb{R}^n, where each component corresponds to a coordinate of the point.

Example: The vector x=(2,1) in x_i \in \mathbb{R}^2.

Examples:

Transpose

If x is a column vector, x^T denotes the corresponding row vector, and vice-versa. Hence, if x is the column vector above:

x^T = (x_1 \cdots x_n)

Sometimes we use the looser, in-line notation x = (x_1, \cdots, x_n), to denote a row or column vector, the orientation being understood from context.

Matlab syntax

A column vector x = (2, 3.1, -4) and its transpose y can be declared in Matlab’s workspace as follows. Here, no room for loose notation: we use a semicolon to separate the components of a column vector, while we use commas for row vectors.

Matlab syntax: declare and transpose a vector
>> x = [2; 3.1; -4]; % declare a column vector using ";".
>> y = x'; % the prime operator ' transposes the vector.
>> y = [2,3.1,-4]; % can also declare a row vector with commas.
>> x(2) % this produces the second component of x.
>> x([1,3]) % this produces the 2-vector with the first and the third component of x.

Independence

A set of vectors {x_1, x_2, \cdots, x_m} in \mathbb{R}^m , i = 1, \cdots, m.  is said to be independent if and only if the following condition on a vector \lambda \in \mathbb{R}^m:

\sum\limits_{i=1}^{m} \lambda_i x_i = 0

implies \lambda = 0. This means that no vector in the set can be expressed as a linear combination of the others.

Example: the vectors x^1 = [1, 2, 3] and x^2 = [3, 6, 9] are not independent, since 3x^1-x^2=0.

Subspace, span, affine sets

A subspace of \mathbb{R}^n is a subset that is closed under addition and scalar multiplication. Geometrically, subspaces are ‘‘flat’’ (like a line or plane in 3D) and pass through the origin.

An important result of linear algebra, which we will prove later, says that a subspace {\bf S} can always be represented as the span of a set of vectors x_i \in \mathbb{R}^n , i = 1, \cdots, m, that is, as a set of the form

{\bf S} = {\bf span}(x_1, \cdots, x_m):= \left\{\sum\limits_{i=1}^{m} \lambda_i x_i: \lambda \in \mathbb{R}^m\right\}

An affine set is a translation of a subspace — it is ‘‘flat’’ but does not necessarily pass through 0, as a subspace would. (Think for example of a line, or a plane, that does not go through the origin.) So an affine set {\bf A} can always be represented as the translation of the subspace spanned by some vectors:

{\bf A} = \left\{x_0 + \sum\limits_{i=1}^{m} \lambda_i x_i: \lambda \in \mathbb{R}^m\right\},

for some vectors {x_0, x_1, x_2, \cdots, x_m}. In shorthand notation, we write {\bf A} = x_0 + {\bf S},

alt text Example: In \mathbb{R}^3 , the span {\bf S} of the two vectors

(2)   \begin{align*} u &= \begin{bmatrix} -1 \\ 2 \\ 0.5 \end{bmatrix} \end{align*}

[latex],\begin{align} v &= \begin{bmatrix} 1 \\ 3 \\ 0.1 \end{bmatrix} \end{align}[/latex]

is the plane passing through the origin pictured in blue.

When {\bf S} is the span of a single non-zero vector, the set {\bf A} is called a line passing through the point x_0. Thus, lines have the form \{x_0 + tu : t\in \mathbb{R}\}

where u determines the direction of the line, and x_0 is a point through which it passes.

alt text Example: A line in \mathbb{R}^2 passing through the point x_0 = (2,0), with direction u=(0.8944, 0.4472).

Basis, dimension

Basis in \mathbb{R}^n

A basis of \mathbb{R}^n is a set of n independent vectors. If the vectors u_1, \cdots, u_n form a basis, we can express any vector as a linear combination of the u_i‘s:

x = \sum\limits_{i=1}^{n} \lambda_i u_i

for appropriate numbers \lambda_1, \cdots, \lambda_n.

The standard basis (alternatively, natural basis) in \mathbb{R}^n consists of the vectors e_i, where e_i‘s components are all zero, except the i-th, which is equal to 1. In \mathbb{R}^3 , we have

(3)   \begin{align*} e_1 &:= \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} \end{align*}

[latex],\begin{align} e_2 &:= \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix} \end{align}[/latex]

(4)   \begin{align*} e_3 &:= \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \end{align*}

Example: A basis in \mathbb{R}^3 .

Basis of a subspace

The basis of a given subspace {\bf S} \in \mathbb{R}^n  is any independent set of vectors whose span is {\bf S}. If the vectors u_1, \cdots, u_r form a basis of {\bf S}, we can express any vector as a linear combination of the u_i‘s:

x = \sum\limits_{i=1}^{r} \lambda_i u_i
for appropriate numbers \lambda_1, \cdots, \lambda_r.

The number of vectors in the basis is actually independent of the choice of the basis (for example, in \mathbb{R}^3  you need two independent vectors to describe a plane containing the origin). This number is called the dimension of {\bf S} . We can accordingly define the dimension of an affine subspace, as that of the linear subspace of which it is a translation.

Examples:

License

Hyper-Textbook: Optimization Models and Applications Copyright © by L. El Ghaoui. All Rights Reserved.

Share This Book