Returning to the example involving power laws, we ask the question of finding the ‘‘best’’ model of the form

[latex]\begin{align*} y &= C x_1^{a_1} \cdots x_n^{a_n}, \end{align*}[/latex]

given experiments with several input vectors [latex]x_{i}[/latex] and associated outputs [latex]y_i[/latex], [latex]i=1,\cdots,m[/latex]. Here the variables of our problem are [latex]C[/latex], and the vector [latex]a \in \mathbb{R}^n[/latex]. Taking logarithms, we obtain

[latex]\begin{align*} \log(y_i) &= \log(C) + a_1 \log(x_{i1}) + \cdots + a_n \log(x_{in}), \end{align*}[/latex]

which can be rearranged to the linear form

[latex]\begin{align*} \tilde{y}_i &= a^T \tilde{x}_{i} +b, \quad i=1,\cdots, m, \end{align*}[/latex]

where [latex]b = \log(C)[/latex], and [latex]\tilde{x}_i[/latex] and [latex]\tilde{y}_i[/latex] are the logarithms of [latex]x_i[/latex] and [latex]y_i[/latex], respectively. We can represent the above linear equations compactly as

[latex]\begin{align*} \begin{pmatrix} \tilde{y}_1 \\ \vdots \\ \tilde{y}_m \end{pmatrix} &= \begin{pmatrix} \tilde{x}_1^T & 1 \\ \vdots & \vdots \\ \tilde{x}_m^T & 1 \end{pmatrix} \begin{pmatrix} a \\ b \end{pmatrix}. \end{align*}[/latex]

In practice, the power law model is only an approximate representation of reality. Finding the best fit can be formulated as the optimization problem

[latex]\begin{align*} \min\limits_z ||X^Tz-\tilde{y}||_2^2, \end{align*}[/latex]

[latexpage] where \( z = \begin{pmatrix} a \\ b \end{pmatrix} \in \mathbb{R}^{n+1} \), \( X \in \mathbb{R}^{(n+1) \times m} \), with the \(i\)-th column of \( X \) given by \( \begin{pmatrix} \tilde{x}_{i} \\ 1 \end{pmatrix} \), and \(\tilde{y} = \begin{pmatrix} \tilde{y}_1 \\ \vdots \\ \tilde{y}_m \end{pmatrix}\).

See also: Power laws.