Quadratic Approximation of the Log-Sum-Exp Function

L. El Ghaoui

Quadratic Approximation of the Log-Sum-Exp Function

As seen here, the log-sum-exp function $lse: \mathbb{R}^2 \rightarrow \mathbb{R}$ , with values

$lse(x):= \log(e^{x_1}+e^{x_2}).$

admits the following gradient and Hessian at a point $x$ :

$\nabla lse(x) = \frac{1}{z_1 + z_2}\left(\begin{array}{c} z_1 \\ z_2 \end{array}\right), \nabla^2 lse(x) = \frac{z_1 z_2}{(z_1 +z_2)^2}\left(\begin{array}{cc} 1 & -1 \\-1 & 1 \end{array}\right),$ where $z_i: = e^{x_i}$ , $i=1,2$ .

Hence, the quadratic approximation of the log-sum-exp function at a point $x = (x_1, x_2)$ is given by

$\begin{aligned}lse(x+h) &\approx lse(x) + h^T \nabla lse(x) + \frac{1}{2} h^T \nabla^2 lse(x)h \\ &= lse(x) + \frac{h_1 e^{x_1} + h_2 e^{x_2}}{e^{x_1}+e^{x_2}} + \frac{e^{x_1+x_2}}{2(e^{x_1}+e^{x_2})^2} (h_1 - h_2)^2. \end{aligned}$

Quadratic Approximation of the Log-Sum-Exp Function

License

Share This Book