The first-order derivative of $f $ denoted $Df $ is
$$Df \triangleq \left[\pder{ f }{ x_1 }, \pder{ f }{ x_2 }, … , \pder{ f }{ x_n }\right]$$
Note that the gradient $\nabla f $ is just the transpose of $Df $; that is
$$\nabla f = \transpose{ Df } $$
$$\b{F}(\b{x}) = D^2 \triangleq {\Mee{ \pderw{ f }{ x_1 } (\b{x})}{ … }{ \pderws{ f }{ x_n }{ x_1 } (\b{x})}{ \vdots }{ }{ \vdots }{ \pderws{ f }{ x_1 }{ x_n } (\b{x})}{ … }{ \pderw{ f }{ x_n } (\b{x})}}$$
A vector $\b{d} \in \R^n, \b{d} \neq \b{0} $, is a feasible direction at $\b{x} \in \Omega$ if
$$\exists \alpha_0 > 0\text{ such that }\forall \alpha \in [0, \alpha_0], \pare{\b{x} + \alpha \b{d}} \in \Omega$$
Let $f: \R^n \to \R $ be a real-valued function and let $\b{d} $ be a feasible direction at $\b{x} \in \Omega $. The directional derivative of $f$ in the direction $\b{d} $, denoted $\pder{ f }{ \b{d}} $, is the real-valued function defined by
$$\pder{ f }{ \b{d}} (\b{x}) \triangleq \limu{ \alpha }{ 0 } \frac{ f(\b{x} + \alpha \b{d}) - f(\b{x})}{ \alpha }$$
In particular, if $\norm{ \b{d}} = 1$, then $\pder{ f }{ \b{d}} $ is the rate of increase of $f $ at $\b{x} $ in the direction $\b{d} $.
To compute the above derivative, suppose that $\b{x} $ and $\b{d} $ are given. Then $f(x + \alpha \b{d}) $ is a function of $\alpha $, and
$$\pder{ f }{ \b{d}} (\b{x}) = \der{ }{ \alpha } f(\b{x} + \alpha \b{d}){\biggr\rvert}_{ \alpha = 0 } = \nabla \transpose{ f(\b{x})} \b{d} = \inner{ \nabla f(x)}{\b{d}} = \transpose{ \b{d}} \nabla f(\b{x})$$
In summary, if $\b{d} $ is a unit vector, then $\inner{ \nabla f(x)}{\b{d}} $ is the rate of increase of $f $ at the point $\b{x} $ in the direction $\b{d} $.
$\Omega \subseteq \R^n, f \in \mathcal{C}^1(\Omega)$.
If $\b{x}^{*}$ is a local minimizer of $f$ over $\Omega$, then for any feasible direction $\b{d}$ at $\b{x}^{*}$, we have
$$\inner{ \nabla f(\b{x}^{*})}{ \b{d}} = \transpose{ \b{d}} \nabla f(\b{x}^*) \geq 0$$
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
$\Omega \subseteq \R^n, f \in \mathcal{C}^1(\Omega)$.
If $\b{x}^* $ is a local minimizer of $f $ over $\Omega $ and if $\b{x}^* $ is an interior point of $\Omega $, then
$$\nabla f(\b{x}^*) = \b{0} $$
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
Let $\Omega \subseteq \R^n $, $f \in \mathcal{C}^2(\Omega) $, $\b{x}^* $ a local minimizer of $f $ over $\Omega $, and $\b{d} $ a feasible direction at $\b{x}^* $. If $\transpose{ \b{d}} f(\b{x}^*) = 0 $, then
$$\transpose{ \b{d}} \b{F}(\b{x}^*) \b{d} \geq 0$$
where $\b{F} $ is the Hessian of $f$.
by contradiction
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
Let $x^* $ be an interior point of $\Omega \subseteq \R^n $. If $\b{x}^* $ is a local minimizer of $f: \Omega \to \R, f \in \mathcal{C}^2 $, then
$$\nabla f(\b{x}^*) = \b{0} $$
and $\b{F}(\b{x}^*) $ is positive semidefinite ($F(\b{x}^*) \geq 0$); that is, for all $\b{d} \in \R^n $
$$\transpose{ \b{d}} \b{F}(\b{x}^*) \b{d} \geq 0 $$
Let $f\in\mathcal{C}^2$ be defined on a region where $\b{x}^*$ is an interior point. Suppose that
- $\nabla f(\b{x}^*) = \b{0}$; and
- $\b{F}(\b{x}^*) > 0$.
Then $\b{x}^*$ is a strict local minimizer of $f$.