$\newcommand{\br}{\\}$ $\newcommand{\R}{\mathbb{R}}$ $\newcommand{\Q}{\mathbb{Q}}$ $\newcommand{\Z}{\mathbb{Z}}$ $\newcommand{\N}{\mathbb{N}}$ $\newcommand{\C}{\mathbb{C}}$ $\newcommand{\P}{\mathbb{P}}$ $\newcommand{\F}{\mathbb{F}}$ $\newcommand{\L}{\mathcal{L}}$ $\newcommand{\spa}[1]{\text{span}(#1)}$ $\newcommand{\dist}[1]{\text{dist}(#1)}$ $\newcommand{\max}[1]{\text{max}(#1)}$ $\newcommand{\min}[1]{\text{min}(#1)}$ $\newcommand{\supr}[1]{\text{sup}(#1)}$ $\newcommand{\infi}[1]{\text{inf}(#1)}$ $\newcommand{\set}[1]{\left\{#1\right\}}$ $\newcommand{\emptyset}{\varnothing}$ $\newcommand{\otherwise}{\text{ otherwise }}$ $\newcommand{\if}{\text{ if }}$ $\newcommand{\proj}{\text{proj}}$ $\newcommand{\union}{\cup}$ $\newcommand{\intercept}{\cap}$ $\newcommand{\abs}[1]{\left| #1 \right|}$ $\newcommand{\norm}[1]{\left\lVert#1\right\rVert}$ $\newcommand{\pare}[1]{\left(#1\right)}$ $\newcommand{\t}[1]{\text{ #1 }}$ $\newcommand{\head}{\text H}$ $\newcommand{\tail}{\text T}$ $\newcommand{\d}{\text d}$ $\newcommand{\limu}[2]{\underset{#1 \to #2}\lim}$ $\newcommand{\inv}[1]{{#1}^{-1}}$ $\newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $\newcommand{\nullity}[1]{\text{nullity}(#1)}$ $\newcommand{\rank}[1]{\text{rank }#1}$ $\newcommand{\var}[1]{\text{var}(#1)}$ $\newcommand{\tr}[1]{\text{tr}(#1)}$ $\newcommand{\oto}{\text{ one-to-one }}$ $\newcommand{\ot}{\text{ onto }}$ $\newcommand{\ceil}[1]{\lceil#1\rceil}$ $\newcommand{\floor}[1]{\lfloor#1\rfloor}$ $\newcommand{\Re}[1]{\text{Re}(#1)}$ $\newcommand{\Im}[1]{\text{Im}(#1)}$ $\newcommand{\dom}[1]{\text{dom}(#1)}$ $\newcommand{\fnext}[1]{\overset{\sim}{#1}}$ $\newcommand{\transpose}[1]{#1^{\text{T}}}$ $\newcommand{\Vcw}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Vce}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Vcr}[4]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \end{bmatrix}}$ $\newcommand{\Vct}[5]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \end{bmatrix}}$ $\newcommand{\Vcy}[6]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \end{bmatrix}}$ $\newcommand{\Vcu}[7]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \br #7 \end{bmatrix}}$ $\newcommand{\vcw}[2]{\begin{matrix} #1 \br #2 \end{matrix}}$ $\newcommand{\vce}[3]{\begin{matrix} #1 \br #2 \br #3 \end{matrix}}$ $\newcommand{\vcr}[4]{\begin{matrix} #1 \br #2 \br #3 \br #4 \end{matrix}}$ $\newcommand{\vct}[5]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \end{matrix}}$ $\newcommand{\vcy}[6]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \end{matrix}}$ $\newcommand{\vcu}[7]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \br #7 \end{matrix}}$ $\newcommand{\Mqw}[2]{\begin{bmatrix} #1 & #2 \end{bmatrix}}$ $\newcommand{\Mqe}[3]{\begin{bmatrix} #1 & #2 & #3 \end{bmatrix}}$ $\newcommand{\Mqr}[4]{\begin{bmatrix} #1 & #2 & #3 & #4 \end{bmatrix}}$ $\newcommand{\Mqt}[5]{\begin{bmatrix} #1 & #2 & #3 & #4 & #5 \end{bmatrix}}$ $\newcommand{\Mwq}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Meq}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Mrq}[4]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \end{bmatrix}}$ $\newcommand{\Mtq}[5]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \end{bmatrix}}$ $\newcommand{\Mqw}[2]{\begin{bmatrix} #1 & #2 \end{bmatrix}}$ $\newcommand{\Mwq}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Mww}[4]{\begin{bmatrix} #1 & #2 \br #3 & #4 \end{bmatrix}}$ $\newcommand{\Mqe}[3]{\begin{bmatrix} #1 & #2 & #3 \end{bmatrix}}$ $\newcommand{\Meq}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Mwe}[6]{\begin{bmatrix} #1 & #2 & #3\br #4 & #5 & #6 \end{bmatrix}}$ $\newcommand{\Mew}[6]{\begin{bmatrix} #1 & #2 \br #3 & #4 \br #5 & #6 \end{bmatrix}}$ $\newcommand{\Mee}[9]{\begin{bmatrix} #1 & #2 & #3 \br #4 & #5 & #6 \br #7 & #8 & #9 \end{bmatrix}}$ $\newcommand{\b}[1]{\boldsymbol{#1}}$
Definition: Quadratic Form

A quadratic form $f: \R^n \to \R $ is a function

$$f(x) = \transpose{ x }Qx $$

where $Q $ is an $n \times n$ real matrix. There is no loss of generality in assuming $Q$ to be symmetric, that is, $Q = \transpose{ Q }$. For if the matrix $Q $ is not symmetric, we can always replace it with the symmetric matrix.

$$Q_0 = \transpose{ Q_0 } = \frac{ 1 }{ 2 } (Q + \transpose{ Q }) $$

Note that

$$\transpose{ x } Q x = \transpose{ x } Q_0 x = \transpose{ x } (\frac{ 1 }{ 2 }Q + \frac{ 1 }{ 2 } \transpose{ Q })x $$

Definition: Positive (Semi)definite / Negative (Semi)definite

A quadratic form $\transpose{ \b{x}} \b{Q} \b{x}, \b{Q} = \transpose{ \b{Q}} $, is said to be

  • positive definite if $\forall \b{x}, \transpose{ \b{x}} \b{Q} \b{x} > 0 $;
  • positive semidefinite if $\forall \b{x}, \transpose{ \b{x}} \b{Q} \b{x} \geq 0$;
  • negative definite if $\forall \b{x}, \transpose{ \b{x}} \b{Q} \b{x} < 0$;
  • negative semidefinite if $\forall \b{x}, \transpose{ \b{x}} \b{Q} \b{x} \leq 0$.
Definition: Principal Minors

The principle minors are $\det \b{Q} $ itself and the determinants of matrices obtained by successively removing an $i$th row and an $i$th column. That is, the principal minors are:

$$\det {\Mqr{{\vcr{ q_{i_1i_1}}{ q_{i_2i_1}}{ \vdots }{ q_{i_pi_1}}}}{{\vcr{ q_{i_1i_2}}{ q_{i_2i_2}}{ \vdots }{q_{i_pi_1}}}}{{\vcr{ … }{ … }{ }{ … }}}{{\vcr{ q_{i_1i_p}}{ q_{i_2i_p}}{ \vdots }{q_{i_pi_p}}}}}\ \ , p < n, \forall 1 \leq k \leq p, 1 \leq i_k \leq n $$

Definition: Leading Principal Minors

The leading principal minors are $\det \b{Q} $ and the minors obtained by successively removing the last row and the last column. That is, the leading principal minors are:

$$\Delta_1 = q_{11}, \Delta_2 = \det {\Mww{ q_{11}}{ q_{12}}{ q_{21}}{ q_{22}}}, \Delta_3 = \det {\Mee{ q_{11}}{ q_{12}}{ q_{13}}{ q_{21}}{ q_{22}}{ q_{23}}{ q_{31}}{ q_{32}}{ q_{33}}}, … , \Delta_n = \det \b{Q} $$

Theorem : Sylvester's Criterion

A quadratic form $\transpose{\b{x}}\b{Q}\b{x} = \transpose{\b{Q}}$ is positive definite

$\iff$ the leading principal minors of $\b{Q}$ are positive.