$F = \R$ or $\C$.
Let $V $be a vector space over a $F$,
A function $\inner{}{} : V \times V \to F$ is called an inner product if it satisfies the following properties:
- $\inner{ x + z }{ y } = \inner{ x }{ y } + \inner{ z }{ y }, \forall x, y, z \in V $
- $\inner{ cx }{ y } = c \inner{ x }{ y }, \forall x, y \in V, \forall c \in F $
- $\inner{ x }{ y } = \overline{ \inner{ y }{ x }}$, where $\overline{ z } $ for $z \in F $ means complex conjugate.
- $\inner{ x }{ x } > 0 \if x \neq 0 $
$V = \R ^n , F = \R$
$x = (a_1, a_2, …, a_{ n }) $
$y = (b_1, (b_2, …, (b_{ n }) $
Define $\inner{ x }{ y } = \sum_{ i=1 }^{ n } a_ib_i $
$z = (c_1, _2, …, _{ n }) $
$\begin{align*} \inner{ x+z }{ y } &= \sum_{ i=1 }^{ n } (a_i + c_i) b_i \br &= \inner{ x }{ y } + \inner{ z }{ y } \end{align*}$
$V = \C^n, F = \C $
$x = (a_1, (a_2, …, (a_{ n }) , a_i \in \C$
$y = (b_1, (b_2, …, (b_{ n }), b_i \in \C $
Define $\inner{ x }{ y } = \sum_{ i=1 }^{ n } a_ib_i $
All first three axioms stands,
(iv) $\inner{ x }{ x } = \sum_{ i = 1 }^{ n } a_i \overline{ a_i } = \sum_{ i=1 }^{ n } \abs{ a_i }^2 $
$Continuous [0, 1] / \R = f: [0, 1] \to \R, f$is continuous on $[0, 1] $
$f,g \in continuous [0, 1] $
$\inner{ f }{ g } = \int_{ 0 }^{ 1 } f(x) g(x) \d x $
$\P_n(x) / \R$
$\inner{ f(x)}{ g(x)} = \int_{ 0 }^{ 1 } f(x) g(x) \d x$
$\P_n(x)/ \R \cong \R^{n+1} $
$a_0 + a_1x + … + a_nx^n $i.e. ${\Vcr{ a_1 }{ a_2 }{ … }{ a_n }} $
$f(x), g(x) \in \P_n(x)$
$\inner{ f(x)}{ g(x)} = \sum_{ i=1 }^{ n } a_ib_i $
$f(x) = \sum_{ i=0 }^{ n } a_i x^i$
$\inner{ f(x)}{ g(x)} = \sum_{ i=1 }^{ n } a_ib_i$
$f(x) = \sum_{ i=0 }^{ n } a_i x^i $
$g(x)= \sum_{ i=0 }^{ n } b_i x^i $
Let $\inner{ }{ }$ be an inner product space over $F$, then
- $\inner{x}{y + w} = \inner{x}{y} + \inner{x}{w} , \forall x, y, w \in V, \forall x, y \in V, \forall c \in F$
- $\inner{x}{cy} = \overline{c} \inner{x}{y} , \forall x, y \in V, \forall c \in F$
- $\inner{0}{x} = \inner{x}{0} = 0, \forall x \in V, 0 \in V $
- $\inner{x}{x} = 0 \iff x = 0$
- $\if \inner{x}{y} = \inner{x}{z}, \forall x \in V$, then $y = z $(notice: for all $x$ in $V$)
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
3. $\inner{0}{ x} = \inner{x}{ 0} = 0, \forall x \in V, 0 \in V $
$\begin{align*} \inner{0}{ x} &= \inner{0 \cdot x}{ x} \br &= 0 \inner{x}{ x} = 0 \end{align*}$
4. $\inner{x}{ x} = 0 \iff x = 0$
Assume $\inner{x}{ x} = 0 $
$x = 0 \implies \inner{0}{ 0} = 0$ by $(3) x= 2x \implies x= 0$
By contradiction assume that $x \neq 0 $, then by definition of inner product $\inner{x}{ y} > 0y$.
5. $\if \inner{x}{ y} = \inner{x}{ z} \forall x \in V$, then $y = z $
$\implies \inner{ x}{ y} - \inner{x}{ z} = 0 $
$\implies \inner{x}{ y-z} = 0 , \forall x \in V$
choose $x = y -z $
$\implies \inner{y -z}{ y -z} = 0$
$\implies y - z= 0$ by $(4) $
$\implies = z $
$\inner{ y}{ x} = \inner{z }{ x } \forall x \in V $
$\implies y = 2 $
$V $is a inner product space over a field $F $, let $x \in V$. The norm of $x$ is defined as $\norm{ x } = \sqrt{ \inner{x}{ x}} $
$\R^n$, inner product space with usual dot product. $x = (a_1, _2, …, _{ n }) $
$\norm{ x } = \sqrt{ a^2_1 + a^2_2 + … + a^2_n } $
$\begin{align*} \norm{ x } &= \sqrt{ \inner{ x}{ x}} \br &= \sqrt{ \sum_{ i = 1 }{n} a_i a_i} \br &= \sqrt{ \sum_{ i = 1 }{n} a_i^2} \br \end{align*}$
$M_{n \times n}(\R) $
$\inner{A}{ B} = \tr(B^* A) = \tr(\inv{ B }A) $
$\inner{A}{ A} = \sum_{ j=1 }^{ n } \sum_{ i = 1 }^{ n } a_{ij}^2, a_{ij} \in \R, \overline{ B } = B, B^* = (\overline{ B })^t = B^t $
$\norm{ A } = \sqrt{A, A} = \sqrt{ \sum_{ j=1 }^{ n } \sum_{ i = 1 }^{ n } a_{ij}^2 }$
Frobenius norm of a matrix.
Let $A \in M_{m \times n} (F)$. We define the conjugate transpose or adjoint of A to be the $n \times m$ matrix $A^* $such that $(A^*)_{ij} = \overline{A_{ji}}, \forall i, j$.
Let $V = M_{n \times n} (F)$, and define $\inner{ A }{ B } = tr(B^* A)$ for $A, B \in V. $(Recall that the trace of a matrix $A $ is defined by $\tr{ A } = \sum_{ i=1 }^{ n } A_{ii} $). This inner product on $M_{n \times n} (F) $ is called the Frobenius inner product.
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
A vector space $V $ over $F $ endowed with a specific inner product is called an inner product space.
If $F = \C $, we call $V $ a complex inner product space.
If $F = \R $, we call $V $ a real inner product space.
$V $is inner product space over $F $.
- $\norm{ cx } = \abs{ c } \norm{ x } \forall x \in V, v \in F$
- $\norm{ x } = 0 \iff x = 0$
- (Cauchy-Schwarz Inequality) $\abs{ \inner{x}{ y}} \leq \norm{ x } \norm{ y }, \forall x, y \in V$
- (Triangle Inequality) $\norm{ x + y } \leq \norm{ x } + \norm{ y }, \forall x, y \in V$
$\begin{align*} \norm{ x } &= \sqrt{\inner{ x}{ x}} \br &= \sqrt{ c \cdot \overline{ c } \inner{ x}{ x}} \br &= \sqrt{ \abs{ c }^2 \inner{x}{ x}} \br &= \abs{ c } \inner{x}{x} \end{align*}$
$x = 0 \iff \inner{x}{ x} = 0 \iff \sqrt{ \inner{x}{ x}} = 0 \iff \norm{ x }$
Let $V $ be a inner product space over a field $F $.
A set of vector $S \subseteq V $ called an orthogonal is $\inner{ v}{ w} = 0, \forall v, w \in S$ and $v \neq w$.
A vector $x \in V$ is called a unit vector if $\norm{ x } = 1 $
A set of vectors $S \subseteq V $ is called a orthogonal set if $S $is orthogonal i.e. $\inner{v}{ w} = 0 \forall v, u \in S, v \neq w$ and all the vectors of $S $are unit vectors i.e. $\norm{ v } = 1 v, v \in S $.
We know that $\inner{0}{ x} = \inner{x}{ 0} = 0, \forall x \in V$
i.e. $0$ vector is orthogonal to every other vector.
Because of this a orthogonal set $S$ may contain the null vector, In particular orthogonal set may not be linearly independent in general.
However, if $S $ is orthonormal set, then $0 \notin S$, because $\norm{ 0 } \neq 1$.