Let $X$ be a random variable, we define the moment generating function (or transform) of $X$ to be the function $M_X: \R \to \R $ such that
$$M_X(t) = E\brac{ e^{Xt}}, t \in \R $$
Discrete:
$$M_X(t) = \sum_{x \in \R} e^{xt} p_X(t)$$
Continuous:
$$M_X(t) = \int_{ \R } e^{xt} f_X(x) \d x$$
Let $X$ be a random variable with $\begin{align*} \frac{ 1 }{ 2 } & x = 2 \br \frac{ 1 }{ 6 } & x = 3 \br \frac{ 1 }{ 3 } & x = 5 \br 0 & \otherwise \end{align*}$
$M_X(t) = E\brac{ e^{Xt}} = \frac{ 1 }{ 2 } e^{2t} + \frac{ 1 }{ 6 } e^{3t} + \frac{ 1 }{ 3 } e^{5t}$
Let $M_X(s)$ be the transform associated with a random variable $X$. Consider a new random variable $Y = aX + b$, $ a, b \in \R $.
$$M_Y(t) = e^{bt} M_X(at)$$
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
$Y \text{~} N(\mu, \sigma^2), X \text{~} N(0, 1)$
$Y = \sigma X + \mu $
$f_X(x) = \frac{ 1 }{ \sqrt[ ]{ 2 \pi }} e ^{- \frac{ x^2 }{ 2 }} $
$M_X(t) = \int_{ \R } e^{xt} f_X(x) \d x = \int_{ \R } \frac{ 1 }{ \sqrt[ ]{ 2 \pi } e ^{ - \frac{ x^2 }{ 2 }}} \d x = \int_{ \R } e^{ - (\frac{ x^2 }{ 2 } - xt + \frac{ t^2 }{ 2 })} e^{ \frac{ t^2 }{ 2 }} \d x = e^ { \frac{ t^2 }{ 2 }}$
$X \text{~} \text{ Poisson}(\lambda) , p_X(k) = \begin{cases} \frac{\lambda^k}{k!} e^{-\lambda} & k = 0, 1, 2, … \br 0 & \otherwise \end{cases}$
$M_X(t) = \sum_{ k = 0 }^{ \infty } e^{kt} \frac{ \lambda^k e^{- \lambda}}{ k! } = e ^{ - \lambda } e^{ e^t \lambda } = e^{ \lambda (e^t - 1)}, \forall t \in \R$
$M_X(t)$ may not be defined for all $t \in \R $
$f_X(x) = \begin{cases} \frac{ 1 }{ x^2 } & x \geq 1\br 0, & x < 1 \end{cases}$
$M_X(t) = \int_{ 1 }^{ \infty } \frac{ e^{xt}}{ x^2 } \d x $ does not converge for $t > 0 $
$\int_{ \R } x \d x = \int_{ 1 }^{ \infty } \frac{ 1 }{ x^2 } \d x = 1$
Moment generating function is only defined for $(-\infty, 0]$.
For $t = 0 $, $M_X(0) = E\brac{ e^{0x}} = E\brac{ 1 } = 1$
If $h: \R \to \R $ such that $h(0) \neq 1 $, then $h $ cannot be the MGF function for some random variable $X$.
That is, here is no random variable $X$, such that $M_X(t) = h(t), \forall t \in \R$.
Let $X $ and $Y $ be two random variable with $M_X(t) $ and $M_Y(t) $ defined for all $t \in \R $ (for all $t$ in some open interval $(a, b)$ in $\R$). If $M_X(t) = M_Y(t), \forall t \in \R$($\forall t \in (a, b)$).
Then $F_X(s) = F_Y(s) \forall s \in R$. $X $ and $Y $ have the same distribution.
Suppose a random variable $X $ has MGF given as
$$M_X(t) = \frac{ 1 }{ y } e^{-t} + \frac{ 1 }{ 2 } + \frac{ 1 }{ 8 } e^{yt} + \frac{ 1 }{ 8 } e^{5t} $$
What is the distribution of $X $
$$ p_X(x) = \begin{cases} \frac{ 1 }{ 4 } & x = -1 \br \frac{ 1 }{ 2 } & x= 0 \br \frac{ 1 }{ 8 } & x = 4 \br \frac{ 1 }{ 8 } & x = 5 \br 0 & \otherwise \end{cases} $$
Recall that the nth moment of a random variable $X $ is $E\brac{ X^n } $ from MGF to moments.
$\der{ }{ t } M_X(t) = \der{ }{ t } E\brac{ e^{xt}} = E\brac{ \der{ }{ t } e^{xt}} = E\brac{ x e^{xt}} $
$\der{ }{ t } M_X(t) \big |_{t = 0} = E\brac{ X } $
$E\brac{ e^{xt}} = E\brac{ 1 + \frac{ xt }{ 1 ! } + \frac{ \pare{xt}^2 }{ 2! } + … } = \sum_{ n = 0 }^{ \infty } E\brac{ \frac{(xt)^n }{ n! }} = \sum_{ n = 0 }^{ \infty } \frac{ E\brac{ X } t^n }{ n! } $
This is the “exponential” quality function for the sequence $E\brac{ X^2 } \big |^ \infty _{n = 0} $
$\frac{ \d ^n }{ \d t^n } E\brac{ e^{xt}} \big |_{t = 0} = \frac{ \d ^n }{ \d t^n } \frac{ \pare{E\brac{ x^n } t^n }}{ n! } \big |_ {t = 0} = E\brac{ X^n } $
If $M_X(t) $ is defined in a neighborhood of $0$ then
$$\frac{ \d ^n }{ \d t^n } M_X(t) \big | _{t = 0} = E\brac{ X^n }$$
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
$X \text{~}\text{ Exp}(\lambda), \lambda > 0$
$f_X(x) = \begin{cases} \lambda e^{ - \lambda }x & x \geq 0 \br 0 & \otherwise \end{cases}$
$M_X(x) = \int_{ x \in \R } e^{xt} f_X(x) \d x = \int_{ 0 }^{ \infty } e^{xt} (\lambda e^{ - \lambda x }) \d x = \lambda \int_{ 0 }^{ \infty } e^{x (t - \lambda)} \d x $
for $t < \lambda $
$= \frac{ \lambda }{ t - \lambda } e^{x (t - \lambda)} \big | ^ \infty _0 = \frac{ \lambda^0 }{ t - \lambda } (0 - 1) = \frac{ \lambda }{ \lambda - t } $
$\der{ }{ t } M_X(t) \big | _{t = 0} = \frac{ \lambda }{(\lambda - t)^2 } \big | _{t = 0} = \frac{ 1 }{ \lambda } = E\brac{ X } $
$\frac{ \d ^n }{ \d t^n } M_X(t) \big | _{t = 0} = \frac{ n! }{ \lambda^n }$(by induction)
Note that $\frac{ \lambda }{ \lambda - t } = \frac{ 1 }{ 1 - \frac{ t }{ \lambda }} = 1 + \frac{ t }{ \lambda } + \frac{ t^2 }{ \lambda^2 } + … = \sum_{ n=0 }^{ \infty } \frac{ t^n }{ \lambda^n }$