Let $X$ be an (arbitrary) random variable. The distribution function $F_X: \R \to [0, 1]$ of $X$ is defined by $F_X(x) = \b{P}(X \leq x)$, for $x \in \R$. We say two random variables $X, Y$ are identically distributed if they have the same distribution function.
The random variable $X$ and $Y$ are said to be identically distributed if $F_X = F_Y$.
A function $f: \R \to [0, \infty)$ is called a density function if it satisfies the following conditions.
- For any $-\infty \leq a < b < \infty $, the integral $\int_{ a }^{ b } f_X(x) \d x $ exists.
- $\int_{ -\infty }^{ +\infty } f_X(x) \d x = 1$.
Note that density functions are nonnegative.
A random variable $X$ is said to have a continuous distribution (we also say that $X$ is continuous) if there exists a density function $f_X$ such that
$$\b{P}(X \in B) = \int_B f_X(x)\d x$$
for every subset $B$ of the real line.
We call $f_X$ the probability density function of $X$, or PDF for short.
In particular the probability that the value of $X$ falls within an interval is
$$\b{P}(a \leq X \leq b) = \int_{ a }^{ b } f_X(x) \d x $$
and can be interpreted as the area under the graph of the PDF.
For any continuous random variable $X$ and any single value $a$, we have
$$\b{P}(X = a) = \int_{ a }^{ a } f_X(x) \d x = 0$$
Hence, including or excluding the endpoint of an interval has no effect on its probability. That is
$$\b{P}(a \leq X \leq b) = \b{P}(a < X < b) = \b{P}(a \leq X < b) = \b{P}(a < X \leq b)$$
Suppose that $X$ is a continuous random variable with density function $f_X$. The Fundamental Theorem of Calculus implies that the distribution function
$$F_X(x) = \b{P}(X \leq x) = \int_{ -\infty }^{ x } f_X(t) \d t $$
is continuous at any $x \in \R$.
If $f_X$ is continuous at a point $x \in \R$, then $F_X$ is differentiable at $X $, and $F'_X(x) = f_X(x)$.
Let $X$ be a continuous random variable, and $h: \R \to \R $ a function. Then the expected value of $h(X) $ is defined by
$$E[h(X)] = \int_{ - \infty }^{ + \infty } x f_X(x) \d x$$
For an arbitrary random variable $X$, the expected value is defined as the following Lebesgue integral
$$E[X] = \int_{ \Omega } X(\omega) \d \b{P}(\omega)$$
Let $X$ be an arbitrary random variable on the sample space $\Omega $ and $a, b \in \R$ be constants. Then
$$E[aX+b] = a E[X] + b$$
If random variables $X_1, _2, …, _n $ are independent, then
$$E[ X_1, X_2, …, X_n ] = E[X_1] E[X_2] … E[X_n]$$
If a discrete random variable $X$ on the sample space $\Omega $ is non-negative, that is, $X \geq 0$, and integer valued, then
$$E\brac{ X } = \sum_{ k=1 }^{ \infty } \b{P}(X \geq k)$$
Let $X \geq 0$, that is $X : \Omega \to [0, \infty)$ . Let $h$ be a continuous differentiable function with $h(0) = 0 $. Then
$$E\brac{h(X)} = \int_{ \R } \b{P}(X > x) \d x$$
Let $X$ be an arbitrary random variable. The expectation $E\brac{ X^n } $, denoted by $\mu_n $, is called the $n$-th moment (around 0).
Let $X$ be a continuous random variable, the variance of $X $ is defined the same way as before, i.e.
$$var(X) = E[(X - E(X))^2]$$
Let $g: \R \to \R,$ then $E[g(X)] = \int_{ - \infty }^{ + \infty } g(x) f_X(x) \d x$.
- $var(aX+b) = a^2 var(X)$
- $var(X) = E[X^2] - E ^2$
Let $\lambda > 0, f_X(x) = \begin{cases} \lambda e^{- \lambda x} & \text{ for } x> 0 \br 0 & \otherwise \end{cases}$ , find expectation.
$X $ is called an exponential random variable.
</span>
</span>
<span class="proof__expand"><a>[expand]</a></span>
CDF of exponential random variable,
For $a \geq 0 $
$\begin{align*} F_X(a) &= \b{P}(X \leq a) \br &= 1- \b{P}(X > a) \br &= 1 - \int_{ a }^{ + \infty } f_X(x) \d x \br &= 1 - \int_{ a }^{ \infty } \lambda e^{- \lambda x} \br &= 1 - \int_{ \lambda a }^{ \infty } e^{-u} \d u \br &= 1 - e^{- \lambda a} \end{align*}$
$\begin{align*} E[X] &= \int_{ 0 }^{ \infty } x f(x) \d x \br &= [-xd^{- \lambda x}]_0^{ \infty } - \int_{ 0 }^{ \infty } [-e^{- \lambda x}] \d x \br &= [-0 - (-0)] + [ \frac{ -e^{- \lambda x}}{ \lambda } ]_0^\infty \br &= \frac{ 1 }{ \lambda } \end{align*}$
Time $T $ until a new light bulb burns out is an exponential random variable with parameter $\lambda $. Ariadne turns the light on, leaves the room, and when she returns, $t $ time units later, finds that the light bulb is still on, which corresponds to the event $A = {T > t} $. Let $X $ be the additional time until the light bulb burns out. What is the conditional CDF of $X $. Given the event $A $?
We have for $x \geq 0 $.
$\b{P}(X > x \mid A) = \b{P}(T > t+x \mid T > t) = \frac{ \b{P}(T > t+x\text{ and }T > t)}{ \b{P}(T > t)} = \frac{ \b{P}(T > t+x)}{ \b{P}(T > t)} = \frac{ e^{ - \lambda(t + x)}}{ e^{ - \lambda t }} = e^{ - \lambda x }$
Thus the conditional $CDF$ of $X$ is exponential with parameter $\lambda$, regardless of the time $t$ that elapsed between the lighting of the bulb and Ariadne’s arrival. This is known as the memorylessness property of the exponential. Generally, if we model the time to complete a certain operation by an exponential random variable $X$, this property implies that as long as the operation has not been completed, the remaining time up to completion has the same exponential CDF, no matter when the operation started.
Let $Y $ be geometric random variable with parameter $p$ for $k = 1, 2, 3, … $
Thus
$\begin{align*} \b{P}(Y = k) = (1 - p)^{k -1} p \end{align*}$
so,
$$\begin{align*} F_Y(n) = \b{P}(Y \leq n) &= \sum_{ k=1 }^{ n } (1 - p)^{k-1}p \br &= p \sum_{ k=1 }^{ n } (1 - p)^{k - 1} \br &= p (\frac{ 1- (1 - p)^n }{ 1- (1 - p)}) \br &= 1- (1-p)^n,\text{ for} n = 1, 2, … \end{align*}$$
So CDF of geometric random variable is:
$$F_Y(x) = \begin{cases} 1 - (1 - p)^n, \if x \leq [n, n + 1), n \geq 1 \br 0, \if x < 1 \end{cases}$$