$\newcommand{\br}{\\}$ $\newcommand{\R}{\mathbb{R}}$ $\newcommand{\Q}{\mathbb{Q}}$ $\newcommand{\Z}{\mathbb{Z}}$ $\newcommand{\N}{\mathbb{N}}$ $\newcommand{\C}{\mathbb{C}}$ $\newcommand{\P}{\mathbb{P}}$ $\newcommand{\F}{\mathbb{F}}$ $\newcommand{\L}{\mathcal{L}}$ $\newcommand{\spa}[1]{\text{span}(#1)}$ $\newcommand{\dist}[1]{\text{dist}(#1)}$ $\newcommand{\max}[1]{\text{max}(#1)}$ $\newcommand{\min}[1]{\text{min}(#1)}$ $\newcommand{\supr}{\text{sup}}$ $\newcommand{\infi}{\text{inf}}$ $\newcommand{\ite}[1]{\text{int}(#1)}$ $\newcommand{\ext}[1]{\text{ext}(#1)}$ $\newcommand{\bdry}[1]{\partial #1}$ $\newcommand{\argmax}[1]{\underset{#1}{\text{argmax }}}$ $\newcommand{\argmin}[1]{\underset{#1}{\text{argmin }}}$ $\newcommand{\set}[1]{\left\{#1\right\}}$ $\newcommand{\emptyset}{\varnothing}$ $\newcommand{\tilde}{\text{~}}$ $\newcommand{\otherwise}{\text{ otherwise }}$ $\newcommand{\if}{\text{ if }}$ $\newcommand{\proj}{\text{proj}}$ $\newcommand{\union}{\cup}$ $\newcommand{\intercept}{\cap}$ $\newcommand{\abs}[1]{\left| #1 \right|}$ $\newcommand{\norm}[1]{\left\lVert#1\right\rVert}$ $\newcommand{\pare}[1]{\left(#1\right)}$ $\newcommand{\brac}[1]{\left[#1\right]}$ $\newcommand{\t}[1]{\text{ #1 }}$ $\newcommand{\head}{\text H}$ $\newcommand{\tail}{\text T}$ $\newcommand{\d}{\text d}$ $\newcommand{\limu}[2]{\underset{#1 \to #2}\lim}$ $\newcommand{\limsupu}[2]{\underset{#1 \to #2}{\lim\supr}}$ $\newcommand{\liminfu}[2]{\underset{#1 \to #2}{\lim\infi}}$ $\newcommand{\limd}[3]{\underset{#1 \to #2; #3}\lim}$ $\newcommand{\der}[2]{\frac{\d #1}{\d #2}}$ $\newcommand{\derw}[2]{\frac{\d #1^2}{\d^2 #2}}$ $\newcommand{\pder}[2]{\frac{\partial #1}{\partial #2}}$ $\newcommand{\pderw}[2]{\frac{\partial^2 #1}{\partial #2^2}}$ $\newcommand{\pderws}[3]{\frac{\partial^2 #1}{\partial #2 \partial #3}}$ $\newcommand{\inv}[1]{{#1}^{-1}}$ $\newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $\newcommand{\nullity}[1]{\text{nullity}(#1)}$ $\newcommand{\rank}[1]{\text{rank }#1}$ $\newcommand{\nullspace}[1]{\mathcal{N}\pare{#1}}$ $\newcommand{\range}[1]{\mathcal{R}\pare{#1}}$ $\newcommand{\var}[1]{\text{var}\pare{#1}}$ $\newcommand{\cov}[2]{\text{cov}(#1, #2)}$ $\newcommand{\tr}[1]{\text{tr}(#1)}$ $\newcommand{\oto}{\text{ one-to-one }}$ $\newcommand{\ot}{\text{ onto }}$ $\newcommand{\ipto}{\overset{\text{i.p.}}\longrightarrow}$ $\newcommand{\asto}{\overset{\text{a.s.}}\longrightarrow}$ $\newcommand{\expdist}[1]{\text{ ~ Exp}(#1)}$ $\newcommand{\binomdist}[2]{\text{ ~ B}(#1, #2)}$ $\newcommand{\unifdist}[1]{\text{ ~ Unif}(#1)}$ $\newcommand{\normdist}[2]{\text{ ~ N}(#1,#2)}$ $\newcommand{\berndist}[2]{\text{ ~ Bernoulli}(#1,#2)}$ $\newcommand{\geodist}[1]{\text{ ~ Geometric}(#1)}$ $\newcommand{\poissondist}[1]{\text{ ~ Poisson}(#1)}$ $\newcommand{\ceil}[1]{\lceil#1\rceil}$ $\newcommand{\floor}[1]{\lfloor#1\rfloor}$ $\newcommand{\Re}[1]{\text{Re}(#1)}$ $\newcommand{\Im}[1]{\text{Im}(#1)}$ $\newcommand{\dom}[1]{\text{dom}(#1)}$ $\newcommand{\fnext}[1]{\overset{\sim}{#1}}$ $\newcommand{\transpose}[1]{{#1}^{\text{T}}}$ $\newcommand{\b}[1]{\boldsymbol{#1}}$ $\newcommand{\None}[1]{}$ $\newcommand{\Vcw}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Vce}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Vcr}[4]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \end{bmatrix}}$ $\newcommand{\Vct}[5]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \end{bmatrix}}$ $\newcommand{\Vcy}[6]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \end{bmatrix}}$ $\newcommand{\Vcu}[7]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \br #7 \end{bmatrix}}$ $\newcommand{\vcw}[2]{\begin{matrix} #1 \br #2 \end{matrix}}$ $\newcommand{\vce}[3]{\begin{matrix} #1 \br #2 \br #3 \end{matrix}}$ $\newcommand{\vcr}[4]{\begin{matrix} #1 \br #2 \br #3 \br #4 \end{matrix}}$ $\newcommand{\vct}[5]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \end{matrix}}$ $\newcommand{\vcy}[6]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \end{matrix}}$ $\newcommand{\vcu}[7]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \br #7 \end{matrix}}$ $\newcommand{\Mqw}[2]{\begin{bmatrix} #1 & #2 \end{bmatrix}}$ $\newcommand{\Mqe}[3]{\begin{bmatrix} #1 & #2 & #3 \end{bmatrix}}$ $\newcommand{\Mqr}[4]{\begin{bmatrix} #1 & #2 & #3 & #4 \end{bmatrix}}$ $\newcommand{\Mqt}[5]{\begin{bmatrix} #1 & #2 & #3 & #4 & #5 \end{bmatrix}}$ $\newcommand{\Mwq}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Meq}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Mrq}[4]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \end{bmatrix}}$ $\newcommand{\Mtq}[5]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \end{bmatrix}}$ $\newcommand{\Mqw}[2]{\begin{bmatrix} #1 & #2 \end{bmatrix}}$ $\newcommand{\Mwq}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Mww}[4]{\begin{bmatrix} #1 & #2 \br #3 & #4 \end{bmatrix}}$ $\newcommand{\Mqe}[3]{\begin{bmatrix} #1 & #2 & #3 \end{bmatrix}}$ $\newcommand{\Meq}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Mwe}[6]{\begin{bmatrix} #1 & #2 & #3\br #4 & #5 & #6 \end{bmatrix}}$ $\newcommand{\Mew}[6]{\begin{bmatrix} #1 & #2 \br #3 & #4 \br #5 & #6 \end{bmatrix}}$ $\newcommand{\Mee}[9]{\begin{bmatrix} #1 & #2 & #3 \br #4 & #5 & #6 \br #7 & #8 & #9 \end{bmatrix}}$
Definition: Poisson Process

An arrival process is called a Poisson process with rate $\lambda$ if it has the following properties:

  1. (Time-homogeneity) The probability $P(k, \tau) $ of $k $ arrivals is the same for all intervals of the same length $\tau $.

  2. (Independence) The number of arrivals during a particular interval is independent of the history of arrivals outside this interval.

  3. (Small interval probabilities) The probabilities $P(k, \tau) $ satisfy

$$\begin{align*} P(0, \tau) &= 1 - \lambda \tau + o(\tau), \br P(1, \tau) &= \lambda \tau + o_1(\tau), \br P(k, \tau) &= o_k(\tau),\text{ for } k = 2, 3, … \end{align*}$$

Here, $o(\tau) $ and $o_k(\tau) $ are functions of $\tau $ that satisfy

$$\limu{ \tau }{ o } \frac{ o(\tau)}{ \tau } = 0, \limu{ \tau }{ o } \frac{ o_k(\tau)}{ \tau } = 0 $$

Remarks

The first property states that arrivals are “equally likely” at all times.
The arrivals during any time interval of length $\tau$ have the same distribution.
This a counterpart to the assumption that the success probability $p$ in a Bernoulli process is the same for all trials.

The second property, consider a particular interval $[t, t']$, of length $t' - t$.
The unconditional probability of $k$ arrivals during that interval is $P(k, t'-t)$.
Suppose now that we are given complete or partial information on the arrivals outside this interval, property $2$ states that this information is irrelevant: the conditional probability of $k$ arrivals during $[t, t']$ remains equal to the unconditional probability $P(k,t'-t)$. This property is analogous to the independence of trials in a Bernoulli process.

The third property is critical. The $o(\tau) $ and $o_k(\tau) $ terms are meant to be negligible in comparison to $\tau $, when the interval length $\tau $ is very small. They can be thought of as the $O(\tau^2) $ terms in a Taylor series expansion of $P(k, \tau) $.
Thus for small $\tau $, the probability of a single arrival is roughly $\lambda \tau $, plus a negligible term. Similarly, for small $\tau $, the probability of zero arrivals is roughly $1 - \lambda \tau $.
Finally, the probability of two or more arrivals is negligible in comparison to $P(1, \tau) $, as $\tau $ becomes smaller.

Note: Number of Arrivals in an Interval

Consider a fixed time interval of length $\tau $, and partition it into $\tau / \delta $ periods of length $\delta $, where $\delta $ is a very small number. The probability of more than two arrivals during any period can be neglected, because of property $(3)$ and the preceding discussion.

Different periods are independent, by property $(2)$.

Furthermore, each period has one arrival with probability approximately equal to $\lambda \delta$, or zero arrivals with probability approximately equal to $1 - \lambda \delta $.

Therefore, the process being studied can be approximated by a Bernoulli process, with the approximation becoming more and more accurate as $\delta $ becomes smaller.

The probability $P(k, \tau) $ of $k $ arrivals in time $\tau $ is approximately the same as the (binomial) probability of $k$ successes in $n = r / \delta $ independent Bernoulli trials with success probability $p = \lambda \delta$ at each trial. While keeping the length $\tau $ of the number $n$ of periods goes to infinity, while the product $np$ remains constant and equal to $\lambda \tau$.

Under these, circustances, we saw in the previous section that the binomial PMF converges to a Poisson PMF with parameter $\lambda \tau $. We are then led to the important conclusion that

$$P(k, \tau) = e^{- \lambda \tau} \frac{(\lambda \tau)^k }{ k! }, k = 0, 1, 2, … $$

Note that a Taylor series expansion of $e^{- \lambda \tau} $ yields

$$\begin{align*} P(0, \tau) &= e^{- \lambda \tau} = 1 - \lambda \tau + o(\tau) \br P(1, \tau) &= \lambda \tau e^{- \lambda \tau} = \lambda \tau - \lambda^2 \tau^2 + O(\tau^3) = \lambda \tau + o_1(\tau) \end{align*}$$

consistent with property $(3)$.

We already know the mean and variance of the Poisson PMF.

$$E[N_ \tau] = \lambda \tau, \var{ N_ \tau } = \lambda \tau$$

where $N_ \tau $ denotes the number of arrivals during a time interval of length $\tau $.

Properties: Random Variables Associated With the Poisson Process and Their Properties
  • The Poisson with parameter $\lambda \tau$, the number $N_ \tau $ of arrivals in a Poisson process with rate $\lambda $, over an interval of length $\tau $.

$$p_{N_ \tau}(k) = P(k, \tau) = e^{ - \lambda \tau } \frac{(\lambda \tau)^k }{ k! }, k = 0, 1, 2, … $$

$$E[N_ \tau] = \lambda \tau, \var{ N_ \tau } = \lambda \tau $$

  • The exponential with parameter $\lambda$, the time $T$ until the first arrival.

$$f_T(t) = \lambda e^{ - \lambda t }, t \geq 0$$

$$E [ T ] = \frac{ 1 }{ \lambda }, \var{ T } = \frac{ 1 }{ \lambda^2 } $$

Example: 0 or 1 arrivals

You get email according to a Poisson process at a rate of $\lambda = 0.2 $ messages per hour. You check your email every hour. What is the probability of finding $0$ and $1$ new messages?

The probabilities can be found using the Poisson PMF $\frac{ e^{ - \lambda \tau } (\lambda \tau)^k }{ k! } $. We have $\tau=1$.

$$P(0, 1) = e^{-0.2} = 0.819, P(1,1) = 0.2 e^{ -0.2 } = 0.164 $$

Example: The Sum of Independent Poisson Random Variables is Poisson

Arrivals of customers at the local supermarket are modeled by a Poisson process with a rate of $\lambda = 10$ customers per minute. Let $M$ be the number of customers arriving between $9:00 $ and $9:10$. Also, let $N$ be the number of customers arriving between $9:30$ and $9:35$. What is the distribution of $M+N$?