$\newcommand{\br}{\\}$ $\newcommand{\R}{\mathbb{R}}$ $\newcommand{\Q}{\mathbb{Q}}$ $\newcommand{\Z}{\mathbb{Z}}$ $\newcommand{\N}{\mathbb{N}}$ $\newcommand{\C}{\mathbb{C}}$ $\newcommand{\P}{\mathbb{P}}$ $\newcommand{\F}{\mathbb{F}}$ $\newcommand{\L}{\mathcal{L}}$ $\newcommand{\spa}[1]{\text{span}(#1)}$ $\newcommand{\dist}[1]{\text{dist}(#1)}$ $\newcommand{\max}[1]{\text{max}(#1)}$ $\newcommand{\min}[1]{\text{min}(#1)}$ $\newcommand{\supr}[1]{\text{sup}(#1)}$ $\newcommand{\infi}[1]{\text{inf}(#1)}$ $\newcommand{\set}[1]{\{#1\}}$ $\newcommand{\emptyset}{\varnothing}$ $\newcommand{\otherwise}{\text{ otherwise }}$ $\newcommand{\if}{\text{ if }}$ $\newcommand{\proj}{\text{proj}}$ $\newcommand{\union}{\cup}$ $\newcommand{\intercept}{\cap}$ $\newcommand{\abs}[1]{| #1 |}$ $\newcommand{\norm}[1]{\left\lVert#1\right\rVert}$ $\newcommand{\pare}[1]{\left(#1\right)}$ $\newcommand{\t}[1]{\text{ #1 }}$ $\newcommand{\head}{\text H}$ $\newcommand{\tail}{\text T}$ $\newcommand{\d}{\text d}$ $\newcommand{\limu}[2]{\underset{#1 \to #2}\lim}$ $\newcommand{\inv}[1]{{#1}^{-1}}$ $\newcommand{\inner}[2]{\langle #1, #2 \rangle}$ $\newcommand{\nullity}[1]{\text{nullity}(#1)}$ $\newcommand{\rank}[1]{\text{rank }#1}$ $\newcommand{\var}[1]{\text{var}(#1)}$ $\newcommand{\tr}[1]{\text{tr}(#1)}$ $\newcommand{\oto}{\text{ one-to-one }}$ $\newcommand{\ot}{\text{ onto }}$ $\newcommand{\Re}[1]{\text{Re}(#1)}$ $\newcommand{\Im}[1]{\text{Im}(#1)}$ $\newcommand{\Vcw}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Vce}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Vcr}[4]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \end{bmatrix}}$ $\newcommand{\Vct}[5]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \end{bmatrix}}$ $\newcommand{\Vcy}[6]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \end{bmatrix}}$ $\newcommand{\Vcu}[7]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \br #7 \end{bmatrix}}$ $\newcommand{\vcw}[2]{\begin{matrix} #1 \br #2 \end{matrix}}$ $\newcommand{\vce}[3]{\begin{matrix} #1 \br #2 \br #3 \end{matrix}}$ $\newcommand{\vcr}[4]{\begin{matrix} #1 \br #2 \br #3 \br #4 \end{matrix}}$ $\newcommand{\vct}[5]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \end{matrix}}$ $\newcommand{\vcy}[6]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \end{matrix}}$ $\newcommand{\vcu}[7]{\begin{matrix} #1 \br #2 \br #3 \br #4 \br #5 \br #6 \br #7 \end{matrix}}$ $\newcommand{\Mqw}[2]{\begin{bmatrix} #1 & #2 \end{bmatrix}}$ $\newcommand{\Mqe}[3]{\begin{bmatrix} #1 & #2 & #3 \end{bmatrix}}$ $\newcommand{\Mqr}[4]{\begin{bmatrix} #1 & #2 & #3 & #4 \end{bmatrix}}$ $\newcommand{\Mqt}[5]{\begin{bmatrix} #1 & #2 & #3 & #4 & #5 \end{bmatrix}}$ $\newcommand{\Mwq}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Meq}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Mrq}[4]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \end{bmatrix}}$ $\newcommand{\Mtq}[5]{\begin{bmatrix} #1 \br #2 \br #3 \br #4 \br #5 \end{bmatrix}}$ $\newcommand{\Mqw}[2]{\begin{bmatrix} #1 & #2 \end{bmatrix}}$ $\newcommand{\Mwq}[2]{\begin{bmatrix} #1 \br #2 \end{bmatrix}}$ $\newcommand{\Mww}[4]{\begin{bmatrix} #1 & #2 \br #3 & #4 \end{bmatrix}}$ $\newcommand{\Mqe}[3]{\begin{bmatrix} #1 & #2 & #3 \end{bmatrix}}$ $\newcommand{\Meq}[3]{\begin{bmatrix} #1 \br #2 \br #3 \end{bmatrix}}$ $\newcommand{\Mwe}[6]{\begin{bmatrix} #1 & #2 & #3\br #4 & #5 & #6 \end{bmatrix}}$ $\newcommand{\Mew}[6]{\begin{bmatrix} #1 & #2 \br #3 & #4 \br #5 & #6 \end{bmatrix}}$ $\newcommand{\Mee}[9]{\begin{bmatrix} #1 & #2 & #3 \br #4 & #5 & #6 \br #7 & #8 & #9 \end{bmatrix}}$
Definition: Monotonic Sequences

A sequence $(s_n) $ of real numbers is called an increasing sequence if $s_n \leq s_{n+1} $ for all $n $, and $(s_n) $ is called a decreasing sequence if $s_n \geq s_{n+1} $ for all $n $. Note that if $(s_n) $ is increasing, then $s_n \leq s_m$ whenever $n < m$. A sequence that is increasing or decreasing will be called a monotone sequence or a monotonic sequence.

Theorem

All bounded monotone sequences converge.

Proof [expand]

Let $(s_n) $ be a bounded increasing sequence.

Let $S $ denote the set $\set{ s_n : n \in \N } $.

Want to show $\lim s_n = u$.

Let $\epsilon > 0 $. Since $\supr{ S } - \epsilon $ is not an upper bound for $S $, there exists $N $ such that $s_N > u - \epsilon $.

Since $(s_n) $ is increasing, we have $s_N \leq s_n $ for all $n \geq \N $.

Also, $s_n \leq \supr{ S } $ for all $n $.

$\therefore \supr{ S } - \epsilon < s_n \leq u $, which implies $\abs{ s_n - u } < \epsilon$.

This shows $\lim s_n = u$.

Similar for decreasing sequences.

Theorem

If $(s_n)$ is an unbounded increasing sequence, then $\lim s_n = +\infty $. If $(s_n)$ is an unbounded decreasing sequence, then $\lim s_n = -\infty $.

Proof [expand]

Let $(s_n) $ be an unbounded increasing sequence. Let $M > 0 $.

Since the set $\set{ s_n: n \in \N }$ is unbounded above, for some $N $ in $\N $ we have $s_N > M $.

Clearly $n > N$ implies $s_n \geq s_N > M $, so $\lim s_n = +\infty$.

Corollary

If $s_n $ is a monotone sequence, then the sequence either converges, diverges to $+\infty$ or diverges to $-\infty $. Thus $\lim s_n $ always exists for monotone sequences.

Definition

Let $(s_n) $ be a sequence in $\R $. We define

$\lim \supr{ s_n } = \limu{ N }{ \infty } \supr{ \set{ s_n : n > N }} $

$\lim \infi{ s_n } = \limu{ N }{ \infty } \infi{ \set{ s_n : n > N }} $

Note

In this definition we do not restrict $(s_n) $ to be bounded. However, we adopt the following conventions. If $(s_n) $ is not bounded above, $\supr{ s_n : n > N } = +\infty$ for all $N $ and we decree $\lim \supr{ s_n } = +\infty$. Likewise, if $(s_n) $ is not bounded below, $\infi{ \set{ s_n : n > N }} = -\infty $ for all $N $ and we decree $\lim \infi{ s_n } = -\infty $.

$\lim \supr{ s_n }$ may not equal $\supr{ s_n : n \in \N } $, but $\lim \supr{ s_n } \leq \supr{ \set{ s_n : n \in \N }}$.

Theorem

Let $(s_n) $ be a sequence in $\R $.

1) If $\lim s_n $ exists, then $\lim \infi{ s_n } = \lim s_n = \lim \supr{ s_n } $.

2) If $\lim s_n = \lim \supr{ s_n }$, then $\lim s_n$ exists and $\lim s_n = \lim \infi{ s_n } = \lim \supr{ s_n } $.

Definition

A sequence $(s_n) $ of real numbers is called a Cauchy sequence if

for each $\epsilon > 0 $ there exists a number $N $ such that

$$m,n > N \implies \abs{ s_n - s_m } < \epsilon $$

Lemma

Convergent sequences are Cauchy sequences.

Proof [expand]

Suppose $\lim s_n = s$.

$$\abs{ s_n - s_m } = \abs{ s_n - s + s - s_m } \leq \abs{ s_n - s } + \abs{ s - s_m }$$

Let $\epsilon > 0 $, then there exists $N$ such that

$n > N $ implies $\abs{ s_n - s } < \epsilon /2 $

$m > N $ implies $\abs{ s_m - s } < \epsilon /2 $

so

$m, n > N $ implies $\abs{ s_n - s_m } \leq \abs{ s_n - s } + \abs{ s - s_m } < \frac{ \epsilon }{ 2 } + \frac{ \epsilon }{ 2 } = \epsilon$

Thus $(s_n)$ is a Cauchy sequence.

Lemma

Cauchy sequences are bounded.

Proof [expand]

Applying definition of Cauchy sequence with $\epsilon = 1 $ we obtain $N $ in $\N $ so that

$$m, n > N \implies \abs{ s_n - s_m } < 1 $$

In particular, $\abs{ s_n - s_{N+1}} < 1$ for $n > N $, so $\abs{ s_n } < \abs{ s_{N+1} + 1}$ for $n > N $. If $M = \max{ \set{ \abs{ s_{N+1} + 1}, \abs{ s_1 }, \abs{ s_2 }, …, \abs{ s_N }}}$ for all $n \in \N $.

Theorem

Let $(s_n) $ be a sequence in $\R $.

$(s_n)$ is a convergent sequence if and only if it is a Cauchy sequence.

Proof [expand]

We have already proved that A convergent sequence is a Cauchy sequence.

Now we need to prove that a Cauchy sequence is a convergent sequence.

We have also proved that Cauchy sequences are bounded, we now only need to prove that $$\lim \infi{ s_n } = \lim \supr{ s_n }$$ by theorem 10.7.

Let $\epsilon > 0 $. Since $(s_n) $ is a Cauchy sequence, there exists $N $ so that $m,n > N $ implies $\abs{ s_n - s_m } < \epsilon $.

In particular, $s_n < s_m + \epsilon$ for all $m, n > N$.

Then $s_m + \epsilon $ is an upper bound for $\set{ s_n : n > N }$, so $v_N = \supr{ s_n : n > N } \leq s_m + \epsilon $ for $m > N $.

Since $s_m > \supr{ s_n : n > N } - \epsilon $, $u_N = v_N - \epsilon $ is a lower bound for $\set{ s_m : m > N }$.

Thus,

$$\lim \supr{ s_n } \leq v_N \leq u_N + \epsilon \leq \lim \infi{ s_n } + \epsilon $$.

Since this holds for all $\epsilon > 0 $, we have $\lim \supr{ s_n } \leq \lim \infi{ s_n } $. By switching $m $ and $n $ in this proof, the opposite of this inequality also holds, then we have

$$\lim \infi{ s_n } = \lim \supr{ s_n }$$