Limits and Differentiation

.title[
# Limits and Differentiation
]
.author[
### <a href="https://jmclip.github.io/MACSS_math_camp/">MACS 33000</a> <br /> University of Chicago
]

---

# Learning objectives
* Define and evaluate limits
* Define continuity and relate it to limits
* Define the slope of a line
* Summarize tangent lines, rates of change, and derivatives
* Define derivative rules for common functions
* Apply the product, quotient, and chain rules for differentiation
* Summarize the exponential function and natural logarithms
* Identify properties of derivatives helpful to statistical methods

---

# What is calculus?

* Calculus
*   Differential calculus
*   Integral calculus
* Underlies most statistical and computational methods
* Crucial for **optimization**

## Potential uses

* Given data, what is the most likely value of a parameter(s)?
* Game theory: given another player's strategy, what is the action that maximizes utility?

---

# Limits

* Sequences `$\leadsto$` limits of functions (see Bonus slides/content for refresher on sequences!)
* Calculus/Real Analysis: study of functions on the **real line**
* Limit of a function: how does a function behave as it gets close to a particular point?

---

## Uses of limits

* Derivatives
* Asymptotics 
* Game Theory

---

# Limits

We need to talk about our functions and how they behave. We're going to be interested in talking about what they do, but we need to make sure they are 'predictable' in a way that makes sense for our analysis.

We've seen how things do/not converge, and so we want to have a way to talk about this in the land of functions.

---

# Limit: Visual
We can think about our definition as speaking to what a function is doing around any of its points in the domain. For a limit, if we make some bounds around our variable of interest (e.g. in the domain), and similarly 'arbitrarily small' bounds around our corresponding y, then the graph/function/line for y will lie within the bounds of our range of our variable.

--
<img src="https://math.libretexts.org/@api/deki/files/9980/CNX_Calc_Figure_02_05_001.jpeg?revision=1" alt="" width="100%" style="display: block; margin: auto;" />

source: [math 210](https://math.libretexts.org/Courses/Monroe_Community_College/MTH_210_Calculus_I_%28Professor_Dean%29/Chapter_2_Limits/2.7%3A_The_Precise_Definition_of_a_Limit)

---

# Limits, cont'd

* Limits are about the behavior of functions at **points**, here we will often use `$x_{0}$`
* `$\epsilon$` defines an **error rate**
* `$\delta$` defines an area around `$x_{0}$` where `$f(x)$` is going to be within our error rate

---

# Examples of limits

* The function `$f(x) = x + 1$` has a limit of `$1$` at `$x_{0} = 0$`

---

# How we use limits

* Understand what is happening big picture
* See how they underpin math later
* Fall in love with continuity

---

# Proof (sketch) for limits (L):

* **Without loss of generality** (WLOG) choose `$\epsilon >0$`
* Show that there is `$\delta_{\epsilon}$` such that

`$$|f(x) - L| < \epsilon \, \text{for all } \, x \, \text{such that} \, 0 < |x - x_0 | < \delta$$`

`$$\begin{aligned}|(x + 1) - 2| < \epsilon \, \text{for all } \, x \, &\text{such that} \, 0 < |x - 0 | < \delta \\|x| < \epsilon \, \text{for all} \, x \, &\text{such that} \, 0 < |x | < \delta \\\end{aligned}$$`

* But if `$\delta_{\epsilon}  = \epsilon$` then this holds

---

# `$f(x_{0} ) \neq L$`

* A function can have a limit of `$L$` at `$x_{0}$` even if `$f(x_{0} ) \neq L$`

* The function `$f(x) = \frac{x^2 - 1}{x - 1}$` has a limit of `$2$` at `$x_{0} = 1$`

---

# `$f(x_{0} ) \neq L$`

* Reminder: `$$|f(x) - L| < \epsilon \, \text{for all } \, x \, \text{such that} \, 0 < |x - x_0 | < \delta$$`

* For all `$x \neq 1$`,

`$$\begin{aligned}\frac{x^2 - 1}{x - 1} & = \frac{(x + 1)(x - 1) }{x - 1} \\													& = x + 1 \end{aligned}$$`
  
* Choose `$\epsilon >0$` and set `$x_{0}=1$`. Recall that `$L=2$`.  Then, we're looking for `$\delta_{\epsilon}$` such that

`$$\begin{aligned}|(x + 1) -2 | < \epsilon \, \text{for all } \, x \, &\text{such that } \, 0 < |x - 1 | < \delta \\|x - 1 | < \epsilon \, \text{for all } \, x \, &\text{such that } \, 0 < |x - 1 | < \delta \\\end{aligned}$$`

* If `$\delta_{\epsilon} = \epsilon$`, then this is satisfied

---

# Not all functions have limits

Consider `$f:(0,1) \rightarrow \mathbb{R}$`, `$f(x) = \frac{1}{x}$`.  `$f(x)$` does not have a limit at `$x_{0}=0$` as we approach from the right.

---
# Not all functions have limits (sketch) [not tested/just giving background]

* Take `$\epsilon = 1$`, and suppose some `$\delta>0$` worked. Then for every `$x$`
  with `$0<|x|<\delta$`, the triangle inequality gives

$$
`\begin{aligned}
\Big|\frac{1}{x}\Big|
  &= \Big|\Big(\frac{1}{x}-L\Big) + L\Big| \\
  &\le \Big|\frac{1}{x}-L\Big| + |L| \\
  &< 1 + |L|
\end{aligned}`
$$

* and since both sides are positive we may invert:

`$$|x| > \frac{1}{1+|L|}.$$`

---

# Not all functions have limits (sketch)

* The `$\epsilon$`-condition forces every `$x$` in the punctured interval to stay
  *away* from `$0$`, but the `$\delta$`-condition applies to *all* `$x$` with
  `$0<|x|<\delta$` — including points closer to `$0$` than `$\frac{1}{1+|L|}$`.

* Let's think about using `$\frac{\delta}{2}$` as a possible `$x_0$`, since it
  will be less than `$\delta$`.

* But if `$\delta$` is large, `$\frac{\delta}{2}$` needn't be small enough to
  break the bound. So take whichever is smaller:

`$$x_0 = \min\left(\frac{\delta}{2},\ \frac{1}{1+|L|}\right)$$`

* This still satisfies `$0<|x_0|<\delta$`.

---

## Limits sketch: conclusion

* Since `$0<|x_0|<\delta$`, the previous slide forces
  `$|x_0| > \frac{1}{1+|L|}$`.

* But `$|x_0| \le \frac{1}{1+|L|}$`, since `$\min(a,b)\le b$` always — either
  `$x_0 = \frac{\delta}{2} \le \frac{1}{1+|L|}$`, or `$x_0 = \frac{1}{1+|L|}$`.

* `$\le$` and `$>$` are exact negations, so both cannot hold. Contradiction.

* Since `$\delta>0$` was arbitrary and `$x_0$` was built *from* it, no `$\delta$`
  works for `$\epsilon = 1$`. And `$L$` was arbitrary, so no real number is the
  limit. 
  
---
# When there are no limits!
<img src="https://media.giphy.com/media/7JvlHfd7C2GDr7zfZF/giphy.gif" alt="" width="80%" style="display: block; margin: auto;" />
--

When there is no limit, this has a lot of implications for us w/r/t (with respect to) how we can expect the function(s) to behave and what we can do with the function(s) mathematically.

---

# Intuitive definition of a limit

`$$\lim_{x \rightarrow x_{0}} f(x) = L$$`

## Right-hand limits

`$$\lim_{x \rightarrow x_{0}^{+} } f(x) = L$$`

## Left-hand limits

`$$\lim_{x \rightarrow x_{0}^{-} } f(x) = L$$`

---

# Algebra of limits

Suppose `$f:\mathbb{R} \rightarrow \mathbb{R}$` and `$g: \mathbb{R} \rightarrow \mathbb{R}$` with limits `$A$` and `$B$` at `$x_{0}$`. Then,

$$
`\begin{aligned}
\text{i.) } \lim_{x \rightarrow x_{0} } (f(x) + g(x) ) & = \lim_{x \rightarrow x_{0}} f(x) + \lim_{x \rightarrow x_{0}} g(x)  = A + B \\
\text{ii.) }\lim_{x \rightarrow x_{0} } f(x) g(x) & = \lim_{x \rightarrow x_{0}} f(x) \lim_{x\rightarrow x_{0}} g(x)  = A B 
\end{aligned}`
$$

Suppose `$g(x) \neq 0$` for all `$x \in \mathbb{R}$` and `$B \neq 0$` then `$\frac{f(x)}{g(x)}$` has a limit at `$x_{0}$` and

`$$\lim_{x \rightarrow x_{0}} \frac{f(x)}{g(x)} =  \frac{\lim_{x\rightarrow x_{0} } f(x) }{\lim_{x \rightarrow x_{0} } g(x) } = \frac{A}{B}$$`

---

# Continuity

* Limit exists at 1 
* But hole in function 
* Fails the **pencil** test, **discontinuous** at 1

---

# Defining continuity

* Suppose `$f:\mathbb{R} \rightarrow \mathbb{R}$` and consider `$x_{0} \in \mathbb{R}$`
* `$f$` is continuous at `$x_{0}$` if for each `$\epsilon>0$` there is a `$\delta>0$` such that if,

`$$\begin{aligned}|x - x_{0} | & < \delta \text{ for all  } x \in \mathbb{R} \text{ then } \nonumber \\|f(x) - f(x_{0})| & < \epsilon \nonumber \end{aligned}$$`

* `$L$` is replaced by `$f(x_{0})$` 
* Now: `$f(x)$` has to converge on itself at `$x_{0}$`
* **Continuity is more restrictive than limit**

---

# Examples of continuity

---

# Examples of continuity

---

# Examples of continuity

---

# Measuring incumbency advantage

* Incumbency advantage
* U.S. House of Representatives
* Win rate of incumbent parties `$>90\%$`
* Win rate of incumbent candidates `$\approx 90\%$` (conditional on running for election)
    * `$88\%$` probability of running for reelection
--
* Runners-up
    * `$3\%$` chance of winning the next election
    * Only `$20\%$` chance of running in the next election
* Is there an electoral advantage to incumbency?
* Can this be proven through observational study?

---

# Ideal experiment

* Randomly assign incumbent parties in a district between Democrats and Republicans
* Keep all other factors constant
* Corresponding increase in Democratic electoral success in the next election would represent the overall electoral benefit due to being the incumbent party in the district
* Realistic?

---

# Regression discontinuity design

* Dichotomous treatment is a deterministic function of a single, continuous covariate
* Treatment is assigned to those individuals whose score crosses a known threshold
* If you know the score, you can reverse-engineer the treatment assignment
* Assumes as-if random assignment in the local neighborhood around a probability of `$50\%$`

---

# RDD + incumbency advantage

Source: [Lee, D. S. (2008). Randomized experiments from non-random selection in US House elections. *Journal of Econometrics*, 142(2), 675-697.](https://www.sciencedirect.com/science/article/pii/S0304407607001121)

---

# Continuity and limits

* Let `$f: \mathbb{R} \rightarrow \mathbb{R}$` with `$x_{0} \in \mathbb{R}$`
* Then `$f$` is continuous at `$x_{0}$` if and only if `$f$` has a limit at `$x_{0}$` and that `$\lim_{x \rightarrow x_{0} } f(x) = f(x_{0})$`

* Suppose `$f$` is continuous at `$x_{0}$`
    * This implies that `$0 < |f(x) - f(x_0)| < \epsilon \, \text{for all} \, x \, \text{such that} \, 0<|x - x_0 | < \delta$`
    * Definition of a **limit**, with `$L = f(x_{0})$`
--

* Suppose `$f$` has a limit at `$x_{0}$` and that limit is `$f(x_{0})$`
    * This implies that `$|f(x) - f(x_0)| < \epsilon \, \text{for all} \, x \, \text{such that} \, |x - x_0 | < \delta$`
    * This is the definition of **continuity**

---

# Algebra of continuous functions

Suppose `$f:\mathbb{R} \rightarrow \mathbb{R}$` and `$g:\mathbb{R} \rightarrow \mathbb{R}$` are continuous at `$x_{0}$`. Then,

1. `$f(x) + g(x)$` is continuous at `$x_{0}$`
1. `$f(x) g(x)$` is continuous at `$x_{0}$`
1. If `$g(x_0) \neq 0$`, then `$\frac{f(x) } {g(x) }$` is continuous at `$x_{0}$`

---

# How functions change

* Derivatives
* A special limit
* Cover three broad ideas 
    1. Geometric interpretation/intuition
    1. Formulas/algebra derivatives
    1. Famous theorems

---

# The tangent as a limit

---

# The tangent as a limit

]

$$
`\begin{aligned}
P &= (x_0, y_0) \\
Q &= (x_1, y_1) \\
\text{slope of } L &= \frac{y_1 - y_0}{x_1 - x_0}
\end{aligned}`
$$

]

---

# The tangent as a limit

$$
`\begin{aligned}
P &= (x_0, y_0) \\
Q &= (x_1, y_1) \\
\text{slope of } L &= \frac{y_1 - y_0}{x_1 - x_0}
\end{aligned}`
$$

### Substitute values

$$
`\begin{aligned}
h &= x_1 - x_0 \\
x_1 &= x_0 + h \\
y_0 &= f(x_0) \\
y_1 &= f(x_0 + h) \\
\text{slope of } L &= \frac{f(x_0 + h) - f(x_0)}{h}
\end{aligned}`
$$

---

# Derivative

Suppose `$f:\mathbb{R} \rightarrow \mathbb{R}$`. Measure rate of change at a point `$x_{0}$` with a function `$R(x)$`,

`$$R(x) = \frac{f(x) - f(x_{0}) }{ x- x_{0} }$$`

* `$R(x)$` defines the rate of change
* A derivative will examine what happens with a small perturbation at `$x_{0}$`

---
class: center, middle, inverse

# Derivatives

## You need to know how to calculate a derivative. For our class, we will not ask you to name the theorems we use here. It's only to provide context.

---

# Derivative

* Let `$f:\mathbb{R} \rightarrow \mathbb{R}$`. If the limit

`$$\begin{aligned} \lim_{x\rightarrow x_{0}} R(x) & = \lim_{x\rightarrow x_{0}} \frac{f(x) - f(x_{0}) }{x - x_{0}} \\ & = f^{'}(x_{0}) \end{aligned}$$`

exists then we say that `$f$` is **differentiable** at `$x_{0}$`
* If `$f^{'}(x)$` exists for all `$x \in \text{Domain}$`, then we say that `$f$` is differentiable

---

# Derivative

* Let `$f$` be a function whose domain includes an open interval containing the point `$x$`
* The derivative of `$f$` at `$x$` is given by

`$$\frac{d}{dx}f(x) =\lim\limits_{h\to 0} \frac{f(x+h)-f(x)}{(x+h)-x} = \lim\limits_{h\to 0} \frac{f(x+h)-f(x)}{h}$$`

## Derivative notation

* Leibniz notation: `$\frac{d}{dx}(f(x))$`
* Prime or Lagrange notation: `$f'(x)$`

---

# Example derivatives

<div class="figure" style="text-align: center">
<img src="03-differentiation_files/figure-html/derivsimple-1.png" alt="The Derivative as a Slope" width="85%" />
<p class="caption">The Derivative as a Slope</p>
</div>

* Reminder: If `$f'(x)$` exists at a point `$x_0$`, then `$f$` is said to be **differentiable** at `$x_0$`
* Implies continuity of `$f(x)$` at `$x_0$`

---

# Rates of change in a function

* Consider `$y = f(x)$`
* As `$x$` changes from `$x_0$` to `$x_0 + h$`, the value of the function changes from `$f(x_0)$` to `$f(x_0 + h)$`
* Change in `$x$` is `$h$`
* Change in `$f(x)$` is `$f(x_0 + h) - f(x_0)$`
* Rate of change of `$f(x)$` is defined to be

`$$\frac{f(x_0 + h) - f(x_0)}{h}$$`

* Same thing as a derivative

---

# Rates of change in a function

* Rate of change `$\leadsto$` return on vote share for dollars invested
* Instantaneous rate of change `$\leadsto$` increase in vote share in response to infinitesimally small increase in spending
* A type of **limit**

---

# Examples of derivatives

* `$f(x) = x^2$` and consider `$x_{0} = 1$`

$$
`\begin{aligned}
\lim_{x\rightarrow 1}R(x) & = \lim_{x\rightarrow 1} \frac{x^2 - 1^2}{x - 1}  \\
 	& = \lim_{x\rightarrow 1} \frac{(x- 1)(x + 1) }{ x- 1}   \\
	& =  \lim_{x\rightarrow 1} x + 1  \\
	& = 2
\end{aligned}`
$$

---

# Examples of derivatives

* `$f(x) = |x|$` and consider `$x_{0} = 0$`

`$$\lim_{x\rightarrow 0} R(x) = \lim_{x\rightarrow 0} \frac{ |x| } {x}$$`

* `$\lim_{x \rightarrow 0^{-}} R(x) = -1$`
* `$\lim_{x \rightarrow 0^{+}} R(x) = 1$`
* Not differentiable at `$0$`

---

# Continuity and derivatives

* `$f(x) = |x|$` is **continuous** but not differentiable
* Change is **too abrupt**
* Suggests differentiability is a stronger condition

Let `$f:\mathbb{R} \rightarrow \mathbb{R}$` be differentiable at `$x_{0}$`. Then `$f$` is continuous at `$x_{0}$`.

---

# What goes wrong?

$$
`\begin{aligned}
f(x)  & = x^{2} \text{ for all  } x \in \mathbb{R} \setminus \{0\} \\
f(x) & = 1000  \text{ for  } x = 0
\end{aligned}`
$$

By definition,

$$
`\begin{aligned}
f'(0)
  &= \lim_{h \to 0} \frac{f(0+h)-f(0)}{h} \\
  &= \lim_{h \to 0} \frac{f(h)-1000}{h} \\
  &= \lim_{h \to 0} \frac{h^{2}-1000}{h} \\
  &= \lim_{h \to 0} \left( h - \frac{1000}{h} \right)
\end{aligned}`
$$
---

# What goes wrong?

* The third line uses `$f(h) = h^{2}$`, valid because `$h \neq 0$` throughout the
  limit — the value `$f(0)=1000$` enters only through the `$-f(0)$` term.

* Now `$h \to 0$`, but `$-\frac{1000}{h}$` gets huge. So `$f'(0)$` does not exist.
---

# Calculating derivatives

* Rarely (basically never) will we take a limit to calculate derivative
* Rather, rely on **rules** and properties of derivatives
* **Important**: do not forget core intuition

## Strategy

* Algebra theorems
* Some specific derivatives
* Work on problems

---

# Derivative rules: WRITE THESE DOWN

$$
`\begin{aligned}
f(x) &= x & \quad f^{'}(x) &= 1 \\
f(x) &= a x^{k} & \quad f^{'}(x) &= (a) (k) x ^{k-1} \\
f(x) &= e^{x } & \quad f^{'} (x) &= e^{x} \\
f(x) &= \sin(x) & \quad f^{'} (x) &= \cos (x) \\
f(x) &= \cos(x) & \quad f^{'} (x) &= - \sin(x) \\
\end{aligned}`
$$

---

# Derivative rules

##### Constant rule `$\left[k f(x)\right]' = k f'(x)$`

##### Sum rule `$\left[f(x)\pm g(x)\right]' = f'(x)\pm g'(x)$`

##### Product rule `$\left[f(x)g(x)\right]' = f'(x)g(x)+f(x)g'(x)$`

##### Quotient rule `$\left[\frac{f(x)}{g(x)}\right]' = \frac{f'(x)g(x)-f(x)g'(x)}{[g(x)]^2}, g(x)\neq 0$`

##### Power rule `$\left[x^k\right]' = k x^{k-1}$`

---

# Derivatives with a friend

Differentiate the following functions:

1. `$f(x)= x^3 + 5 x^2  + 4 x$`, <span style="color: gray;"> at `$x_{0} = 2$`</span>
1. `$f(x) = \sin(x) x^3$`, <span style="color: gray;"> at `$x_{0} = 2$`</span>
1. `$h(x) = \dfrac{e^{x}}{x^3}$`, <span style="color: gray;"> at `$x_{0} = 2$` </span>
1. `$h(x) = \log (x) x^3$`, <span style="color: gray;"> at `$x_{0} = e$` </span>

---

# Composite functions

* Differentiating functions of functions

`$$x^2 + 1^2 \quad \text{vs.} \quad (x^2 + 1)^2$$`

* Composite functions

`$$f \circ g=f[g(x)]$$`

* Range of `$g$` must be contained (at least in part) within the domain of `$f$`
* Domain of `$f \circ g$` consists of all the points in the domain of `$g$` for which `$g(x)$` is in the domain of `$f$`

---

# Composite functions

$$
`\begin{aligned}
f(x) &= \log x, x \in (0, \infty) \\
g(x) &= x^2, x \in (0, \infty)
\end{aligned}`
$$

`$$f[g(x)] = \log x^2 \in (-\infty, \infty) - \{0\}$$`

`$$g[f(x)] = [\log x]^2 \in (0, \infty)$$`

---

# Chain rule

* `$y = f[g(x)]$`
* Derivative of `$y$` with respect to `$x$` is

`$$\frac{d}{dx} \{ f[g(x)] \} = f'[g(x)] g'(x)$$`

* Derivative of the "outside" times the derivative of the "inside"
* "Outside" evaluated at the value of the inside function

---

# Chain rule

$$
`\begin{aligned}
h(x) &= e^{2x} \\
g(x) &= e^{x} \\
f(x) &= 2x
\end{aligned}`
$$

`$$h(x) = g(f(x)) = g(2x) = e^{2x}$$`

`$$h^{'}(x) = g^{'}(f(x))f^{'}(x) = e^{2x}2$$`

---

# Chain rule

$$
`\begin{aligned}
h(x) &= \log(\cos(x) ) \\
g(x) &= \log(x) \\
f(x) &= \cos(x)
\end{aligned}`
$$

`$$h(x) = g(f(x)) = g( \cos(x)) = \log(\cos(x))$$`

`$$h^{'}(x) = g^{'}(f(x))f^{'}(x) = \frac{-1}{\cos(x)} \sin(x) = -\tan (x)$$`

---

# Generalized power rule

* If `$f(x)=[g(x)]^p$` for any rational number `$p$`

`$$f^\prime(x) =p[g(x)]^{p-1}g^\prime(x)$$`

---

# Exponential function

---

# Derivative of exponential function

`$$\frac{d}{dx}(e^x) = e^x$$`

* Why?

$$
`\begin{aligned}
\frac{d}{dx}f(x) & = \lim\limits_{h\to 0} \frac{f(x+h)-f(x)}{h} \\
&= \lim\limits_{h\to 0} \frac{e^{x + h} - e^x}{h} \\
&= \lim\limits_{h\to 0} \frac{e^x e^h - e^x}{h} \\
&= \lim\limits_{h\to 0} \frac{e^x(e^h - 1)}{h} \\
&= e^x \lim\limits_{h\to 0} \frac{e^h - 1}{h}
\end{aligned}`
$$

---

# Derivative of exponential function

---

# Derivative of exponential function

---

# Derivative of exponential function

<div class="figure" style="text-align: center">
<img src="03-differentiation_files/figure-html/fig-derivexponent-1.png" alt="Derivative of the Exponential Function" width="85%" />
<p class="caption">Derivative of the Exponential Function</p>
</div>

---

# Derivative of the natural logarithm

* Logarithm to base `$e$` of `$x$`, where `$e$` is defined as **Euler's number**

`$$y = \log_e (x) \iff x = e^y$$`

* Relationship between `$e^x$` and `$\log_e(x)$`

`$$\begin{aligned} e^{\log(x)} &= x \, \text{for every positive number} \, x \\ \log(e^y) &= y \, \text{for every real number} \, y \\ \end{aligned}$$`

---

# Exponential function and natural logarithm

<div class="figure" style="text-align: center">
<img src="03-differentiation_files/figure-html/exp-log-1.png" alt="Exponential function and natural logarithm" width="85%" />
<p class="caption">Exponential function and natural logarithm</p>
</div>

---

# Derivative of a natural logarithm
    
  $$(f^{-1})'(x) = \frac{1}{f'(f^{-1}(x))} $$
    
Let's find the derivative of `$\log(x)$`. We already know the derivative of its inverse, `$e^x$`, is `$e^x$`. So, we begin with `$f(x) = e^x$` (and `$f'(x) = e^x$`) and want to know `$(f^{-1})'(x)$`.
  
We know `$f'(f^{-1}(x)) = f'(\log(x)) = e^{\log(x)} = x$`
  
--

Therefore,
  `$$\begin{aligned} (f^{-1})'(x) &= \frac{1}{f'(f^{-1}(x))} \\
   &= \frac{1}{(e^{\log(x)})} \\
  &= \frac{1}{x} \end{aligned}$$`

---

# Derivative of a natural logarithm

<div class="figure" style="text-align: center">
<img src="03-differentiation_files/figure-html/fig-derivlog-1.png" alt="Derivative of the Natural Log" width="864" />
<p class="caption">Derivative of the Natural Log</p>
</div>

---

# Relevance of `$e^x$` and `$\log(x)$`

* Growth over time (e.g. compounding interest)
* Elasticity models
* Variable transformation

---

# Derivatives and properties of functions

* Often used to **optimize** a function (tomorrow)
* But also reveal **average rates of change**
* Or crucial properties of functions

---

# Fermat's interior extremum

> Suppose `$f:[a, b] \rightarrow \mathbb{R}$`.  Suppose `$f$` has a relative maxima or minima on `$(a,b)$` and call that `$c \in (a, b)$`.  Then `$f'(c) = 0$`.

---

# Fermat's interior extremum

> Suppose `$f:[a, b] \rightarrow \mathbb{R}$`.  Suppose `$f$` has a relative maxima or minima on `$(a,b)$` and call that `$c \in (a, b)$`.  Then `$f'(c) = 0$`.

* Consider (without loss of generality) a relative maximum `$c$`
* Consider the left-hand and right-hand limits

`$$\begin{aligned} \lim_{x \rightarrow c^{-}} \frac{f(x) - f(c) }{x - c } & \geq 0  \\ \lim_{x \rightarrow c^{+}} \frac{f(x) - f(c) } {x - c }  & \leq 0 \end{aligned}$$`

* But we also know that

`$$\begin{aligned} \lim_{x \rightarrow c^{-}} \frac{f(x) - f(c ) }{x - c } & = f^{'}(c)  \\ \lim_{x \rightarrow c^{+}} \frac{f(x) - f(c) } {x - c }  &  =  f^{'}(c) \end{aligned}$$`

---

# Fermat's interior extremum and Rolle's theorem

* The only way, then, that we can have something `$\geq 0$` equal to `$\leq 0$` is if `$f^{'}(c) = 0$` :
`$$\lim_{x \rightarrow c^{-}} \frac{f(x) - f(c) }{x -c}  = \lim_{x \rightarrow c^{+}} \frac{f(x) - f(c) } {x - c}$$`

* Rolle's theorem adds the additional condition that if `$f(a) = f(b)$` then there must be somewhere where `$f'(c)=0$`. 
---

# Mean value theorem

* If `$f:[a,b] \rightarrow \mathbb{R}$` is continuous on `$[a,b]$` and differentiable on `$(a,b)$`, then there is a `$c \in (a,b)$` such that

`$$f^{'}(c) = \frac{f(b) - f(a) } { b - a}$$`

* Rolle's theorem is a special case of the MVT with the `$f(a) = f(b)$` restriction, yielding `$f'(c) = 0$`
* MVT is more general (not requring a value for c)

---

# Mean value theorem

---

# Applications of the mean value theorem

* This will come up in a formal theory article. You'll at least know where to look
* It allows us to say lots of powerful stuff about functions
* Especially useful for approximating derivatives (see next set of slides)

---

# Applications of the mean value theorem

Suppose that `$f:[a,b] \rightarrow \mathbb{R}$` is continuous on `$[a,b]$` and differentiable on `$(a,b).$` Then,

1. If `$f^{'}(x) \neq 0$` for all `$x \in (a,b)$` then `$f$` is 1-1
1. If `$f^{'}(x) = 0$` then `$f(x)$` is constant 
1. If `$f^{'}(x)> 0$` for all `$x \in (a,b)$` then `$f$` is strictly increasing
1. If `$f^{'}(x)<0$` for all `$x \in (a,b)$` then `$f$` is strictly decreasing

---

# If `$f^{'}(x) \neq 0$` for all `$x \in (a,b)$` then `$f$` is 1-1

---

# If `$f^{'}(x) \neq 0$` for all `$x \in (a,b)$` then `$f$` is 1-1

* Suppose that `$f$` is not 1-1. Then there are `$x, y \in (a,b)$` with `$x \neq y$` such that `$f(x) = f(y)$`. Then,

`$$f'(c) = \frac{f(x) - f(y)}{x- y} = \frac{0}{x -y}  = 0$$`

* This means `$f'(x) = 0$` for some point, a **contradiction** to our starting assumption that `$f'(x) \neq 0$`.

---

# If `$f^{'}(x) = 0$` then `$f(x)$` is constant

* By way of contradiction, suppose that there is `$x, y \in (a,b)$` such that `$f(x) \neq f(y)$`. But then,

`$$f'(c) = \frac{f(x) - f(y) } {x - y} \neq 0$$`

---

## If `$f^{'}(x)> 0$` for all `$x \in (a,b)$` then `$f$` is strictly `$\uparrow$`

* By way of contradiction, suppose that there is `$x, y \in (a,b)$` with `$y<x$` but `$f(y)>f(x)$`. But then,

`$$f'(c) = \frac{f(x) - f(y) }{x - y } < 0,$$` which contradicts `$f^{'}(x)> 0$`
* Proof for strictly decreasing is the reverse of this

---

# Cauchy mean value theorem

* Suppose `$f$` and `$g$` are differentiable functions and `$a$` and `$b$` are real numbers such that `$a < b$`
* Suppose also that `$g'(x) \neq 0$` for all `$x$` such that `$a < x < b$`
* There exists a real number `$c$` such that `$a < c < b$` and

`$$\frac{f'(c)}{g'(c)} = \frac{f(b) - f(a)}{g(b) - g(a)}$$`

* The classical mean value theorem is the special case where `$g(x) = x$` for all `$x$`

---

# Implications and a little help from Cauchy

Suppose we want to find

`$$\lim_{x \rightarrow a} \frac{f(x)}{g(x)}$$`

where `$f$` and `$g$` are continuous functions

* Suppose `$g(a) \neq 0$`, then we can do this:

`$$\lim_{x \rightarrow a} \frac{f(x)}{g(x)} = \frac{f(a)}{g(a)}$$`

* BUT if `$f(a) = g(a) = 0$`... OH NO! What do we do with an indeterminate form (i.e. `$\frac{0}{0}$`)?

---

# New bestie: L'Hôpital's Rule

* Suppose that `$f(a) = g(a) = 0$` and `$g'(x) \neq 0$` if `$x$` is close but not equal to `$a$` (also applies for `$\frac{\pm\infty}{\pm\infty}$`).
* Then

`$$\lim_{x \rightarrow a} \frac{f(x)}{g(x)} = \lim_{x \rightarrow a} \frac{f'(x)}{g'(x)}$$`

$$
`\begin{aligned}
\frac{f(x) - f(a)}{g(x) - g(a)} &= \frac{f'(c)}{g'(c)} \\
\frac{f(x) - 0}{g(x) - 0} &= \frac{f'(c)}{g'(c)} \\
\frac{f(x)}{g(x)} &= \frac{f'(c)}{g'(c)}
\end{aligned}`
$$

---

# Example of L'Hôpital's Rule

.pull-left[
<img src="03-differentiation_files/figure-html/lhopital-1-1.png" alt="" width="100%" style="display: block; margin: auto;" />
]

.pull-right[
$$
`\begin{aligned}
f(x) &= (1 + x)^{1/3} - 1 \\
g(x) &= x - x^2 \\
f'(x) &= \tfrac{1}{3} (1 + x)^{-2/3} \\
f'(0) &= \tfrac{1}{3} (1)^{-2/3} = \tfrac{1}{3} \\
g'(x) &= 1 - 2x \\
g'(0) &= 1 - 2(0) = 1
\end{aligned}`
$$
---
# Example of L'Hôpital's Rule

Since `$f(0) = g(0) = 0$`, this is `$\tfrac{0}{0}$`, so L'Hôpital applies:

`$$\lim_{x \rightarrow 0} \frac{f(x)}{g(x)} = \lim_{x \rightarrow 0} \frac{f'(x)}{g'(x)} = \frac{1/3}{1} = \frac{1}{3}$$`
]

---

# Example of L'Hôpital's Rule

.pull-left[
<img src="03-differentiation_files/figure-html/lhopital-2-1.png" alt="" width="75%" style="display: block; margin: auto;" />

]

.pull-right[
$$
`\begin{aligned}
f(x) &= x - \log(1 + x) \\
f'(x) &= 1 - \frac{1}{1 + x} \\
g(x) &= x^2 \\
g'(x) &= 2x
\end{aligned}`
$$

]

---

# Simplify the expression

$$
`\begin{aligned}
L &= \lim_{x \rightarrow 0} \frac{1 - \frac{1}{1 + x}}{2x} \\
&= \lim_{x \rightarrow 0} \frac{1}{2x} - \frac{\frac{1}{1 + x}}{2x} = \lim_{x \rightarrow 0} \frac{1}{2x} - \frac{1}{2x(1 + x)} \\
&= \lim_{x \rightarrow 0} \frac{1(1 + x)}{2x(1 + x)} - \frac{1}{2x(1 + x)} \\
&= \lim_{x \rightarrow 0} \frac{1(1 + x) - 1}{2x(1 + x)} \\
&= \lim_{x \rightarrow 0} \frac{1 + x - 1}{2x(1 + x)} \\
&= \lim_{x \rightarrow 0} \frac{x}{2x(1 + x)} = \lim_{x \rightarrow 0} \frac{1}{2(1 + x)} \\
&= \lim_{x \rightarrow 0} \frac{1}{2(1 + 0)} = \frac{1}{2}
\end{aligned}`
$$

---

# Iterative application

`$$\lim_{x \rightarrow a} \frac{f(x)}{g(x)} = \lim_{x \rightarrow a} \frac{f'(x)}{g'(x)} = \lim_{x \rightarrow a} \frac{f''(x)}{g''(x)} = \ldots$$`

$$
`\begin{aligned}
f''(x) &= \frac{1}{(1 +x)^{2}} \\
g''(x) &= 2 \\
\lim_{x \rightarrow 0} \frac{f''(x)}{g''(x)} &= \lim_{x \rightarrow 0}\frac{(1 + x)^{-2}}{2} = \frac{1^{-2}}{2} = \frac{1}{2}
\end{aligned}`
$$

---

# RECAP

---
## RECAP:

* Good chunk of calc 1
* Limits
  * continuity
  * what they tell us
  * how to find them
* Rate of change
* Intuition for derivative
* Mathematical foundation for derivative
* Rules for derivatives
* Making magic: MVT + implications

---

## Helpful formulas

---

# Derivative rules: WRITE THESE DOWN

Suppose `$a$` is some constant, `$f(x)$` and `$g(x)$` are functions:

---

# Derivative rules

##### Constant rule `$\left[k f(x)\right]' = k f'(x)$`

##### Sum rule `$\left[f(x)\pm g(x)\right]' = f'(x)\pm g'(x)$`

##### Product rule `$\left[f(x)g(x)\right]' = f'(x)g(x)+f(x)g'(x)$`

##### Quotient rule `$\left[\frac{f(x)}{g(x)}\right]' = \frac{f'(x)g(x)-f(x)g'(x)}{[g(x)]^2}, g(x)\neq 0$`

##### Power rule `$\left[x^k\right]' = k x^{k-1}$`

---

### TA TEAM!

* Saaket Raman: saaket@uchicago.edu
* Xulan Jiang: xulanj@uchicago.edu
* Yanjing Li: yanjingli@uchicago.edu

---
class: center, middle, inverse
## Let's get to know each other a bit more – Name, pronouns, subfield/research area, where you are currently, something fun/interesting about you and/or your hobbies