However, the option price is convex in regard to the stock price, thus the amount of stock held $\frac{\partial V}{\partial S}$ decreases when $S$ decreases and increase when $S$ increases. In other words: the market maker always buys high and sells low, making volatility costly for him. To compensate for this, an option will lose value over time, compensating for the volatility experienced during that time, and leaving the market maker with the risk-free return when averaged over many different underlyings.

The famous Black-Scholes formula quantizes this relationship between the time evolution of the option price and its second derivative with respect to $S$, allowing one to calculate the time premium for a given expected volatility.

Random Walk

Assume one takes steps of equal size $s$ after a fixed interval $\tau$, with the direction of each step decided randomly between noth and south. After the time $t$ there will have been made $\frac{t}{\tau}$ steps, and the probability of then beeing at point $x(t)$ will after many steps, by the central limit theorem, follow a normal distribution with expectation value $\mathbb{E}[x] = \frac{t}{\tau} (p_\uparrow-p_\downarrow) s =: \mu t$ and variance $\mathbb{V}[x] = \frac{t}{\tau} s^2 =: \sigma^2 t$.

Taking the limit $\tau \rightarrow 0$ while keeping $\mu$ and $\sigma^2$ constant results in a so-called Wiener process. A change $\Delta x$ after time $\Delta t$ is then again drawn from a normal distribution $\Delta x \sim \mathcal{N}(\Delta t \mu, \Delta t \sigma^2) \sim \mu\Delta t + \sigma\sqrt{\Delta t}\,\mathcal{N}(0, 1)$, making the derivative $\frac{\Delta x}{\Delta t} \sim \mu + \frac{\sigma}{\sqrt{\Delta t}}\mathcal{N}(0, 1)$ divergent in the limit $\Delta t \rightarrow 0$ and leaving the function $x(t)$ continous everywhere but differentiable nowhere.

Itô’s lemma

If one now assumes, that the price of a stock is governed by a Wiener process, and the option price $V(S, t)$ depends only on the stock price $S$ and time $t$, with the other parameters $\mu, \sigma, r$ assumed to be constant, then one can look at the change $\Delta V(S, t)$ after the time $\Delta t$ by using the Taylor expansion of $V$:

Compared to the usual chain rule, there is now an additional term $\frac{1}{2}\frac{\partial^2 V}{\partial S^2}(\Delta S)^2$ due to the fact that $\Delta S$ is of the order $\sqrt{\Delta t}$, requiring one to include the second order term $(\Delta S)^2$ when looking at changes of the order $\Delta t$.

Delta hedging

Since there is still a $\Delta S$ term in the expression for $\Delta V$, the time derivative of the option price is also divergent. But this is not the case when looking at the delta-hedged portfolio $V - \frac{\partial V}{\partial S}S$:

Since the first-order price-swings of the portfolio have been eliminated and only the (small) second-order changes remain, one can require the portfolio to grow at the risk-free interest rate $r$. Further, one can maintain a portfolio of many different delta-hedged options where the remaining second-order changes (hopefully) cancel each other, making the combined portfolio resemble a risk-free portfolio even more.

Bachelier model

The only remaining part is the expression for $(\Delta S)^2$. Assuming the stock price itself is a Wiener process, then $\mathbb{E}[(\Delta S)^2] = \sigma^2\Delta t$ and $\mathbb{V}[(\Delta S)^2] = 2\sigma^4(\Delta t)^2$, up to leading order in $\Delta t$. Here the random part behaves much nicer and causes no divergence in the limit $\Delta t \rightarrow 0$. Because of this, we can assume by the central limit theorem that for the sum / integral the random parts will cancel each other and we can replace $(\Delta S)^2$ by its expectation value $\mathbb{E}[(\Delta S)^2] = \sigma^2\Delta t$.

This model was first described in 1900 by Louis Bachelier, predating Black and Scholes by seven decades. The resulting differential equation is then given by:

Which is solved by:

$$e^{r\tau}V = (F-K)\phi\left(\frac{F-K}{\sqrt{\tau}\sigma}\right)+\sqrt{\tau}\sigma\phi'\left(\frac{F-K}{\sqrt{\tau}\sigma}\right)$$

Here $\tau = T - t$ is the remaining time, $F = e^{r\tau}S$ the forward price of the stock, $K$ the strike price of the option and $\phi(x) = \pm\int_{-\infty}^{\pm x} \frac{e^{-t^2/2}}{\sqrt{2\pi}}\mathrm{d}t$ with positive sign for the call and negative for the put option.

In this model, the stock price itself follows a normal-distribution, whereas in a more appropriate model one would expect the stock prices to follow a lognormal-distribution, avoiding the possibility of negative prices.

Black-Scholes model

To get to the lognormal distribution, one needs to instead assume that the logarithm of the stock price follows a Wiener process: $\Delta \ln S \sim \mathcal{N}(t \mu, t \sigma^2)$. Using $\Delta \ln S = \frac{\Delta \ln S}{\Delta S}\Delta S = - \frac{\Delta S}{S} \implies (\Delta S)^2 = (S\Delta \ln S)^2$ and again using the expectation value $\mathbb{E}[(\Delta S)^2] = S^2\sigma^2 \Delta t$ yields the Black-Scholes equation:

This differential equation is solved by:

$$e^{r\tau}V = F\phi\left(\frac{\ln\left(\frac{F}{K}\right)+\frac{1}{2}\tau\sigma^2}{\sqrt{\tau}\sigma}\right)-K\phi\left(\frac{\ln\left(\frac{F}{K}\right)-\frac{1}{2}\tau\sigma^2}{\sqrt{\tau}\sigma}\right)$$

Here $\tau = T - t$ is the remaining time, $F = e^{r\tau}S$ the forward price of the stock, $K$ the strike price of the option and $\phi(x) = \pm\int_{-\infty}^{\pm x} \frac{e^{-t^2/2}}{\sqrt{2\pi}}\mathrm{d}t$ with positive sign for the call and negative for the put option.

Expectation value

Instead of delta hedging one could alternatively just calculate the expected value of the option at expiration, using e.g. a normal or log-normal distribution for the outcomes:

Where $\tau = T - t$ is the remaining time, $F = e^{r\tau}S$ the forward price of the stock and $K$ the strike price of the option. The result is:

Where $\phi(x) = \pm\int_{-\infty}^{\pm x} \frac{e^{-t^2/2}}{\sqrt{2\pi}}\mathrm{d}t$ with positive sign for the call and negative for the put option.

One can see that the solutions to the Bachalier and Black-Scholes equation are exactly the expectation value of the option when using $\mu = 0$ instead of the actual value $\mu$. This is what is meant when one states that the value of the option equals the expectation value under a risk-neutral measure.

For the normal distribution one can partially integrate:

$$\begin{aligned} e^{r\tau}V &= \pm\int_{\pm K}^\infty (F-K)\phi'\left(\frac{F-F_0-\mu\tau}{\sqrt{\tau}\sigma}\right) \frac{\mathrm{d}F}{\sqrt{\tau}\sigma} \\ &= \left[(F-K)\phi\left(\frac{F-F_0-\mu\tau}{\sqrt{\tau}\sigma}\right)\right]_{\pm K}^\infty - \sqrt{\tau}\sigma\int_{\pm K}^\infty \phi\left(\frac{F-F_0-\mu\tau}{\sqrt{\tau}\sigma}\right) \frac{\mathrm{d}F}{\sqrt{\tau}\sigma} \\ &= (F_0-K+\mu\tau)\phi\left(\frac{F_0-K+\mu\tau}{\sqrt{\tau}\sigma}\right)+\sqrt{\tau}\sigma\phi'\left(\frac{F_0-K+\mu\tau}{\sqrt{\tau}\sigma}\right) \end{aligned}$$

For the log-normal distribution one can complete the square in the exponential function:

$$\begin{aligned} e^{r\tau}V &= \pm\int_{\pm K}^\infty (F-K)\phi'\left(\frac{\ln F-\ln F_0-\mu\tau+\frac{1}{2}\tau\sigma^2}{\sqrt{\tau}\sigma}\right) \frac{\mathrm{d}\ln F}{\sqrt{\tau}\sigma} \\ &= \pm \int_{\pm \ln K}^\infty e^u\phi'\left(\frac{u-\ln F_0-\mu\tau}{\sqrt{\tau}\sigma}\right) \frac{\mathrm{d}u}{\sqrt{\tau}\sigma} \mp K\int_{\pm K}^\infty \phi'\left(\frac{\ln F-\ln F_0-\mu\tau+\frac{1}{2}\tau\sigma^2}{\sqrt{\tau}\sigma}\right)\frac{\mathrm{d}\ln F}{\sqrt{\tau}\sigma} \\ &= e^{\mu\tau}F_0 \phi\left(\frac{\ln F_0-\ln K+\mu\tau+\frac{1}{2}\tau\sigma^2}{\sqrt{\tau}\sigma}\right)-K \phi\left(\frac{\ln F_0-\ln K+\mu\tau-\frac{1}{2}\tau\sigma^2}{\sqrt{\tau}\sigma}\right) \end{aligned}$$

Table of contents

• Part 1: Bayesian
  • Bayes’ Theorem
  • Asymptotic behaviour
  • Choice of Prior
  • Credible intervals
  • Loss functions
  • Example: Normal distribution
• Part 2: Frequentist
  • Maximum Likelihood
  • Confidence Intervals
  • Mean squared error
  • Cramer Rao bound

Part 1: Bayesian

Bayes’ Theorem

From the definition of the conditional probability $P(X|Y) \equiv \frac{P(X \cap Y)}{P(Y)}$ one gets:

This theorem - called Bayes’ Theorem - can be used to estimate the probability $\textrm{p}(\theta|x)$ of a parameter being $\theta$ after measuring a value $x$ drawn from a probability distribution $p_{\theta}(x) \equiv \textrm{p}(x|\theta)$:

Where the fact that conditional probabilities sum up to one was used, implying that:

The four probability densities are then:

• the posterior $\textrm{p}(\theta|x)$ which is the probability of the parameter being $\theta$ considering that $x$ was measured
• the likelihood $\textrm{p}(x|\theta)$ which is the probability of measuring $x$ for a given parameter $\theta$
• the prior $\textrm{p}(\theta)$ which is the assumed distribution of the parameter $\theta$ for many repeated experiments
• the marginal probability $\textrm{p}(x)$ of measuring $x$ averaged over all possible parameters $\theta$

After the posterior has been found, the new probability $\textrm{p}(x|\mathbf{x}) = \int \textrm{p}(x|\theta)\textrm{p}(\theta|\mathbf{x})\mathrm{d}\theta$ is called posterior predictive distribution.

Since the likelihood $\textrm{p}(\mathbf{x}|\theta) = \prod_i \textrm{p}(x_i|\theta)$ becomes narrower and narrower for more measurements, the posterior becomes more independent of the prior with more measurements.

Asymptotic behaviour of the likelihood

That the likelihood becomes narrower with more measurements can be seen by rewriting the likelihood function:

Taylor expanding the exponent around the maximum of the likelihood (where $\theta_{\textrm{ML}}$ maximizes the (log) likelihood, which - by definition - is the maximum likelihood estimate of $\theta$):

The first term is just a multiplicative constant that is removed by the normalisation of the posterior. By the law of large numbers the second term will converge to the expectation value of the second derivative:

The expectation value of the second derivative of the log likelihood is also known as the Fisher information.

The likelihood function thus asymptotically becomes a normal distribution with mean $\mu = \theta_{\textrm{ML}}$ and variance $\sigma^2 = \frac{1}{NI(\theta)}$:

A more rigerous proof of this is known as the Bernstein–von Mises theorem.

Choice of Prior

A flat prior $\textrm{p}(\theta)=1$ seems like a sensible choice when having no prior information, but is not invariant under reparametrizations: For example $\textrm{p}(\sigma)\mathrm{d}\sigma=1$ implies $\textrm{p}(\sigma^2)\mathrm{d}\sigma^2=\frac{1}{2\sigma}\mathrm{d}\sigma^2$, even though they are the same parameter. A more sensible choice of prior is one that maximizes the difference between prior and posterior distribution. This difference can be measured by the Kullback-Leibler divergence:

A prior that maximizes this difference is the Jeffreys prior, which is given by the square root of the Fisher information:

Credible intervals

From the posterior $\textrm{p}(\theta|\mathbf{x})$ one can determine a range $[\theta_{\textrm{min}}, \theta_{\textrm{max}}]$ in which $\theta$ lies with probability $1-\alpha$, called credible interval:

Loss functions

Instead of stating the full posterior $\textrm{p}(\theta|\mathbf{x})$, one can characterise the distribution by a value $\hat{\theta}(\mathbf{x})$ that minimizes on average a loss function $\mathbb{E}[L] \equiv \int L(\hat{\theta}(\mathbf{x}),\theta)\textrm{p}(\theta|\mathbf{x})\mathrm{d}\theta$ over the posterior $\textrm{p}(\theta|x)$. Possible loss functions include:

Mean squared error

The loss function $L=\left|\hat{\theta}(\mathbf{x})-\theta\right|^2$ is minimized by $\hat{\theta}(\mathbf{x}) = \textrm{mean}[\textrm{p}(\theta|x)]$ since one requires:

Where was used that $\hat{\theta}(\mathbf{x})$ must be independent of $\theta$. Plugging the mean back into the average squared error shows that the error is then given by the variance of the posterior.

This can alternatively be seen directly from the average loss function:

$$\mathbb{E}[(\hat{\theta}(\mathbf{x})-\theta)^2] = \mathbb{E}[(\hat{\theta}(\mathbf{x})-\mathbb{E}[\theta])^2 + (\theta - \mathbb{E}[\theta])^2] \geq \mathbb{E}[(\theta - \mathbb{E}[\theta])^2] = \mathbb{V}[\theta]$$

Mean absolute error

The loss function $L=\left|\hat{\theta}(\mathbf{x})-\theta\right|^1$ is minimized by $\hat{\theta}(\mathbf{x}) = \textrm{median}[\textrm{p}(\theta|x)]$ since one requires:

Where by definition the median is the value for that the cumulative probabilities above and below are equal.

Mean 0-1 error

The loss function $L=\left|\hat{\theta}(\mathbf{x})-\theta\right|^0$ with $0^0 \equiv 0$ is minimized by $\hat{\theta}(\mathbf{x}) = \textrm{mode}[\textrm{p}(\theta|x)]$ since one requires:

Luckily for unimodal and symmetric distributions (e.g. the normal distribution) the mean, median and mode are all the same. In the following we will concentrate on the mean squared error.

Example: Estimating the parameters of a normal distribution

The likelihood function $\textrm{p}(x|\mu,\sigma)$ can be written with $\overline{x} = \sum_i \frac{x_i}{n}$ as:

Using the Jeffreys prior $\textrm{p}(\mu) = 1$, $\textrm{p}(\sigma) = \frac{1}{\sqrt{2\pi\sigma^2}}$ and integrating out $\mu$ and $\sigma$ yields:

The posterior of the mean is a Student’s t distribution with $n-1$ degrees of freedom:

The posterior of the variance is an inverse $\chi^2$ distribution with $n-1$ degrees of freedom:

Estimating the parameter $p$ of a binomial trial with Jeffreys prior $\textrm{p}(p)=\sqrt{\textrm{p}(1-p)}^{-1}$ yields a mean of:

$$\hat{p}(x) = \frac{\int p p^x(1-p)^{n-x} \textrm{p}(p)\mathrm{d}p}{\int p^x(1-p)^{n-x} \textrm{p}(p)\mathrm{d}p} = \frac{x+\frac{1}{2}}{n+1}$$

And a mean squared error of:

$$(\Delta \hat{p})^2 = \frac{\int p^2 p^x(1-p)^{n-x} \textrm{p}(p)\mathrm{d}p - \left(\int p p^x(1-p)^{n-x} \textrm{p}(p)\mathrm{d}p\right)^2}{\int p^x(1-p)^{n-x} \textrm{p}(p)\mathrm{d}p} = \frac{(n-x+\frac{1}{2})(x+\frac{1}{2})}{(n+2)(n+1)^2}$$

Part 2: Frequentist

An alternative to the Bayesian method is to not assume a prior/posterior distribution for $\theta$, instead relying on the mode of the likelihood function and looking at the distribution of the possible measurements $x$ instead of $\theta$.

Maximum Likelihood

A way of parameter estimation is the maximum likelihood method where the estimator $\hat{\theta}(\mathbf{x})$ is given by the condition that $\hat{\theta}(\mathbf{x})$ maximizes the likelihood, implying $\frac{\partial \textrm{p}(\mathbf{x}|\hat{\theta}(\mathbf{x}))}{\partial \theta} \overset{!}{=} 0$. One can Taylor expand the likelihood around the true parameter $\theta_0$:

This difference converges by the central limit theorem, the law of large numbers and Slutsky’s theorem to the following normal distribution:

Where was used that the derivative of the logarithm vanishes in expectation:

Similarly the expecation values in the nominator and denominator are in fact both equal to the Fisher information:

If the estimator converges to the true value $\hat{\theta}(x) \rightarrow \theta_0$, then:

Confidence Intervals

The confidence interval $[\hat{\theta}_\textrm{min},\hat{\theta}_\textrm{max}]$ is the analog of the credible interval, where after repeated measurements the true parameter $\theta$ is included $1-\alpha$ of the time. For a given $x$ the confidence interval can be found from:

Mean squared error

The mean squared error is taken as the squared difference between estimated parameter $\hat{\theta}(\mathbf{x})$ and the true parameter $\theta$ over all possible $x$:

For unbiased estimators the mean squared error equals the variance of the estimator.

Estimating for example the parameter $p$ of a binomial trial with the maximum likelihood estimator yields:

$$\hat{p}(x) = \frac{x}{n}$$ $$(\Delta \hat{p})^2 = \sum_x\binom{n}{x}p^x(1-p)^{n-x}\left(\frac{x}{n}-p\right)^2 = \frac{p-p^2}{n}$$

Cramer Rao bound

Defining the scalar product $\langle f(x)g(x) \rangle \equiv \mathbb{E}\left[f(x)g(x)\right]$ one gets from the Cauchy–Schwarz inequality that the variance of an estimator $\hat{\theta}(\mathbf{x})$ is bounded by the Fisher information:

Where the first equality follows from the interchange of integration and differentiation:

For unbiased estimators $\left(\mathbb{E}[\hat{\theta}(\mathbf{x})] = \theta \right)$ this gives a lower bound on the mean squared error as the inverse of the fisher information $I(\theta)$:

The Cauchy-Schwarz inequality follows from:

$$\frac{1}{2}\ \mathbb{E}\left[\left(\frac{x}{\sqrt{\mathbb{E}[x^2]}} - \frac{y}{\sqrt{\mathbb{E}[y^2]}}\right)^2\right] = 1 - \frac{\mathbb{E}[xy]}{\sqrt{\mathbb{E}[x^2]\mathbb{E}[y^2]}} \geq 0$$

• Sum of random variables
• Normal distribution
• $\chi$ distribution
• Student-t distribution
• Multinomial distribution
• Poisson distribution
• Erlang distribution

Probability of finding a sum of random variables

The probability density $\textrm{p}(\overline{x})$ of finding the average $\overline{x} = \frac{\sum_i x_i}{n}$ of $n$ random variables can be written using the Delta function as:

Calculating the Fourier transform $\hat{\textrm{p}}(k)$ yields:

The original $\textrm{p}(\overline{x})$ can now be recovered using the inverse Fourier transform:

Where the last equality holds for identically distributed random variables.

Normal distribution

Taylor expanding any probability density $\hat{\textrm{p}}_i(k/n)$ around $k = 0$ yields:

Comparing up to first order in $\frac{1}{n}$ one gets:

Writing $\mu \equiv \mathbb{E}[x_i]$ and $\sigma^2 \equiv \mathbb{E}[x_i^2]-\mathbb{E}[x_i]^2$, the original $\textrm{p}(\overline{x})$ can now be recovered using the inverse Fourier transform:

The average of $n$ identically distributed random variables will for large $n$ become normally distributed with the same mean $\mu$ and lower variance $\sigma^2/n$ compared to the original distribution.

$\chi$ distribution

The probability of finding the root of the sum of squares of $n$ standard-normal distributed random variables is proportional to the surface area of a $n$ dimensional sphere:

The surface area of the n-dimensional sphere can be calculated from:

$$1=\prod_i \int\frac{e^{-x_i^2/2}}{\sqrt{2\pi}}\mathrm{d}x_i = \int \mathrm{d}A \int \frac{e^{-r^2/2}r^{n-1}}{\sqrt{2\pi}^n}\mathrm{d}r = \int \mathrm{d}A \int \frac{e^{-t}t^{\frac{n}{2}-1}}{2\sqrt{\pi^n}}\mathrm{d}t \equiv \frac{\Gamma\left(\frac{n}{2}\right)}{2\sqrt{\pi^n}} \int \mathrm{d}A$$

Where the gamma function is defined as $\Gamma(x) \equiv \int_0^\infty e^{-t}t^{x-1}\mathrm{d}t$.

$\chi^2$ distribution

A change of variables yields the distribution for the sum of squares, called the $\chi^2$ distribution:

Student-t distribution

When estimating the true mean $\mu$ of a normally-distributed random variable $x$ by the sample mean $\overline{x} \equiv \frac{\sum_i x_i}{n}$, the deviation $\overline{x}-\mu$ is again normally-distributed with a standard deviation of $\frac{\sigma}{\sqrt{n}}$, making $\delta \equiv \frac{\overline{x}-\mu}{\sigma/\sqrt{n}}$ standard-normal distributed. By Cochran’s theorem the ratio $\chi^2 \equiv \frac{(n-1)s^2}{\sigma^2}$ between sample variance $s^2 \equiv \frac{\sum_i (x_i-\overline{x})^2}{n-1}$ and true variance $\sigma^2$ follows a $\chi^2$-distribution of degree $n-1$. The deviation $t \equiv \frac{\overline{x}-\mu}{s/\sqrt{n}} = \frac{\delta}{\chi/\sqrt{n-1}}$ will then depend only on the measurable quantities $\overline{x}$ and $s^2$, following a so-called Student-t distribution of degree $n-1$: