## Exercises

1. This question is about the normal distribution, and how it relates to the classification rule provided by linear discriminant analysis.
1. (1)Write down the density function for a bivariate normal distribution ($$p=2$$), with mean $$\mu_k$$ and variance $$\Sigma$$.

$f(x) = \frac{1}{2\pi|\Sigma|}\exp\{-\frac{1}{2}(x-\mu)'\Sigma^{-1}(x-\mu)\}$
where $$x=(x_1, x_2)$$, $$\mu=(\mu_1, \mu_2)$$, $$\Sigma=\left[ \begin{array}{cc} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{array} \right]$$

1. (2)Show that the linear discriminant rule for two groups ($$K=2$$), $$\pi_1=\pi_2$$ and

$\Sigma_1=\Sigma_2 = \Sigma = \left[\begin{array}{cc} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{array}\right]$ where $$\rho$$ is the population correlation between the two variables, and $$\sigma_1, \sigma_2$$ are the population standard deviations of the two variables, respectively, is equal to: Assign a new observation $$x_0$$ to group 1 if $x_0'\Sigma^{-1}(\mu_1-\mu_2) > \frac{1}{2}(\mu_1 + \mu_2)'\Sigma^{-1}(\mu_1-\mu_2)$ The rule is to assign observation to the group with the highest probability, so work from the density functions for the tewo groups. Let

$$x=(x_1, x_2)$$, $$\mu_1=(\mu_{11}, \mu_{12})$$, $$\mu_2=(\mu_{21}, \mu_{22})$$ then

$\frac{1}{2\pi|\Sigma|}\exp\{-\frac{1}{2}(x-\mu_1)'\Sigma^{-1}(x-\mu_1)\}>\frac{1}{2\pi|\Sigma|}\exp\{-\frac{1}{2}(x-\mu_2)'\Sigma^{-1}(x-\mu_2)\}$

$\leadsto ~~~ (x-\mu_1)'\Sigma^{-1}(x-\mu_1) > (x-\mu_2)'\Sigma^{-1}(x-\mu_2)$ $\leadsto ~~~~~~~~~~~~~~~ x'\Sigma^{-1}x -x'\Sigma^{-1}\mu_1 - \mu_1'\Sigma^{-1}x+\mu_1'\Sigma^{-1}\mu_1 > x'\Sigma^{-1}x -x'\Sigma^{-1}\mu_2 - \mu_2'\Sigma^{-1}x+\mu_2'\Sigma^{-1}\mu_2$ $\leadsto ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~-x'\Sigma^{-1}\mu_1 - \mu_1'\Sigma^{-1}x+\mu_1'\Sigma^{-1}\mu_1> -x'\Sigma^{-1}\mu_2 - \mu_2'\Sigma^{-1}x+\mu_2'\Sigma^{-1}\mu_2$ $\leadsto ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~x'\Sigma^{-1}\mu_1 +x'\Sigma^{-1}\mu_2 + \mu_1'\Sigma^{-1}x + \mu_2'\Sigma^{-1}x > - \mu_1'\Sigma^{-1}\mu_1 + \mu_2'\Sigma^{-1}\mu_2$ $\leadsto ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~2x'\Sigma^{-1}(\mu_1-\mu_2) > (\mu_1+\mu_2)'\Sigma^{-1}(\mu_1-\mu_2)$ $\leadsto ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~x'\Sigma^{-1}(\mu_1-\mu_2) > \frac{1}{2}(\mu_1+\mu_2)'\Sigma^{-1}(\mu_1-\mu_2)$ c. (2)By generating a grid of values, draw the boundary between two groups, in the 2D space. Use these values for $$\mu_1, \mu_2$$ and $$\sigma$$. $\mu_1 = \left[\begin{array}{r} -2 \\ 2 \end{array}\right], ~~~\mu_2 = \left[\begin{array}{r} 2 \\ -2 \end{array}\right]$ $\Sigma = \left[\begin{array}{rr} 4 & 3 \\ 3 & 5 \end{array}\right]$