1
$\begingroup$

What is the maximum likelihood function of an independent Bivariate normal sample $(x_i, y_i)$, where the mean is known as a vector of $(\mu_x, \mu_y)$, and the variance is known to be some sort of a Variance-Covariance matrix?

I'm a little bit confused in how Bivariate normals work, and stuck with figuring out a likelihood function.

Thanks in advance if anyone can reach out and help!

$\endgroup$
3
  • $\begingroup$ When you say "independent" do you mean to say that $X_i$ and $Y_i$ are independently distributed? (this would imply their covariance is zero) or are you referring to the fact that the sample is independently distributed. i.e. that $(X_i,Y_i)$ is independent of $(X_j,Y_j)$ for $i \neq j$? I think you mean the later, in which case you should ask, "what is the likelihood function of an iid bivariate normal sample?" where iid, stands for independantly and identically distributed. just to make the question clearer $\endgroup$ Commented Dec 14, 2015 at 5:44
  • $\begingroup$ @ZacharyBlumenfeld Yes, it should be iid. Sorry I didn't make that clear enough. $\endgroup$
    – user98139
    Commented Dec 14, 2015 at 5:48
  • $\begingroup$ So both the density and log-likelihood of the multivariate normal are given on Wikipedia en.wikipedia.org/wiki/… and can be found in numerous other places. Where are you having trouble? Do you know how to maximize the likelihood of an iid univariate normal sample? Is it that you don't understand how to derive the multivariate likelihood? how to show first order conditions? how to maximize it?, or are you looking for some sort of intuition? The more detail on exactly where you are stuck, the better. $\endgroup$ Commented Dec 14, 2015 at 5:58

1 Answer 1

1
$\begingroup$

The bivariate normal density can expresed as follows

p(x)=$\frac{1}{\sqrt{2\pi}\|\Sigma|}exp(-\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu))$

where $\mu$ is mean vector and $\Sigma$ is variance-covariance matrix.

First you need to understand what is $\Sigma$. This is a matric of which $i,j^{th}$ element is the covariance between $i^{th}$ and $j^{th}$ variable. As in univariate case the shape of distribution depends upon variance $\sigma$, in bivariate case shape depends upon $\Sigma$. In univariate case density at a point depends upon distance of that point from $\mu$ with respect to $\sigma$. The same thing happens in bivariate case but here you can move away from mean in infinite directions. If you move in a direction in which variance is low your density would decrease faster compared with if you move in a direction in which variance is high.

For example suppose the variance of first variable is $\sigma_1^{2}$ and variance of second variable is $\sigma_2^{2}$. And the covarince between variable 1 and variable 2 is zero. Then you would get an ellitical density depending upon the values of $\sigma_1^{2}$ and $\sigma_2^{2}$. If $\sigma_1^{2}$ > $\sigma_2^{2}$; then the major axis of ellipse would be in the direction of variable 1 and minor axis ib the direction of variable 2.

Now that you understand bivariate normal density you can write likelihood function as in single variable case.

L($\mu, \Sigma$) = $\Pi_i p(x_i)$

$\endgroup$
1
  • 1
    $\begingroup$ The normalizing constant for $k$-variate normal is $1/\sqrt{(2\pi)^k |\Sigma|}$, but for the likelihood it is not needed so can be dropped. $\endgroup$
    – Tim
    Commented Jun 24, 2020 at 8:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.