# Likelihood at MLE and transformations, the multivariate normal case

Given a univariate sample $\vec X = X_1, ..., X_n$ with standard deviation 1 and a strictly monotone transformation $t: R \to R$ with the property that the standard deviation of $t(\vec X)$ is also 1 (where $t(\vec X)$ is $t$ applied to each $X_i$). If one know fits a normal distribution to $\vec X$ and $t(\vec X)$, one observes the following fact: the likelihood at the respective MLEs of $\vec X$ and $t(\vec X)$ is the same. The reason for that is, that the standard deviation is the MLE for the $\sigma$ parameter of the normal distribution, so just the mean changes, which does not change the likelihood.

In some computations with multivariate data I observed the same fact, using transformations such that $\det(cov(t(\vec X)))=1)$. But I did not see a straightforward proof for that! Note that in the multivariate case, the MLEs for the variance parameters do change, even if the above determinant is always 1.

I am sure I am not the first to notice, but my (somewhat limited) literature on the multivariate normal distribution does not give me a clue how to prove the above statement, i.e. that the multivariate normal likelihood does only depend on the determinant of the $cov(t(\vec X))$, regardless of choice of $t$.

A proof boils down to looking at the terms of the form $x'\Sigma x$ for $\det \Sigma = 1$.

Do you have more clue?

Thanks, Philipp

This is due to the invariance property of MLEs. It is an elementary exercise in calculus to show that if $E_{MLE}$ maximises the function $F(E)$ then any monotonic function of $E$, say $g=g(E)$ will also be maximised at $g_{MLE}=g(E_{MLE})$.

$F(E)=F[g^{-1}(E)]$ and therefore the maximum occurs at $F_{MAX}=F(E_{MLE})=F[g^{-1}(E_{MLE})]$, which means that $g_{MLE}=g(E_{MLE})$.

If $g(.)$ were not monotone (1-to-1 so that $g^{-1}(.)$ is well defined), then the above argument would not hold.

EDIT: I think this is why the MLE's change for the determinant transformation, perhaps the $t(.)$ function is not a 1-to-1 function. It is all about the functional properties of your $t(.)$ function (whether it is 1-to-1 or not).

EDIT#2: After some discussion, the transformation $t(.)$ may be 1-to-1, but if the inverse transformation $t^{-1}(.)$ doesn't have a Jacobian of 1, then the value of the likelihood at its maximum will be different. Fixing the variance to be 1 may one way of achieving this property.

The class of transformations $t(X)$ has a approximate (exact for normals?) covariance of $$cov[t(X)]\approx (\frac{\partial t(x)}{\partial t})_{x=X_{MLE}}^T cov[X](\frac{\partial t(x)}{\partial t})_{x=X_{MLE}}$$.
Fixing the determinant to 1 fixes (approximately) $$|\frac{\partial t(x)}{\partial t}|^{-2}_{x=X_{MLE}}\approx |cov[X]|$$ Now if $t(.)$ is invertible, the jacobian of the inverse transformation is just the inverse of the jacobian. SO we have: $$|\frac{\partial t(x)}{\partial t}|^{-1}_{x=X_{MLE}}=|\frac{\partial t^{-1}(x)}{\partial t}|_{x=X_{MLE}}\approx |cov[X]|^{\frac{1}{2}}$$

Now in the normal distribution, the likelihood evaluated at the MLE, the exponential part just becomes 1. So the maximised likelihood for X is given by

$$f_X(x_{MLE}) = \frac{1}{(2\pi)^{\frac{N}{2}} |cov[X]|^{\frac{1}{2}}}$$

using the Jacobian rule for transforming to $T=t(X)$ gives

$$f_T(t_{MLE}) = \frac{|\frac{\partial t^{-1}(x)}{\partial t}|_{x=X_{MLE}}}{(2\pi)^{\frac{N}{2}} |cov(X)|} \approx\frac{|cov[X]|^{\frac{1}{2}}}{(2\pi)^{\frac{N}{2}} |cov[X]|^{\frac{1}{2}}} = (2\pi)^{-\frac{N}{2}}$$

Which is a constant, as your analysis shows (you could even check my calculations, by comparing the likelihood you get with $(2\pi)^{-\frac{N}{2}}$, and see if it is right). I would imagine that either the approximation is exact for normals, or exact for certain transformations $t(.)$, or becomes exact as $N\rightarrow\infty$, or the approximation error is in decimal places which you checked (or can't calculate), OR the computer you use to get the MLEs of $T$ uses the same approximation technique as I have (called the delta method).

• I am aware of te invariance property. My question refered to the fact, that the likelihood at the MLEs does not change. If you for example consider a trivial transform like $x \mapsto 0.1 * x$, then this transform will reduce variance because all the points move closer together. Hence the likelihood will increase, because of the factors of the form $1/ \sqrt(2 \pi \sigma^2)$. – Philipp Jan 26 '11 at 0:16
• Nope, the likelihood will remain exactly the same because a jacobian factor will come out of the transformation to make them equal (dx to 0.1 dx) – probabilityislogic Jan 26 '11 at 1:09
• I am not talking about the likelihood of $x$ under the new distribution obtained by plugging in the transformation, but about the likelihood of $t(x)$! That is something different. – Philipp Jan 26 '11 at 15:59
• Okay, here's how it goes, start with CDF $F_{T}(t_0)=Pr(T<t_0)$, then we know $T=t(X)$ sub this in to get $F_{T}(t_0)=Pr(t(X)<t_0)=Pr(X<t^{-1}(t_0))=F_{X}(t^{-1}(t_0))$ To get the pdf you have to differentiate *with respect to $t_0$, which leaves us with $f_{T}(t_0)=f_{X}(t^{-1}(t_0)) \times |\frac{\partial t^{-1}(t_0)}{\partial t_0}|$. It is this Jacobian I was taking about, which makes the "maximum" of the likelihood equal, whenever $t(x)$ is a 1-to-1 function. – probabilityislogic Jan 26 '11 at 23:29
• ...In your example we have $T=0.1X$ so $t(x)=0.1x$ and $t^{-1}(x)=10x$ with derivative 10, and $f_{T}(t)=f_{X}(10t)10$ Which, in the normal case you will have $f_{T}(t)=\frac{10}{\sigma \sqrt{2\pi}}exp(-\frac{1}{2\sigma^2}(10t-\mu)^2)$. So it would appear you're right. My thinking now, is that in fixing the variance, you are also fixing the Jacobian to be equal to 1 for certain functions $t(.)$ (i.e. functions who's inverse have derivative 1). This may explain why it doesn't always work. – probabilityislogic Jan 26 '11 at 23:50