# Correlation matrix as maximum likelihood estimator under constraint

## The Problem

Although it seems to be straight forward I am struggling to prove the following statement. Assume, we have $$p$$-variate Gaussian observations $$\left\{x_1, \ldots, x_N \right\} \subset \mathbb{R}^p$$ with known mean $$\mu = 0$$. We encounter the sufficient statistic $$S = \frac{1}{N} \sum_{l=1}^N x_l x_l^t.$$ We are looking for the solution of the following optimization problem, which is the constrained log-likelihood estimator for the normal distribution $$\hat{\Sigma} = argmax_{\Sigma_{ii} = 1 \forall i = 1, \ldots, p} -\log \left(|\Sigma|\right) - tr\left(\Sigma^{-1} S\right).$$ and additionally $$\hat{\Sigma}$$ shall be positiv-semidefinit (p.s.d). It seems to be clear that $$\hat{\Sigma} = \Sigma_0 := D_S^{-\frac{1}{2}} S D_S^{-\frac{1}{2}}$$, where $$D_S$$ is the diagonal matrix with the diagonal elements of $$S$$. However, I can not prove this.

## What I have tried so far

In my oppinion the problem can be tackled from two sides.

• We know that the maximum likelihood estimator for the transformed observations $$x_l' = D_S^{-1}x_l, l = 1, \ldots, N$$ is equal to $$\Sigma_0$$. However, it is not obvious to me, that the problem is invariant to a scaling of the observations.
• By defining $$\Omega = \Sigma^{-1}$$, we can transform the problem to a convex optimization problem, where the search space is also convex. I.e. we are then looking for $$\hat{\Omega} = argmin_{\left(\Omega^{-1}\right)_{ii} = 1 \forall i = 1, \ldots, p} -\log \left(|\Omega|\right) + tr\left(\Omega S\right).$$ The Lagrange function is then $$L(\Omega, \lambda) = -\log \left(|\Omega|\right) + tr\left(\Omega S\right) + \sum_{j=1}^p \lambda_j\left(tr\left(\Omega^{-1} e_j e_j^T\right) - 1\right),$$ where $$e_j$$ is the $$j-th$$ unit column vector. Setting its derivative to $$0$$, I ended up with $$\frac{\partial L}{\partial \Omega} = -\Omega^{-1} + S -\Omega^{-1}\left(\sum_{j=1}^{p} \lambda_j \left( e_j e_j^{T} \right) \right) \Omega^{-1} \overset{!}{=} 0,$$ where $$\lambda_1, \ldots, \lambda_p$$ can be chosen arbitrarily. Unfortunately, I am not able to prove that this holds for $$\Omega = \Sigma_0^{-1}$$. Also, I do not use that $$\Omega$$ is p.s.d.. Optimizing with respect to the square root of $$A := \Omega^\frac{1}{2}$$ (which can be done to ensure the search space is p.s.d.), I can't write down the derivative with respect to $$tr\left(A^{-1} A^{{T}^{-1}} e_j e_j^T\right) - 1$$.

Can somebody give a rigorous argument why $$\Sigma_0$$ solves the problem? Thank you very much! I am looking desperately for it!

• It turns out, that $\Sigma_0$ is in fact not the maximizer. Apparently, there is also no closed form solution (as indicated for two dimensions in this paper on page 969). However, I would be curious if that is all we can say about the problem. Sep 15 at 13:37