# Degenerate multivariate normal in Maximum Likelihood Estimator (Akaike's Info Criterion, BIC, LR Test usage)

Let's suppose that the considered set of random variable has a covariance matrix which is psd. Therefore the Gaussian pdf must be written in its degenerate form, where the determninat of the covariance matrix is replaced by the pseudodeterminant (i.e. the product of non-zero egeinvalues) and the inverse of the covariance matrix is replaced by the pseudoinverse (Wikipedia link2 to stats.stackexchange).

My question is whether we are still able to make use of the information criteria and tests used to specify the model in the non-degenerate case, like the AIC, BIC, LR Test (Wald test, etc...). More precisely, given a number of observed samples t and a number of parameters k used in the model

$$AIC=-2Loglik+2k$$ $$BIC=-2Loglik+ln(T)2k$$ $$LRstatistic=2(Loglik_{fullmodel}-Loglik_{restrictedmodel})$$

Can these metrics be used in the current form reported above, even when a degenerate multivariate normal distribution used to compute the Loglik? For example, does the LRstatistic in this case preserve its Chi-sqaured distribution as for the standard non-degenerate case?

Clearly, a good hint on the way to solve the problem is to analyze the empirical distribution of LR test statistic using a simulation and the derivation of Akaike's and other ICs to see whether the non-singularity of the varcov matrix is strictly necessary to the derivation of the ICs. As far as the latter point is concerned, I have checked the derivation of Akaike's IC (which is also available here) and, in my opinion, I do not see that the non-singularity of the varcov matrix is strictly necessary. But I would like to hear the forum opinion on the point.

EDIT: notice that here we are talking about a probability model and the support of the varcov matrix is potentially allowed to vary across different specifications to be found with ICs/LR test

• Please provide some context for understanding this question: exactly what do you mean by "the considered set of random variables" and what is your "covariance matrix"? For instance, are you supposing some kind of probability model with a degenerate covariance matrix, or is it perhaps the case that the estimated covariance matrix for a set of estimates is nondegenerate? And in the first case, is the support of the covariance always the same subspace for every instance of the model or does the support vary?
– whuber
Commented Aug 2, 2019 at 14:42
• @whuber I am supposing a probability model and allowing the support to vary across different possible model specifications (I upvoted because the question was useful)
– Fr1
Commented Aug 2, 2019 at 14:45
• Can you maybe write down the model and it's implied covariance matrix and elaborate a bit on the reason for the degeneracy? Commented Aug 2, 2019 at 16:03
• @StoryTeller0815 yes also in my context usually we use constraints for a pd varcov, but on this occasion we cannot set constraints because, for reasons a bit long to explain here, the residuals must by definition have a psd varcov. I read that going Bayesian is a good way to overcome the issue. However, working both analytically (i.e. assuming my varcov is any varcov and solving with symbols of each scalar in the varcov) and numerically, I solved the problem reaching the expected solution. So I was wondering, why not staying in the degenerate MLE case? Especially if it worked numerically.
– Fr1
Commented Aug 2, 2019 at 17:11
• For the numerical resolution I used Python Scipy: indeed in the docs they state that if the varcov is singular then the formulas used will be automatically updated to the degenerate case in order to allow a numerical resolution docs.scipy.org/doc/scipy-0.14.0/reference/generated/… and I must admit that the numerical solution of the optimization converged exactly where we were expecting for all the experiments and simulations
– Fr1
Commented Aug 2, 2019 at 17:12