Covariance of Constrained Maximum Likelihood Estimators

Question

I plan to numerically estimate the parameters of a GLM but with constraints imposed on some of the parameters. In this case, does the general approach of estimating the covariance matrix of my MLE estimators still apply? For example, those given by: https://www.statlect.com/fundamentals-of-statistics/maximum-likelihood-covariance-matrix-estimation

Answer 1

If the ML estimators in the restricted parameter space happens to be on the edge of that space (that is, your restrictions are active at the optimum), then the usual approximate distribution theory for the ML estimators is invalid. In practice, there will be problems also when the estimate is close to the edge.

So the problem is not so much about estimating the covariance matrix. The covariance matrix is just a tool to be used with the normal approximation. When the parameter is on (or close) to the boundary, the normal approximation is no longer valid, so you should think about some other way of doing inference. Maybe using the likelihood function directly, as with profile likelihood.

EDIT

How to do likelihood inference when the parameter is on (or close to) the boundary is a big question, and really should have its own question (needing long answers). For the moment, I will only give a few references: Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard Conditions, and Likelihood Ratio, Score, and Wald Tests in a Constrained Parameter Space and Statistical Inference Using Maximum Likelihood Estimation and the Generalized Likelihood Ratio When the True Parameter Is on the Boundary of the Parameter Space