I plan to numerically estimate the parameters of a GLM but with constraints imposed on some of the parameters. In this case, does the general approach of estimating the covariance matrix of my MLE estimators still apply? For example, those given by: https://www.statlect.com/fundamentals-of-statistics/maximum-likelihood-covariance-matrix-estimation


If the ML estimators in the restricted parameter space happens to be on the edge of that space (that is, your restrictions are active at the optimum), then the usual approximate distribution theory for the ML estimators is invalid. In practice, there will be problems also when the estimate is close to the edge.

So the problem is not so much about estimating the covariance matrix. The covariance matrix is just a tool to be used with the normal approximation. When the parameter is on (or close) to the boundary, the normal approximation is no longer valid, so you should think about some other way of doing inference. Maybe using the likelihood function directly, as with profile likelihood.


How to do likelihood inference when the parameter is on (or close to) the boundary is a big question, and really should have its own question (needing long answers). For the moment, I will only give a few references: Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard Conditions, and Likelihood Ratio, Score, and Wald Tests in a Constrained Parameter Space and Statistical Inference Using Maximum Likelihood Estimation and the Generalized Likelihood Ratio When the True Parameter Is on the Boundary of the Parameter Space

  • 1
    $\begingroup$ Thanks @kjetil b halvorsen, your answer is concise and intuitive. As for profile likelihood, if I require all the parameters, profile likelihood wouldnt apply would it? My understanding of profile likelihood is from this source:stats.stackexchange.com/questions/28671/… $\endgroup$ – krenova Dec 30 '18 at 7:55
  • $\begingroup$ thanks again @kjetil b halvorsen, the references would really help :) $\endgroup$ – krenova Dec 31 '18 at 6:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.