I recall from my graduate school days that the Gauss-Markov (GM) theorem states that the Best Linear Unbiased Estimator (BLUE) in a linear regression is $\vec{\beta}=(X^TX)^{-1}X^T\vec{y}$. An amazing aspect of the proof is that you do not need distributional assumptions to prove the claim.

I've been studying up on GLMs (and using them) now for several years and you can definitely draw various analogies between GLMs and the linear model.

I've been trying to understand how the GM theorem-or a generalization of it-fits into the GLM theory. It seems like the only place the distribution assumption is used-and even then only sometimes used-is for the link choice. Notably hypothesis testing does not use the distributional assumption instead usually using a Gaussian, or large sample approximation.

My question is if for estimation purposes there is an analogous result to the GM theorem stating BLUE? This may require a generalized definition of linearity but should still be stateable and provable. If anyone is aware of this result a reference would be appreciated.

  • 1
    $\begingroup$ It's worth noting that MLE estimates of GLM parameters are generally biased: stats.stackexchange.com/questions/60723/… $\endgroup$ – Matthew Drury Sep 19 '17 at 4:59
  • 1
    $\begingroup$ @Matthew Drury -> For finite samples the GLM estimates are biased but they are asymptotically unbiased. I guess the equivalent statement to BLUE in GLM world would be an asymptotic statement, good point. $\endgroup$ – Lucas Roberts Sep 19 '17 at 13:14
  • $\begingroup$ I wonder if this (and the comments) does not answer your question? stats.stackexchange.com/questions/346304/… $\endgroup$ – Patrick May 16 '18 at 8:55
  • $\begingroup$ @Patrick, not quite, that post has a Gaussian error assumption. I'm looking for an analogous proof of for a general GLM. The proof may have more conditions (e.g. an asymptotic statement only) but there should be an analogous statement for GLMs too. $\endgroup$ – Lucas Roberts May 16 '18 at 13:38
  • 1
    $\begingroup$ @Digio, FWIW I think the answer is it doesn't really matter other than some conventions but no mathematical theory. For example, a gamma distribution conventionally uses a log-link but that is not the canonical link. As I indicated in the question, the hypothesis tests do not really rely on distribution b/c they are asymptotic and normal cutoff values. Kind of difficult to prove that this doesn't matter though. $\endgroup$ – Lucas Roberts Feb 14 at 21:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.