I am trying to derive the expression for the variance/standard errors of the alpha parameter in the simple regression framework.

The model is:

$$y_i = \alpha + \beta x_i + \epsilon_i$$ for $i=1,...,N$

We assume that the errors are normally and independently distributed with: $$E[\epsilon_i|x_i]=0$$ $$Var[\epsilon_i|x_i]=\sigma^2$$

Accordingly, we get the following log-likelihood function:

$$L(\alpha,\beta,\sigma^2)=-N/2log(2\pi)-N/2log(\sigma^2) -1/(2\sigma^2)\sum_{i=1}^{N} (y_i-\alpha -\beta x_i)^2$$

The partial first derivative w.r.t. alpha is: (1) $$\partial L/\partial \alpha=N/\sigma^{2}\sum_{i=1}^{N}(y_i-\alpha-\beta x_{i}) $$

From which, by setting it equal to 0, we get the MLE of alpha:


Where $\hat{y}=1/N\sum_{i=1}^{N} y_i$ and $\hat{x}=1/N\sum_{n=1}^{N} x_i$

It is known that the variance of the estimator of alpha is :

$$Var[\hat{\alpha}] = (\sigma^2\sum_{i=1}^{N}\hat{x}^2)/(N\sum_{n=1}^{N} (x_i-\hat{x})^2)$$

I know that the variance of the estimators can be derived using the inverse of the Fisher information matrix. However, if I compute the derivative of (1), namely the second derivative of the LogLik w.r.t. alpha, change sign and then take its expectation, I can not obtain the expression of the variance claimed by the authors. Can you help me with this last step, please?

  • $\begingroup$ You don't describe actually inverting the Information matrix. It seems like you're only inverting one of its diagonal elements. That's not the same thing. $\endgroup$ – whuber Aug 16 '19 at 12:43
  • $\begingroup$ How should I proceed in order to get the expression for the std of the alpha via MLE? $\endgroup$ – Alchemy Aug 16 '19 at 14:07
  • $\begingroup$ Exactly as you described: use the inverse of the full $2\times 2$ information matrix. $\endgroup$ – whuber Aug 16 '19 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.