3
$\begingroup$

I was reading about (simple) linear regression through origin and I have the following questions:

  1. What are the standard assumptions of such a regression model? I am asking this of the true model not the fitted model. In specific if you were to write down before you even fit the model what are assumptions of the error term, assumptions for the distribution of the parameters for inference and so on.
  2. How can you derive the MLE (NOT LSE) of the standard (Simple) regression model through the origin. And by derive I do not mean saying that MLE=LSE in this case and just minimizing the least Squares. I would be quite confused on whether to even start with normal distribution as I am still uncertain of the assumptions of the model through the origin.
Improve this question
$\endgroup$
6
  • 2
    $\begingroup$ Have you tried searching for answers to this question? I only suggest this because there are many many answers. Here are some that might be of help: stats.stackexchange.com/questions/13747/… stats.stackexchange.com/questions/54794/… stats.stackexchange.com/questions/133028/… $\endgroup$ – ilanman Sep 30 '16 at 23:48
  • 4
    $\begingroup$ Likelihood-based methods require that you assume a distribution, and if the assumed distribution is normal then you end up with least squares. The objective function just changes to $\sum_{i=1}^{n} (y_i - \beta x_i)^2$ if you don't have an intercept. $\endgroup$ – dsaxton Oct 1 '16 at 0:01
  • $\begingroup$ How about deriving mle for the sigma?..Thanks! $\endgroup$ – NoIdea Oct 1 '16 at 0:01
  • $\begingroup$ Also are we assuming that the error terms here are normally distributed with mean 0 and constant variance? I am just having trouble picturing the exact list of assumptions one would write down for a such a model $\endgroup$ – NoIdea Oct 1 '16 at 0:05
  • 1
    $\begingroup$ Generally in regression problems we don't use the MLE for $\sigma^2$ because it tends to be shrunk towards zero. We prefer to use the unbiased estimate $\sum_{i=1}^{n} (y_i - \hat{y}_i)^2 / (n - p)$. Yes, the error distribution is assumed to have zero mean, but the actual residuals will in general not sum to zero as they do when there's an intercept. The assumptions are basically the same as when you have an intercept. $\endgroup$ – dsaxton Oct 1 '16 at 0:16

Browse other questions tagged or ask your own question.