4
$\begingroup$

So to estimate the parameters of a model using MLE one must write the likelihood function of having observed the data sample at hand by assuming that it came from a particular distribution.

In order to assume that the data came from a particular distribution, do I have to assume or know the probability distribution function of the dependent variable or of the error term?

I know there is a property, for linear regression models, where if the error follows a standard normal distribution then the dependent value, the intercept and the coefficients will also follow a standard normal distribution as well. Can someone elaborate on this, though?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

In the context of arbitrary parametric models, in which the theory of MLE is developed, there is no such thing as an error term. Some models have parts that are described as error terms, but not all do. All that MLE requires is a probabilistic model of the dependent variable in terms of the model's parameters. Whether or not this model contains an error term is irrelevant.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Ok, but what about in those models where there is an error term say in a linear regression. How is assuming an error is normally distributed affect our assumptions about the distribution of the dependent variable and the regressors? $\endgroup$
    – StatsScared
    Commented Mar 17, 2015 at 20:15
  • 1
    $\begingroup$ Linear regression doesn't need any assumptions about the distributions of the predictors (indeed, the predictors aren't even construed as random variables), nor about the marginal distribution of the dependent variable. Rather, the normality assumption is about the dependent variable conditional on all predictors. That conditional distribution is exactly the error term, because once you know the model, the coefficient values, and the predictor values, all the variability that remains in the dependent variable must be error, by definition. $\endgroup$
    – Kodiologist
    Commented Mar 18, 2015 at 1:11
  • $\begingroup$ Is there a proof that shows that indeed the conditional distribution of the dependent variable given the independent variable(s) (ie. y|x1,x2,...) equals the error term? $\endgroup$
    – StatsScared
    Commented Mar 19, 2015 at 21:44
  • 2
    $\begingroup$ I'm not sure which part you think calls for proof. Two ways in which a linear regression model can be characterized are: (1) The dependent variable (DV) is distributed as a linear combination of predictors plus an intercept plus a normally distributed error term with mean 0 and constant variance. (2) The DV is normally distributed with mean equal to a linear combination of predictors plus an intercept and constant variance. The equivalence of these definitions follows from the fact that A ~ Normal(0, σ) iff A + μ ~ Normal(μ, σ) for all real μ. This fact in turn follows from linearity of means. $\endgroup$
    – Kodiologist
    Commented Mar 19, 2015 at 22:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.