16
$\begingroup$

What is null model in regression and whats the relationship between null model and null hypothesis?

For my understanding, does it mean

  • Using "average of the response variable" to predict continuous response variable ?
  • Using the "label distribution" in predicting discrete response variables?

If that is the case, it seems there is a missing the connections between null hypothesis.

$\endgroup$
  • 4
    $\begingroup$ Note, in R, you can try fit = lm(formula = y ~ 1, data) and you should see the mean of y. Also, see MorganBall's answer. I would agree with his response the most. Also, a null model can be a model with $p$ predictors, with an alternative model being one with $p+k$, where k can be 1,2,... additional covariates. $\endgroup$ – Jon Feb 2 '17 at 20:26
  • 3
    $\begingroup$ Here's a reference for you: onlinecourses.science.psu.edu/stat501/node/295 $\endgroup$ – Jon Feb 2 '17 at 20:31
11
$\begingroup$

No, I would say "null model" essentially has the same meaning as "null hypothesis": the model if the null hypothesis is true. What this means, in a particular case, of course depends upon the concrete null hypothesis.

Your interpretations as "the average value" (you probably want to say "the marginal distribution on response variable") not taking into account any predictors, is one possibility, corresponding to the null hypothesis of an "omnibus test", testing all the parameters (except the intercept) simultaneously.

But interest could well focus on a model of the form $$ y_i = \beta_0 + \beta_1^T x_{1i} + \beta_2^T x_{2i} + \epsilon_i $$ where $x_1$ contains the predictors you know are affecting the outcome, so are not wanting to test, while $x_2$ contains the predictors you are testing.

So the null hypothesis will be $\beta_2 =0$ and the null model would be $y_i = \beta_0 + \beta_1^T x_{1i} + \epsilon_i$. So it depends.

$\endgroup$
  • 2
    $\begingroup$ The null hypothesis is usually something specific about parameter values; I'd say the null model would be the null hypothesis plus all the accompanying assumptions under which the null distribution of the test statistic would be derived -- its the assumptions that contain most of the model. For example the null hypothesis doesn't mention independence, but I'd definitely say it's part of the null model. $\endgroup$ – Glen_b Sep 17 '17 at 1:42
17
$\begingroup$

A null model is related to a null hypothesis. Take the following univariate model:

$Y=\alpha+\beta_{1}X + \epsilon$

My null hypothesis would normally be that $\beta_{1}$ is statistically no different from zero.

$H_{0}: \beta_{1}=0$ (null hypothesis)

$H_{A}: \beta_{1}\neq 0$ (alternative hypothesis)

For a univariate linear model, such as the above, if we were to reject the alternative hypothesis then we could drop $\beta_{1}X$ from the linear model and we'd be left with

$Y = \alpha + \epsilon$

Which is your Null model and the same as mean of $Y$.

$\endgroup$
  • 1
    $\begingroup$ To the last point, yes that is correct. In R, you can see this by comparing the intercept of lm(y ~ 1, data) and mean(y). $\endgroup$ – Jon Feb 2 '17 at 20:29
  • 2
    $\begingroup$ +1 Nice answer Morgan! I have taken the liberty to edit your notation a tad bit, because it looked odd. $\endgroup$ – Alexis Feb 3 '17 at 0:36
9
$\begingroup$

In regression as described partially in the other two answers the null model is the null hypothesis that all the regression parameters are 0. So you can interpret this as saying that under the null hypothesis there is no trend and the best estimate/predictor of a new observation is the mean which is 0 in the case of no intercept.

$\endgroup$
  • 1
    $\begingroup$ This answer helped me to understand null = 0 in coefficients (other than intercept), Thanks! $\endgroup$ – Haitao Du Feb 2 '17 at 20:03
  • 1
    $\begingroup$ also, the model can be the intercept only model, compared to another model. $\endgroup$ – D_Williams Feb 2 '17 at 20:06
  • 1
    $\begingroup$ +1, this is a useful addition to the thread. However, I would say that this is a specific & very restrictive use of the term "null model". The term is often (most of the time in my guess) used more loosely. $\endgroup$ – gung - Reinstate Monica Sep 19 '17 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.