1
$\begingroup$

I need to recreate some of the results of R in a python library, specifically LL-Null. I know that it describes

The null deviance shows how well the response is predicted by the model with nothing but an intercept.

as per Interpreting Residual and Null Deviance in GLM R

Question

Does this mean that I use an already-fit model and 0 out all the non-intercept weights? If not, how would this go? I need to implement it custom for an internal library I am building

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

The documentation describes a null model that is just an intercept, without any features; in other words, the null model estimates the intercept that minimizes the loss.

For instance, in an OLS model, you can show that the value $\beta_0$ that minimizes the square error $\sum_i (y_i - \beta_0)^2$ is $\beta_0=\bar{y}$ the sample mean of $y$.

It's generally the case that the sample mean of $y$ is different from the intercept of a regression model that includes one or more features/independent variables.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Ahh, gotcha! So, you're basically saying "take the average" of the output variable? $\endgroup$
    – IanQ
    Commented Mar 17, 2021 at 22:08
  • $\begingroup$ Not necessarily. Different models will have different losses or parameterizations, such which might imply that the optimal value of the intercept in the null model takes an alternative form. Your question references GLMs, for instance. $\endgroup$
    – Sycorax ♦
    Commented Mar 17, 2021 at 22:23
  • $\begingroup$ Hmm, could you elaborate on that? Say I've got GLMs all using least-squares. How would I derive what the null model for each of them? E.g a Poisson, a sigmoid, probit, etc. Do you know of any articles or resources? $\endgroup$
    – IanQ
    Commented Mar 17, 2021 at 22:57
  • $\begingroup$ The R documentation for GLMs, which you cite, contains four references. $\endgroup$
    – Sycorax ♦
    Commented Mar 17, 2021 at 23:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.