2
$\begingroup$

I am struggling with the understanding of a saturated model.

As far as I know, the saturated model is the model that have as many parameter as the data points. But I don't know how to build it or what is the exact form of the saturated model.

For illustration, I have the example as follow:

$(Y, X_1, X_2, X_3, X_4)$ = $(1, 2, 3, 4,5); (2, 3,4, 5,6); (3,4,5, 6,7), (3, 5, 6,7,8)$

where $Y$ is a Poisson distribution, $X_1, X_2, X_3, X_4$ are the independent variable. The link function is log.

I think the saturated model may have the form:
$log(Y) = coef_1 * X_1 + coef_2 * X_2 + coef_3 * X_3 + coef_4 * ???$

(4 coefficients/ parameters as we have 4 data points)

And to fill in $???$, I think we have many way to choose : $X_1 * X_2, X_1/X_3$ or even $X_1* X_2 * X_3 / X_4$...

So I would like to have 2 question please:

  1. How many saturated model are their given a dataset for a GLM model ? (i.e. we fix the hypothesis that Y follow some distributions and also fix the dataset) and what is its form ? Is there any general principle to construct the saturated model ?

  2. If there are more than 1 saturated models, what is the "real saturated" model ? (because as I know, the saturated model is defined to be the model that fit perfectly the dataset)

Thank you very much for your help!

$\endgroup$
3
$\begingroup$

The question is not specific to GLMs, or to Poisson models. This applies also to any regression model.

A saturated model is one in which there are as many estimated parameters as observations, as you say. By definition, this will lead to a perfect fit, but will be of little use statistically.

  1. How many saturated model are their given a dataset for a GLM model ?

Given an arbitrary dataset, you can construct as many saturated models as you wish. If there are insufficient variables, then you can add higher order terms, interactions, or other derived variables such a logarithms or fractional powers. Once the number of parameters equals the number of observations, the model will be saturated.

  1. If there are more than 1 saturated models, what is the "real saturated" model ? (because as I know, the saturated model is defined to be the model that fit perfectly the dataset)

I don't think the term "real saturated" is well-defined. If you have $n$ observations and $p$ variables, then (assuming a model with no intercept) with $n >p$ then you can include all $p$ variables, and just add further terms until you reach saturation. With $n = p$ then you can include all $p$ variables, and the model will be saturated, and with $n<p$ you can choose whatever variables you want (as well as derived terms) to achieve saturation.

$\endgroup$
12
  • $\begingroup$ Hi, thank you very much for your answer. It helps me a lot. Actually the purpose behind this question is to know how to compute the log-likelihood of a saturated model (in order to calculate the deviance eventually). My concerne is that if we have so many saturated models, I think we will have so many log likelihood of the saturated model. But it seems to me there is only 1 log likelihood of the saturated model right ? Or if there is more than 1, do you know which one the statistical software use to calculate the deviance ? Thank you!! $\endgroup$ – InTheSearchForKnowledge Jun 7 at 19:46
  • $\begingroup$ You're welcome. I'm not sure what you mean. There is no unique saturated model. What software are you referring to that calculates a "saturated model deviance" ? Could you provide an example ? $\endgroup$ – Robert Long Jun 7 at 19:48
  • $\begingroup$ I use R. The definition of deviance of a GLM Model is $2 * ( log L_{Model} - log L_{Saturated Model}$ I know that we can compute $log L_{Model}$. So to compute the deviance, we have to compute $log L_{Saturated Model}$, but since there are many, I don't know how to compute it. $\endgroup$ – InTheSearchForKnowledge Jun 7 at 19:56
  • $\begingroup$ Ahh, OK I see what you mean. So, we usually use deviance to compare 2 models. The saturated deviance (however we define it) will be the same for all models fitted to this dataset so we can think of it as a constant, when comparing the deviance of two models, the constant obviously cancels out. So I don't think you need to actually compute the likelihood of the saturated model. $\endgroup$ – Robert Long Jun 7 at 20:04
  • 1
    $\begingroup$ It confuses me too. I was actually thinking about a logistic model and the answer by @Taylor in that thread does show that it is zero for a logistic model (with ungrouped) data, but you have poisson data....... $\endgroup$ – Robert Long Jun 7 at 20:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.