1
$\begingroup$

In the context of a generalized linear model I have heard three different amounts of parameters which belong to a saturated model:

  1. The number of observations
  2. The number of unique predictor observations
  3. The number of unique linear predictors $\beta_1 X_1 + \ldots + \beta_p X_p$

I am considering a binary response variable $Y \in \lbrace 0,1 \rbrace$ and I am interested in fitting logistic regression models with respect to categorical predictors. Now I am wondering how to properly calculate the deviance if multiple data points share the same predictor values (but not necessarily the same response). So for example the three data points $$(Y=0, X_1 = 1, X_2 = 0, X_3 = 1) \\ (Y=1, X_1 = 1, X_2 = 0, X_3 = 1) \\ (Y=1, X_1 = 1, X_2 = 0, X_3 = 1)$$ could belong to my data set.

If the first case would be true, then the fitted log likelihood of the saturated would be equal to 0. In the second case however, there would be a parameter for each unique predictor setting which would lead to the log likelihood of the saturated model not being 0 if the response takes different values for such a setting.

So which of the three amounts is correct?

$\endgroup$
0
$\begingroup$

After reading some chapters of Alan Agresti's book "Categorical Data Analysis" (Third Edition), I am now pretty confident that the answer to my question depends on the data type.

In chapter 4.5.3 he states that the saturated model does have a parameter for every unique predictor combination if you are considering a grouped data set. This means that for every unique predictor observation there is one data point with the response being equal to the total count of successes for this specific predictor vector, i.e. one is considering a binomial distribution for every unique predictor setting.

On the other hand if you consider a data point for every single observation with $Y \in \lbrace 0,1 \rbrace$ (as I do), you are looking at many Bernoulli distributed outcomes. Agresti states in chapter 4.5.3 that one has to consider a parameter for each observation in this ungrouped case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.