Consider the single response variable $Y\sim$Bin$(n,p)$.

The MLE estimate of $p$ is given by $\hat{p} = \frac{y}{n}$. I want to find the deviance: $$ 2[l(p_{\text{max}}) - l(\hat{p})] $$ where $p_{\text{max}}$ is the estimate of p under the saturated model. Maybe I am a little confused about the definition of the saturated model, but since there is only one observation, wouldn't $D=0$ ?

update If we consider the model $E(Y) = \hat{y} = x_i^T \beta = np$

Then the log likelihood is given by: $$ l(\beta) = \log \pmatrix{n\\y} + y \log p + (n-y) \log (1-p) $$ For the saturated model, there is a $p_i$ for each $y_i$, in this case, there is only one $y_i$, so only one $p_i = p$, differentiating the log likelihood wrt p, I get: $$ \hat{\beta} = p_{max} = \frac{y}{n} =y $$

For any other model with parameters less than the number of observations (what is this model in this case?), we have the MLE estimate given by: $$ \hat{\beta}=(X^TX)^{-1}X^Ty $$ so we have that $n p_{mle} =\hat{y} $, therefore, the deviance is given as:

$$ 2 \left( y \log p_{\max} + (n-y) \log(1-p_{\max}) - y \log p_{MLE} -(n-y)\log(1-p_{MLE}) \right ) $$ and this simplifies to: $$ 2 \left [ y\log(\frac{p_{MAX}}{p_{MLE}}) + (n-y) \log \frac{1-p_{MAX}}{1-p_{MLE}} \right ] $$

and this is the correct answer, what I'm really confused about is:

  1. Why is $n=1$ in the full model, but $n \neq 1$ in the second model? I feel like I am confusing the number of predictors and the number of observations, but isn't the number of predictors bounded by observations?

  2. Why isn't $p_{MAX} = p_{MLE}$ in this case since we only have one observation

  • 1
    $\begingroup$ Do you know the definition of the deviance? Just plug it in! $\endgroup$ – probabilityislogic Apr 16 '16 at 7:02
  • $\begingroup$ @probabilityislogic i thought that was the definition of the deviance $\endgroup$ – dimebucker91 Apr 16 '16 at 7:05
  • $\begingroup$ @probabilityislogic the saturated model is a model with a parameter for every observation, so that the data are fitted exactly, in this case, we have one observation, so the MLE estimate of the saturated model is the same as the MLE estimate of any nested model? $\endgroup$ – dimebucker91 Apr 16 '16 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.