AIC for logistic regression

Question

On page 231 of The Elements of Statistical Learning AIC is defined as follows in (7.30)

Given a set of models $f_\alpha(x)$ indexed by a tuning parameter $\alpha$, denote by $\overline{err}(\alpha)$ and $d(\alpha)$ the training error and number parameters for each model. Then for a set of models we define

$$AIC(\alpha) = \overline{err}(\alpha) + 2 \cdot \frac{d(\alpha)}{N}\hat{\sigma_\epsilon}^2$$

Where $\overline{err}$, the training error, is $\frac{1}{N}\sum_{i=1}^NL(y_i,\hat{f}(x_i))$.

On the same page it is stated (7.29) that

For the logistic regression model, using the binomial log-likelihood, we have

$$AIC = -\frac{2}{N} \cdot \text{loglik} + 2 \cdot \frac{d}{N}$$

where "$\text{loglik}$" is the maximised log-likelihood.

The book also mentions that $\hat{\sigma_\epsilon}^2$ is an estimate of the noise variance, obtained from the mean-squared error of a low-bias model.

It is not clear to me how the first equation leads to the second in the case of logistic regression? In particular what happens to the $\hat{\sigma_\epsilon}^2$ term?

Edit I found in a later example in the book (on page 241) the authors use AIC in an example and say

For misclassification error we used $\hat{\sigma_{\epsilon}}^2=[N/(N-d)] \cdot \overline{err}(\alpha)$ for the least restrictive model ...

This doesn't answer my question as it doesn't link the two aforementioned expressions of AIC, but it does seem to indicate that $\hat{\sigma_{\epsilon}}^2$ is not simply set to $1$ as stated in Demetri's answer.

Answer 1

I imagine they made an approximation. $\sigma^2_\epsilon$ is the residual variance of the outcome conditioned on the variables $x_i$. When the outcome is binary, as in logistic regression, $\sigma^2<1$.

When we compare models with AIC, only the absolute differences between models matter, so using the approximation $\sigma^2=1$ for all models isn't so offensive. Let me demonstrate

$$\Delta AIC = AIC_1 - AIC_2 = \dfrac{-2}{N}(\text{loglik}_1 - \text{loglik}_2) + \dfrac{2}{N}(d_1 - d_2) $$

Because we made the assumption that $\sigma^2_\epsilon$ was the same for each model (namely, it was 1) it would factor out of the difference between models' effective number of parameters. Setting $\sigma^2_\epsilon=1$ isn't arbitrary, it is an upper bound on the variance of a binary variable. A least upper bound would be 0.25 and it it isn't quite clear to me why that wasn't chosen, but again the choice of $\sigma^2_\epsilon$ seems only to affect the AIC values and not the differences between model AIC, which is what we're really after.