AIC formula in Introduction to Statistical Learning I'm a little puzzled by a formula presented in Hastie's "Introduction to Statistical Learning". In Chapter 6, page 212 (sixth printing, available here), it is stated that:
$AIC = \frac{RSS}{n\hat\sigma^2} + \frac{2d}{n} $
For linear models with Gaussian noise, $d$ being the number of predictors and $\hat\sigma$ being the estimate of error variance. However,
$\hat\sigma^2 = \frac{RSS}{(n-2)}$
Which is stated in Chapter 3, page 66.
Which would imply:
$AIC = \frac{(n-2)}{n} + \frac{2d}{n} $
Which can't be right. Can someone point out what I'm doing incorrectly?
 A: Unfortunately this will be a rather unsatisfying answer...
First of all usually for the AIC calculation you will use the Maximum Likelihood estimate of $\sigma^2$ which would be biased. So that would reduce to $\sigma^2 = \frac{RSS}{n}$ and ultimately the calculation you do would reduce to $1+2\frac{d}{n}$. Second I would refer you to the Wikipedia article on AIC in particular in the equivariance cases section. As you see there it is clear that most derivations omit a constant $C$. This constant is irrelevant for model comparison purposes so it is omitted. It is somewhat common to see contradictory derivations of AIC because exactly of that issue. For example Johnson & Wichern's Applied Multivariate Statistical Analysis, 6th edition give AIC as: $n \log(\frac{RSS}{N}) + 2d$ (Chapt. 7.6), which clearly does not equate the definition of James et al. you are using. Neither book is wrong per se. Just people using different constants. In the case of the James et al. book it seems they do not allude this point. In other books eg. Ravishanker and Dey's A First Course in Linear Model Theory this is even more profound as the authors write:
\begin{align} 
AIC(p) &= -2l(y; X, \hat{\beta}_{ML}, \hat{\sigma}_{ML}^2) + 2p \\
&= -N \log(\hat{\sigma}_{ML}^2)/2 - N/2 + 2p \qquad (7.5.10)
\end{align}
which interestingly it cannot be concurrently true either. As Burnham & Anderson (1998) Chapt 2.2 write: "In the special case of least squares (LS) estimation with
normally distributed errors, and apart from an arbitrary additive constant, AIC can be expressed as a simple function of the residual sum of squares."; B&A  suggest the same AIC variant that J&W use. What messes you up is that particular constant (and the fact you were not using the ML estimate for the residuals.) Looking at M. Bishop's Pattern Recognition and Machine Learning (2006) I find an even more contradictory definition as:
\begin{align} 
AIC  &= l(D|w_{ML}) - M  \qquad (1.73)
\end{align}
which is funny because it not only omits the multiplier from the original paper but also goes ahead to tumble the signs so it can use AIC based selection as a maximization problem...
I would recommend sticking with the old fashioned definition $−2\log(L)+2p$ if you want to do theoretical derivations. This is the one Akaike states in his original paper. All the other intermediate formulas tend to be messy and/or make some implicit assumptions. If it is any consolation, you "did nothing wrong".
A: I think that you are confusing the two residual sum of squares that you have. You have one RSS to estimate the $\hat{\sigma}^2$ in the formula, this RSS is in some sense independent of the number of parameters, $p$. This $\hat{\sigma}^2$ should be estimated using all your covariates, giving you a baseline unit of error. You should call the RSS in the formula for AIC: $\text{RSS}_{p_i}$, meaning that it corresponds to model $i$ with $p$ parameters, (There may be many models with $p$ parameters). So the RSS in the formula is calculated for a specific model, while the RSS for $\hat{\sigma}^2$ is for the full model.
This is also noted in the page before, where $\hat{\sigma}^2$ is introduced for $C_p$.
So the RSS for the formula in AIC is not indepednent of $p$, it is calculated for a given model. Introducing $\hat{\sigma}^2$ to all of this is just to have a baseline unit for the error, such that there is a "fair" comparison between the number of parameters and the reduction in error. You need to compare the number of parameters to something that is scaled w.r.t. the magnitude of the error.
If you would not scale the RSS by the baseline error, it might be that the RSS is dropping much more than the number of variables introduced and thus you become more greedy in adding in more variables. If you scale it to some unit, the comparison to the number of parameters is independent of the magnitude of the baseline error.
This is not the general way to calculate AIC, but it essentially boils down to something similar to this in cases where it is possible to derive simpler versions of the formula.
