This might have a very simple answer, but I am doing an analysis of a financial series and have decided to use regression in order to predict a particular revenue given a set of input variables. Since this will ultimately be reported to a budget committee, we have aggregated all of the data to a monthly level so that we can use the model to make predictions of monthly totals.
Since we are working with monthly totals, we have far fewer observations than I would like (~11,000 records aggregates down to just 41 monthly totals); fitting my model with 19 predictors gives:
F(19, 21) = 11.21, p < 0.000, r2 = 0.910
However, when I presented this to my supervisor, he said that the good fit was probably spurious because the model contains so many predictors and so few free observations. Out of curiosity, I fit the model to the unaggregated data (individual records), getting:
F(19, 1227) = 268.3, p < 0.000, r2 = 0.806
On the basis of r2 alone, I'm convinced that overfitting does not seem to be the cause of the good fit at the monthly level (otherwise why would it fit a much larger dataset with only a small penalty?).
So, are there common tests I could perform or statistics to present that would help me convince myself (or my supervisor) that the monthly model is not overfitted?