Whether to use r-square or adjusted r-square with a small sample size that may represent the entire population?

Question

I read online that it is only necessary to use adjusted-$R^2$ when you are working with a sample rather than the entire population.

The data I'm working with is information on a series of live educational seminars. Each datapoint represents a single seminar that was held in the past, and contains various information on that program's characteristics.

In trying to decide whether to use $R^2$ or adjusted-$R^2$, I can see two different sides to the coin.

1. Since my dataset contains every seminar we've held to date, I'm working with the entire population, so I should go with regular old $R^2$.

2. The population of interest is really all possible seminars, including those that haven't happened yet, especially since my goal in this model is to better understand the relationship of factors going forward. Therefore I am looking at a sample, and I should use adjusted-$R^2$.

Which logic is correct, and which measure of correlation should I use?

Answer 1

I think you have two different viewpoints and no correct or incorrect answer. But I would be more inclined to go with 2. Although in 1 you said you have included every seminar held to date it seems that your universe includes future seminars as well.

But accepting 2 does not settle the issue between R square and adjusted R square. The reason adjusted R square is included in the first place is that if the size of the model parameters or covariates is large relative to the size of the sample the ordinary R square will tend to overestimate the amount of variation that the model explains. It is the percentage of variance explained by the model for the observed data set but it overestimates the amount of variation the model will explain on a new data set randomly sampled from the population. The adjusted R square makes an effort to account for this bias. But if the sample size is very large relative to the number of covariates R square and adjusted R square won't differ much and choosing adjusted R square is far less important than if the sample size was only slightly larger than the number of parameters estimated in the model.

So I see the choice of adjusted R square over R square being more a matter of the relative size of the sample size to the number of parameters rather than whether or not the sample represent the enitre population or just a random piece of it.

Answer 2

It is not the case that one uses R-squared for entire populations and adjusted R-squared for samples. They each give different information. R-squared is the proportion of variability in the data accounted for by your model. Adjusted R-squared takes into account (i.e., adjusts for) the number of explanatory terms in your model. R-squared can never be decreased by adding additional terms because you can't explain less variation with more predictors. On the other hand, the adjusted R-squared, will increase only if the added predictors benefit the model. Conceptually, you can think of adjusted R-squared as penalizing complexity. Only if that added complexity significantly increases (i.e., more than would be expected by chance) the predictive power of the model will adjusted R-squared increase. Adding poor predictors can actually decrease adjusted R-squared. The real question then, is what are you trying to answer using this information?