Generate fake data consistent with adjusted R^2 pattern

Question

Is it possible to specify a vector of adjusted $R^2$ values (or any other measure like AIC, BIC, $C_p$) for the set of all possible models in a data set, and then generate data that is consistent with that pattern of adjusted $R^2$? What restrictions would we have to place on the adjusted $R^2$ vector?

For example, say there are 3 independent variables. That means there are $2^3=8$ possible models (no interactions or transformations of the IV's). Can I specify a vector of adjusted $R^2$ values that correspond to each of the 8 possible models and then generate data that is consistent with that vector?

My intuition is that it should be possible. Adjusted $R^2$ reduces to some function of $RSS$ (and the number of parameters and n, which we can specify). Then, we can create any $X$ matrix we want and solve for $y$ values and the true $\beta$'s.

It will be hard, and the process will likely generate some finite sample correlation between the $X$'s and $\varepsilon$'s, even if they're drawn independently. But in expectation with large N, it seems possible to converge to that pattern of $R^2$.

Is this possible? And what does a sketch of the procedure look like?

Answer 1

Not without some constraints. Let's just consider two independent variables. Let ${R_1}^2$ be the (unadjusted) coefficient of determination for a single variable model and ${R_2}^2$ be that of the model updated with one additional variable added. Use a bar, $\bar{R}^2$, to denote the adjusted $R^2$. For $n \ge 4$,

\begin{align} \bar{R_2}^2 &= {R_2}^2-(1-{R_2}^2)\frac{2}{n-3} \\ &\ge {R_2}^2-2(1-{R_2}^2) \\ &= 3{R_2}^2-2 \\ &\ge 3{R_1}^2-2 \\ &\gt 3\bar{R_1}^2-2 \end{align}

So, for example, the combination $\bar{R_1}^2 \ge 0.8$ and $\bar{R_2}^2 \lt 0.4$ is impossible.

Answer 2

You have 8 models, each with 3+intercept+error variance=5 parameters, i.e. 5x8=40 total parameters to estimate. With N observations, you have 3xN array of dependent variables X. Additionally all y's must be the same, i.e. you have 7*N restrictions on independent variables.

So, you have in total 3N+40 free parameters to fit 8 $R^2$ numbers + 7*N restrictions on y's. This means that when 3N+40 > 7N+8, or N<8, you have infinite number of ways to fit.

When N=8, there's exactly one way to fit.

When N>8, you'd have to fit using some sort of curve fitting, such as least squares

If X matrix is fully specified, then you have only 40 free parameters, and 7N+8 restrictions. The same logic will apply except the threshold on N is 32/7.