3
$\begingroup$

Is it possible to specify a vector of adjusted $R^2$ values (or any other measure like AIC, BIC, $C_p$) for the set of all possible models in a data set, and then generate data that is consistent with that pattern of adjusted $R^2$? What restrictions would we have to place on the adjusted $R^2$ vector?

For example, say there are 3 independent variables. That means there are $2^3=8$ possible models (no interactions or transformations of the IV's). Can I specify a vector of adjusted $R^2$ values that correspond to each of the 8 possible models and then generate data that is consistent with that vector?

My intuition is that it should be possible. Adjusted $R^2$ reduces to some function of $RSS$ (and the number of parameters and n, which we can specify). Then, we can create any $X$ matrix we want and solve for $y$ values and the true $\beta$'s.

It will be hard, and the process will likely generate some finite sample correlation between the $X$'s and $\varepsilon$'s, even if they're drawn independently. But in expectation with large N, it seems possible to converge to that pattern of $R^2$.

Is this possible? And what does a sketch of the procedure look like?

$\endgroup$
2
$\begingroup$

Not without some constraints. Let's just consider two independent variables. Let ${R_1}^2$ be the (unadjusted) coefficient of determination for a single variable model and ${R_2}^2$ be that of the model updated with one additional variable added. Use a bar, $\bar{R}^2$, to denote the adjusted $R^2$. For $n \ge 4$,

\begin{align} \bar{R_2}^2 &= {R_2}^2-(1-{R_2}^2)\frac{2}{n-3} \\ &\ge {R_2}^2-2(1-{R_2}^2) \\ &= 3{R_2}^2-2 \\ &\ge 3{R_1}^2-2 \\ &\gt 3\bar{R_1}^2-2 \end{align}

So, for example, the combination $\bar{R_1}^2 \ge 0.8$ and $\bar{R_2}^2 \lt 0.4$ is impossible.

$\endgroup$
  • $\begingroup$ This is a very good answer. So, it's clear that there are restrictions on the adjusted $R^2$ vector among nested models. Are there any other restrictions we might be missing? $\endgroup$ – Devon Jan 30 '15 at 1:51
  • $\begingroup$ The entries of the covariance matrix of the dependent and all independent variables can be used to recover all the regressions. The matrix is constrained to be positive semi-definite. The penalties complicates things. $\endgroup$ – A. Webb Jan 30 '15 at 2:45
0
$\begingroup$

You have 8 models, each with 3+intercept+error variance=5 parameters, i.e. 5x8=40 total parameters to estimate. With N observations, you have 3xN array of dependent variables X. Additionally all y's must be the same, i.e. you have 7*N restrictions on independent variables.

So, you have in total 3N+40 free parameters to fit 8 $R^2$ numbers + 7*N restrictions on y's. This means that when 3N+40 > 7N+8, or N<8, you have infinite number of ways to fit.

When N=8, there's exactly one way to fit.

When N>8, you'd have to fit using some sort of curve fitting, such as least squares

If X matrix is fully specified, then you have only 40 free parameters, and 7N+8 restrictions. The same logic will apply except the threshold on N is 32/7.

$\endgroup$
  • $\begingroup$ If I specify the $X$ matrix, would that provide sufficient restrictions? $\endgroup$ – Devon Jan 29 '15 at 16:10
  • $\begingroup$ Can you explain why each model has 3+2=5 parameters? I would think that some models have less b/c all X's aren't being used (e.g., the intercept-only model). Maybe I'm confused... $\endgroup$ – Devon Jan 29 '15 at 16:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.