Testing difference between two (adjusted) r^2 Let's say I have two regression models, one with three variables and one with four.  Each spits out an adjusted r^2, which I can compare directly.
Obviously, the model with the higher adjusted r^2 is the better fit, but is there way to test the difference between the two adjusted r^2 and get a p-value?
I know you can do Chow test to test the difference between slopes, but this is variance, so I don't think that's what I'm looking for.
Edit: One model does not simply contain a subset of variables from the other model, or else I would probably use stepwise regression.
In model 1, I have four variables: W, X, Y, and Z.
In model 2, I have three variables: W, X, and (Y+Z)/2.
The idea is that if Y and Z are conceptually similar, the model may make better predictions by grouping these two variables together prior to entering them into the model.
 A: As whuber stated this actually is a case of nested models, and hence one can apply a likelihood-ratio test. Because it is still not exactly clear what models you are specifying I will just rewrite them in this example;
So model 1 can be:
$Y = a_1 + B_{11}(X) + B_{12}(W) + B_{13}(Z) + e_1$
And model 2 can be (I ignore the division by 2, but this action has no consequence for your question):
$Y = a_2 + B_{21}(X) + B_{22}(W+Z) + e_2$
Which can be rewritten as:
$Y = a_2 + B_{21}(X) + B_{22}(W) + B_{22}(Z)+ e_2$
And hence model 2 is a specific case of model 1 in which $B_{12}$ and $B_{13}$ are equal. One can use the likelihood-ratio test between these two models to assign a p-value to the fit of model 1 compared to model 2. There are good reasons in practice to do this, especially if the correlation between W and Z are quite large (multicollinearity). As I stated previously, whether you divide by two does not matter for testing the fit of the models, although if it is easier to interpret $\frac{W+Z}{2}$ then $W+Z$ by all means use the average of the two variables.
Model fit statistics (such as Mallow's CP already mentioned by bill_080, and other examples are AIC and BIC), are frequently used to assess non-nested models. Those statistics do not follow known distributions (like the log-likelihood does, Chi-square) and hence the differences in those statistics between models can not be given a p-value. 
A: Take a look at Mallow's Cp:
Mallow's Cp
Here's a related question:
Is there a way to optimize regression according to a specific criterion?
A: Given the setup in Andy W answer, if one estimates the model
$Y = a_3 + B_{31}(X) + B_{32}(W+Z) + B_{33}(Z) +  e_3$
the test associated with $B_{33}$ gives you the test that model 1 is different from model 2. The reason is that $B_{33}$ is exactly (a part from the sign) the difference between $B_{12}$ and $B_{13}$. Thus, if their difference is not significant, keeping W and Z in the model (model 1) does not help in terms of variance explained as compared with combining them in one variable (model 2). If $B_{33}$ is significant, model 1 is better.  
