Choosing between two regression models I have two regression models:
Model A:
569 observations
13 predictors
Model B:
569 observations
14 predictors
Model diagnostics are:
R^2 value of Model A < R^2 value of Model B
F-statistic of Model A > F-statistic of Model B
p-value for Model A > p-value of Model B
Residual variance of Model A > Residual variance of Model B
Both models are similarly plausible, physically.
Which model should I pick?
 A: You have not given us enough information to tell you which model should be preferred.
First, it may be that neither model is a good model. An analogy: is an aluminum boat with a big hole in it better than a wooden boat with a big hole in it? Well on one hand, the aluminum boat is lighter, more fire resistant, etc. On the other hand, neither boat floats. Have you checked that both models satisfy the assumptions of linear regression (normality or least symmetry of residuals, no heteroskedasticity or strong outliers, etc.)?
$R^2$ is in general, not a very good metric for choosing among models or even describing models. There any many, many instances in general linear modeling where $R^2$ is misleading or just plain bogus. See Anscombe's quartet for one example. Any time you add a predictor you will get a lower $R^2$, up to adding a predictor for every single data point which will yield a perfect $R^2$ of 1 and a model which is entirely useless for prediction or description.
What are the F-statistic and p-value of? Without more information these are meaningless comparisons. An F-statistics and a p-value relate to a specific hypothesis, so you'd have to specify what that is before we could tell you how to interpret. But in general, the difference between statistically significant and not statistically significant, is not itself statistically significant. This means that there is no meaningful distinction between a p-value of 0.0501 and 0.0499, or between a p-value of 0.001 and 0.002. P-values should not,in general, be treated on a continuous scale or compared to one another. Source.
So in short, none of the information you have given is helpful in determining which model is better description of the data.
Some recommendations:
In general, more parsimonious models are preferred over less parsimonious model. This is rationale behind information criterion statistics that are so in vogue these days. You could consider adding AIC or BIC to the mix of criteria you have listed.
Lastly if these are nested model, i.e. they only differ by the addition of one predictor (you haven't specified, but I'm going to go out on a limb and assume they are) then you can perform a likelihood ratio test. That will tell you directly whether the addition on one more predictor produces a meaningful difference in the descriptive power of the model. Which is better means of testing the hypothesis that these two models are different than a comparison of summary statistics.
A: Based on this, I would go with B.  Of course, I'm sure there is some conceivable situation where A might be better because of some nonlinear cost function, but in general, these characteristics suggest B.
Why?
R^2 is essentially a measure of how much of the variation in the output can be accounted for using the inputs.  Higher R^2 means more variation is accounted for with B.
Similarly, lower residual variance with B means more of the variance is accounted for and is thus part of the model.
A lower p-value means it is less likely that the model coefficients are the results of general randomness rather than true system characteristics.
The F-statistic is actually higher for A, which does point that direction (A being a better model).  But unless it is massively larger and all others are only slightly skewed in favor of B, I would let this one go, and generally go the other way (i.e. B).
You could make this question a bit more complex by considering a cost function.  Is it harder (more time consuming, expensive, etc..) to collect the 14 predictors or the 13?  How much does it save you to predict more of the variation?  Is the reduction in residual variation worth the increase in collection costs.
