0
$\begingroup$

I've read about how F-tests can be used to compare models and to decide whether an additional variable should be included in the regression. However, I want to test whether A vs. B are better predictors of Y. (In the case of my actual project, I have two models, A vs. B, that attempt to predict some phenomena and I want to test which is a stronger predictor). So I run a linear regression:

Y ~ A + B

This gives me an ANOVA table showing that the F-value associated with A and B are both significant. However, the F-value of A is a powerful 20, but the F-value of B is a wimpier 5.

How should I compare the predictive powers of A vs. B? I want to definitively say that one is more predictive than the other one (preferably using non-Bayesian statistics). Comparing the slopes of the regression seems not appropriate since the value distributions of A and B may have different variances.

Should I take the SquaredSum(A) / SquaredSum(B) = my new F-value? If I do this, should the F-critical value have DF1 = n-2, DF2 = n-2, where n = number of subjects? None of this would change if I was doing a logistic regression and/or a multilevel model, right? Would this answer be most elegantly framed in terms of AIC or BIC?

Thanks

(I don't want to use Bayesian statistics for simplicity's sake if I'm explaining results to others. If I can do this all with a straightforward F-test, that would be nice.)

$\endgroup$
  • $\begingroup$ 1. What do you mean by "predictive power"? Do you mean which is more strongly-related to the outcome in your logistic regression model? Or do you mean which is going to be a better predictor of future cases? 2. Are A and B on the same scale? That is, are they both 1-7 scales or are they both1/0 variables etc.? $\endgroup$ – Mark White May 26 at 23:30
  • $\begingroup$ Sorry for that.... "Predictive power" is clearly bad phrasing. I meant this: "Do you mean which is more strongly-related to the outcome in your logistic regression model?" "Are A and B on the same scale?" In my project, yes. Although, I would be curious about situations where they are not? $\endgroup$ – Paul May 27 at 17:07
  • 1
    $\begingroup$ Your question seems to deal with both linear regression/ANOVA and logistic regression. There is no F test in logistic regression, so please clarify what kind of model you are asking about. There are also plenty of other Q&A's on this site dealing with this question, e.g. here, here, here. $\endgroup$ – AlexK May 28 at 7:08
  • $\begingroup$ "There is no F test in logistic regression, so please clarify what kind of model you are asking about." Ah, okay. I wasn't aware of this since summary(glmerModel) gives me some F-values. Thank you for these links. $\endgroup$ – Paul Jun 1 at 21:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.