0
$\begingroup$

I am trying to get my head around interaction terms. Suppose if I fit a model and try to predict pass or fail for state exams. The independent terms are: rank of school, results in midterm and over GPA score. Would I be right to say I believe that there could be a interaction term between rank of school and GPA score? If what i am saying is true, how would I test that the new model with the interaction term was better than the old model?

$\endgroup$
3
$\begingroup$

Analysis of deviance

Since the models you're interested in comparing are nested - your interaction model is a special case of your non-interacted model - you could also do an 'analysis of deviance' (like an analysis of variance, but suitable for generalized linear models such as logistic regression). That would test whether it was worth putting the interaction term in.

Out of sample performance

If one of the two models you are comparing is not a special case of the other then you'll certainly need to look at model comparison statistics like AIC or BIC, or possibly to something like cross-validation. These statistics (AIC and cross-validation at least) are trying to give you an idea of what you could expect from the model on new data. If this is what counts as a 'good' in a model, then these are your statistics.

The cost of mistakes

Another, very general way to compare the two logistic regression models (nested or not) would be to compare the ROC curves for them. That would be a measure of model fit under all possible assumptions about the relative cost of mistaking a passing student for a failing student versus the opposite. Typically some models perform well when the costs skew one way and other when it skews the other. A model whose ROC curve always dominates another is better under all loss structures.

There are, inevitably, some caveats to all these, but it's a start.

$\endgroup$
3
$\begingroup$

To elaborate on @Michael Chernick's apt answer: You might first fit a model with predictors A, B, and AxB. If the interaction term adds value, more likely than not that term will appear significant in the model.

Next, you can compare the 'full' (A, B, AxB) model versus a 'reduced' model including only A and B. The model selection criteria @Michael Chernick recommends (AIC or alternatively, BIC) allows you to compare the fit of both models while correcting for the fact that models with more terms usually always fit better than models with fewer terms.

$\endgroup$
0
$\begingroup$

To choose the better model you would use a variable selection criterion such as AIC. The interaction term is plausible since the better (higher ranking) schools should make it more likely that a student will pass the state exam.

$\endgroup$
  • 1
    $\begingroup$ For an R example of using AIC in non-hierarchical models, have a look here. The next question you will probably have is how to draw inference from the interaction terms. For that you can see here and here $\endgroup$ – Simon Nov 29 '16 at 1:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.