1
$\begingroup$

I have a question regarding interactions in GLM. I run a Poisson regression with the purpose of predicting claims in insurance.

I have the following problem :

model 1 : CLAIMS ~ X1 + X2
model 2 : CLAIMS ~ X1 + X2 + X1:X2

X1 has 10 levels, X2 has 4 levels

When I do a deviance test (Chi Square), my result is that model 2 is significantly better compared to model 1 (at the 5% level).

On the other side the AIC of model 2 is worse. (higher)

When I look at the individual parameter estimates and their corresponding standard error, I see that only 3 out of 27 levels from X1:X2 differ significantly from the base level.

I am not sure how to interpret these observations. Can someone help me and say something meaningful about it? And more important, how to deal with it?

My attempt would be: "There are interactions in the model that are meaningful, however most of them are not. This explains the increasing AIC, although the deviance test says that model 2 is the better one." Should I try to construct a few dummy variables for these three interactions that seem significant? (However, I am not sure this is a correct approach, because it is only significant relative to the base level...)

$\endgroup$
2
$\begingroup$

You seem to have "meaningful" confused with "significant". A significant variable can be unmeaningful and a nonsignificant variable can be meaningful.

The decreasing deviance is because the new terms improve fit. The increasing AIC is because the interactions add a lot of complexity, so much that it is not "worth it" to include the terms.

However, prior to all this is whether the interaction is, in fact, meaningful and important. This is not really a statistical question - or not purely a statistical question. It is also substantive. For your purposes, this means "does it improve my prediction?" and the best way to answer this is probably by dividing the data into training, test and validation sets and then seeing what happens.

$\endgroup$
2
  • $\begingroup$ I plan to do the following then. Train three models : model 1 from my example, model 2 from my example and model 3 (which is model 1 + three dummy variables corresponding with the three interactions that are significant). And the I will see which one performes best... Does this seem reasonable to you? $\endgroup$ – Roger Jan 8 '14 at 2:16
  • $\begingroup$ The usual practice is to use the training set to develop models then test them on the test data set. Including interactions only at a few levels is probably not right. $\endgroup$ – Peter Flom Jan 8 '14 at 10:21
1
$\begingroup$

One question is whether the 4 and 10 leveled variables $X_1$ and $X_2$ are ordinal or categorical. If they are categorical, then observing 3 out of the 27 to be positive findings strongly indicates that you should be correcting for multiple testing since such a number is consistent with a 0.05 Type I error rate under the null hypothesis.

The word "meaningful" is misleading as far as interpretation is concerned. It's important to recall that the parameters in Poisson GLMs are interpreted as relative rates comparing groups differing by 1 unit in the predictor. What, then, would a significant interaction actually mean about the relative rates for claims comparing units differing in $X_1$ and/or $X_2$?

Tests of dispersion are possibly worth considering here, however you seem to have a vague understanding of what it achieves. It is not necessarily a diagnostic test, nor are diagnostic tests requisite for confidently reporting model estimates. The dispersion parameter is a proportionality parameter that is estimated in quasilikelihood models which handle small, unobserved sources of covariation between observations. For an illustrative example, consult Alan Agresti's Categorical Data Analysis and the section on horseshoe crabs.

Alternately, AIC is a measure of predictive ability. It is used to compare two possibly non-nested models, however in this case you've chosen to compare nested models. If indeed the parameters are categorical, it explains why the addition of 27 parameters in this model dramatically increased the AIC and led to the conclusion that model 2 did not out perform model 1. However, testing for interaction is not best achieved with comparing AICs and you should consider a better approach.

$\endgroup$
1
  • $\begingroup$ It is categorical. Thanx for your response. $\endgroup$ – Roger Jan 8 '14 at 2:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.