I realize this might be a too general question, so I'll describe what I'm doing right now first.
I'm working for a virtual insurance company and I have this dataset. It has severity (meaning payment/number of claims), type of car, gender, marital status, age of car, risk type (high and low), company name (ours and others), deductible range. All the variables are categorical. The question I'm trying to answer is: Is our company paying out more on claims than other companies do?
First I suppose I should choose main effects. The way I do it is that, if I can come up with a story or theory, then I include it. For example, I choose to include marital status because probably married people drive more cautiously since they have families. I include company because...well, this is what I need to study.
Now here is my question:
After reading many posts, it occurred to me that merely looking at the test statistics is not a good idea. Sometimes even if the p-value is greater than 0.05, we should still keep the variable. My question is, is there a rule-of-thumb or something? For example, if the p-value o test statistics of marital_status is 0.8, then I guess it's obvious that we should not include it. But what about 0.08? 0.07?
After selecting main effects, I might should start considering interactions between these variables. My question is, how do I even start? My variable type_of_car has around 20 different values (20 different types of car). I thought about running a regression with all possible interactions, and then drop those with insignificant p-values. That didn't really work out because everytime I drop one, the p-value of the variables in the new model would alter drastically.
I apologize for such a long post. I don't have a solid background in statistics (I'm a math major) and I really have to teach myself all the stuff. Any suggestion on how to proceed would be appreciated!