6
$\begingroup$

Is it possible to compare probabilities of 2 logistic different models? For example if I have one model that returns the probability that someone answer a phone call on Mondays, and then I have another model for Tuesday, and another for Wednesday and so on... Then for the same input I run the first model and I get that the probability for that person of being contacted is .8 while for the model of Tuesdays is .6 and the for the rest of the days is also less than 0.8. Would it be ok to compare those, and say that is more probable to contact this person on Tuesdays or those probabilities are not comparable?

I think that they are not because those models might have for example different error rates. If this is the case, how would you do a model that gives you do the best time to contact someone? I would really appreciate some light in this subject. thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

Indeed, you cannot reliably compare across logit models with different underlying data. Without repeating what has been written before, this post has a very good answer (or see this paper).

In your case, combine the data from different days, and model this:

$answer=\alpha+\beta_1Tues+\beta_2Wed+\beta_3Thurs+\beta_4Fri+\beta_5Sat+\beta_6Sun$

You can do simple Wald tests or likelihood ratio tests to compare whether the coefficients for each day are statistically different. You may find, for example, that there is no statistical difference between Sat and Sun, in which case you could update your model:

$answer=\alpha+\beta_1Tues+\beta_2Wed+\beta_3Thurs+\beta_4Fri+\beta_5Weekend$

You can also estimate the marginal effects of each day, as odds ratios can be confusing or misleading depending on what you are really interested in.

If you have time of day, that can be a multiplying effect, which may moderate the day, though interpreting interaction terms in logit models can be confusing.

In addition, other variables may mediate the effect of the specific day - employment status, marital and parental status, etc. If you have these you may want to include them as controls.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hi. thanks. But my result should say "this person is more likely to answer on Saturdays while this other should be called on Mondays". With only one model, I can only say the probability of being contacted any day... or with the coefficients I can say that for example trying Mondays increases the probability of contact more than Saturdays.... but both are for every person. I need to get different coefficients for different types of persons. $\endgroup$
    – GabyLP
    Commented Jan 17, 2015 at 19:33
  • 1
    $\begingroup$ The coefficients for a model with all data pooled will tell you which days for your population are better to be contacted. As I pointed out, if you have specifics on subgroups you can identify whether the overall population model holds for subgroups, by using interaction effects or something like seemingly-unrelated regression, among other methods. The issue with a logit model (if you read the links) is that you can't safely compare coefficients across different populations. So you have no choice but to pool the data. At the minimum include a dummy for which subgroup the data come from. $\endgroup$
    – robin.datadrivers
    Commented Jan 17, 2015 at 19:52
  • $\begingroup$ Maybe a dummy for each subgroup combine with the dummy for each day... So you can see how much increases the chances of being contacted the group-day combination.... if that's the idea I would end up with a lot of variables. If I have 6 days, and 10 variables to describe the groups, I will end up with 60+10+6 variables.... am I right? $\endgroup$
    – GabyLP
    Commented Jan 17, 2015 at 21:02
  • 1
    $\begingroup$ There are different approaches you can take. How many observations do you have? Here are some options: - Machine-learning/exploratory perspective: try all combinations (you will have inflated p-values, but that may not be an issue if all you are doing is exploring to have a model you can test with more data later). - A more confirmatory approach, where you start with the variables that make the most sense conceptually. - Conduct cluster analysis on the 10 variables to define groups, and model them. If you need more guidance, you should put a new posting up with more details than you have here. $\endgroup$
    – robin.datadrivers
    Commented Jan 17, 2015 at 21:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.