1
$\begingroup$

In short, my question is, if you have a population which you divide into two segments (with different target rates) and build logistic regression models on each, will the predictions from both models be comparable?

Here's an example in R:

library(ggplot2)
library(data.table)

data("diamonds")
diamonds <- data.table(diamonds)

# Create target variable
diamonds[,target:=ifelse(price>10000,1,0)]
diamonds[,mean(target)]
# 0.09681127

diamonds_defg <- diamonds[color %in% c("D","E","F","G")]
diamonds_defg[,mean(target)]
# 0.07720687

diamonds_hij <- diamonds[color %in% c("H","I","J")]
diamonds_hij[,mean(target)]
# 0.1411637

glm_defg <- glm(data=diamonds_defg,factor(target) ~ depth + table,family="binomial")
glm_hij <- glm(data=diamonds_hij,factor(target) ~ depth + table,family="binomial")

mean(predict(glm_defg,diamonds_defg,type="response"))
# 0.07720687
mean(predict(glm_hij,diamonds_hij,type="response"))
# 0.1411637

mean(c(predict(glm_defg,diamonds_defg,type="response"),predict(glm_hij,diamonds_hij,type="response")))
# 0.09681127

The average of the propensities for each segment equals that segment's target rate and when you add the two lists of propensities together, the average of that equals the total population target rate. However, I'm not 100% convinced by the above that you can safely combine the two model outputs and treat as one without any modifications. e.g. does a 0.2 from one model equal an 0.2 from the other.

Any good arguments either way out there?

$\endgroup$
1
$\begingroup$

Let's denote color_class as whether the diamond is in D-G or H-J. If you run a logistic regression including your two predictors, color_class, and the interaction between color_class and each of the predictors, you will get the same predicted values as you have generated with the method you proposed. That is, the following will give you the same predicted values:

glm(factor(target) ~ depth + table + color_class + color_class*depth + color_class*table, 
    data = diamonds, family = "binomial")

Clearly this formulation is acceptable; your method is just a roundabout way of doing the same thing. In propensity score analysis, this method can be used in the context of moderation; see Green & Stuart (2014).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.