In short, my question is, if you have a population which you divide into two segments (with different target rates) and build logistic regression models on each, will the predictions from both models be comparable?
Here's an example in R:
library(ggplot2) library(data.table) data("diamonds") diamonds <- data.table(diamonds) # Create target variable diamonds[,target:=ifelse(price>10000,1,0)] diamonds[,mean(target)] # 0.09681127 diamonds_defg <- diamonds[color %in% c("D","E","F","G")] diamonds_defg[,mean(target)] # 0.07720687 diamonds_hij <- diamonds[color %in% c("H","I","J")] diamonds_hij[,mean(target)] # 0.1411637 glm_defg <- glm(data=diamonds_defg,factor(target) ~ depth + table,family="binomial") glm_hij <- glm(data=diamonds_hij,factor(target) ~ depth + table,family="binomial") mean(predict(glm_defg,diamonds_defg,type="response")) # 0.07720687 mean(predict(glm_hij,diamonds_hij,type="response")) # 0.1411637 mean(c(predict(glm_defg,diamonds_defg,type="response"),predict(glm_hij,diamonds_hij,type="response"))) # 0.09681127
The average of the propensities for each segment equals that segment's target rate and when you add the two lists of propensities together, the average of that equals the total population target rate. However, I'm not 100% convinced by the above that you can safely combine the two model outputs and treat as one without any modifications. e.g. does a 0.2 from one model equal an 0.2 from the other.
Any good arguments either way out there?