Testing whether we should build a model on the union of data sets Suppose I have 4 massive data sets representing specific products all belonging to the same "product class". A priori we expect that these will all behave similarly and that a grouping will make sense. Our task is to predict a binary response associated to each observation in a product. The current implementation is to fit a logistic regression to each product separately. For various reasons we would prefer to merge the data and fit one model to it, unless the data suggests strongly that this is not appropriate.
Is there a well-defined procedure or statistical test to judge whether fitting one model to the combined data is a good or bad idea?
I can see that this question is tricky in general, and will depend on subject knowledge. However, if we assume that all subject knowledge indicates that these datasets should behave in highly similar ways, a priori, how can we test this?
 A: A way to do this is:


*

*Create your stacked data set. That is, which combines the data for all of the observations, which each data set on top of each other. When creating this, make sure that you include a variable that indicates which data file each case came from. 

*Compute the pooled data model (i.e., estimating a single effect for each of your predictor variables). 

*Compute a second model, where you include interactions between each of your predictors and the data file memberhip variable. This variable needs to be treated as categorical (e.g., in R, you need it to be a factor, in SPSS, you need to be selecting it as categorical when specifying the model).

*Use an F-Test or the AIC to compare the two models. If using R, aov is the easiest way to do this. If the pooled model is not rejected, you can use it.

*If the pooled model is rejected, I would then estimate the models in each data set and compare the coefficients and work out if the variation between them is problematic in your domain, as with big data sets, it is easy to reject the pooled model, even though the differences between the coefficients are trivial. If this is your situation, you can check your judgment by computing a pseudo-R-square of some kind (e.g., McFadden's Rho-square), and checking that the value from the pooled model is not too different from the model with the interactions.

