I have two sets of data from two groups of participants that are identical in all but one way: one set of data were collected using software A and the other set was collected with software B. It's possible that the outcome variables are roughly equivalent across groups, and it's possible that they're not. (The groups are quite similar otherwise; I've run chi-square tests on gender and race and there are no differences by the latter and only minor differences by the former. Regardless, such differences should not be systematic.)
What I want to do is run some preliminary tests to see whether there are significant effects of software engine on my three primary outcome variables. If there are, I'd then treat these as two separate experiments. If there are not, I'd instead group them together.
What I had in mind was to use a straightforward linear regression, as so (I'm using R, but obviously this is not specific to the stats program):
combined <- rbind(data1, data2) reg1 <- lm(outcome1 ~ engine, data=combined) summary(reg1) # and repeat for outcome2 and outcome3
If this regression shows a significant effect of engine for any of these outcome variables, then that would mean that I should keep the groups separate. It also occurred to me to do this with other covariates in the model:
combined <- rbind(data1, data2) reg1 <- lm(outcome1 ~ engine + age + gender + race, data=combined) summary(reg1) # and repeat for outcome2 and outcome3
In this case, I'd be seeing if engine had a significant effect in the presence of other relevant variables.
Any thoughts on which method makes more sense, or if there's a better way?