Compare two regressions

Question

I try to determine India’s fertility decline between 1991 and 2001 in a multivariate regression (OLS). I have used “total fertility rate” as dependent variable and estimate the effects from six explanatory variables (see example below).

I have two questions:

First, How should I compare the result between 1991 and 2001? Which is the best method? Shall I investigate “interaction effects”? Or maybe “fixed-effects” and “random-effects” (seems to be a little bit complicated)? Or is it enough to simply interpret the result for each year in each column and then compare differences and similarities?

Second, I would like to estimate regional effects. How shall I do this? Shall use dummy variables (like South dummy, North dummy, East dummy, West dummy). Or is a better method to use separate analysis? Which means four separate multivariate regressions for each year. Or shall I combine both methods?

Example of Multivariate regression Dependent variable: Total Fertilty Rate (standard error in parenthesis)

                          INDIA 1991          INDIA 2001

Poverty                      0,014***          0,015***
                            (0,003)           (0,002)

Female literacy             -0,015***          0,0021***
                            (0,002)           (0,007)

Answer 1

As others have suggested, the answer to your question depends on clarifying exactly what your research questions are. I am going to answer assuming that you only have two time points to deal with and will try to answer the question with regard to the two primary questions I see you asking.

1. If you are asking: "Overall, do the effects of poverty, female literacy, etc. differ between 1991 and 2001?" then your question is an interaction one. Interactions assess the degree to which a particular slope depends on some second predictor. For example, if you think that the relationship between female literacy and overall fertility rate has decreased between 1991 and 2001 you would want to model an interaction of female literacy and time. It may serve you well to only specify the interaction terms you hypothesize for a few reasons. First, your model will become increasingly complicated as you add more interaction terms. Second, the addition of predictors that do not add explanatory power to your model will serve only to eat up degrees of freedom and reduce power to detect any significant effects. Third, the more effects you estimate without correcting for multiple tests, the more you inflate the probability of detecting a false positive result.

2. If you are asking whether the overall fertility rate has changed from 1991 to 2001 there are a few ways you could address the question, but the simplest would be to regress your outcome variable on a time variable. Dummy coding time would work just fine and your resulting slope would tell you the difference in fertility rate between 1991 and 2001. The intercept in this case would tell the mean fertility rate for whatever year is coded 0 (which may be an informative test depending on your exact research questions). An alternative coding scheme, which can be helpful in the context of models that contain interactions, is contrast coding. A single contrast coded predictor should center around 0 and allows you to ask different questions from the intercept. For example, if you coded 1991 as -.5 and 2001 as .5 the resulting slope would give you the difference in fertility rates between 1991 and 2001 (just as the dummy codes would) and the intercept would be the mean of the fertility means for the two years (which offers at the very least a close approximation of the grand mean fertility rate). If this is your question, you would run separate models examining overall differences in fertility rate from 1991 to 2001 and for testing whether any of your predictors across both years are related to fertility rate.

Modeling regional differences: I would argue that you should include dummy-coded predictors to estimate regional differences in your outcome. Estimating separate models for each outcome will be much simpler and may be helpful for post hoc interpretation of interactions, but will not test whether regional differences are significant. Again, if you questions are whether the relationships between predictors and the outcome are different from region to region, you will need to estimate interactions of region * predictor. If your questions were just about the overall effect of region on the outcome, you would want a model with just the region codes. If your question is about whether there is a "unique" effect of region, you may want to add these dummy codes to the model that includes the other predictors of interest (poverty, female literacy, etc.).

A final note about violations of non-independence. Because I am assuming you are measuring the same places at two different times, a model that treats time as a between location factor is likely violating a key assumption of regression, which is that errors from one's model should not be dependent on any of the model predictors. Since it seems reasonable to assume that observations of the same place would be more similar than observations of different places you should treat time as a within location variable. The treatment of non-independence violations would, however; be a whole other post. However, any intermediate-level statistics text book should have chapters on repeated measures designs. Hope this helps!

Answer 2

If I understood your question correctly, the dependent variable is fertility rate and the independent variables female literacy, poverty, etc. If time is not one of the independent variables, I think it is probably better to keep the analysis simple for now and consider only how the different independent variables impact on the dependent one. At a second stage you can pose a different question (e.g. is the dependent variable of the highest-impact independent variable significantly different between the first and second halves of the data?) and devise a clear test to compare the two (like a p-value-based test).

To estimate regional effects, I would use dummy variables, as they allow an homogeneous treatment of the data and to clearly see the effect of turning them on/off. If you do multiple regressions there is no way to ensure homegeneity, you might be affected by small numbers, and the interpretation is far more complicated. From what I understood, you are using categorical moderation variables, and I allow myself to provide a reference on a textbook I really enjoyed.