Testing Independent variables impact during different decades

Question

I am trying to test the impact that independent variables, Growth, Inflation, and Z (good economy for several quarters), have on the presidential vote received by the incumbent party.

My alternative hypothesis is that over the decades, the amount that these impacted the elections has not been constant. The reason I am testing this is I want to see if I should restrict my data set to the modern era (1980 onwards or so) if the variables have a different impact contemporarily than they did in the early 20th century. If I use such a restricted data set, obviously the beta coefficients will be different and will impact my results. What is the best way to test this?

My ideas: I wanted to do an ANOVA against my model (in R: votePercent~G*P*Z) for every 6 elections and see if certain variables caused more variance in earlier elections.

Alternatively, I could do the R2 of this model against the Dependent var (popular vote) over groups of elections and see how the slope changes.

Finally, I hope it is correct to perform this ANOVA/LinReg against the entire linear model with interaction terms. Any guidance would be appreciated. Please tell me if my reasoning makes sense or if you would approach this differently.

Side Note: In R, anova only returned data on my independent vars and only 1 interaction term of 4 when I used 6 election years as compared to 33 election years, which returned data on all terms in my linear model. Does the number of data points impact the ability of ANOVA to work properly. Does this make anova meaningless when I try to use it in small data sets?

Answer 1

You need to understand that your sample size limits the complexity of the model that it's reasonable to fit. With three independent variables, three interactions between them, an intercept, & a variance, you have eight parameters. If you try to fit a model like this to seven data points, you are not able to estimate the error variance. With a few more, your parameter estimates will still be very uncertain, & wrong assumptions will not easily be caught.

No-one can tell you how to build your model; it depends on your expertise in the subject matter. A rough idea of a useful approach would be to include 'Year' as a independent variable, most likely in a general linear model, perhaps as a linear trend term. You would need to consider very carefully which interactions of the four (now) independent variables to include, taking into account the small sample size, and any correlation between them (e.g. do 'Growth' & 'Z' tell you very different things?).

It's good to try out different models, plot graphs to see what's going on, transform variables in different ways, & so on; but remember that the more you do this the more your analysis becomes exploratory rather than confirmatory.