Is pooling countries together or running a regression model for each country alone is more suitable for comparison?

Question

My research questions are:

1. Does the gap in earnings has changed between individuals with different educational levels between 2007 and 2010 (before and after the 2008 financial crisis).
2. Does that change, if founded, is stronger among countries that were severely hit by the economic crisis compared with countries less touched?

I am analysing two cross-sectional data, one in 2007 and another in 2010. I am aware of the limitations of comparing two cross-sectional data across time. My question is regarding executing the model, as two different strategies give me different results.

1. The first method i ran a regression for each country alone such as:

   lm(earnings ~ education*year + age + gender, data=df)

year is a binary variable (2007 and 2010). The interaction education*year shows whether the gap in earnings increased or decreased between individuals with a different educational background in each country separately.

2)The second method i pool all countries together and introduce a dummy variable for countries. Then, i interpret a three-way interaction between education, countries and year.

lm(earnings ~ education*country*year + age + gender, data=df)

The two different analytical strategies yield somewhat different results. The results of the 2nd method also change depending on which country i specify as a reference category. My question is regarding which method seems more appropriate to my research questions after taking into consideration all the limitation of the cross-sectional data. Multilevel design is not possible as i have only six countries.

Answer 1

The two models yield different results because they are, well, different models.

Clearly you are interested in the "effects" of education and year and naturally you include an interaction between them, which is fine. age and gender are presumably a potential confounder, hence this is also correclty included.

So the question really comes down to how to treat country. First, you have repeated measures within country - earnings in one country are more likely to be similar to earnings in the same country, than other countries and this needs to be accounted for. Pooling will not do this. country might also be considered a confounder, since it seems likely that country will influence both earnings, and education. So your 2nd model is more approriate. You are correct that the output will change depending on the reference level of country, but this is just a simple re-parameterisation. To understand this, you need to realise that the intercept include the reference level, so when you change the reference level, a few things will change, but the overall model is the same.

Since you have only 6 countries, you are rightly concerned about fitting a mixed effects / multilevel model - however there is some consensus in some domains that 6 is (just) OK. So I would also try to fit the model:

earnings ~ education*year + age + gender + (1 | country)

and compare the inferences it with the your 2nd model.