3
$\begingroup$

In my regressions I analyse subgroups of people, i.e. thresholds for certain continuous variables. I am interested in changes at these high values. I was told that it is beneficial to not drop observations but to code this threshold via a dummy and include it via interaction at my effect of interest. That makes sense to me. When the dummy is 0, the interaction is not evaluated. It sounds good to not drop observations.

Here is an arbitrary example, say I am interested in the effect of emp on gsp at high values of emp:

data("Produc", package = "plm")

# Linear Regression    
lm <- lm(log(gsp) ~ log(pcap)*emp + pc + unemp, data = Produc)

# Linear Regression with threshold step dummy
Produc2 <- Produc %>% mutate(emp_dummy = ifelse(emp > 1800, 1, 0))
lm_dummy <- lm(log(gsp) ~ log(pcap)*emp*emp_dummy + pc + unemp, data = Produc2)

# Linear Regression with reduced sample
Produc3 <- Produc %>% filter(emp > 1800)
lm_subgroup <- lm(log(gsp) ~ log(pcap)*emp + pc + unemp, data = Produc3) 

What exactly is the reason why I better should not drop the observations?

$\endgroup$
1
  • $\begingroup$ So in this case the research question is about the association of emp with gsp (I realise this is a made up example) $\endgroup$ Aug 11, 2020 at 15:42

1 Answer 1

2
$\begingroup$

When you use your dummy coding, you are still using the observations below the threshold to estimate the other coefficients like pc and unemp. Therefore it makes use of all data, leading to more precise estimates of your coefficients.

$\endgroup$
2
  • $\begingroup$ I think the prevailing logic is not to drop points because instead you should try to understand why they occur. You may for example be violating a distributional assumption, some unusual event may drive the outlier etc. If you don't explore why the outlier occurs you will miss this. I had a data distribution that was exponential rather than normal. Had I just deleted all the outliers I would not have realized this. $\endgroup$
    – user54285
    Aug 11, 2020 at 21:29
  • $\begingroup$ This sounds plausible to me. Can you give a source for a minimal example or a (not too complicated) mathematical illustration? $\endgroup$
    – Marco
    Aug 12, 2020 at 7:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.