4
$\begingroup$

I'm working on a linear regression problem where I have variables that improve the response, but have very low variance. In these variables almost all of the terms are zero, but the non-zero terms are positively correlated with the response. However, when I regress it I get a negative coefficient.

Most of the information that I saw said that I should discard these variables, but I know they have valuable information. What can I do?

I wrote a R code with a reproducible example of what I am experiencing:

y = 1:100
x = rep(0,100)
x[c(10, 11, 15, 21, 15, 40, 45, 49, 55, 70)] = c(5, 15, 24, 21, 17, 44, 48, 49, 85, 70)
plot(x, y)

enter image description here

lm(y ~ x)
Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
   50.70483     -0.05786  

And if I only get the non-zero observations:

y2 = y[x!=0]
x2 = x[x!=0]

plot(x2, y2)

enter image description here

lm(y2 ~ x2)
Call:
lm(formula = y2 ~ x2)

Coefficients:
(Intercept)           x2  
  5.5357       0.7519 

I apologize if this post is somehow against the rules. It is my first time posting here.

$\endgroup$
3
  • 5
    $\begingroup$ Your issue here is that using the lm function treats your data as if it follows a normal distribution. Based on those graphs it clearly does not, so you will have to use a more sophisticated modelling technique. I would recommend investigating zero-inflated models, which are well supported in R. $\endgroup$ Mar 29, 2017 at 21:13
  • 1
    $\begingroup$ linear regression just assumption for the residual distribution , so you can check it $\endgroup$
    – wolfe
    Mar 30, 2017 at 0:09
  • $\begingroup$ It makes sense, thanks! Any ideas on how to transform this kind of variable so it can result in uniformly distributed residuals? $\endgroup$
    – jcp
    Mar 31, 2017 at 14:30

1 Answer 1

1
$\begingroup$

1) To first address your comment: no transformation will fix this problem. You have a large number of X values that are identical - so whatever transformation you apply, they will remain identical.

2) @ConorNeilson proposed the correct solution, which is to switch to using zero-inflated models (a subset of mixture models in which one portion models the excess zeroes) for your analysis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.