6
$\begingroup$

I'm currently looking at rates for a study that vary between 0 and 100 with most of the rates falling between 0 and 1. I am running a linear regression on 70 dummy variables (coded 0-1) and nearly 100,000 lines of observations. When I run the regression, the coefficients I am getting for each of the dummy variables and intercept is in the region of 10E10 to 10E13. Testing the predicted values of this regression does come out to numbers around the actual rate (somewhere between 0 and 1 for the most part) but I feel like something is wrong with this analysis.

Is there something I might be missing as to why my coefficients for each variable are coming out so high? I'm new to actually implementing regression and don't know if anything is wrong or this is just the result I'm looking for. I'd really appreciate any help with this

Improve this question
$\endgroup$
0

2 Answers 2

Reset to default
6
$\begingroup$

Try to see what happens if you drop those observations that are close to 100 (or anyway have a scale far above the others). This way you will have a better understanding of the situation. If you say that most of the dependent var values are between 0 and 1 it may happen that some high values (extreme values) are skewing the coefficients. Those coefficients seem too high (although it may be theoretically possible if positive coefficients are offset by negative coefficients of the same scale).. check also if some of those 70 variables are too strongly correlated, in which case there could be excessive multicollinearity (maybe you could have extremely high values for the correlations making the estimate unstable)

Improve this answer
$\endgroup$
6
  • $\begingroup$ Upvoted for the excellence of the advice. $\endgroup$
    – James Phillips
    Aug 9, 2019 at 22:23
  • $\begingroup$ @JamesPhillips thanks a lot for the nice words, really appreciated! $\endgroup$
    – Fr1
    Aug 11, 2019 at 17:19
  • $\begingroup$ Thank you for the advice. I tried removing large the large values (even as much as those rates only >2) and found that this helped but my coefficients are now roughly in the range of 10E8 to 10E12. I think there is multicollinearity present but am not exactly sure how to deal with this. Would I be able to use something like VIF on categorical data in this case? $\endgroup$
    – Michael Paolucci
    Aug 12, 2019 at 14:27
  • $\begingroup$ @MichaelPaolucci measure the multicollinearity with a correlation matrix, identify those features that are too highly correlated and drop them, then re-estimate the model and see what happens. The rationale is that if you have two variables that show a very high correlation then they represent repeated info in the dataset, so you can just use one of the two $\endgroup$
    – Fr1
    Aug 12, 2019 at 14:35
  • $\begingroup$ @Fr1 when I tried out the correlations between my dummy variables, none of the correlation matrix had a value higher than 0.26. Do you think that maybe using Ridge/ Lasso regression would be a good way to handle the data then if none of the variables seem to be correlated? $\endgroup$
    – Michael Paolucci
    Aug 12, 2019 at 15:13
1
$\begingroup$

You are right that something is wrong with this analysis. If a given predictor has 99,999 "0"s and a single "1," it is likely to produce an astronomical coefficient and standard error. Such distributions might be called "extremely narrow" or be said to feature "rare events." It sounds as if you have many predictors like this; they will be of no use in your regression. See also Large standard error for one predictor.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.