1
$\begingroup$

Doing my thesis I came up on a problem where I can't find the answer on. I have a dataset with only categorical predictors with sometimes many levels and a numeric outcome variable bounded between zero and 1. The outcome variable is a given discount percent variable. It is not normally distributed with more then two modes:

enter image description here

In this case I used 5 predictor variables and Nrows around 850.000 I tried several different algorithms ( betareg, lm, anova, random forest, svr ) But every time this residuals vs fitted plot comes up:

enter image description here

Also when I use a smaller training you will see this downwards trend.

My question is how to interpet this plot and is there a way to deal with this since all my predictor variables are categoric?

I try to find out witch variables of a 200 variables wide dataset have a impact on the given discounts.

Thanks,

Improve this question
$\endgroup$
2
  • $\begingroup$ You have non-constant variance in residuals, model behaves more badly where your model predicts bigger discount. $\endgroup$
    – Analyst
    Commented Jun 25, 2018 at 19:38
  • 1
    $\begingroup$ If you use beta regression, then don't worry about residuals being non-normally distributed. That's an assumption of standard linear regression with a Gaussian (identity) link function. $\endgroup$
    – Mark White
    Commented Aug 6, 2018 at 18:07

1 Answer 1

Reset to default
1
$\begingroup$

If your response is bounded between 0 and 1, you don't conform to the assumptions of a linear regression, and it is thus not surprising that you don't get normal residuals.

You can model 0/1 proportional data in a number of ways. A common approach is a beta regression.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.