1
$\begingroup$

I am doing a research on the variables that have an impact on the discount given on 'products'.

The dataset is is large with possible 800.000+ rows in the training set. Also the data consist of many categorical variables with sometimes more then 30+ levels.

The main problem is that the outcome variable is not normal distributed in the way that it has 3 modes. Also the outcome variable is a percentage a value from 0 to 1. The strange thing is that the simple linear model stated below where not all assumptions are met has a better R2 then for example a random forest model.

The question here is: - What type of regression to use for that as this, with maintaining the possibility to check feature importance?

Distribution outcome variable

If a take a subset of my dataset with 5 independent categorical variables and 1 continuous numeric dependent variable and run a multiple linear regression these plots come out of it: I tried to find residuals distributions like this one but could not find it.

The distribution of the residuals do not look normal. And it goes of the line of the normal q-q plot. The question her how to interpretative those two plots in this case?

enter image description here

Thanks in advance

Improve this question
$\endgroup$
2
  • $\begingroup$ Since you are considering a random forest, are your goals a predictive model? $\endgroup$
    – Matthew Drury
    Commented Jun 21, 2018 at 15:32
  • $\begingroup$ There are two goals: 1) To get an answer on which variables contribute most on the given discounts ( variance explained?) 2) Predictive modeling to predict future discounts by these variables. the whole data set has +- 100 different variables(99% categorical features ) $\endgroup$
    – Amsterdam-boats
    Commented Jun 22, 2018 at 7:49

0

Reset to default

Browse other questions tagged or ask your own question.