1
$\begingroup$

I have a data on some overall conversion rates (i.e. out of x users visiting, y buy something hence y/x is my conversion rate, essentially proportions) over a time period, now this overall proportion can be broken by if they came from channel 1, channel 2 or channel 3 and for each channel there would be again similar proportions. My objective is to see how these proportions from different channels impact the overall proportion

I have run a simple linear regression in R and below is the result. 

Call:
lm(formula = target_variable ~ . - date, data = data_lcr)

Residuals:
  Min        1Q    Median        3Q       Max 
-0.034173 -0.003217 -0.000704  0.002331  0.073845 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -0.0049876  0.0006139  -8.124  7.4e-15 ***
exp1         0.0785438  0.0086230   9.109  < 2e-16 ***
exp2         0.0290531  0.0175517   1.655   0.0987 .  
exp3        -0.1026385  0.0080550 -12.742  < 2e-16 ***
exp4         1.0760312  0.0669632  16.069  < 2e-16 ***
exp5         0.2466149  0.0195844  12.592  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.007503 on 358 degrees of freedom
Multiple R-squared:  0.9843,    Adjusted R-squared:  0.9841 
F-statistic:  4503 on 5 and 358 DF,  p-value: < 2.2e-16

The Model has great R-squared which is significant, all variables turn out to be significant. Next I am checking if my residuals are normally distributed

>  skewness(fitlm$residuals)
[1] 2.863341
> kurtosis(fitlm$residuals)
[1] 33.83711

Shapiro-Wilk normality test

data:  fitlm$residuals
W = 0.72781, p-value < 2.2e-16

Anderson-Darling normality test

data:  fitlm$residuals
A = 17.485, p-value < 2.2e-16

These tests suggest that my residuals are not normally distributed. Should I still consider the model based on R-squared and F-Value or make some corrections? Please suggest

Here is the residual plot: enter image description here

enter image description here

EDIT After removing outliers: enter code here

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ What does the plot of the residuals look like? Also, can you explain the data setup? What is the response here? My most natural instinct is a transformation of the response. $\endgroup$ – Greenparker Mar 9 '16 at 9:17
  • $\begingroup$ As @Greenparker said, how does residual plots look like? See also: stats.stackexchange.com/questions/2492/… $\endgroup$ – Tim♦ Mar 9 '16 at 9:20
  • $\begingroup$ @AnuragH, a plot of the residuals means predicted values on the x axes and the residuals on the y axes. If you used lm() in R, then you can get the diagnostic plot when you do plot(lm.object). Also, it looks like you might have at least one outlier that might be affecting your results. The QQ plot looks a little heavy tailed, but nothing that suggests anything too crazy. $\endgroup$ – Greenparker Mar 9 '16 at 10:13
  • $\begingroup$ @Greenparker I am updating the post with the diagnostic plots and yes I found about 13 outliers using the Box Plot, even after removing them I do not see great improvement. I am attaching the plot(lm.object) after removing the outliers, do let me know what can I do next. Thanks $\endgroup$ – Anurag H Mar 9 '16 at 10:32
  • $\begingroup$ @AnuragH I edited your question to rollback the previous version of the question and add the new plots. Your last edit changed your question into a totally different problem. $\endgroup$ – Tim♦ Mar 9 '16 at 11:12
4
$\begingroup$

It is always good to look at plotted data. In case of regression, it is good to look at residuals plots. In your case residuals seem to come from distribution with longer tails than normal. Distribution of your data is closer to $t$-distribution and actually I was able to produce a similar data example for $t_2$ distribution.

enter image description here

It even produces quite close Shapiro-Wilk estimates:

> shapiro.test(x)

    Shapiro-Wilk normality test

data:  x
W = 0.78051, p-value < 0.00000000000000022

But this is of smaller importance. What the residual plot really shows you is that your distribution has longer tails, i.e. it has some outliers. Now you should ask yourself what are the outlying values? Identify and check the values. Why are they outlying? Is it the measured phenomenon having long tail distribution that produced them, or maybe there was some issues with measurement (are they erroneous)? The outlying values can influence your final estimate, so you have to make a number of decisions on what do do with the outlying values. Check also How should outliers be dealt with in linear regression analysis? and Interpreting the residuals vs. fitted values plot for verifying the assumptions of a linear model threads.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks Tim for the assistance. If I remove the outliers and run the model and I still tend to have non-normal residuals, what is the impact I make in accepting the model given it's goodness of fit to be good $\endgroup$ – Anurag H Mar 9 '16 at 10:56
  • $\begingroup$ @AnuragH I see that you edited your question - it would be better if you rather left the previous plots and added the new ones since your last edit changes your question to totally different one. As about new plots - they seem to show a totally different pattern: with some small "cluster" of outlying values and overall linear trend in residuals. This needs further investigation (search this site for residuals regression for multiple similar cases and examples). $\endgroup$ – Tim♦ Mar 9 '16 at 11:09
1
$\begingroup$

When data are not normally distributed the inferential results on p-values (and on F-statistics) do not hold and so it is not correct to look at them.

However the leasts squares fit does not rely on the normality assumption and so if the fit is good and R squared is high there is no reason to discard the model.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ You are partially right, but notice that $R^2$ can be misleading, e.g. stats.stackexchange.com/questions/13314/… $\endgroup$ – Tim♦ Mar 9 '16 at 11:15
  • $\begingroup$ Yes, R squared is highly misleading, but (according to my knowledge) not for reasons related to lack of normality in the data. Correlation between predictors is one of them for example. $\endgroup$ – adaien Mar 9 '16 at 12:14
  • $\begingroup$ I meant your suggestion to rely of $R^2$ when believing that there is "no reason to discard the model". The example in the question (after we learned about the residuals distribution - what you obviously did not know at the moment of answering) shows that there are issues with residuals - outliers, linearity etc. that can influence the results. $\endgroup$ – Tim♦ Mar 9 '16 at 12:19
  • $\begingroup$ His question is what to do when model has a great fit but data are not normally distributed, my answer is that the inferential results on significance do not hold, but this does not contradict the other results. Of course there could other reasons (like outliers) why the fit should not be good, but this was not his question for what I have understood. $\endgroup$ – adaien Mar 9 '16 at 12:40
  • 1
    $\begingroup$ Completely agree on that $\endgroup$ – adaien Mar 9 '16 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.