I have been recently trying to fit a linear model to my data. The dependent variable is continuous and the independent variable is numeric and discrete. When I first test the assumptions concerning the data structure in order to run a linear regression I observe violations of the assumptions of linearity and normality. The relationship between my independent variable and my dependent variable is nonlinear, and residuals are not normally distributed:

Here a plot of the relationship between y and x (plus a line of predicted values using a linear regression): enter image description here

Here some plots to test linearity and normality assumptions: enter image description here

After that, I tried unsuccessfully to linearize the relationship between the variables by transforming the data (by doing log(y); log(x); sqrt(y). etc...), but I did not find any transformation that satisfactorily increased the linearity.

Thus, I decided not to run a prototypical linear regression but a polynomial regression, using a quadratic polynomial. The model that I have run is as follows:

model <- lm(y ~ x + I(x^2))

By running the polynomial regression I obtain the next residuals and coefficients:

     Min       1Q   Median       3Q      Max 
-2.30465 -0.65161  0.01504  0.60506  1.95826 

              Estimate Std. Error t value Pr(>|t|)    
(Intercept)  2.3617770  0.0036701  643.52   <2e-16 ***
x           -0.0508341  0.0014057  -36.16   <2e-16 ***
I(x^2)      -0.0062881  0.0001141  -55.12   <2e-16 ***
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.8663 on 739197 degrees of freedom
Multiple R-squared:  0.1781,  Adjusted R-squared:  0.1781 
F-statistic: 8.007e+04 on 2 and 739197 DF,  p-value: < 2.2e-16

Here a plot of the polynomial fitting the data: enter image description here

Some questions:

1) By running a linear regression (y~x) I get R2=0.1747. When running the quadratic regression I get R2=0.1781. Giving this R2 and giving that there is a violation of the linearity assumption: should I keep the quadratic regression as a better fit of my data?

2) In a prototypical linear regression (y = b0 + b1*x) the way to read the output of x (our predictor effect or effect size) is quite straightforward: "for every increase of x by 1, the effect of x on y is b1". However, for the quadratic regression, the interpretation of the effect of x on y seems to be a bit more complicated. How could I report (read, explain) the effect of my independent variable (in the current case) in an elegant way?

I really appreciate suggestions.

  • 3
    $\begingroup$ As a comment that in no way is intended to answer your question - with ~739,400 observations and only 11 values of the independent variable, I personally wouldn't run any regression at all, I'd just use the mean of $y$ given each of the 11 values of $x$. The standard errors of the $\bar{y}_i, i = 1, \dots, 11$ are going to be very small even with a quite uneven distribution of your samples across $i=1,\dots,11$, and I don't see that trying to force a parametric function on the data is adding anything of value. $\endgroup$ – jbowman Feb 5 '18 at 20:23
  • 1
    $\begingroup$ To add to @jbowman's comment, you have so much data and the values of the independent variable are so limited that it is likely you could accurately estimate the entire conditional distribution for every one of the 11 independent values. Moreover, it looks like the response actually is discrete: only a limited number of distinct values appears. $\endgroup$ – whuber Feb 5 '18 at 20:49
  • $\begingroup$ Thank you for your comments. I hope anybody else try to answer. In the meantime, I've been trying to plot the conditional distribution by using cdplot. First time that I use it, and it looks good and it seems to facilitate the analysis of my data. I wonder if both quadratic regression and conditional density analysis can be run when having a discrete response (as shown in my dependent variable). Any limitation or "red line" on this? $\endgroup$ – pyring Feb 6 '18 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.