Linear regression is meant for linear relationships right, so, if I don't trust linear correlation and want to find out if random variables $y$ and $x$ have a non-linear relationship, can't I just regress $y$ against $x$ without intercept like so, $y = \beta x + \epsilon$, and then use a QQ plot of the residuals to see that a divergence from the 45-degree line indicates non-linearity. Otherwise, maybe some other related post-regression test?
$\begingroup$
$\endgroup$
2
-
6$\begingroup$ Linear regression means linear in the parameters. Just plotting the original data is a more direct alternative here. $\endgroup$– Nick CoxSep 11, 2020 at 9:41
-
1$\begingroup$ linear in parameters, not linear in variables, ok $\endgroup$– develaristSep 11, 2020 at 17:22
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
3
No, the QQ plot doesn't tell you about the relationship between y and x's, it tells you about the distribution of the residuals (which should reflect the errors if the model is otherwise correct, if the QQ plot doesn't look fairly close to linear the residuals are not close to what you'd expect if the errors were normally distributed).
Residual plot(s) tell you about non-linearity is the relationship between y and the corresponding x variable(s).
Here's what you could see for an example set of data with mild non-linearity in it.
Here there is a bit of non-linearity (because I put it in the data), but it is not totally obvious in the plot of y vs x.
If the linear model were correct the residuals should appear to be randomly scattered above and below 0 at each x-value.
It is not the case here -- in the plot of residuals vs x you can see the curvature clearly. I marked in a quadratic curve but you'd more typically look at a smooth fit to the residuals for such a purpose.
The QQ plot looks linear here but it is not readily interpretable because of the issue in the residual plot.
(There are better things to plot than raw residuals but let's get the more basic concepts clear to begin with.)
-
$\begingroup$ so how to infer presence of nonlinearity from residual plots instead $\endgroup$ Sep 11, 2020 at 4:28
-
-
$\begingroup$ python mainly. in the second graph with the red regression line, how were you able to run linear regression (
lm) and produce a curved non-linear regression line? $\endgroup$ Sep 22, 2020 at 4:58
$\begingroup$
$\endgroup$
7
Before even getting to regression modelling, if you just have two scalar variables then you should start with a scatterplot of $x$ and $y$. That is likely to immediately tell you if they have a "linear" (or perhaps "affine") relationship. It is of course possible to follow this up with a formal regression model to test for non-linearity between the two variables; one way to do this is to fit a polynomial regression and test whether the higher-order coefficients (and constant term) are all equal to zero.
-
$\begingroup$ what in a scatter plot of two time series would indicate non-linearity? $\endgroup$ Sep 11, 2020 at 4:37
-
$\begingroup$ Linearity would mean that $x$ and $y$ roughly follow a straight line through the origin, so anything that looks like it violates that would be evidence of non-linearity. (If you actually mean an affine relationship then the straight line does not have to go through the origin.) $\endgroup$– BenSep 11, 2020 at 4:40
-
2$\begingroup$ @develarist Be very careful about naive use of regression with time series data. $\endgroup$– Glen_bSep 11, 2020 at 6:50
-
$\begingroup$ how about scatter plot on time series data like what you were saying $\endgroup$ Sep 11, 2020 at 11:21
-
1$\begingroup$ @develarist See economics.stackexchange.com/a/14261/8891 for an example of time series having an apparent linear relationship where there was little or no relationship between the changes of the two variables $\endgroup$– HenrySep 11, 2020 at 16:31