5
$\begingroup$

I have problems with the interpretation of a scatter plot in a multiple linear regression (OLS method). I have posted an image below of the scatter plot of the standardized residuals vs the predicted value of my dependent variable (C).

My question is: in this graph can I assume that my linear regression model is good? The disposition of the residuals is suspect and this often is trace of nonlinear relationship between the variables. What do you think of my case? Thanks to everyone.enter image description here

EDIT: here is the dataset https://mega.nz/#!ehtEERJA!_3OMnu2GutFmM9R9fZjfQIthF7bzCNMaT_g1Q2033ko

C is the dependent variable, the other variables are the indipendent. Durbin Watson stat: 1,241603582 Shapiro-Wilk test shows that residuals are normally distributed

EDIT 2: here is the qq plot for residuals enter image description here

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ Definitely looks like non-constant variance, but I'd hesitate to comment further right away. If you posted your data, that might help people assess the situation. $\endgroup$
    – Michael Hardy
    Jul 3, 2016 at 18:52
  • $\begingroup$ From your plot I can see that as your predicted values increase, so does your residuals (you have a clear "cone" pattern). This is bad but without more information on your independent and dependent variables I can't give any advice on what to do next. $\endgroup$
    – Mud Warrior
    Jul 3, 2016 at 18:52
  • $\begingroup$ I put a link where you can download my dataset, I hope you can help me. $\endgroup$
    – Kolmogorovwannabe
    Jul 3, 2016 at 19:28
  • $\begingroup$ Which variable are you regressing on which others? $\qquad$ $\endgroup$
    – Michael Hardy
    Jul 3, 2016 at 19:28
  • $\begingroup$ C is my dependent, the other variables are the indipendent. $\endgroup$
    – Kolmogorovwannabe
    Jul 3, 2016 at 19:29

2 Answers 2

Reset to default
10
$\begingroup$

No, this does not look good. You appear to have a problem with heteroscedasticity as there is increasing variance of residuals with increasing predicted values. Constant variance is an important condition for OLS regression in order to perform valid inference. This might be resolved by log-transforming the response variable.

There is also a hint of autocorrelation but this is hard to assess with so few data points.

Edit, after downloading the data:

Log-transforming C helps with heteroscedasticity, though there are few data points so I would advise some caution: while it seems to help with these data, it may not be the case with more observations. There could be other non-linearities that should be accounted for.

enter image description here

However, all your independent variables are highly correlated with each other, which is not good at all for model interpretation:

      years    Y    W  SSW    G    T   TR    D
years  1.00 0.95 0.96 0.96 0.98 0.98 1.00 0.98
Y      0.95 1.00 0.99 0.95 0.97 0.98 0.95 0.87
W      0.96 0.99 1.00 0.97 0.98 0.98 0.96 0.89
SSW    0.96 0.95 0.97 1.00 0.98 0.97 0.97 0.93
G      0.98 0.97 0.98 0.98 1.00 0.99 0.99 0.95
T      0.98 0.98 0.98 0.97 0.99 1.00 0.98 0.93
TR     1.00 0.95 0.96 0.97 0.99 0.98 1.00 0.98
D      0.98 0.87 0.89 0.93 0.95 0.93 0.98 1.00
Improve this answer
$\endgroup$
10
  • $\begingroup$ I edited the post with some further informations. I hope you can help me, I'm trying to find a way to post my datas not as an image $\endgroup$
    – Kolmogorovwannabe
    Jul 3, 2016 at 19:07
  • $\begingroup$ What format is your data in ? If you load it in R you could try using dput and post it in your question. Otherwise put it in a file sharing site as a CSV file. $\endgroup$
    – Robert Long
    Jul 3, 2016 at 19:17
  • $\begingroup$ I resolved, I put a link where you can download my dataset! $\endgroup$
    – Kolmogorovwannabe
    Jul 3, 2016 at 19:27
  • $\begingroup$ The collinearity is a serious problem, but it's not my principle fear in my case. I'm doing regression for an economic experiment where I have to read the values of the coefficients, I studied my variables with linear regression, but your analysis shows me that a linear model maybe is not good in this case, right? $\endgroup$
    – Kolmogorovwannabe
    Jul 3, 2016 at 19:54
  • $\begingroup$ So you suggested me to log-trasforming C. But I need another advice. I'm new with log-traforming, can you help me? I need to trasform only C or all my variables in log form? I mean, I have to use this equation: logC= logY+logW+logSSW+logG+logT+logTR+logD or only logC= Y+W+SSW+G+T+TR+D? After log-traforming I run as always the OLS regression? $\endgroup$
    – Kolmogorovwannabe
    Jul 3, 2016 at 19:59
4
$\begingroup$

Visually at first it seemed to me that your residuals look like they were heteroskedastic (non-constant variance), autocorrelated (not independent), and non-Normally distributed. Those are actually issues that could be resolved anyway. It turns out that you tested your residuals, and they appear to have done ok on all those counts. However, those issues are minute vs. the multicollinearity issue uncovered by Long. All your independent variables are very highly correlated with positive correlation coefficients ranging between 0.9 and 1.0. The multicollinearity is a huge problem. You indicated not being very concerned about it. But, you should. This problem does not go away just by wishful thinking. And, logging the variables will certainly not solve that. It may actually exacerbate it. A symptom of multicollinearity is that your variables' regression coefficients are likely very unstable. Rerun your regression by omitting some of the data to test the stability of the coefficients. They are likely to be very unstable. Another symptom is that the statistical significance of some of your variables may be questionable. It does not make sense to have 8 independent variables in a model that are all very highly correlated. They simply impart to your model almost the exact same type of info in terms of explaining the variance (or behavior) of your dependent variable.

I think you need to rebuild this model by starting by selecting one single of the best time-oriented independent variable you have. And, then adding other variables that provide information other than time. One type of time-oriented variable you may add to this model without running into excessive multicollinearity issues are seasonality dummy variable. But, this is not a sure thing. You would have to test for that (the multicollinearity bit).

Improve this answer
$\endgroup$
7
  • $\begingroup$ I know that the multicollinearity is a huge problem, but I can't depart from this model because I'm basing my work on the model propose in scientific literature. I executed other tests using less variables but the collinearity it's still present. i can't have more observations because the data sources are limitated. Unfortunately I have no way to deal with collinearity so I'm using these variables and I'm reading the predicted values with huge prudence. Do you have any other suggestions? Thank you anyway. $\endgroup$
    – Kolmogorovwannabe
    Jul 4, 2016 at 12:10
  • $\begingroup$ There is something that is not clear to me. I've done the Shapiro-Wilk test on the residuals and it turns out that they are normally distributed. The Durbin-Watson test was not conclusive so I don't know why they are autocorrelated. I notice that the test of Breusch-Pagan to test heteroskedasticy show that there is Homoscedasticity. Why all these test failed? Can you explain me these points? $\endgroup$
    – Kolmogorovwannabe
    Jul 4, 2016 at 13:31
  • $\begingroup$ I have added to my question the qq plot for residuals to study the hp of normal distribution. $\endgroup$
    – Kolmogorovwannabe
    Jul 4, 2016 at 15:07
  • $\begingroup$ Giuseppe, my assessment of the residuals were just based on visual observations. Your residuals did not look quite right on a few counts. But, you tested them and they essentially passed all conditions. So, on those mentioned counts they are all fine. However, on the multicollinearity bit I again suggest you rebuild your model. You have 8 ind. variable, and 7 of them appear to be completely wasted. When your ind. variable are all correlated to nearly 1.0 level they are duplicative, and superfluous. Rebuild yourself a more parsimonious and informative model. $\endgroup$
    – Sympa
    Jul 4, 2016 at 18:03
  • $\begingroup$ Giuseppe, also related to multicollinearity, you have not addressed the issues of the stability of the regression coefficients and the statistical significance of the variables. I would also add to check for the directional sign of your variables. Do they all make sense? Given your model structure, I expect you will run into some issues on some of those counts. Recommendation is the same, rebuild your model by just keeping one of your original 8 independent variables and add different types of variables that are not multicollinear with each other. $\endgroup$
    – Sympa
    Jul 4, 2016 at 18:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.