2
$\begingroup$

My question is about the implications of the violation of homoscedasticity/linearity for multiple linear regression. I have tried to find the answer in multiple sources but could not figure it out.

I have performed three multiple linear regressions on a dataset with $4,000$ participants. There are $4$ independent variables. $2$ dependent variables are ordinal $(1-7, 7-21)$. I performed the following assumption checks:

  • Analysis of standard residuals to identify outliers ($-3.29 <$ Std. Residual $< 3.29$) (DV1 $0$ outliers, DV2 $6$ outliers, DV3 $31$ outliers).
  • Collinearity assumption is met (no multicollinearity, VIF values are okay).
  • Assumption of independent errors is met (Durbin-Watson).
  • Distribution of errors with histogram of standardized residuals and PP-plot of standardized residuals, not met in all three will show below the one violated as I cannot post all my links.
  • Assumption of linearity and homoscedasticity seems violated.

I consulted statistics help from our university who said that for all three MLR analyses the assumption of linearity is violated and that I should just mention that in my discussion. My first thesis tutor (from other university) does not find these assumptions important. However, my second from my own university does and wants me to report more about it, f.e. what the consequences are of these violated assumptions and how I will deal with them. When I google or look into literature (Field's SPSS book & Regression & Linear Modeling: Best Practices and Modern Methods). I only find very different examples of the violation of linearity, which look more like a rainbow than a linear line. The line I have added in SPSS is a Loess line. I am not so sure about what to report exactly on this violation, more than that the assumption has not been met. It takes a long time to make an appointment with the statistic help so that is why I am here. I hope someone can help me with an answer or a source where I can find an answer/examples).

What implications do these violations have for my results?

DV1: enter image description here

DV2 enter image description here

DV3 enter image description here enter image description here enter image description here

Scattermatrix: enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Simply reporting the model is nonlinear without doing something about it really isn't helpful in my opinion. Your outcome variable appears to be very discrete (thats not obvious from your question but the residuals seem to paint that picture). Is your DV also a likert scale test? I would provide raw scatterplots of your data (these appear to be fitted x residual diagnostic plots) to get a much better sense of what is going on. Simply showing the diagnostic plots doesn't really answer the core questions here. The scatterplots of the actual data will. $\endgroup$ Oct 27, 2023 at 12:21
  • $\begingroup$ Thank you for your answer. Oh, excuse me, the DV 1 & 3 are on likert scales! Will edit the post. I have a scatter matrix which I will also add. $\endgroup$
    – Morin
    Oct 27, 2023 at 21:33
  • $\begingroup$ I have added the scatter matrix, F1-F4 are the four independent variables (factors). The statistic advisor at the university said that transforming the data is out of the scope for a master thesis at this university. The current fit of the linear models is that for DV1 12% of the data can be explained, for DV2 24% and for DV3 2%. As they are psychological constructs, the statistic advisor told me that especially for DV2 the fit is quite OK. $\endgroup$
    – Morin
    Oct 27, 2023 at 21:43

2 Answers 2

2
$\begingroup$

For data with such clear discreetness of Y with floor and ceiling effects I would have never entertained a linear model. This problem has ordinal semiparametric regression (e.g., proportional odds ordinal logistic model) written all over it. Resources for such models may be found here.

$\endgroup$
0
$\begingroup$

On Nonlinearity and Diagnostics

First off, I wan to begin by saying that ignoring linearity is not advisable. It is, mathematically, the most important assumption to meet in a linear regression (Gelman et al., 2022). What the statistics consultant said on this point seems to gloss over the importance of this assumption. If your data can be sufficiently represented as linear, fine. But the consultant and your plots seem to indicate that it may be more useful to model this without a linear assocation.

Whatever DV3 represents, it basically has no relation to your IV. The scatterplots clearly show that there is no relation, and whatever concerns there are about linearity or homoscedascity of errors is not important in that sense. DV1 also seems to be fairly irrelevant, even with the factor shifts in the association (I assume thats what F means) shown in the scatterplot. Perhaps jittering the points on the scatterplot can show why the loess curves are weighted different between factors, but I'm not convinced it matters. Loess curves are typically not as flexible with data as they seem to show given they use a weighted least squares estimation of the data (Jacoby, 2000; Simpson, 2018), and in my experience can often overfit heavily weighted regions of the data distribution.

The most interesting association here is DV2. There definitely appears to be a nonlinear association, but not a complex one. Something you can consider is a polynomial regression. This maps nonlinear relationships in the data via squared, cubed, or quadratic terms (anything above that tends to over-interpolate). If that doesn't fit your data well, a spline-based method could also work, and often does. If you seek to do so within a multiple linear regression framework, generalized additive models (GAMs) may be your friend, because they are typically more flexible. I would also consider modeling this data with an error distribution that is not Gaussian (the default). Since your most relevant variable D2 seems right skewed, I would consider a beta distribution.

After doing all of that, I would move away from typical residual plots for discrete values like yours and try to use a simulation-based approach, such as posterior prediction checks or DHARMa residuals. Residual plots for non-Gaussian models can be wonky, particularly when random effects are introduced into the model. Using a simulation based approach typically provides better inference.

It may also help to quantify what zero means in your DV. I think perhaps it may be okay, but if they actually mean an absence of a measure, you may want to consider how to model/interpret your data. I'm guessing because it is likert scaled, the assumption is more that somebody lacks something in the measure rather than a total absence. A couple references I add below discuss some particulars about likert scaled data that may be helpful to consider.

References

  • Gelman, A., Hill, J., & Vehtari, A. (2022). Regression and other stories. Cambridge University Press.
  • Jacoby, W. G. (2000). Loess: A nonparametric, graphical tool for depicting relationships between variables. Electoral Studies, 19, 577–613. https://doi.org/10.1016/S0261-3794(99)00028-1
  • Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in Health Sciences Education, 15(5), 625–632. https://doi.org/10.1007/s10459-010-9222-y
  • Simpson, G. L. (2018). Modelling palaeoecological time series using generalised additive models. Frontiers in Ecology and Evolution, 6(149), 1–21. https://doi.org/10.3389/fevo.2018.00149
  • Sullivan, G. M., & Artino, A. R. (2013). Analyzing and interpreting data from likert-type scales. Journal of Graduate Medical Education, 5(4), 541–542. https://doi.org/10.4300/JGME-5-4-18
$\endgroup$
2
  • $\begingroup$ Thank you for your time and large explanation. This is really helpful! $\endgroup$
    – Morin
    Oct 30, 2023 at 10:16
  • $\begingroup$ No prob. If you feel that Frank or I have answered your question, feel free to accept one of the answers here. $\endgroup$ Oct 30, 2023 at 10:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.