I'm trying to use a residual plot in order to get a better understanding of my dataset and what I should be doing.

My data is a series of scored labels with a bunch of distance features used to predict them. I'm hoping to use linear regression.
When I generated my residual plot I ended up with the following unusual graph which I haven't seen before.

Residual Plot

In my limited knowledge I'm aware that ideally a Residual plot should contain a cluster of points with a mean of 0. In a bad case scenario the plot is non random, most examples indicate a spiked residual error value at a given predicted value.

My graph however looks nothing like examples I have seen and I'm not certain how to interpret it. Could is perhaps be that my dataset is not suited for a regression model? What does the residual plot indicate about my data?

Additonal Information that may be useful:

Predicted Values vs Ground Truth

r2 value: 0.32812704493994505

  • $\begingroup$ It looks to me as if the model is missing some predictive factor, in part because the regression line does not pass through two groups of points on the plots. I suggest visually inspecting scatterplots of the raw data to see if a non-linear relationship might be apparent. $\endgroup$ – James Phillips May 1 '18 at 11:43
  • $\begingroup$ It sounds like a "scored label" might merely be an arbitrary assignment of numbers to distinguish different values of a response. If so, your model is unlikely to work well (and the residuals show that). Could you explain more about these "scores"? $\endgroup$ – whuber May 1 '18 at 11:48
  • $\begingroup$ Thanks for the advice about investigating scatterplots. A non-linear relationship did occur to me. The scores are indeed arbitrary. They represent the level of match between two users as labelled by someone based on some values. Is this not a application for a regression model? I've been sceptical about whether this would work from the onset however I was told to do it anyway. The only data available was a series of lists for each user consisting of roles they picked. These roles were then turned into distance values by way of turning them into a tree like structure and measuring distances. $\endgroup$ – harrison May 1 '18 at 12:06
  • $\begingroup$ It looks like you have a categorical predictor with five levels and one or more continuous predictors. Can you give more details about your model? $\endgroup$ – mdewey May 1 '18 at 12:30
  • 1
    $\begingroup$ Consider an ordinal regression model. It will automatically handle the nonlinearity. $\endgroup$ – whuber May 1 '18 at 12:35

It seems to me that you have 5 outcome values, 1 to 5. However, if you use ordinary linear regression in this case, your predicted values will not be constrained to take one of the values 1 to 5. This is why there are the 5 horizontal lines in the second graph, each with a spread of predicted values. This also explains the first graph: the two graphs are in fact equivalent, since the residual equals the predicted label minus the training label.

Instead of performing ordinary linear regression (which requires a continuous outcome variable), I'd instead perfom ordinal regression. I can advise you how to do this in R - could I ask what software you're using?

  • $\begingroup$ I'm currently using Python. I'm investigating using Ordinal Regression right now using the mord package. I'll let you know how it goes. Thankyou. $\endgroup$ – harrison May 1 '18 at 13:37
  • $\begingroup$ After some experimenting Ordinal Regression is indeed the way forward. Thankyou for your answer. $\endgroup$ – harrison May 3 '18 at 9:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.