2
$\begingroup$

My question concerns two methods for plotting regression residuals against fitted values.

The standard method: You make a scatterplot with the fitted values (or regressor values, etc.) on one axis (generally $x$-axis), and the residuals on the other (generally $y$). This is what I've always been taught.

The sorted method: I recently saw someone plotting residuals in a slightly different way: he would sort the residuals using the fitted values as a key, and then plot the residuals in sorted order. In this case, the $x$-axis would be the ranked ordering of the residuals by fitted values, $i=1, 2, ..., n$ for $n$ residuals. To be clear this is not a normal probability plot, where we have to sort the residuals by their own values.

My question is, is this sorted method valid, or does it present any benefits not captured in the standard method?

To illustrate a scenario where the difference is very apparent, say I have a data set where a value is repeated quite a bit.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

np.random.seed(0)

df = {}
df['x'] = list(np.random.randint(1, 20, 20)) + ([0] * 10) #Fitted values
df['y'] = np.random.uniform(-2, 2, 30) #Residuals
df = pd.DataFrame(df)

ax = df.plot.scatter('x', 'y')
ax.set_title('Standard method')
ax.set_xlabel('Fitted value')
ax.set_ylabel('Residual')
plt.show()
plt.close()

keyed_values = sorted(zip(df['x'], df['y']), key=lambda x: x[ 0])
sorted_residuals = [x[ 1] for x in keyed_values]
fig = plt.figure(figsize=(6, 4))
ax = fig.add_subplot(1, 1, 1)
ax.plot(range(0, len(residuals)), sorted_residuals, '.', alpha=0.75, markersize=10, color='C0')
ax.set_title('Sorted method')
ax.set_xlabel('Ranked order of residual by fitted value')
ax.set_ylabel('Residual')
plt.show()
plt.close()

enter image description here

Same residuals, same fitted values, two very different plots that could lead to different interpretations. The sorted method guarantees that no two values would ever be plotted at the same value of the $x$-axis, but breaks ties somewhat arbitrarily. It also forces equal spacing of residuals over the $x$-axis. Is this a valid way to reveal a pattern otherwise not shown by the standard method?

$\endgroup$
  • 1
    $\begingroup$ The x-axis label of the second plot should be "Ranked order of fitted value", no? $\endgroup$ – Nuclear Wang Aug 12 '19 at 19:15
  • $\begingroup$ You're correct, fixed to 'Ranked order of residual by fitted value' in code and plot. $\endgroup$ – Julian Drago Aug 12 '19 at 19:18
  • 1
    $\begingroup$ Look at the top plot's residual errors for a fitted value of zero. This clearly shows that the model has its widest range of error at fitted values of zero. Compare this to the second plot, which clearly shows... um... ahh... hmm, not sure about that one... $\endgroup$ – James Phillips Aug 12 '19 at 20:56
5
$\begingroup$

On the contrary I would say that the standard method is way better. The "ranked method", which is not a standard method and should probably not be called a "method" at all, masks the true relationship between predicted values and residuals.

If in a certain interval your model is really bad, you would notice that you have many high (in absolute value) residuals. By ranking them (and breaking the ties in the ranks randomly - if they were any), you visually spread these residuals which makes them appear less grave. Moreover, it would be more difficult to identify where your model is bad, since you have only ranks and no values on the response scale. Lastly, it would also be more difficult to identify heteroscedasticity...

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ upvoted for the excellence of the answer. $\endgroup$ – James Phillips Aug 12 '19 at 20:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.