Residual Plots for Model Comparison

Question

Other than using AIC and BIC, I'd like to compare two different models via graphical interpretation, therefore my question is, is it possible to compare two regression models with each one having the same set of independent variables but different response variables with residual plots?

---

My primary aim is to set up two different regression models with different response variables, and both of the responses are the results of diagnostic tests for a specific disease. I'd like to find out the one that fits the data most to be able to suggest this model for calculating risk scores.

Answer 1

Assuming y1 and y2 as the response variables, predict the response variables using the corresponding models, say mod1 and mod2 and then plot the scatterplot between the actual and predicted in the same plot and same scales. In R that could be

py1 <- predict(mod1)
py2 <- predict(mod2)

plot(y1, py1, col='blue', 
     xaxs="i", yaxs="i", 
     xlim = c(0, max(y1)), ylim = c(0, max(py1)), 
     xlab = 'Y Actual', ylab = 'Y Predicted')
par(mfrow=c(1, 1), new=TRUE)
plot(y2, py2, col='red',
     xaxs="i", yaxs="i", 
     xlab ="", ylab ="", 
     xlim = c(0, max(y1)), ylim = c(0, max(py1)))

For residual plots

library(car)
residualPlot(mod1, col='blue', col.quad='blue', 
  xlim=c(min(mod1$fitted.values, mod2$fitted.values), 
  max(mod1$fitted.values, mod2$fitted.values)), 
  ylim=c(min(mod1$residuals, mod2$residuals), 
  max(mod1$residuals, mod2$residuals)))
par(mfrow=c(1,1), new=TRUE)

residualPlot(mod2, col='red', col.quad='red', 
  xlim=c(min(mod1$fitted.values, mod2$fitted.values), 
   max(mod1$fitted.values, mod2$fitted.values)), 
   ylim=c(min(mod1$residuals, mod2$residuals), 
   max(mod1$residuals, mod2$residuals)),
             xlab="", ylab="")