Loess line interpretation I'm sorry for this noob question, but I'm following a practical to draw a plot of Boston Housing data set after using Gradient Boosting Machine to train the data, but I don't understand how to interpret the blue line and the red line.
> data.frame(Predicted = pred, Observed = data.test.z$medv) %>%
+   ggplot(aes(x = Observed, y = Predicted))+
+   geom_point(size = 1, alpha = 0.5)+
+   geom_smooth(method = "loess", col = "red")+
+   geom_smooth(method = "lm")


 A: You haven't provided information about the GBM model and in a way the details are irrelevant. We can assume that the predictions are generated by a black-box model and still interpret the plot of observed vs predicted values in a meaningful way.
Both the lm and the loess lines are mostly a distraction from an useful interpretation.
Overall, the black-box model is a good fit to the data: it predicts higher prices for more expensive houses. Then in the mid-range (price between 15k and 30k) the black-box model seems to under-predict: the actual prices tend to be higher than the model predicts. That is, the model has some bias in the mid-price range.
You don't need to fit any lines to observe the bias. Instead, plot the observed vs fitted values, fix the aspect ratio at 1:1 and plot the y = x line.
data.frame(
  Predicted = pred, Observed = data.test$medv
) %>%
  ggplot(aes(x = Observed, y = Predicted)) +
  geom_abline(slope = 1, intercept = 0) +
  geom_point() +
  coord_fixed()

The loess fit, which adapts locally to the quality of the predictions, might be more informative once the y = x is added, perhaps less so in the high-price range where there are very few houses.
Aside: It's more common to plot the observed values against the residuals, Residuals = Observed - Predicted. The two plots contain the same information but the second version is easier to interpret as it more clearly visualizes the errors that the model makes.
