Deciding between linear and non-linear approach in a predictive model when the relationship 'looks linear'

Question

When we have a relatively small number of samples it is easy to see on a plot what is intuitively happening when we fit a regression line; we can see how far each of the individual sample points are from the regression line.

But when there are thousands of samples, they become too clustered to tell them apart on a plot.

Consider the image below. Pressure appears to decrease with temperature and the relationship 'looks' linear (the blue line is the regression line by the way). However, if we also plot a loess smoothed line (red curve) it appears the relationship may be better described with a nonlinear function.

However, I am not just interested in fitting a line, ultimately my primary goal is prediction. And I understand that the more complicated non-linear function may not generalize as well for out-of sample errors. On the other hand, we have a large amount of data here so I'm not sure if overfitting is such an issue..maybe the more complicated non-linear function will generalize fine here.

So if an experienced statistician was developing a predictive model and saw this plot, would they decide to model the relationship with the linear fit (blue line) or the the non-linear function (red curve)?

Would an experienced statistician always use some computational technique to systematically decide which model to use or would they 'go with their instinct'?

Answer 1

An experienced statistician would look at the plot you shared and comment that neither the linear fit nor the loess fit seem to do justice to the data at the lower and upper end of your temperature range.😜

Notice how most of the pressure measurements located at the lower temperature range are situated above the fitted line and fitted loess curve; most of the pressure measurements located at the higher temperature range are situated below the fitted line and fitted loess curve.

We don't know how your data came to be and if we can assume observations to be independent of each other - that makes it difficult to make concrete recommendations. We also don't know the underlying processes involved - is it feasible to expect different behaviour for the pressure - temperature relationship at lower and higher temperatures?

In any event, you can look into using the ols() function in the rms package of R (presuming you have independent observations), as it makes it easy to validate your prediction model and also calibrate it for achieving better out-of-sample predictions. Check out the validate() and calibrate() functions in rms. Ideally, you will consider both a linear and a non-linear model, compare their performance via bootstrap validation, pick the model with the best performance, etc. Your work needs to withstand external scrutiny so you have to be able to justify all your modelling choices objectively.