5
$\begingroup$

I have data with 2 variables: X - Area Size of a field, Y - Average Production Rate. I need to check the relation between the two. I have plotted the data, and got the following two graphs:

enter image description here

and

enter image description here

Now I have a dilemma. Should I fit a linear regression model, or should I take into account all the little fluctuations in the trend, as shows in the smoothed curve. Which model is better, linear regression or spline fit (which I am not too familiar with). Won't I get overfitting by using the spline model here?

Improve this question
$\endgroup$
7
  • 4
    $\begingroup$ What questions are you trying to answer? In general these plots are very suboptimal: figures without labelled axes are next to impossible to interpreter. $\endgroup$
    – usεr11852
    Commented Jan 14, 2017 at 15:06
  • $\begingroup$ I didn't add values to the axis on purpose, I can't publish the data. I am trying to test if there is a relation (clearly there is), and to make predictions, if possible. Regression analysis. I just noticed the fluctuations. $\endgroup$
    – BlueSigma
    Commented Jan 14, 2017 at 15:20
  • 1
    $\begingroup$ The issue of overfitting requires knowledge of sample size. There may be other alternative to consider such as nonlinear regression. You need to provide more details. $\endgroup$
    – Michael R. Chernick
    Commented Jan 14, 2017 at 15:57
  • $\begingroup$ The sample size is N=70. The mean of X is 92.1 with SD of 128, the mean of Y is 318,521 with SD of 475,588. The range of X is 552. If I fit a linear regression, I get an R squared of 0.8. $\endgroup$
    – BlueSigma
    Commented Jan 14, 2017 at 16:24
  • 2
    $\begingroup$ If you cannot publish our data, you should publish plots of them either (annotated or not). Having said that, you do say that: "X - Area Size of a field, Y - Average Production Rate" in your first sentence already so we infer that much; I mostly ask for units and the overall modelling task. In any case, talk about the problem you try to solve. Generally speaking a spline seems overly flexible for an "area-to-production" relation but currently there is not enough information to do an informed suggestion. $\endgroup$
    – usεr11852
    Commented Jan 15, 2017 at 0:59

1 Answer 1

Reset to default
3
$\begingroup$

Polynomials are notoriously bad at extrapolating trends. They are mathematically bound to go to $\pm \infty$ whereas our intuition of trends does not. This should be a cautionary tale for anyone considering splines for inference or prediction. Looking at your graphic, I am certain that is displaying overfitting, which applies to your question whether it is for inferential or predictive statistics.

You can penalize your splines, like with a LOESS smoother, if your desire is to generate a descriptive non-parametric summary of a possibly non-linear trend.

In your comments you note you transform $Y$ to log and the trend is non-linear. It seems to me your visual inspection of these data is heavily weighted by the high influence/leverage points in the tails. It seems less than 5% of the data lie in the right 2/3rds of the graphic. I would suggest graphing these data with a log transform applied to the $X$ axis. If you would like to find a single polynomial function which relates $X$ to $Y$ in this graphic, consider the use of fractional polynomials, rather than fitting scads of models higgledy piggledy.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ May I directly ask: What is the advantage of polynomials then? I assume extrapolating is the most popular case? Despite of that: Isn't it, that polynomials display the underlying/physical behaviour which is most the case? $\endgroup$
    – Ben
    Commented Jan 30, 2018 at 12:05
  • 2
    $\begingroup$ @Ben in inferential statistics and in prediction, we use the polynomials to interpolate values. Forecasting differs from prediction in that the objective of the model is predicting means and uncertainty for values that lie beyond the range of observed values. $\endgroup$
    – AdamO
    Commented Jan 30, 2018 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.