In the image below, the orange curve is the true function, the dots are samples from the true function plus a unit variance gaussian, while the blue curve are the estimated function using linear regression with polynomial basis.


I want to find a meaningful way to compute the true error between the blue and orange curves. I used $$ \epsilon=\|f(x)-\hat f(x)\|_2 $$ where $f(x), \hat f(x)$ are the vector of all the outputs in the interval and $x$ consists of small increments (points) in the X-axis. The problem is that the true error increases as I let the increments become small to smoothen the plot of the curve. What shall I divide with my true error function to make them more meaningful like staying between 0 and 1 for example?

  • $\begingroup$ Are you multiplying your point-estimates by the "dx" you are using? If not, that would certainly be a reason your error is diverging with decreasing stepsize. $\endgroup$
    – costrom
    Jul 19, 2022 at 15:39
  • $\begingroup$ @costrom unfortunately no, I am simply using the function values obtained on those many x points in my error function. May I ask where should I place those dx in my function? $\endgroup$
    – wd violet
    Jul 19, 2022 at 15:40
  • $\begingroup$ This question may be useful: math.stackexchange.com/questions/675833/… $\endgroup$
    – costrom
    Jul 19, 2022 at 20:32
  • $\begingroup$ Could you please explain what you mean by "true error"? After all, you already have an estimate of the error between your polynomial fit and the data. $\endgroup$
    – whuber
    Jul 19, 2022 at 20:34
  • 1
    $\begingroup$ That "true error" is not defined until (1) you specify a compact domain for the function and (2) you specify the metric you are using to compare two functions. $\endgroup$
    – whuber
    Jul 20, 2022 at 12:50

1 Answer 1


The "total error" you are looking for may be something like the distance between functions.

You are currently computing:

$$error = \Sigma (f(x_i) -g(x_i))^2 $$

This will, as you have found, not converge to a single answer when you vary your sampling 'increment'.

What you should be computing is something more like:

$$error = \Sigma (f(x_i) -g(x_i))^2 \Delta x $$

where $\Delta x$ is the 'increment' in $x$ you use to estimate the error.

This should produce the converging results that you expected.

  • $\begingroup$ That error function approaches 0 as I let $\Delta x$ approach 0, is that behavior expected? $\endgroup$
    – wd violet
    Jul 20, 2022 at 3:31
  • $\begingroup$ I would not expect it to go to zero, no. Can you provide the functions that produce the blue and orange curves? $\endgroup$
    – costrom
    Jul 22, 2022 at 20:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.