# Why does a Least squares distribution look like a parabola?

For regression analysis, one often uses the least squares method to minimize the quadratic differences between data and a model function f as follows:

$$\chi^2=\sum_i \frac{(\text{data}\, _i-f_i(p))^2}{\sigma_i^2}$$

with the parameter to be fitted $$p$$ (see question for a plot and discussion).

Why (and under which assumptions) does $$\chi^2(p)$$ look like a parabola?

In my work, due to computational intensity (of $$f$$), I can't use common tools to find the minimum and have a small number of data points, which is why I then fit a parabola to the $$\chi^2(p)$$ distribution to find the estimated minimum that way.

Edit1: In my case, $$\text{data}\, _i$$ and $$f_i$$ are produced by the same numerical procedure (I do sensitivity tests of parameters), which is why I can assume that the two distributions look very similar for small variations of $$p$$.

## 1 Answer

If $$f(p)$$ is a linear function of the parameters you'll get a parabola, since we are squaring it.

As for the why, we are looking for something whose solution is easy to find and possibly can be found explicitly and assigning positive weight to any difference between data and model. We could use the absolute value, but it then becomes annoying to minimize.

• thank you for your answer. in my case $f$ is not linear in $p$. But maybe, as I am varying $p$ only slighty, this assumption might be a good approximation still. regardin the why, I was more interested in the general case. I believe (in general) for stronger variations, the valley to minimize doesn't have the form of a parabola. there could even be local minima etc. So is it only a parable because I am looking very closely at small variations? Commented Jan 13, 2021 at 10:09