I report one of the definiton of Fisher's $F$

Suppose that $\chi^2_A$ has the chi-square distribution with $\nu_A$ degrees of freedom and $\chi^2_B$ has the chi-square distribution with $\nu_B$ degrees of freedom, where $\nu_A,\nu_B∈(0,∞)$. Assume also that $\chi^2_A$ and $\chi^2_B$ are independent. The distribution of $$\mathrm{F}=\frac{\chi^2_A/\nu_A}{\chi^2_B/\nu_B}$$

is the $\mathrm{F}$ distribution with $\nu_A$ degrees of freedom in the numerator and $\nu_B$ degrees of freedom in the denominator.

Consider the following situation: I have a set of data and I try to use Least Square Method with two different functions, say a line and a parabola, to fit these data.

Then I calculate $\chi^2$ in both cases and in both cases I get a value smaller than the critical one, so I do not refuse any of the two fits.

(In order to choose between the two of course I can see where $\chi^2$ is smaller, but that can be a fluctuation and let's not consider that).

I heard that a more suitable way to understand if the two fits are both good is to use $F$, as defined above, to do a $F-$test.

Anyway I do not understand what the null hypotesis of the test would be in that case, neither I found anything on textbooks.

So how can one use $F$ to understand if two $\chi^2$ from different fits of the same data can be considered both "good"? In particular what the null hypotesis for that test would be?

My guess (surely wrong and confused) would be that, if the two $\chi^2$ are both "good" then $F \approx 1$ but "the two $\chi^2$ are both good" does not seem a real null hypotesis, since $F$ is defined as $\chi^2$s ratio indipendently from the fact that both the $\chi^2$s come from fits of the same set of data or not. As far as I understand the two $\chi^2$s could be any (as long as indipendent) and also from different sets of data.

  • $\begingroup$ Could you validate my answer since I provided (for the line and parabola you mentioned) a test statistic and a code comparing the F-distribution with the sampling distribution of the test statistic? $\endgroup$ – Adrien Renaud Sep 8 '16 at 12:35

Your two models (parabola and line) are nested (Two statistical models are nested if the first model can be transformed into the second model by imposing constraints on the parameters of the first model) so you can indeed use an F-test constructed from the two least square fits.

I wrote a rather detailed answer on that: Converting several t-statistics to a single F-statistic? It even contains a Python code providing a sampling distribution of the F-stat in the case of a linear relationship fitted with a linear model and a quadratic polynomial ! And the sampling distribution is in good agreement with the F-distribution !

Your null hypothesis will be that the parabola does not provide a significantly better fit than the line. With respect to the link I gave, you just need to change the two models, model 1 will be the line and model 2 the parabola.

  • $\begingroup$ Thanks for the answer, I read your useful answer and the pdf you linked there. If I may, I still don't understand two main things. Firstly in the example you made and in the pdf $F$ is defined as $$F=\frac{(\chi^2_A-\chi^2_B)/(\nu_A+\nu_B)}{\chi^2_B/\nu_B}$$ And not just $$F=\frac{\chi^2_A/\nu_A}{\chi^2_B/\nu_B}$$ Why is this necessary? And can it be defined as the form in my question in some cases (maybe when the models are not nested?)? $\endgroup$ – Sørën Sep 6 '16 at 17:52
  • $\begingroup$ Secondly, still concerning the null hypotesis for the test, from your answer: "Under the null hypothesis that model 2 does not provide a significantly better fit than model 1, $F$ will have an $F$ distribution". Here is what I do not get: how can this null hypotesis make that ratio (however it is defined, as in my question or in your answer) follow an $F$ distribution (and don't follow it otherwise), what is the "key" in the hypotesis that transform that ratio in a ratio that follows a $F$ distribution? Or would it follow a $F$ distribution in any case? $\endgroup$ – Sørën Sep 6 '16 at 17:54
  • $\begingroup$ Thanks for feedback. 1) My version is necessary because one or both of the two $\chi^2$ could not follow a $\chi^2$ distribution. Suppose you fit a parabola relationship with a line... 2) I'm not sure about that but you could investigate using the provided code. $\endgroup$ – Adrien Renaud Sep 6 '16 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.