Here on chi-squared distribution wikipedia page is mentioned that many statistical tests use chi-squared distribution. I would like to ask why? What is so special about chi-squared that predetermines it to be used with (those goodness of fit) tests? There are many other tests and I would like to ask tests if they are related somehow and what is their main difference? Is it just type of data on which they should be applied? I've also seen many times that chi-square is used with statistical hypothesis testing to make inferences. In following video it is said that when data does not follows normal distribution one should be care which test to choose. For example in following video the guy explains the logic behind hypothesis testing and he chooses so called "null model" which seems to have the normal distribution. What would happen if chosen test/null model is wrong for given data? Is there any comparative study which will show how good is particular test for given data which follows given distribution? I would like also ask why there are so many distributions? If I understand it correct then all of those distribution is the result of observing some patterns. For example normal distribution is result of observing e.g. height of peoples (there are a few short people then a lot of people with normal height and again a few very tall people) Can you please explain this to non native english and non statistical guy and use as much examples as you can (if it is possible)? Thank you very much
2 Answers
Here on chi-squared distribution wikipedia page is mentioned that many statistical tests use chi-squared distribution. I would like to ask why?
There might be others, but I think the main reason for the frequency of chi-squared distribution is because of the likelihood ratio test. Under certain conditions, $-2ln(R)$ asymptotically follows a chi-squared distribution where R is the ratio of the likelihood of two models with one being a nested version of the other. The number of degree of freedom for this distribution is how much degree of freedom the nested model fixed.
There are many other tests and I would like to ask tests if they are related somehow and what is their main difference?
This is a very broad question, but as they depends on many things the choice of a test for a specific case is not very large...Ok, there are still some choices as Whuber in the comments pointed out, but at least not very large compared to overall amount of existing test. The reason is that they differ on their application scope. Among the things that matters to choose a test, you have the numbers of (dependent or independent) variables in the test, their types (category, ordered category, numerical), the assumptions that can be made on the data like their distribution (which partly answers to your next question),the experimental design comes also into play, for example if you have multiple measurements, and other things..
What would happen if chosen test/null model is wrong for given data? Is there any comparative study which will show how good is particular test for given data which follows given distribution?
Yes there are, browsing randomly crossvalidated I found this Q&A summarizing how good was the wilcoxon test compared to the student t test when the normality assumption required for the unpaired t-test was missing and the variable was following another distribution.
-
$\begingroup$ I think you are essentially correct in the reason why $\chi^2$ is so prevalent. I found two other pronouncements surprising, though. First, the choice of test is a deep, complicated issue and in almost every case there is an enormous array of possible tests that could be conducted--the opposite of "not very large." Second, the reason there are "so many [named] distributions" is because of mathematics, but not in the way you describe. The world is a rich, complicated, and varied place. Compared to how it works, there are extremely few distributions studied by mathematicians. $\endgroup$– whuber ♦Commented Jun 20, 2015 at 14:37
-
$\begingroup$ As always, thanks for your instructive comments. I ll try to update my post in light of these. $\endgroup$– brumarCommented Jun 20, 2015 at 14:42
-
$\begingroup$ I deleted the last part. I think it's too tricky to be answered, it's almost a philosophical question which could hardly be answered in an objective way. Concerning the amount of test, I added a nuance, but I still consider that it was necessary, in the context of the OP first steps in statistical inference, to highlight that all the tests does not test the same things and that many factors reduce the number of available tests for a specific case. $\endgroup$– brumarCommented Jun 20, 2015 at 14:56
-
$\begingroup$ I think I may have a different point of view about testing (there are infinitely many tests to choose from, most of them are poor, but there are still as many optimal ones as there are combinations of statistical models and loss functions, which is--even as a practical matter--still infinite), but I still appreciate your edits. (+1) $\endgroup$– whuber ♦Commented Jun 20, 2015 at 19:28
It seems there are a number of topics you concerned but I can briefly explain you only one about the relation between common distributions.
The fact is that there are a lot of distribution depending on what you want. You can design a new one but the common distributions have general concept and assumptions so it can be applied to many circumstance. You are right that the patterns you observe determine statistical distribution you choose. However, when you get a number of groups of samples, the distribution of average value of groups is approximately Normal distribution,see central limit theorem , regardless to your distribution of population. So, in general, we can often assume that any average of data has normal distribution.
When comes to the Chi-square, it has relation with normal distribution. If you have free time you can try to get a set of number from normal distribution with mean=0, variance =1, get the square of those number and then plot the histogram, you will see it is chi-square distribution. Briefly, if a variable has normal distribution, the square of the variable has chi-square distribution with degree of freedom =1 and sum of N chi-square distribution is chi-square with d.f. = N. You might notice that, in the goodness of test, the statistic come from sum of square of standardized value. The data may be assumed to be normal. The standardization makes them standard normal. The square makes them chi-square with d.f.=1. Finally the sum change d.f. from 1 to N.
If you can do some degree of programming, I encourage you to try it and see it yourself.
