3
$\begingroup$

In statistics people gather a lot of data (ie heights of people or gene expression levels) to get some insight. Then in order to perform statistical analyses they try to fit their data to a theoretical distribution (ie Normal Distribution) by computing some parameters.

How do we know that our data follows such a distribution? If we were able to measure all objects in a population and draw a distribution we could probably get something different, a different shape than the theoretical distribution we thought of. Aren’t our calculations wrong then if we use a theoretical distribution?

Please correct me if I am wrong and tell me what you think of this. I think this is a very basic concept in statistics and I have to clarify it.

$\endgroup$
  • 1
    $\begingroup$ "Fit their data to a theoretical distribution" sounds backwards to me. Some people fit a theoretical distribution to their data in order to describe or understand the data, but extremely few people actually modify their data to conform to distributional assumptions (which is what your use of "fit" really means)--and their work is generally considered fraudulent. $\endgroup$ – whuber Jul 16 '13 at 14:17
  • $\begingroup$ Yes this is what I meant. To fit a theoritical distribution to the data by calculating parameters. $\endgroup$ – SKY Jul 17 '13 at 11:15
4
$\begingroup$

I think you're using "theoretical" in the way it's used in common speech, to mean "remotely possible". These distributions are theoretical because they are based on theory. The theory specifies conditions, and if these conditions apply, the theory is applicable.

So you don't just figure out what distribution can fit your data. That's a common rookie mistake that's easy in an era where you can easily get a program and ask it to fit 100 different curves to some data. It's possible that you may be able to do this and then inject some domain knowledge into the situation, "Hmmm... this data is biological in nature and it does appear to fit a growth curve that's common in biology, so it gives me some clues as to an underlying mechanism for the data", but without the application of domain knowledge to the problem, you're just doing curve-fitting voodoo.

Rather, a scientist looks at the conditions under which the measurements were made, at the underlying mechanisms that are plausible, and chooses distributions which would be applicable. Of course, a distribution is used because there is variation and uncertainty in the data, so you shouldn't expect that every single point of the data should fall exactly on some curve

Since things don't match exactly, are the calculations all wrong? Yes. The question is "how wrong and for what reason?" If the answer is "not very wrong, and mainly because of small measurement errors", it's good enough. No application of any theory is going to fit perfectly in the real world, but if properly applied it will be close enough to do what you need to do.

Which raises the question, "what do you intend to do with the data?" If you're happy assuming that you've measured the entire population with no error, and you will not speak of anything outside of the population, and don't care to speculate on underlying mechanisms, you don't need distributions. If you want to measure the heights of the members of your family (the population) and make statements like "the tallest person in our family is X tall, and the shortest person is Y tall, and half of the people in our family are taller than Z and half are shorter.", great for you. No need for distributions at all -- assuming you're ignoring measurement error, of course.

If you're going to go beyond this, if you can't measure the entire population, you'll need to use distributions to account for variability and uncertainty. Not arbitrary distributions, but ones that are applicable based on a knowledge of your goals, your data, your assumed mechanisms (model), etc.

$\endgroup$
2
$\begingroup$

People gather data for lots of reasons in statistics; sometimes it's to see if a sample fits a particular distribution. In this case, there are both statistical tests (e.g. Kolmogorov Smirnov) and graphical methods (quantile plots against a theoretical disribution. Is that what you meant>

More often, we gather data not to see if it fits a distribution but to see if (to be general) "something is happening". We may wish to test a hypothesis (e.g men are taller than women) or we may wish to explore (e.g. how different in height are people from different ethnic groups?) For this we may model the data. Some models (e.g. ordinary least square regression) make assumptions, and sometimes these involve a distribution (e.g. OLS regression wants the residuals to be normally distributed). There are, again, ways to see if the assumptions are met. Other methods make few or no assumptions about the distributions (e.g. permutation tests).

$\endgroup$
1
$\begingroup$

An amazing paper on the subject was written by Leo Breiman, the father of random forests, see Statistical Modeling: The Two Cultures.

It would not be correct to say that all statistical inference algorithms are based on statistical models, but this is generally true. This question was asked by others and led to the development of the machine learning community.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.