32
votes
$\begingroup$

This is a similar question to the one here, but different enough I think to be worthwhile asking.

I thought I'd put as a starter, what I think one of the hardest to grasp is.

Mine is the difference between probability and frequency. One is at the level of "knowledge of reality" (probability), while the other is at the level "reality itself" (frequency). This almost always makes me confused if I think about it too much.

Edwin Jaynes Coined a term called the "mind projection fallacy" to describe getting these things mixed up.

Any thoughts on any other tough concepts to grasp?

$\endgroup$
6
  • $\begingroup$ (I don't know enough to put this as an answer, hence adding a comment.) I always thought it was strange that PI crops up in statistical equations. I mean - what's PI got to do with statistics? :) $\endgroup$
    – Reinstate Monica - Goodbye SE
    Jan 27, 2011 at 9:16
  • 2
    $\begingroup$ I'd agree (In my surprisal) - I think its that $\pi$ pops up in many mathematical analysis. Just a note you can write $\pi$ by with Latex commands as $\text{\pi}$ enclosed within $ signs. I use the wiki page to get the syntax en.wikibooks.org/wiki/LaTeX/Mathematics . Another trick is to "right click" on an equation you see on this site, and select "show source" to get the commands that were used. $\endgroup$
    – probabilityislogic
    Jan 27, 2011 at 10:59
  • $\begingroup$ @Wiki If you accept that $\pi$ crops up when you go from measuring the length of a straigh piece of line to the length of a piece of circle, I don't see why it would not appear while going from measuring a probability to fall down on a segment to measuring the probability to fall down in a piece of circle ? $\endgroup$
    – robin girard
    Jan 27, 2011 at 12:19
  • $\begingroup$ @Wiki Whenever you have trigonometric funcions (sine, cosine, tangent etc.) you risk having $\pi$ pop up. And remember that whenever you derive a function you're actually finding a tangent. What is surprising is that $\pi$ doesn't appear more often. $\endgroup$
    – Carlos Accioly
    Jan 28, 2011 at 18:36
  • $\begingroup$ @Carlos I suspect the prevalence of $2\pi$ is mostly due to the use of the $\ell^2$ metric, leading to n-spheres. In the same vein, I would expect it's $e$ whose prevalence is due to analysis. $\endgroup$
    – sesqu
    Jan 29, 2011 at 22:00

12 Answers 12

Reset to default
31
votes
$\begingroup$

for some reason, people have difficulty grasping what a p-value really is.

$\endgroup$
37
  • 3
    $\begingroup$ @shabbychef: Most of the people grasp it in the worst possible way i.e. probability of making Type I error. $\endgroup$
    – suncoolsu
    Jan 27, 2011 at 7:19
  • 2
    $\begingroup$ I think that's mostly related to how p-values are explained in classes (i.e.: just by giving a quick definition and without specifying what p-values are NOT) $\endgroup$
    – nico
    Jan 27, 2011 at 7:34
  • $\begingroup$ I think this is mainly to do with how it is introduced. For me, it was an "add-on" to the classical hypothesis test - so it appears as though its just another way to do a hypothesis test. The other problem is that it is usually only taught with respect to a normal distribution, where everything "works nice" (e.g. p-value is a measure of evidence in testing a normal mean). Generalising the p-value is not easy as there is no specific principles to guide the generalisation (e.g. there is no general agreement on how a p-value should vary with the sample size & multiple comparisons) $\endgroup$
    – probabilityislogic
    Jan 27, 2011 at 11:12
  • $\begingroup$ @shabbychef +1 though student often have difficulties with p-values (roughly because the concept in testing is a bit more subtle than a binary decision process and be cause "inverting a function" is not easy to aprehend). When you say "for some reason" do you mean it is unclear for you why people have difficulties ? PS: If I could, I would try to make statistics on this site about the relation between "being a top answer" and "talking about p-value" :) . I also even ask myself if the hardest statistical concept to grasp can have the most upvote (if it is difficult to grasp ... :) ) $\endgroup$
    – robin girard
    Jan 27, 2011 at 12:28
  • 1
    $\begingroup$ @eduardo - yes a small enough p-value is sufficient to cast doubt on the null hypothesis: but it is calculated in complete isolation to an alternative. Using p-values alone, you can never formally "reject" $H_0$, because no alternative has been specified. If you formally reject $H_0$, then you must also reject the calculations which was based on the assumption of $H_0$ being true, which means you must reject the calculation of the p-value that was derived under this assumption (it messes with your head, but it is the only way to reason consistently). $\endgroup$
    – probabilityislogic
    Jan 30, 2011 at 5:14
23
votes
$\begingroup$

Similar to shabbychef's answer, it is difficult to understand the meaning of a confidence interval in frequentist statistics. I think the biggest obstacle is that a confidence interval doesn't answer the question that we would like to answer. We'd like to know, "what's the chance that the true value is inside this particular interval?" Instead, we can only answer, "what's the chance that a randomly chosen interval created in this way contains the true parameter?" The latter is obviously less satisfying.

$\endgroup$
1
  • 1
    $\begingroup$ The more I think about confidence intervals, the harder it is for me to think of what kind of question they can answer at a conceptual level that cannot be answered by asking for "the chance a true value is within an interval, given one's state of knowledge". If I were to ask "what is the chance (conditional on my information) that the average income in 2010 was between 10,000 and 50,000?" I don't think the theory of confidence intervals can give an answer to this question. $\endgroup$
    – probabilityislogic
    Jan 27, 2011 at 11:28
21
votes
$\begingroup$

What is the meaning of "degrees of freedom"? How about df that are not whole numbers?

$\endgroup$
13
votes
$\begingroup$

Conditional probability probably leads to most mistakes in everyday experience. There are many harder concepts to grasp, of course, but people usually don't have to worry about them--this one they can't get away from & is a source of rampant misadventure.

$\endgroup$
2
  • $\begingroup$ +1; could you add an example or two, favourite or current ? $\endgroup$
    – denis
    Jan 31, 2011 at 14:53
  • 1
    $\begingroup$ For starters: P(you have the disease|test is positive) != P(test is positive|you have the disease). $\endgroup$
    – xmjx
    Sep 23, 2011 at 6:39
9
votes
$\begingroup$

I think that very few scientists understand this basic point: It is only possible to interpret results of statistical analyses at face value, if every step was planned in advance. Specifically:

  • Sample size has to be picked in advance. It is not ok to keep analyzing the data as more subjects are added, stopping when the results looks good.
  • Any methods used to normalize the data or exclude outliers must also be decided in advance. It isn't ok to analyze various subsets of the data until you find results you like.
  • And finally, of course, the statistical methods must be decided in advance. Is it not ok to analyze the data via parametric and nonparametric methods, and pick the results you like.

Exploratory methods can be useful to, well, explore. But then you can't turn around and run regular statistical tests and interpret the results in the usual way.

$\endgroup$
8
  • 5
    $\begingroup$ I think John Tukey might disagree en.wikipedia.org/wiki/Exploratory_data_analysis ;o) $\endgroup$
    – Dikran Marsupial
    Jan 27, 2011 at 17:32
  • 3
    $\begingroup$ I would partially disagree here. I think the caveat that people miss is that the appropriate conditioning operations are easy to ignore for these kinds of issues. Each of these operations change the conditions of the inference, and hence, they change the conditions of it applicability (and therefore to its generality). These is definitely only applicable to "confirmatory analysis", where a well defined model and question have been constructed. In exploratory phase, not looking to answer definite questions - more looking to build a model and come up with hypothesis for the data. $\endgroup$
    – probabilityislogic
    Jan 27, 2011 at 18:05
  • $\begingroup$ I edited my answer a bit to take into account the comments of Dikran and probabilityislogic. Thanks. $\endgroup$
    – Harvey Motulsky
    Jan 28, 2011 at 14:52
  • 1
    $\begingroup$ For me, the "excluding outliers" is not as clearly wrong as your answer implies. For example, you may only be interested in the relationships at a certain range of responses, and excluding outliers actually helps this kind of analysis. For example, if you want to model "middle class" income, then excluding the super rich and impoverished outliers is a good idea. It is only the outliers within your frame of inference (e.g. "strange" middle class observations) were your comments apply $\endgroup$
    – probabilityislogic
    Jan 29, 2011 at 6:10
  • 2
    $\begingroup$ Ultimately the real problem with the issues raised in the initial answer is that they (at least partially) invalidate p-values. If you are interested in quantifying an observed effect, one should be able to do any and all of the above with impunity. $\endgroup$
    – russellpierce
    Jan 29, 2011 at 19:24
9
votes
$\begingroup$

Tongue firmly in cheek: For frequentists, the Bayesian concept of probability; for Bayesians, the frequentist concept of probability. ;o)

Both have merit of course, but it can be very difficult to understand why one framework is interesting/useful/valid if your grasp of the other is too firm. Cross-validated is a good remedy as asking questions and listening to answers is a good way to learn.

$\endgroup$
2
  • 2
    $\begingroup$ I rule I use to remember: Use probabilities to predict frequencies. Once the frequencies have been observed, use them to evaluate the probabilities you assigned. The unfortunately confusing thing is that, often the probability you assign is equal to a frequency you have observed. One thing I have always found odd is why do frequentists even use the word probability? wouldn't it make their concepts easier to understand if the phrase "the frequency of an event" was used instead of "the probability of an event"? $\endgroup$
    – probabilityislogic
    Jan 28, 2011 at 16:58
  • $\begingroup$ Interestingly, cross validation can be seen as a Monte Carlo approximation to the integral of a loss function in Decision Theory. You have an integral $\int p(x) L(\textbf{x}_{n},x) dx$ and you approximate it by $\sum_{i=1}^{i=n} L(\textbf{x}_{[n-i]},x_i)$ Where $\textbf{x}_{n}$ is data vector, and $\textbf{x}_{[n-i]}$ is the data vector with the ith observation $x_i$ removed $\endgroup$
    – probabilityislogic
    Jan 30, 2011 at 1:30
8
votes
$\begingroup$

From my personal experience the concept of likelihood can also cause quite a lot of stir, especially for non-statisticians. As wikipedia says, it is very often mixed up with the concept of probability, which is not exactly correct.

$\endgroup$
7
votes
$\begingroup$

Fiducial inference. Even Fisher admitted he didn't understand what it does, and he invented it.

$\endgroup$
6
votes
$\begingroup$

What do the different distributions really represent, besides than how they are used.

$\endgroup$
3
  • 3
    $\begingroup$ This was the question I found most distracting after statistics 101. I would encounter many distributions with no motivation for them beyond "properties" that were relevant to topics at hand. It took unacceptably long to find out what any represented. $\endgroup$
    – sesqu
    Jan 29, 2011 at 22:12
  • 1
    $\begingroup$ Maximum entropy "thinking" is one method which helps understand what a distribution is, namely a state of knowledge (or a description of uncertainty about something). This is the only definition that has made sense to me in all situations $\endgroup$
    – probabilityislogic
    Jan 30, 2011 at 4:57
  • $\begingroup$ Ben Bolker provides a good overview of this in the 'beastiary of distributions' section of Ecological Models and Data in R $\endgroup$
    – David LeBauer
    Sep 23, 2011 at 5:04
5
votes
$\begingroup$

I think the question is interpretable in two ways, which will give very different answers:

1) For people studying statistics, particularly at a relatively advanced level, what is the hardest concept to grasp?

2) Which statistical concept is misunderstood by the most people?

For 1) I don't know the answer at all. Something from measure theory, maybe? Some type of integration? I don't know.

For 2) p-value, hands down.

$\endgroup$
1
  • $\begingroup$ Measure theory is neither a field of statistics nor hard. Some types of integration are hard, but, once again, that isn't statistics. $\endgroup$
    – isekaijin
    Jan 29, 2011 at 22:12
5
votes
$\begingroup$

Confidence interval in non-Bayesian tradition is a difficult one.

$\endgroup$
5
votes
$\begingroup$

I think people miss the boat on pretty much everything the first time around. I think what most students don't understand is that they're usually estimating parameters based on samples. They don't know the difference between a sample statistic and a population parameter. If you beat these ideas into their head, the other stuff should follow a little bit easier. I'm sure most students don't understand the crux of the CLT either.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.