If ever there was a case where this become clear is with the Monty Hall problem. Even the great Paul Erdos got fooled by this problem. My question which may be difficult to answer is what is it about probability that we can be so confident of an answer we get uwing an intuitive argument and yet be so wrong. Benford law on first digits and the waiting time paradox are other famous examples like this.
There are two main approaches to understanding this question. The first (and I believe most successful) is the literature on cognitive biases (see this LessWrong link).
Much has been written on this topic and it would be too presumptuous to summarize it here. In general, this just means that the cognitive machinery humans are endowed with through the evolutionary process employs lots of heuristics and shortcuts to make survival decisions more efficiently. These survival decisions mostly applied to ancestral environments which we rarely face anymore, and so the frequency with which we face scenarios where our heuristics fail might be expected to increase.
Humans, for example, are great at generating beliefs. If positing a new belief costs very little, but failing to employ a belief that would have lead to survival has high cost (even if the belief is in general incorrect), then one would expect to see lots of rationalization and low evidence barriers to believing propositions (which is what we do see with humans). You also get behaviors such as probability matching for similar reasons.
One could go on at length describing all of the fascinating ways that we deviate from opimal decision making. Check Kahneman's recent book Thinking, Fast and Slow and Dan Ariely's book Predictably Irrational for popular, readable accounts with lots of examples. I recommend reading some of the sequences at LessWrong for more principled discussion of cognitive bias, and lots of interesting academic literature reviews regarding steps one can take to avoid these biases in certain circumstances.
The other approach to this problem is (I think) far more tenuous. This is the notion that probability is itself not the correct normative theory for dealing with uncertainty. I don't have time to annotate some sources for this now, but I will update my answer later with some discussion of this view.