Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Taleb's book "The Black Swan" was a New York Times best seller when it came out several years ago. The book is now in its second edition. After meeting with statisticians at a JSM (an annual statistical conference), Taleb toned down his criticism of statistics somewhat. But the thrust of the book is that statistics is not very useful because it relies on the normal distribution and very rare events: "Black Swans" don't have normal distributions.

Do you think this is valid criticism? Is Taleb missing some important aspects of statistical modeling? Can rare events be predicted at least in the sense that probabilities of occurrences can be estimated?

share|improve this question
1  
possible duplicate of What is the community's take on the Fourth Quadrant? –  Andy W Sep 9 '12 at 13:53
1  
Also IMO I don't think a "black-swans" tag will be very useful. Sort of inside jargon to this particular author that should IMO be avoided. Rare-events seems sufficient to me, but you would know the lingo better than me for sure. –  Andy W Sep 9 '12 at 13:56
    
@AndyW While black swans may be a term coined by Taleb it is becoming a commonly used term for rare events and so may pertain more broadly than just Taleb's book. –  Michael Chernick Sep 9 '12 at 14:03
1  
I don't necessarily have a problem w/ creating a 'black-swans' tag or a 'rare-events' tag, however, I strongly encourage people to create a tag wiki excerpt, at a minimum, when creating a new tag. Future users will need some guidance regarding the meaning & proper use of the tag. It also might be useful to create both & immediately make b-s a synonym for r-e, to avoid running into this issue accidentally in the future. –  gung Sep 10 '12 at 20:22
    
@gung Thanks. I am too new to be very adept at understanding all the intricacies of tagging. –  Michael Chernick Sep 10 '12 at 20:51
show 1 more comment

9 Answers

up vote 35 down vote accepted

I read the Black Swan a couple of years ago. The Black Swan idea is good and the attack on the ludic fallacy (seeing things as though they are dice games, with knowable probabilities) is good but statistics is outrageously misrepresented, with the central problem being the wrong claim that all statistics falls apart if variables are not normally distributed. I was sufficiently annoyed by this aspect to write Taleb the letter below:

Dear Dr Taleb

I recently read "The Black Swan". Like you, I am a fan of Karl Popper, and I found myself agreeing with much that is in it. I think your exposition of the ludic fallacy is basically sound, and draws attention to a real and common problem. However, I think that much of Part III lets your overall argument down badly, even to the point of possibly discrediting the rest of the book. This is a shame, as I think the arguments with regard to Black Swans and "unknown unknowns" stand on their merits without relying on some of the errors in Part III.

The main issue I wish to point out - and seek your response on, particularly if I have misunderstood issues - is your misrepresentation of the field of applied statistics. In my judgement, chapters 14, 15 and 16 depend largely upon a straw man argument, misrepresenting statistics and econometrics. The field of econometrics that you describe is not the one that I was taught when I studied applied statistics, econometrics, and actuarial risk theory (at the Australian National University, but using texts that seemed pretty standard). The issues that you raise (such as the limitations of Gaussian distributions) are well and truly understood and taught, even at the undergraduate level.

For example, you go to some lengths to show how income distribution does not follow a normal distribution, and present this as an argument against statistical practice in general. No competent statistician would ever claim that it does, and ways of dealing with this issue are well established. Just using techniques from the very most basic "first year econometrics" level, for example, transforming the variable by taking its logarithm would make your numerical examples look much less convincing. Such a transformation would in fact invalidate much of what you say, because then the variance of the original variable does increase as its mean increases.

I am sure there are some incompetent econometricians who do OLS regressions etc with an untransformed response variable the way you say, but that just makes them incompetent and using techniques which are well established to be inappropriate. They would certainly have been failed even in undergraduate courses, which spend much time looking for more appropriate ways of modelling variables such as income, reflecting the actual observed (non-Gaussian) distribution.

The family of Generalized Linear Models is one set of techniques developed in part to get around the problems you raise. Many of the exponential family of distributions (eg Gamma, Exponential, and Poisson distributions) are assymetrical and have variance that increases as the centre of the distribution increases, getting around the problem you point out with using the Gaussian distribution. If this is still too limiting, it is possible to drop a pre-existing "shape" altogether and simply specify a relationship between the mean of a distribution and its variance (eg allowing the variance to increase proportionately to the square of the mean), using the "quasi-likelihood" method of estimation.

Of course, you could argue that this form of modelling is still too simplistic and an intellectual trap that lulls us into thinking the future will be like the past. You may be correct, and I think the strength of your book is to make people like me consider this. But you need different arguments to those that you use in chapters 14-16. The great weight you place on the fact that the variance of the Gaussian distribution is constant regardless of its mean (which causes problems with scalability), for instance, is invalid. So is your emphasis on the fact that real-life distributions tend to be assymetric rather than bell-curves.

Basically, you have taken one over-simplification of the most basic approach to statistics (naïve modelling of raw variables as having Gaussian distributions) and shown, at great length, (correctly) the shortcomings of such an oversimplified approach. You then use this to make the gap to discredit the whole field. This is either a serious lapse in logic, or a propaganda technique. It is unfortunate because it detracts from your overall argument, much of which (as I said) I found valid and persuasive.

I would be interested to hear what you say in response. I doubt I am the first to have raised this issue.

Yours sincerely

PE

share|improve this answer
13  
Did you receive a response? –  cardinal Sep 10 '12 at 19:45
    
Yes. Many statisticians have criticized the normal distribution before! Just one example: The famous danish statistician Georg Rasch (known for Rasch models in psychometrics!) was known to say, when he had drunk to much, that "all books mentioning the normal distribution should be burnt"! –  kjetil b halvorsen Sep 10 '12 at 19:46
1  
++ Peter. A very good letter!! –  Michael Chernick Sep 10 '12 at 20:44
6  
@cardinal - I got an automatic response to the effect of "since the global financial crisis I have been receiving too many emails to respond to". –  Peter Ellis Sep 17 '12 at 19:48
add comment

Saying that " the thrust of the book is that statistics is not very useful " is inaccurate, I think. Having read the book, what he appears to be saying is that things like quantitative finance or any sort of securities trading that assumes a normal distribution is fundamentally flawed (actually, in the book, he calls people who claim to use these models to make predictions, "charlatans"). According to Taleb, while the normal distribution does a great job of modelling the values of tangible/physical things (eg. height, weight, life span etc.), systems like the markets are often driven by human emotion and thus, are prone to large swings that normal distributions cannot accurately predict.

I don't understand statistics well, and until reading the answers here, I'd never heard of things like extreme value theory. Regardless, The Black Swan and Fooled By Randomness seem to have similar premises, which is "normal distribution not always OK". I don't recall him defaming the entire field of statistics.

share|improve this answer
8  
(+1) For the first sentence. However, Taleb is more of a (self-absorbed) polemicist than a serious intellectual. I only have the first edition of BS; his commentary on statistics is overstated and uninformed in many places, but the attempted thesis of the text is more than what's quoted in the first sentence, as you point out. –  cardinal Sep 9 '12 at 20:18
1  
+1 I think the key is when talking about finance. A link in the NY Times that quotes from the first chapter, I believe: nytimes.com/2007/04/22/books/chapters/0422-1st-tale.html –  Wayne Sep 9 '12 at 20:23
4  
Option pricing for instance started with normal assumptions on log-returns but know a days many people account for kurtosis with more complex jump diffusion/stochastic volatility models. –  muratoa Sep 9 '12 at 21:08
2  
+1 Welcome to our site! Thank you very much for sharing your thoughts. –  whuber Sep 10 '12 at 19:40
2  
Having read the book and written my own critique (I may have an amazon custoner's review on it along wtih thousands of others) I think that Taleb has finance and the stock market as his prime examples but he does take a more general view of the so-called Black Swans and takes a very uninformed view of statistics and the statistical profession (at least in the first edition). Misuse of the normal distribution can be a valid criticism for how some individuals may model rare events. But many of us do it the right way and there is some value in the results from the proper approach. –  Michael Chernick Sep 10 '12 at 20:42
show 1 more comment

I've not read the book, but as stated the criticism seems pretty unreasonable to me. If extreme events are important, then statistics has appropriate tools in the toolbox, such as extreme value theory, and a good statistician will know how to use them (or at least find out how to use them and will be sufficiently engaged with the purpose of the analysis to look). The criticism seems to be "statistics is bad because there are bad statisticians that only know about normal distributions".

share|improve this answer
add comment

I haven't read the Black Swan, but if his criticism of statistics is really as simple as you say, then it's ridiculous. Obviously some statistics relies on the Normal distribution, but much does not.

Can rare events be modeled? Of course they can. The real question is how well they can be modeled. And that question will have different answers in different fields, based on how much we know about the rare events and their antecedents.

In today's NY Times Magazine there's an interesting article by Nate Silver on how weather forecasting has improved in the last decade or so. This includes better modeling of rare events such as hurricanes.

Is the book worth reading?

share|improve this answer
1  
I have read the book and made similiar counterarguments such as yours and Dikran's. Taleb seemed very naive. There was a session involving him at the JSM a few years ago. I think it was in Washington. The second edition came out after that and is a little more reasonable. Taleb has some interesting things to say about specific "Black Swans" and he knows a lot about economics. I think it is worth reading and the second edition is better. –  Michael Chernick Sep 9 '12 at 13:39
add comment

I also have not read the book, but there is no way that his point can be as simplistic as saying that there are distributions with fatter tails than the normal distribution. This would be a comment to the other answers, but I have not accumulated enough accolades on this website.

From Wikipedia:

"He states that statistics is fundamentally incomplete as a field as it cannot predict the risk of rare events..."

This question is also quite similar to What is the community's take on the Fourth Quadrant?

share|improve this answer
    
Nice find, that question/answer may in fact be a duplicate. –  Andy W Sep 9 '12 at 13:50
1  
I wasn't aware of the post on "Fourth Quadrant". There John Cook points to the JSM where Taleb spoke and provides a link to his blog comments on the talk. The post is neasrly a duplicate to mine but the discussion is short there. So I think it is worth continuing this one. –  Michael Chernick Sep 9 '12 at 13:57
1  
I don't think it is true that statistics can't predict the risk of rare event. It is difficult because there generally isn't much information that is useful for this task in the data in the same way that there is for estimating central tendency. So it isn't so much a problem with statistics as with the data. –  Dikran Marsupial Sep 9 '12 at 15:03
2  
@dikran: I agree with you, and I think his books are troll books. But I would still lose horribly in a debate against him, in the same way that I would lose against an experienced intelligent design debater. –  draft Sep 9 '12 at 19:27
2  
There is more to the book than just the statisitcs issue - his arguments about "unknown unknowns" (so called black swans) and about the "ludic fallacy" (treating the world as though it is a dice game with knwon probabilities) are pretty much independent of his misguided critique of statistics as depending on normal distributions. You could drop all of the statistics chapters and greatly improve the book. –  Peter Ellis Sep 10 '12 at 19:43
show 2 more comments

I did read "The Black Swan", I did enjoy it, and I am a statistician. I didn't find its "criticism of statistics" unbearable, at all. Point by point:

  1. Taleb did not invent the concept of the black swan. It had been a favored example in philosophical thought for quite a while!
  2. Taleb is not so much criticizing "statistics", as certain (bad) applications of it.
  3. The book was a bestseller. It was not directed toward statisticians, but to the general public. It did very well in teaching that public about things statisticians knew very well, but many of the other readers (the majority!) did not. So we could learn a lot from that book about how to "sell" statistics.
  4. Most important (for me), Taleb included a lot of references to ancient Greek skeptical philosophy. Nobody else has mentioned that point here, but I think that inclusion was the real selling point of the book!
  5. The book is a literary work, not a technical work. If you want to criticize Taleb for his technical work, go to his homepage and download some of his technical papers.
share|improve this answer
2  
A big +1 for being the first (oops--second) respondent actually qualified to talk about the book! (And for saying some interesting things about it, too.) –  whuber Sep 10 '12 at 19:36
1  
what about his representation of econometrics and statistics as depending on Gaussian distributions? –  Peter Ellis Sep 10 '12 at 19:40
1  
@kjetilbhalvorsen You say you read the book. If you read it carefully it is not possible to miss the attach on the statistics profession. Having a degree in mathematics means nothing regarding a persons knowledge of statistics. Many mathematicians got their dgrees without taking a single statistics course. Other may only have had one very elementary course. I have know mathematicians that have taught statistics and/or probability and not really been qualified to do so. –  Michael Chernick Sep 10 '12 at 20:48
1  
Michael Chernik: Might be so, but I still stand by criticizing a work by its strong points, at least not only by its weak points! and, a literary work must be read as such. Taleb should be gratulated for making Black Swans into a concept many people understand. It is an important concept. All the journalists ridiculing Rumsfeldt for talking about "unknown unknowns" shows that. Rumsfeldt was only using a concept he had learnt from the military officers! At least they knew about Black Swans. –  kjetil b halvorsen Sep 10 '12 at 21:02
3  
"A literary work" is only an excuse for misrepresenting reality if what Taleb wrote was a novel. Not going into a technical treatment is excusable, misrepresenting something wholesale is less so. –  Fomite Dec 22 '12 at 5:40
show 2 more comments

I don't think Taleb would actually say that statistical techniques relying on the Gaussian distribution are not useful. His point in the book was that they are highly useful for many (but not all) physical or biological processes and modeling. He makes some good points and some bad (The Black Swan and Linked were the beginning of the "everything is a power law!" plague that still haunts us today), but it's important to remember that the book is a collection of literary and philosophical essays meant for the lay person.

That said I think Taleb likes to aggravate people. You can see this in his battle with Myron Scholes. In this case it may have been useful as statistical education at the undergrad level, and sometimes at the graduate level, sort of flits over the assumption of Gaussian distributions. I imagine during his years in finance he came across a large number of quants with a great knowledge of Black-Scholes and other techniques but who did not consider underlying assumptions like the distribution. I suspect Taleb was poking at the educational establishment for a failure to properly educate.

share|improve this answer
    
+1 for your interesting comments. But I disagree about his view of the normal distribution. He seems to think that statisticians use it where it doesn't apply and he is very wrong to characterize statisticians that way. He may know better now. Yes he clearly has a writing style that is intended to provoke and irritate people. –  Michael Chernick Sep 10 '12 at 21:12
    
I don't have the book with me know, so this is from memory. Surely some of his anger comes from bad experience with people. He tells that t some point "somebody" (will edit when I get the book and can find names) yelled at him "I am a member of the national academy of sciences"! That is'nt exactly an argument, and "somebody" needed to be laughed at for using it as such. –  kjetil b halvorsen Sep 10 '12 at 21:59
    
It is possible I put an unconcious positive spin on what I was reading, but I distinctly rmember NTT giving several examples where the Gaussian distribution made sense, such as his coffee cup. I gave the book away so I can't go back and re-evaluate that. Taleb's popular writing is much more polemic than his professional writing, at least what I have read of the latter. –  Fraijo Sep 11 '12 at 16:54
1  
I don't think we are arguing that Taleb thinks the normal distribution never makes sense. It is just that for the examples he thinks are important he thinks it is improper to use it. He is right about that but wrong in thinking that most statisticians do use it in those situations. –  Michael Chernick Sep 11 '12 at 18:23
    
Interesting not just how many commentators haven't read the book (I, for one, skimmed it and that was plenty) but how many have read it, just haven't seen fit to keep it handy. "I gave it away"; "I left it in the attic"; etc. –  rolando2 Dec 22 '12 at 13:17
add comment

I strongly recommend Dennis Lindley's review of this book. It contains a number of devastating arguments against the poor and arrogant exposition of ideas in the book:

http://onlinelibrary.wiley.com/doi/10.1111/j.1740-9713.2008.00281.x/abstract

The Black Swan is another example where being a "Best-seller" does not guarantee high quality content.

share|improve this answer
add comment

Without reading the book I feel that Gaussian bells fail because they have never given a clear definition of "probability density"; besides that, they never give a complete set of points of Lorenz curves that include at the same time the total of distributed variable and the total of populations that perceive the former one. If "density" is used it is necessary to explain with respect to what variable; for example if you speak of kilograms per liter it refers to a density of weight related to volume. That step is not given by Gaussian theory in textbooks. No wonder that young people does not understand properly statistics.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.