I read the Black Swan a couple of years ago. The Black Swan idea is good and the attack on the ludic fallacy (seeing things as though they are dice games, with knowable probabilities) is good but statistics is outrageously misrepresented, with the central problem being the wrong claim that all statistics falls apart if variables are not normally distributed. I was sufficiently annoyed by this aspect to write Taleb the letter below:
Dear Dr Taleb
I recently read "The Black Swan". Like you, I am a fan of Karl Popper, and I found myself agreeing with much that is in it. I think your exposition of the ludic fallacy is basically sound, and draws attention to a real and common problem. However, I think that much of Part III lets your overall argument down badly, even to the point of possibly discrediting the rest of the book. This is a shame, as I think the arguments with regard to Black Swans and "unknown unknowns" stand on their merits without relying on some of the errors in Part III.
The main issue I wish to point out - and seek your response on, particularly if I have misunderstood issues - is your misrepresentation of the field of applied statistics. In my judgement, chapters 14, 15 and 16 depend largely upon a straw man argument, misrepresenting statistics and econometrics. The field of econometrics that you describe is not the one that I was taught when I studied applied statistics, econometrics, and actuarial risk theory (at the Australian National University, but using texts that seemed pretty standard). The issues that you raise (such as the limitations of Gaussian distributions) are well and truly understood and taught, even at the undergraduate level.
For example, you go to some lengths to show how income distribution does not follow a normal distribution, and present this as an argument against statistical practice in general. No competent statistician would ever claim that it does, and ways of dealing with this issue are well established. Just using techniques from the very most basic "first year econometrics" level, for example, transforming the variable by taking its logarithm would make your numerical examples look much less convincing. Such a transformation would in fact invalidate much of what you say, because then the variance of the original variable does increase as its mean increases.
I am sure there are some incompetent econometricians who do OLS regressions etc with an untransformed response variable the way you say, but that just makes them incompetent and using techniques which are well established to be inappropriate. They would certainly have been failed even in undergraduate courses, which spend much time looking for more appropriate ways of modelling variables such as income, reflecting the actual observed (non-Gaussian) distribution.
The family of Generalized Linear Models is one set of techniques developed in part to get around the problems you raise. Many of the exponential family of distributions (eg Gamma, Exponential, and Poisson distributions) are asymmetrical and have variance that increases as the centre of the distribution increases, getting around the problem you point out with using the Gaussian distribution. If this is still too limiting, it is possible to drop a pre-existing "shape" altogether and simply specify a relationship between the mean of a distribution and its variance (eg allowing the variance to increase proportionately to the square of the mean), using the "quasi-likelihood" method of estimation.
Of course, you could argue that this form of modelling is still too simplistic and an intellectual trap that lulls us into thinking the future will be like the past. You may be correct, and I think the strength of your book is to make people like me consider this. But you need different arguments to those that you use in chapters 14-16. The great weight you place on the fact that the variance of the Gaussian distribution is constant regardless of its mean (which causes problems with scalability), for instance, is invalid. So is your emphasis on the fact that real-life distributions tend to be asymmetric rather than bell-curves.
Basically, you have taken one over-simplification of the most basic approach to statistics (naïve modelling of raw variables as having Gaussian distributions) and shown, at great length, (correctly) the shortcomings of such an oversimplified approach. You then use this to make the gap to discredit the whole field. This is either a serious lapse in logic, or a propaganda technique. It is unfortunate because it detracts from your overall argument, much of which (as I said) I found valid and persuasive.
I would be interested to hear what you say in response. I doubt I am the first to have raised this issue.
Yours sincerely
PE
