- What are good online or offline resources that give an overview of the history of statistics and the main breakthrough in statistics until now?
I have read the page on History of Statistics on Wikipedia.
Read Stephen Stigler's books, The History of Statistics and Statistics on the Table. This will give you an overview of the field from 1750 through about 1930. For pre-1750 history, Anders Hald is well known (but I haven't read his work).
The edited volumes of seminal papers, Breakthroughs in Statistics, provide reprints of key papers along with short introductions by eminent modern statisticians explaining the meaning and impacts of these papers.
There are incredible primary resources, especially in Europe, for researching statistical history, especially the more difficult-to-find sources prior to 1750. For example, you can find Fermat's collected works, Huygens' 17th century treatise on probability, and high-resolution scans of 16th and 15th century treatises, such as Tartagia's General trattato di numeri, et misure. Be prepared to read old French, old Italian, Latin, and even Dutch. (Those with a modest command of the modern versions of these languages won't have difficulties.)
Additional information can be obtained at the Journal for the History of Probability and Statistics and in early reviews of the history of probability and statistics. But your best bet is to search. Searches turned up all these resources several years ago, and several of them I had to search for again because their URLs had changed.
David Salsburg "The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century" is an easy and entertaining read. It might not be focused on breakthroughs per se, but it tells the story featuring the Pearsons, Gosset, Fisher, Neyman and others.
I recommend the work of Stephen Stigler.
The Politics of Large Numbers: A History of Statistical Reasoning By Alain Desrosières.
It covers a wide range of topics, in several countries (France, Germany, Great-Britain, United States), from the 17th/18th centuries to about 1970-1980: the history of institutions collecting and treating statistical data, how statistical methods evolved (probabilities, central limit theorem, regression, survey methodology...), the influence of political or social considerations (epidemics, poverty, unemployment, eugenics...), etc.
It was originally published in 1993 in French, and later translated in English. The most recent edition in French is from 2010, and apparently 2002 for the edition in English.