- What are good online or offline resources that give an overview of the history of statistics and the main breakthrough in statistics until now?
I have read the page on History of Statistics on Wikipedia.
Read Stephen Stigler's books, The History of Statistics and Statistics on the Table. This will give you an overview of the field from 1750 through about 1930. For pre-1750 history, Anders Hald is well known (but I haven't read his work).
The edited volumes of seminal papers, Breakthroughs in Statistics, provide reprints of key papers along with short introductions by eminent modern statisticians explaining the meaning and impacts of these papers.
There are incredible primary resources, especially in Europe, for researching statistical history, especially the more difficult-to-find sources prior to 1750. For example, you can find Fermat's collected works, Huygens' 17th century treatise on probability, and high-resolution scans of 16th and 15th century treatises, such as Tartagia's General trattato di numeri, et misure. Be prepared to read old French, old Italian, Latin, and even Dutch. (Those with a modest command of the modern versions of these languages won't have difficulties.)
Additional information can be obtained at the Journal for the History of Probability and Statistics and in early reviews of the history of probability and statistics. But your best bet is to search. Searches turned up all these resources several years ago, and several of them I had to search for again because their URLs had changed.
In addition to Stigler's excellent work, there is a book The Science of Conjecture: Evidence and Probability before Pascal by James Franklin; very interesting, less technical than Stigler's books, at least in part because the material wasn't as technical.