What book is the most thorough treatment of fundamental concepts in statistics? I am not asking for a book on details of the methods of calculations and procedures, I am mainly interested in a book that thoroughly explains the foundational concepts ... an intuitive/illustrated/visual approach to the core ideas ... rather than load of mathematics equations etc. Size of the book is no problem ... even a large multi-volume text would do ... heck even a web resource would do.
It's hard to know exactly what you're looking for based on your post. Maybe you can edit it to clarify a little. I will say that to really understand statistics well, then you'll need to learn some math.
For fairly broad, low-level, introductory concepts, both
- Gonick and Smith, A Cartoon Guide to Statistics, and
- D. Huff, How to Lie with Statistics
are light, easy reads that present a lot of the core ideas. Another book directed at a more "popular" audience that I think every person should read is J. A. Paulos' Innumeracy. It is not about probability or statistics, per se, and has more elementary probability than statistics, but it's framed in a way that I think most people can easily relate to.
If you have some calculus background and want to understand (introductory, frequentist) theoretical statistics, find a copy of Mood, Graybill and Boes, Introduction to the Theory of Statistics, 3rd. ed. It's old, but in my opinion, still better than any of the more "modern" treatments. But, it's a book for which you'll have to be comfortable with mathematical notation.
For a "modern" view of applied statistics and the interface between it and machine learning, along with good examples, and good intuition, then Hastie et al., Elements of Statistical Learning, is the most popular choice. Many people also tend to like Harrell's Regression Modeling Strategies, which is a solid book, though I'm apparently not quite as big of a fan as others tend to be. Again, in both cases, you'll need to at least be comfortable with some calculus, linear algebra, and standard math notation.
If you're interested in the philosophy of Statistics, you can't do much better than Abelson's "Statistics As Principled Argument".
I think Harvey Motulsky's Intuitive Biostatistics is pretty good for non-mathematical "intuitive" explanations to basic statistical methods most commonly employed in the biological and medical sciences.
I like Kennedy's Guide to Econometrics, which treats every topic on three levels, the first of which is a non-technical description, in so far as this is possible.