Books for some kind of special self-education in probability and statistics theory?

Question

I am looking for a textbook (or, more likely, a set of textbooks) in English for self-education in probability and mathematical statistics.

I’m a middle-aged person working as data scientist, primarily in market research. I’ve been doing data mining, machine learning, sometimes business modeling, etc. for many years. Now when I have achieved some degree of professional success, I feel that my lack of theoretical education is holding me back from further development.

For example, I can do some relatively sophisticated factor or cluster analysis with Python (mostly using scikit-learn), and I have some basic idea of underlying math behind applied methods, but I’ve never truly understood this math in depth.

Ideally, I’d like to be able to trace the derivation of each statistical test or other application from the very outset—not only in order to comprehend their workings, but to modify them for my situations and to develop plausible derivative applications as well. For example, I’ve written a little module for nominal-ordinal association testing based on some academic papers, but I was only able to convert the described methods into code without real insight into why they work at all. I’d better like to be able to explain every detail of these methods to myself and to any interested person from, say, a high-school level.

I tried to search for the textbooks myself, mostly by investigating search results by Best Textbooks on [Something], but I got completely lost. It is as I should learn some set theory, some combinatorics, some measure theory, some topology, lots of calculus, lots of linear algebra, and probably something else—and this is just to get on with the probability theory. Even if I somehow choose the “best” book on each topic (based on my goals) and get through it, I will be a mess due to huge amount of information and conceptual incompatibility between books (different approaches, scientific communities, explanation methods, notation etc.).

So, I need your advice on what to start with. I’m not afraid of serious and niche literature as long as I can see it can help in my situation; I’m far more afraid of books like σ-Fields Comic Book for Dummies.

Answer 1

Not really an answer, but I want to share my thought :

Because I am in the process of relearning the subject I have learnt years ago, I am also searching for several good books. I learnt the subject mostly for passing the exam, so I feel, this time, I need to relearn it in the correct way. Here is my list (ordered number may show my preference/favoritism) for my own reading :

1. Probability and Statistics for Engineering and the Sciences by Jay L. Devore
2. Introduction to Probability and Statistics for Engineers and Scientists by Sheldon M. Ross. The author has another well-known book titled "A First Course in Probability by Sheldon M.Ross", but as the title suggests, it only covers Probability, not Statistic.
3. Probability and Statistics by Morris H. DeGroot and Mark J. Schervish
4. Probability and Statistics for Engineers and Scientists by Ronald E. Walpole and Raymond H. Myers
5. Probability and Statistics for Engineers by Miller and Freunds, Richard Johnson
6. Probability and Statistics for Engineers and Scientists by Scheaffer, Mulekar and McClave

It's very likely I will use the 1st one, i.e, Devore's book as the main textbook, because I feel it's the most enjoyable to read for me. I will read others (esp Ross or DeGroots') when Devores' doesn't cover certain topics such as Moment Generating Function. In the course of relearning, the order of preference may change, though, esp among the first three books. This thing happens to other subjects I am relearning or planning to relearn : Physics, Calculus (toward multivariable), Electric Circuits, Thermodynamics, etc.