I’d like to learn about probability theory, measure theory and finally machine learning. Where do I start? [closed]

Question

I’d like to learn about probability theory, measure theory and finally machine learning. My ultimate goal is to use machine learning in a piece of software.

I studied calculus and very basic probability in college but that’s pretty much it. Do you know some online courses or books that I could use to learn about these subjects. I’ve found many resources on the web but they all seem targeted to an expert audience. I know it’s going to take some time but where do I start if I’d like to learn from the beginning?

Answer 1

I think there exists two very good and popular references for you (I started with these ones as well having a background of master in actuarial science):

1. An Introduction to Statistical Learning (with application in R) by Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani. It is freely available on the site, pretty comprehensive and easy to understand with pratical examples. You can start learning many things even without a very strong statistical background, this reference is good for various profils and includes adequate number of popular algorithms together with its implementation in R without going deep into the mathematical details.

2. The Elements of Statistical Learning by Trevor Hastie, Robert Tibshirani, Jerome Friedman. Comparing to the first one, this book goes deeper into the mathematical aspects if you want to explore further on the particular algorithms that you find useful for you. (is is free as well)

And of course Cross Validated is one of the best sources where you can learn many things, for me: best pratices, statistical misunderstanding and misuse, and many more. After several years of learning at schools / universities as well as seft-learning, I found that my knownledge is too limited when I first went to Cross Validated. I continue to go here every day since the first visit and learn so much.

Answer 2

Here are a couple of free online courses that I've heard are highly recommended:

• http://projects.iq.harvard.edu/stat110/home (Depending on your current comfort with probability theory. Dr. Blitzstein's course became very popular at Harvard even for those who weren't into stats/probability. I've watched a few of the lectures for my own review and found them very helpful. )
• https://www.coursera.org/learn/machine-learning (This is the current version of one of Stanford's first massive online courses by Andrew Ng, who ended up co-founding Coursera. I've been meaning to take this course, but haven't had the time.)

Answer 3

you don't need measure theory. Measure theory is used by mathematicians to justify other mathematical procedures eg taking limits of integrals approximations. Most engineers would not have studied measure theory, they would just use the results. The math knowledge required for ML is roughly characterised by being able to integrate a multivariate Gaussian- If you are confident about that then you probably have the multivariable calculus,linear algebra and probability theory background necessary.

I would recommend Think Stats by Allen Downey - which aims to teach probability/statistics to programmers. The idea is to leverage programming expertise to do simulations and therefore understand probability theory/statistical methods. allen downey blog (he has written others ) Think stats (free) pdf)

Answer 4

Since you're interested in machine learning, I'd skip probability and mesaure, and jump right into the ML. Andrew Ng's course is a great place to start. You can literally finish it in two weeks.

Play with what you've learned for a few weeks, then go back to the roots and study some probabilities. If you're an engineer, then I'm puzzled with how you managed to skip in in college. It used to be the required course in engineering. Anyhow, you can catch up by taking MIT OCW course here.

I don't think you need measure theory. Nobody needs measure theory. Those who do, they won't come here to ask, because their advisor will tell them which course to take. If you don't have an advisor then you definitely don't need it. Tautology, but true.

The thing with a measure theory's that you can't learn it by "easy reading". You have to do the exercises and problems, basically, do it hard way. That's virtually impossible outside of the class room, in my opinion. The best option here is to take a class at the local college, if they offer such. Sometimes, PhD level probabilities course will do the measure and probabilities in one class, which is probably the best deal. I would not recommend taking a pure measure theory class in Math department, unless you really want to torture yourself, though in the end you'd be greatly satisfied.

Answer 5

For machine learning, I think Machine Learning: The Art and Science of Algorithms that Make Sense of Data by Peter Flach can be a good resource to start with. It gives a general introduction to machine learning with intuitive examples, and is suitable for beginners. I like this book particularly because of the last chapter, which deals with machine learning experiments. While learning about machine learning, getting to know different models is not enough, and one should be able to compare different machine learning algorithms. I think this book has made it easier to understand how to compare those algorithms. Lecture slides can be found here.

Answer 6

To add to the excellent suggestions above, I would say if you are interested in getting a firm grasp on more basic concepts in probability and statistics, "From Algorithms to Z-Scores: Probabilistic Computing in Statistics" is an excellent primer on using computers to understand some of the most important beginner/intermediate concepts in probability theory and stochastic processes. I'll also second either "An Introduction to Statistical Learning" or "Elements of Statistical Learning" (ESL) as an introduction to machine learning (ML). I think ESL in particular is amazing, but it does take a much more mathematics-heavy look at the ML concepts, so if you only consider yourself "okay" at stats, you might want to give it a read once you've gotten more experience with ML.

If you're interested in Machine Learning for the sake of being employed or solving problems, getting hands-on experience is key. Take some introduction to data science/machine learning courses. Andrew Ng does an amazing introduction to machine learning in his course at Coursera here. I would also suggest you download some datasets and start playing around with them. If you haven't already, download R and RStudio (in my opinion, more friendly to beginners than Python or Matlab), and sign up at kaggle and do some of their beginner problems. They have great walkthroughs that can get you using ML with basically no idea what's actually happening, but it gives you an idea about the kind of steps you'd need to take to actually implement an ML solution.

I'd personally encourage a combination of starting off using ML tools without really knowing what they do (using Kaggle datasets or similar); and learning fundamental concepts like cross-validation, overfitting, using confusion matrices, different measures of how good a model is, etc. To me, it's much more important to know how to use the algorithms, and knowing how to identify when things are working/aren't working, than it is to understand how the algorithms work.