I took a probability course in university a few years ago, but I'm going through some machine learning algorithms now and some of the math is just befuddling.

Specifically right now, I'm learning the EM algorithm (expectation maximization) and it seems that there is a large disconnect between what is required and what I have.

I'm not asking for a book or a website, but what is the way to learn enough of these topics to be able to get a thorough understanding of the algorithms that use them? Is it required to go through a book and do hundreds of exercises? Or is that overkill in this sense?

edit: If this is the wrong location for this question, please vote to migrate :)

  • $\begingroup$ Wasn't aware of CV! Is there a way to migrate the question over to there? $\endgroup$ – coffee Aug 7 '11 at 17:03
  • 2
    $\begingroup$ I've voted to migrate. :) $\endgroup$ – J. M. is not a statistician Aug 7 '11 at 17:04
  • $\begingroup$ The best way to learn something is to break it into small pieces. Why not ask on this site about one or two specific question(s) or concepts that are bothering you? $\endgroup$ – charles.y.zheng Aug 8 '11 at 20:49

Many of the books and the online introductions to machine learning provide a bit of introduction to their needed probability within their content, so I would start with one or a few books to that kind. Off the top of my head I can think of Statistical Pattern Recognition (may be because I learned EM there) and The Elements of Statistical Learning.

My real advice would be the Statistical Data Mining Tutorials by Andrew Moore. That was the site that bridged the gap I had before I started my PhD (coming from an engineering background). I know you said you are not asking about a web site, but do have a look at the Probability for Data Miners there and the other Probability slides before you decide. And have a look at the Gaussian Mixture Models for EM.

Is it required to go through a book and do hundreds of exercises?

I don't think so. Probability calculations in machine learning tend to cluster around a few well known paths. Having a strong grasp of the Gaussian single and multidimensional distribution and studying a few explanations of EM should get you pretty far. And linear algebra. You will need a lot of linear algebra.


Artificial Intelligence has gained importance in the last decade with a lot depending on the development and integration of AI in our daily lives. The progress that AI has already made is astounding with the self-driving cars, medical diagnosis and even betting humans at strategy games like Go and Chess.

The future for AI is extremely promising and it isn’t far from when we have our own robotic companions. This has pushed a lot of developers to start writing codes and start developing for AI and ML programs. However, learning to write algorithms for AI and ML isn’t easy and requires extensive programming and mathematical knowledge.

Mathematics plays an important role as it builds the foundation for programming for these two streams.

There are many reasons why mathematics is important for machine learning. Some of them are below:

Selecting the right algorithm which includes giving considerations to accuracy, training time, model complexity, number of parameters and number of features. Choosing parameter settings and validation strategies. Identifying underfitting and overfitting by understanding the Bias-Variance tradeoff. Estimating the right confidence interval and uncertainty.

What type of math is required for machine learning?

Math is absolutely necessary for the study of Machine Learning or Artificial Intelligence. Any deeper understanding of the concepts and algorithms in ML requires some basic maths knowledge.

Three main mathematical theories: Linear Algebra, Multivariate Calculus and Probability Theory.

Linear Algebra –

Linear algebra notation is used in Machine Learning to describe the parameters and structure of different machine learning algorithms. This makes linear algebra a necessity to understand how neural networks are put together and how they are operating.

It covers topics such as:

Scalars, Vectors, Matrices, Tensors Matrix Norms Special Matrices and Vectors Eigenvalues and Eigenvectors Multivariate Calculus –

This is used to supplement the learning part of machine learning. It is what is used to learn from examples, update the parameters of different models and improve the performance.

It covers topics such as:

Derivatives Integrals Gradients Differential Operators Convex Optimization Probability Theory –

The theories are used to make assumptions about the underlying data when we are designing these deep learning or AI algorithms. It is important for us to understand the key probability distributions,

It covers topics such as:

Elements of Probability Random Variables Distributions Variance and Expectation Special Random Variables How to learn Maths for Machine Learning quickly?

The self-starter way of learning math for data science is to learn by “doing shit.” Even so, you’ll want to learn or review the underlying theory up front. You don’t need to read a whole textbook, but you’ll want to learn the key concepts first.

As soft prerequisites, I assume basic comfortability with linear algebra/matrix calculus (so you don’t get stuck on notation) and introductory probability.

If you want to learn math for machine learning deeply then there are n number of courses available online, such as,

Khan Academy's Linear Algebra, Probability & Statistics, Multivariable Calculus, and Optimization.

Mathematical Foundation For Machine Learning and AI on eduonix

Learn Machine Learning Maths Behind on udemy

Coding the Matrix: Linear Algebra through Computer Science Applications by Philip Klein, Brown University.

Larry Wasserman’s book — All of statistics: A Concise Course in Statistical Inference.

Remember that you learn the best by doing, and sadly these courses don’t contain enough assignments and homework

What I recommend is, Mathematical Foundation For Machine Learning and AI- This course is not a full math curriculum; it’s not designed to replace school or college math education. Instead, it focuses on the key mathematical concepts that you’ll encounter in studies of machine learning.

What you'll learn:

And much more……

At the end of this course, you will not have not only the knowledge to build your own algorithms, but also the confidence to actually start putting your algorithms to use in your next projects.

The course also comes with projects and quizzes to help solidify your knowledge of the mathematical concepts.

It is designed to fill the gaps for students who missed these key concepts as part of their formal education, or who need to refresh their memories after a long break from studying math.

I think this course is a lot better than investing 2 to 3 months skimming through the material at the beginning and then forgetting half of what you learned by the time you encounter it.

Try to understand the basic concepts shown and always remember to have fun!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy