33
$\begingroup$

I'm trying to put together a self-directed math curriculum to prepare for learning data mining and machine learning. This is motivated by starting Andrew Ng's machine learning class on Coursera and feeling that before proceeding I needed to improve my math skills. I graduated from college a while ago so my algebra and statistics (specifically from political science/psychology classes) are rusty.

The answers in the thread Is a strong background in maths a total requisite for ML? only suggest books or classes directly related to machine learning; I have already looked into some of those classes and books and do not know exactly what math subject to study (for instance: what field[s] of math address deriving an equation to "minimize a cost function"?). The other thread suggested (Skills & coursework needed to be a data analyst) only mentions broad categories of skills needed for analyzing data. The thread Introduction to statistics for mathematicians does not apply because I do not already have a degree in math; a similar thread Mathematician wants the equivalent knowledge to a quality stats degree has an incredible list of stats books, but again, I'm looking at starting math from a rusty recollection of algebra and moving up from there.

So, for those that work in machine learning and data mining, what fields of math do you find essential to do your job? What math subjects would you suggest to prepare for data mining and machine learning, and in what order? Here is the list and order I have so far:

  • Algebra
  • Pre-calculus
  • Calculus
  • Linear algebra
  • Probability
  • Statistics (many different sub-fields here, but don't know how to break them out)

As for the data mining and machine learning, through my current job I have access to records on website/app activity, customer/subscription transactions, and real estate data (both static and time-series). I'm hoping to apply the data mining and machine learning to these datasets.

Thank you!

EDIT:

For posterity's sake, I wanted to share a helpful math self-assessment for Geoffrey Gordon's/Alex Smola's Intro to Machine Learning class at CMU.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ In terms of prerequisites for Coursera classes, that info should be available somewhere in their materials. Outside of their classes / more generally, the question of what math you need for stat / ML / DM strikes me as a duplicate. There are several threads on CV that cover this material, including: is-a-strong-background-in-maths-a-requisite-for-ml, & skills-coursework-needed-to-be-a-data-analyst (perhaps among others). $\endgroup$
    – gung - Reinstate Monica
    Commented Aug 30, 2013 at 18:08
  • 1
    $\begingroup$ Please review those threads, the threads that are linked there as closely related, & perhaps search the site. If you still have a question after reading around, come back here & edit this Q to make it more distinctive / specify more precisely what you still need to know that wasn't covered elsewhere. $\endgroup$
    – gung - Reinstate Monica
    Commented Aug 30, 2013 at 18:10

8 Answers 8

Reset to default
17
$\begingroup$

The suggestions that @gung made are certainly worth following up. Having done the coursera course, I think your list is a good start. Some comments:

  1. linear algebra and matrix algebra are the same thing, so drop the latter.
  2. in calculus be sure to include partial differentiation. This is calculus applied to functions of more than one variable (symbolically, if, say, $z$ is a function of $x$ and $y$ then you want $\frac{\partial z}{\partial x}$ rather than $\frac{\rm{d}z}{\rm{d}x}$). Fortunately this isn't difficult.
  3. in calculus you don't need anything beyond basic integration (and maybe not even that). This is fortunate because integration is hard.
  4. add basic optimization, i.e. finding the maximum or minimum of a function, typically a function of more than one variable. An appreciation of gradient descent at the very least is essential.
  5. in terms of difficulty you probably want to be somewhere between the beginning and end of 1st year undergraduate.
  6. try to read some basic probability and statistics texts, online or otherwise, but don't worry too much (basic maths is a prerequisite anyway to understanding probability and statistics). If you do some courses such as the one you suggest you'll figure out what you need to learn and where your interests lie. One thing you don't want to do, at least at first, is spend a lot of time learning about hypothesis testing. You would rather want to steer towards understanding basic statistics — random variables, probability distributions (PFDs, CDFs), descriptive statistics — and then try to understand regression.

I'd add the book Mathematics for Machine Learning by Marc Peter Deisenroth, published 2020, looks like an excellent foundation, including the above and more.

Improve this answer
$\endgroup$
5
$\begingroup$

There are a couple of excellent threads on this forum-- including THIS ONE that I have found particularly helpful for me in terms of developing a conceptual outline of the important skills for data science work.

As mentioned above, there are many online courses available. For example Coursera now has a Data Science Specialization with a number of courses that would probably cover some of the tools you'd need for your work.

Improve this answer
$\endgroup$
3
$\begingroup$

If you are looking to bulk up on machine learning/data mining I would strongly urge optimization/linear algebra/statistics and probability. Here is a list of books for probability. Hope that helps.

Improve this answer
$\endgroup$
3
$\begingroup$

As far as brushing very very basic math skills, i'm using these books:

Elements of Mathematics for Economics and Finance. Mavron, Vassilis C., Phillips, Timothy N This books covers essential math skills (addition substraction), to partial differentiation, integration, matrix and determinants, and a small chapter on optimization, and also differential equation. It's targeted to economics and finance, but it's a small book, the sequence of chapters suits my need, and easy read for me.

Statistical Analysis: Microsoft Excel 2010. Conrad Carlberg Covers basic statistical analysis, to multiple regression, and analysis of covariance, and it uses excel.

Discovering Statistics Using R. Andy Field, Jeremy Miles, Zoë Field. Have not read it yet. It uses R.

Elementary Linear Algebra. Ron Larson, David C. Falvo.

Matrix Methods: Applied Linear Algebra By Richard Bronson, Gabriel B. Costa. covers elementary linear algebra and matrix calculus

Those are the basic math books that i use to relate to data mining / machine learning

Hope this helps

Improve this answer
$\endgroup$
3
$\begingroup$

There are quite a lot of relevant resources listed (and categorized) here, at the so-called "Open Source Data Science Masters".

Specifically for mathematics they list:

  1. Linear Algebra & Programming
  2. Statistics
  3. Differential Equations & Calculus

Pretty generic recommendations, although they do list some textbooks that you might find useful.

Improve this answer
$\endgroup$
2
$\begingroup$
  • Probability and statistics are essential. Some keywords are hypothesis test, multivariate normal distribution, Bayesian inference (joint probability, conditional probability), mean, variance, covariance, Kullback-Leibler divergence, ...
  • Basic linear algebra is essential for machine learning. Topics that you could learn are Eigen decomposition and singular value decomposition. (Of course you should know how to compute a matrix product.)
  • As TooTone already mentioned: optimization is important. You should know what gradient descent is and maybe have a look at Newton's method, Levenberg-Marquardt, Broyden-Fletcher-Goldfarb-Shanno.
  • Calculus is not that important but it might be useful to know how to compute the partial derivatives of functions (Jacobi matrix, Hesse matrix, ...) and you should know what an integral is.
Improve this answer
$\endgroup$
0
$\begingroup$

Linear Algebra, Stats, Calculus. I think you can learn them in tandem w/ ML - or even after the basics. The starter courses / books do a great job with math primer chapters, and you learn the math essentials while learning ML. I made a podcast episode on the math you need for machine learning, and the resources for learning them: Machine Learning Guide #8

Improve this answer
$\endgroup$
0
$\begingroup$

Before Starting any machine learning course go through following mathematics course. Also don't try to dig in single attempt. Learn basic concepts then again brush-up your mathematics skills and repeat:-

Mathematics Topics are as following:-

  • Linear Algebra
  • Probability
  • Basic Calculus
  • Maxima and minima of function
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.