Is a strong background in maths a total requisite for ML?

Question

I'm starting to want to advance my own skillset and I've always been fascinated by machine learning. However, six years ago instead of pursuing this I decided to take a completely unrelated degree to computer science.

I have been developing software and applications for about 8-10 years now, so I have a good handle but I just can't seem to penetrate the maths side of machine learning/probabilities/statistics.

I start looking at learning material and on the first page it might include something which confuses me and immediately sets up a barrier in my learning.

• Is a strong background in maths a total requisite for ML? Should I try and fill in the blanks of my maths before continuing with ML? Can self learning really work for just a developer without any hard computer science background?

Related question:

• Book for reading before Elements of Statistical Learning?

Answer 1

Stanford (Ng) and Caltech (Abu-Mostafa) have put machine learning classes on YouTube. You don't get to see the assignments, but the lectures don't rely on those. I recommend trying to watch those first, as those will help you to find out what math you need to learn. I believe a very similar class with assignments is taught by Andrew Ng on Coursera, which Ng helped to create.

One exception: If I recall correctly, early in the Stanford lectures, Ng does some calculations involving derivatives of traces of products of matrices. Those are rather isolated, so don't worry if you don't follow those calculations. I don't even know what course would cover those.

You do want to have some familiarity with probability, linear algebra, linear programming, and multivariable calculus. However, you need a lot less than what is contained in many complete college classes on those subjects.

Answer 2

Depending on the kind of application, you don't necessarily need a lot of math as a ML practitioner.

As a self-taught programmer (~15 years) and frequent college dropout without much background in math (Calculus III) or statistics, I started with machine learning / data mining with a few resources:

• The book "Mastering Data Mining: The Art and Science of Customer Relationship Management" by Berry and Linoff

• The book "Data Mining Techniques" by the same authors

• R, in particular and the packages party and nnet

I work at a non-profit supporting marketing and operations. Especially in the beginning, I used data mining primarily for direct mail appeals.

Later I took Linear Algebra, Andrew Ng's Machine Learning, Introduction to Statistical Methods (STAT 301) at CSU, etc.

For you I recommend starting with the two books, Andrew Ng's course, and, depending on your application, decision trees (the party package in R).

Answer 3

I think this is a good question actually, and highly topical; I'm not sure if there is an answer however. A recent article stirred a deal of controversy (see here) by suggesting that data science was easy to learn online. One notable thing about most of the case studies mentioned in that article however is that they come from actuarial or mathematical backgrounds.

This is an interesting point, because it indicates that while online courses like Coursera, Stanford and edX are helpful in teaching the specific computer science skills required, it is likely that some mathematical background is essential to understand what the models you're applying are doing. On the other hand, an equally strong argument could be made that these guys were all analytically minded to start with, and this is both why they work in quantitative disciplines as well as why they picked up machine learning easily and won competitions.

I think fundamentally that there is a levels of analysis problem going on here. While mathematical skills are sometimes helpful in understanding the probabilistic roots of the algorithms you're applying, there's an equal argument to be made that good software engineering skills can add just as much by allowing you to do high level analysis and put together parts of algorithms to accomplish your goal even if you don't entirely understand why they are doing that. Generally, data science (and machine learning by association) is an exciting field precisely because of this breadth - you can be a database guy and use brute force to solve problems, or a mathematician who uses simulation, or a computer scientist who leverages well engineered code to put together different algorithms and approaches in an optimal way.

All approaches that add to the prediction are good, so I'd say that learning some mathematics may be a good idea to give you the best chance of success in the field. If you want some good starting points, MIT has a great linear algebra course , with some nice computational applications, that I found easy to understand. They also have other courses on stochastic processes and multivariable calculus that may also be of interest in building up your knowledge.

Answer 4

Is a strong background in maths a total requisite for ML? – an answer and some speculation for ML conceptualized as being statistics ;-)

Around 1990 I had hopes for computer algebra being of assistance, I think it is but it is fairly limited. But it certainly helps with speeding up the learning of math (less need to develope manipulatory skills by practice or try to get by with just being able to do the simple exercises). I found Fred Szabo's Linear Algebra with Mathematica an excellent example of this (but I had already taken an advanced theory level linear algebra course.)

I have been working since 1988 (Utilizing Computer Intensive Methods to "Concretize" Theorems and Principles from Statistics – Precisely) to make the answer no or at least not necessary (for statistics). One will always be able to understand more quickly and more generally with additional mathematical skill and understanding. I think I am starting to get close, but one needs a manipulate-able representation of probability generating models and inference that is valid and useful for more than just toy problems.

Should I try and fill in the blanks of my maths before continuing with ML?

That’s a hard endeavour – in MHO almost everyone who understands statistics got there by being very comfortable manipulating the standard and especially not so standard mathematical representations of probability generating models and mathematical characterizations of inference (the top x% of mathematical statistics Phds). So it’s not just getting the basics but being real comfortable with the math. (As an aside, for me Fourier Theory was essential.)

Why are these representations hard (even with lots of math)?

Gerd Gigerenzer has pretty much established that there is no challenge with the simple disease positive/negative given test positive/negative problem using _natural frequencies”. A reference from the linked question seems to make good use of that http://www.autonlab.org/tutorials/prob18.pdf

Why is this hard to generalize?

For k tests (repeated and or different ) – 2^k

For tests that take v values – v^k

So for binary unknown – 2 * v^k sample path probabilities

For p multiple binary unknowns 2^p * v^k

For p multiple rational unknowns Q^p * v^k

One quickly moves to math with countable and uncountable infinities to cope with this, which even with mathematical expertise leads to many misunderstandings and seeming paradoxes (e.g. Borel’s paradox?)

Additionally there is linear to non-linear hazardous misunderstandings (e.g. Hidden Dangers of Specifying Noninformative Priors Winbugs and other MCMC without information for prior distribution ) and interactions and random effects, etc.