Taxonomy/overview of machine learning techniques [duplicate]

Question

My question: I'm looking for a taxonomy/bestiary/overview of machine learning techniques. I would like to learn 1) how these methods relate to each other, and 2) the relative costs and benefits (and perhaps typical applications) of the different approaches.

---

Background: I'm trained in statistics, and have a reasonably clear mental map of how a range of these techniques relate to each other. Understanding the costs and benefits of different techniques obviously makes it easier to select the best one to apply in different situations.

I would like to develop a similar mental map or taxonomy of the broader machine learning field. It's most important for me to understand the highest levels in the technique taxonomy, but I also recognise that there are major lower-level developments in some areas I should be aware of (e.g. neural networks seem to have a huge number of sub-classes).

I'm not looking for in-depth explanations of each method - though references would be great - but instead a framework I can use to focus my learning efforts in a more informed way.

This question focussed on statistical techniques is similar, in that the goal is to understand relationships between methods. But I'm looking for more than a 'cheat sheet'. I'd like to understand each at least at a basic level, and not just follow a set of rules on a flow chart.

Answer 1

You can find a very good taxonomy of the most important ML methods in the table of contents of the book Machine Learning: a Probabilistic Perspective by Kevin Patrick Murphy.

Given your background in statistics, I'm pretty confident that you will find that book resourceful. It has both introductory descriptions and in-depth explanations of almost any kind of ML method.

Answer 2

Tree-based methods

A group of regression and classification methods built around decision trees. In a decision tree, data is recursively partitioned based on its predictors, and new predictions are generated by averaging data points at the relevant tips ('leaves') of each tree. Weaknesses of standard decision trees (such as overfitting) have been substantially overcome by bagging (in random forests) and boosting (in gradient boosting machines).

Includes: CART, C4.5, random forests, gradient boosting trees

Advantages: Flexible and relatively easy to interpret (importance scores, partial effects)

Support vector machines (SVMs)

Originally an algorithm for binary classification that identifies the hyperplane best separating two groups of data points. Subsequent extensions include multiclass SVMs (which work by reducing multiple class problems to a series of 2-class problems) and support vector regression (which uses the hyperplane to predict continuous values)

Advantages:

Artificial Neural Networks

Computing systems whose network structure is inspired by (and theoretically allows for the flexibility of) biological systems. Composed of a net of artificial neurons that can transmit signals to each other in a defined, usually hierarchical structure. Each artificial neuron takes multiple inputs, sums them based on their separate weights, and produces output based on some activation function. The weights of each input are learnt during the network's training process. Neurons are organised in layers that tend to abstract different features of the system they are trained upon.

Advantages:

Includes: Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Deep Learning