Probabilistic vs. other approaches to machine learning I'm taking a grad course on machine learning in the ECE department of my university. On the first lecture my professor seemed to make it a point to stress the fact that the course would be taking a probabilistic approach to machine learning. I didn't think much of it at the time, but now that I think back on it, what does this really mean? What other approaches are there to machine learning that I can contrast this against? 
 A: The question may be too broad to answer. It is hard to guess another person's perspective. But I think the question is interesting, and I would like to try to answer.
The term "machine learning" can have many definitions. I believe The popular ones are 


*

*Convex optimization (there are tons of papers on NIPS for this topic)

*"Statistics minus any checking of models and assumptions" by Brian D. Ripley


From optimization perspective, the ultimate goal is minimizing the "empirical loss" and try to win it on testing data set. Where we do not emphasize too much on the "statistical model" of the data. Some big black box discriminative model would be perfect examples, such as Gradient Boosting, Random Forest, and Neural Network. These types of work got popular because the way we collect data and process data has been changed. Where we can think we have infinite data and will never over-fit (for example number of images in Internet). All the computational model we can afford would under-fit super complicated data. The goal would be have an effective way to build the model faster and more complex (For example using GPU for deep learning)
On the other hand, from statistical points (probabilistic approach) of view, we may emphasize more on generative models. For example, mixture of Gaussian Model, Bayesian Network, etc. The book by Murphy "machine learning a probabilistic perspective" may give you a better idea on this branch.
A: The term "probabilistic approach" means that the inference and reasoning taught in your class will be rooted in the mature field of probability theory. That term is often (but not always) synonymous with "Bayesian" approaches, so if you have had any exposure to Bayesian inference you should have no problems picking up on the probabilistic approach. 
I don't have enough experience to say what other approaches to machine learning exist, but I can point you towards a couple of great refs for the probabilistic paradigm, one of which is a classic and the other will soon be, I think:


*

*Jaynes, E.T. (2003) Probability Theory: The Logic of Science. Cambridge University Press, New York.

*Murphy, K. (2012) Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge.

