What methods are used to solving regression problems in Machine Learning (like GLMs)?

Question

I'm going through Andrew Ng's lecture notes on Machine Learning.

The notes start with introducing linear regression and intuitively explaining what the cost function for the problem should be and how to minimize it etc. without any framework.

Then the notes proceed to explain Generalize Linear Models, and we see that how using GLMs we can derive ways to solve linear regression/ logistic regression problems. So here, GLMs seem like they're a common framework to solve a variety of machine learning problems.

So I was wondering, if there are other such similar frameworks for solving machine learning problems? And also, is there a name for solving such problems intuitively/logically without the use of a framework? (like the notes explain solving linear regression problems initially)

I realize that I'm just starting this course and I may learn of other more general methods (like SVM or Neural Nets?) ahead, but I just wanted to know what other methods exist? And when to choose which route to solve a given problem (if that's answerable)

Answer 1

It's a bit unclear what you mean by "framework"; for example, would Least Squares be the framework you're looking for when it comes to ordinary linear regression? Following Least Squares, GLMs is a broader "framework" that encompasses least squares regressions and more. Eventually, the later techniques you mentioned like SVMs or ANNs can also be considered frameworks.

If you look ahead into the course syllabus you should see what other methods the course covers. In addition, there's a great flow chart from SciKit Learn that describes many machine learning methods and when to use what.