Is my understanding of Generalized Linear Models correct?

Question

I'm going through Andrew Ng's lecture notes on Machine Learning & I just learnt about Generalized Linear Models there.

I want to check if I know correctly what Generalized Linear Models are.

Generally for Machine Learning problems (classification/prediction or regression) we can base our models on different distributions. GLMs give us a framework to form our models based on different distributions so that we can solve our problem(s). Depending on what distribution we model on, we'd get a different hypothesis function that will allow us to get the solution to our problem.

Is that definition accurate? Would you add anything if you were to define GLMs?

I'm just not entirely sure if I grasp everything about them that I should to proceed with the course videos.

Answer 1

I would say that your understanding still needs some work, because your description is very vague, which tells us that you're unclear as to exactly what to say about GLMs. First, your statement "for machine learning problems we can base our models on different distributions" is somewhat ambigious. For linear regression models this makes more sense; you'll see later in the course that there are many nonlinear metheds that don't follow a pre-defined probability distribution.

Remember that in regression problems, you're modeling the mean of the response variable as a function of the linear combination of predictors. When we perform ordinary least squares, we are restrained to several assumptions - like that the response variable is normally distributed around the mean and that variance of the response is independent of the predictors. This doesn't always hold in reality, so GLMs allow us to relax some of these assumptions by specifying the response variable distribution, a link function, etc. In other words, GLMs are a generalization and extension of least squares; they're still linear regression problems which are only a part of the overall machine learning theme.

Andrew Ng's lecture notes may not be the best introductory source when it comes to this topic. I recommend reading up on some additional sources if you wish to get to know GLMs a bit better, like this chapter from Applied Regression Analysis and GLMs.