I'm going through Andrew Ng's lecture notes on Machine Learning & I just learnt about Generalized Linear Models there.

I want to check if I know correctly what Generalized Linear Models are.

Generally for Machine Learning problems (classification/prediction or regression) we can base our models on different distributions. GLMs give us a framework to form our models based on different distributions so that we can solve our problem(s). Depending on what distribution we model on, we'd get a different hypothesis function that will allow us to get the solution to our problem.

Is that definition accurate? Would you add anything if you were to define GLMs?

I'm just not entirely sure if I grasp everything about them that I should to proceed with the course videos.

  • 3
    $\begingroup$ This is a very vague description and while it applies to GLM it also applies to distinctly non-GLM procedures. I gave a defintion of the GLM here: stats.stackexchange.com/questions/40876/… $\endgroup$
    – Momo
    Jul 20 '15 at 8:57

I would say that your understanding still needs some work, because your description is very vague, which tells us that you're unclear as to exactly what to say about GLMs. First, your statement "for machine learning problems we can base our models on different distributions" is somewhat ambigious. For linear regression models this makes more sense; you'll see later in the course that there are many nonlinear metheds that don't follow a pre-defined probability distribution.

Remember that in regression problems, you're modeling the mean of the response variable as a function of the linear combination of predictors. When we perform ordinary least squares, we are restrained to several assumptions - like that the response variable is normally distributed around the mean and that variance of the response is independent of the predictors. This doesn't always hold in reality, so GLMs allow us to relax some of these assumptions by specifying the response variable distribution, a link function, etc. In other words, GLMs are a generalization and extension of least squares; they're still linear regression problems which are only a part of the overall machine learning theme.

Andrew Ng's lecture notes may not be the best introductory source when it comes to this topic. I recommend reading up on some additional sources if you wish to get to know GLMs a bit better, like this chapter from Applied Regression Analysis and GLMs.

  • 1
    $\begingroup$ You would convey more of the flavor of GLMs by stating that they model some parameter (rather than the mean) of the response as a linear function of the predictors. (It is rare that a GLM models the mean as a linear combination except in the case of OLS.) $\endgroup$
    – whuber
    Jul 20 '15 at 14:21
  • $\begingroup$ "you're modeling the mean of the response variable as a function of a linear combination of predictors" would seem to cover it. $\endgroup$ Jul 20 '15 at 14:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.