GLM Link Function and Distribution

Question

Well, i can see why we need a GLM for non-linear relationships. If we want to catch non-continuous data we can do so by using different link functions. Nevertheless, the thing is, why do we need to use different distributions ? In linear regression we have an error distribution that is gaussian, whereas, in GLM we can choose the distribution of response variable rather than residuals. Using different distributions for the model has an effect on the model ? or it is just for interpreting the response values ?

Answer 1

I can see why we need a GLM for non-linear relationships. If we want to catch non-continuous data we can do so by using different link functions [...]

GLM's are linear in parameters, the point of using link function is to transform the linear predictor $\eta = \mathbf{X}\boldsymbol{\beta}$ into mean $E[y|\mathbf{X}] = g^{-1}(\eta)$.

There are several possible link functions: linear, inverse, inverse squared, logit, log, etc, notice that they do not transform $\eta$ in a way that makes it non-continuous, as those are continuous functions.

Nevertheless, the thing is, why do we need to use different distributions?

Because the point of using GLM's is to be able to model also data where the conditional distribution of $y|X$ is other than Gaussian. If the predicted variable conditionally is binary, you need Bernoulli distribution i.e. logistic regression, if you model counts, you may want to use Poisson regression, if it is non-negative, you may consider Gamma distribution, etc. In all the above cases assuming Gaussians would usually be wrong. For more details, you can check my answer in the How to decide which glm family to use? thread.

Using different distributions for the model has an effect on the model?

It does. Recall that we use maximum likelihood for fitting GLM's. So what you maximize is

$$ \operatorname{arg\,max}_\boldsymbol{\beta} \sum_i \log f(y_i | \mathbf{X}_i, \boldsymbol{\beta}) $$

where $f$ is the probability density function of the distribution. So by using a different distribution, you are also changing the objective function to maximize. It is like using a different loss function to train a model, it will lead to different results.