0
$\begingroup$

Well, i can see why we need a GLM for non-linear relationships. If we want to catch non-continuous data we can do so by using different link functions. Nevertheless, the thing is, why do we need to use different distributions ? In linear regression we have an error distribution that is gaussian, whereas, in GLM we can choose the distribution of response variable rather than residuals. Using different distributions for the model has an effect on the model ? or it is just for interpreting the response values ?

$\endgroup$
2
  • 4
    $\begingroup$ "i can see why we need a GLM for non-linear relationships." ... "Nevertheless, the thing is, why do we need to use different distributions ?" Are you sure you see, then? How would you prefer to model binary or count data without changing the distribution? $\endgroup$ May 6 '21 at 23:11
  • $\begingroup$ @AryaMcCarthy I think i handled it. Thanks for the contribution :) $\endgroup$ May 8 '21 at 19:26
1
$\begingroup$

I can see why we need a GLM for non-linear relationships. If we want to catch non-continuous data we can do so by using different link functions [...]

GLM's are linear in parameters, the point of using link function is to transform the linear predictor $\eta = \mathbf{X}\boldsymbol{\beta}$ into mean $E[y|\mathbf{X}] = g^{-1}(\eta)$.

There are several possible link functions: linear, inverse, inverse squared, logit, log, etc, notice that they do not transform $\eta$ in a way that makes it non-continuous, as those are continuous functions.

Nevertheless, the thing is, why do we need to use different distributions?

Because the point of using GLM's is to be able to model also data where the conditional distribution of $y|X$ is other than Gaussian. If the predicted variable conditionally is binary, you need Bernoulli distribution i.e. logistic regression, if you model counts, you may want to use Poisson regression, if it is non-negative, you may consider Gamma distribution, etc. In all the above cases assuming Gaussians would usually be wrong. For more details, you can check my answer in the How to decide which glm family to use? thread.

Using different distributions for the model has an effect on the model?

It does. Recall that we use maximum likelihood for fitting GLM's. So what you maximize is

$$ \operatorname{arg\,max}_\boldsymbol{\beta} \sum_i \log f(y_i | \mathbf{X}_i, \boldsymbol{\beta}) $$

where $f$ is the probability density function of the distribution. So by using a different distribution, you are also changing the objective function to maximize. It is like using a different loss function to train a model, it will lead to different results.

$\endgroup$
2
  • $\begingroup$ Well, I don't know why but i got it. I've been reading stuffs like that for a while but i couldn't understand the relationship completely. All of a sudden, when i read your post i got it. Perhaps it is because of the reason that i brushed up on my maximum likelihood and statistics before reading it. Thank you :) $\endgroup$ May 8 '21 at 19:25
  • $\begingroup$ @AryaMcCarthy done $\endgroup$ May 9 '21 at 23:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.