1
$\begingroup$

everybody I'm learning about linear regression and GLM's. One of the things I see is the affirmation that: I can make and histogram of the response $y$ and if it don't follows a normal (for example follows a gamma distribution) then I can use a GLM with gamma distribution. My problem is that this affirmation seems wrong to me, I can simulate a linear function with normal error, but sample it so that the response doesn't look's like normal, if I try enough I can make it looks like a gamma, for example:

import pandas as pd
import numpy as np
def funcao_linear(x):
  #Função linear com erro normal
  return 0.5*x+2+np.random.normal(size=1)[0]
#Pegando mais valores em x=5 to make a bias
x=[5]*1000+[6]*100
for i in range(7,100):
  x=x+[i]*int(100-i)
temp=pd.DataFrame({'x':x})
temp
temp['y']=temp.x.apply(funcao_linear)
temp.y.hist(bins=20)
$\endgroup$
1
  • $\begingroup$ For linear regression, the residuals $(y-y_{predicted})$ should be normally distributed. The distribution of the original dependent variable is irrelevant, other than providing a suggest for the better model. $\endgroup$
    – Dave2e
    Apr 9 at 23:39

1 Answer 1

2
$\begingroup$

You are correct that evaluating a histogram to choose the likelihood is ill advised.

The reason is because the likelihood concerns the conditional distribution of the outcome and a histogram represents the marginal distribution. I demonstrate how misleading the histogram of the outcome can be when selecting a likelihood in this answer.

The choice of likelihood must be made using domain knowledge. You have to take into consideration how the outcome behaves (e.g. is it bounded, do we expect the variance to change as the mean changes, etc). Typically, a likelihood is not something you choose by looking at data.

$\endgroup$
2
  • $\begingroup$ "Typically, a likelihood is not something you choose by looking at data." Oh nuts! I've been doing it all wrong! $\endgroup$ Apr 10 at 3:09
  • $\begingroup$ @JohnMadden I don't think one just stumbles across data that they know aboslutely nothing about and must use data to learn how to model it. There is at least some understanding of what the data are about, how the phenomenon is measured, etc etc that can give you some hint as to what approach to use first. In the case you do just stumble on data without any idea of what it is or where it came from, I think there are larger problems than what likelihood you select. $\endgroup$ Apr 10 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.