1
$\begingroup$

OLS: $y = X\beta+ \epsilon$ with $\epsilon \sim N(0,\sigma^2)$

GLM: $g(\mu) = X\beta$ with $y \sim $<Distribution from exponential family>

I have knowledge of OLS and am trying to understand the more general case of GLM. I noticed that for GLMs the assumption of the distribution is made on $y$ and $\epsilon$ is never explicitly shown in any of the equations. I am aware that in OLS the assumption $\epsilon \sim N(0,\sigma^2)$ implies $y \sim N(X\beta,\sigma^2I)$, but I want to know:

  1. Why do we not explicitly show $\epsilon$ in the GLM model (or probit & logit)?
  2. Is $\epsilon$ even present in these models? The model is not perfect and therefore there has to be an error, but where exactly is it? Can you write the GLM model with $\epsilon$ so I can see how it is incorporated in the model?
$\endgroup$

1 Answer 1

1
$\begingroup$
  1. Why do we not explicitly show $\epsilon$ in the GLM model (or probit & logit)?

It can surely be shown, but does not lead to any new insight. For example, for the Bernoulli GLM model, we have an assumption on the $y$, that it is Bernoulli-distributed. The residuals, however, are not (since they are continuous) and should have a more complicated distribution.

  1. Is $\epsilon$ even present in these models? The model is not perfect and therefore there has to be an error, but where exactly is it? Can you write the GLM model with $\epsilon$ so I can see how it is incorporated in the model?

If $E(y|X) = \mu = g^{-1}(\eta)$, where $\eta = \beta X$, then $y_i = \hat{\mu_i} + \epsilon_i$.

$\endgroup$
5
  • $\begingroup$ But in $y = \hat{\mu} + \epsilon$ you claim that the observation $y_i$ equals the mean (or technically an estimator of the mean) plus an error. Shouldn't it be the case that $E(y) = \hat{\mu} + E(\epsilon)$ but then again $E(\epsilon)=0$ and we no longer show $\epsilon)$ explicitly in the equations? $\endgroup$
    – Xtiaan
    Jan 19, 2023 at 9:46
  • $\begingroup$ @Xtiaan Why would you do the expected value of $y$? $\endgroup$
    – Firebug
    Jan 19, 2023 at 13:46
  • $\begingroup$ Because according to the current equation we say that y equals the the predictor of the mean of y (+ an error). Should it not be the case that we say that y equals a predictor of y (+ an error) or that the expectation of y equals a the predictor of the mean of y (+ an error)? $\endgroup$
    – Xtiaan
    Jan 19, 2023 at 14:08
  • $\begingroup$ @Xtiaan the predictor and the conditional expectation are the same thing. $\endgroup$
    – Firebug
    Jan 19, 2023 at 14:23
  • $\begingroup$ So $y_i = \hat{\mu_i} + \epsilon_i$ says that its observation equals $E(y_i)$ plus an error. However, what is $E(y_i)$? Isn't $y_i$ just an observation. Isn't $y$ stochastic and not $y_i$? $\endgroup$
    – Xtiaan
    Jan 19, 2023 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.