2
$\begingroup$

This question already has an answer here:

I'm beginning with the regression analysis and I'm quite confused with the generalized linear regression.

I understand, that the ordinary linear models can be described with a formula

$$ y_i = \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i $$

and when using generalized linear regression I'd expect it to be something like this

$$ g\left(y_i\right) = \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i, $$ $g$ being the link function.

But, instead I came across this formula

$$ g\left(E[\mathbf{y}]\right) = \mathbf{x'}\mathbf{\beta} $$

and I don't get, what's the expected value in the left doing there.

Could you, please, explain it a little?

$\endgroup$

marked as duplicate by Glen_b Mar 31 '17 at 9:06

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2
$\begingroup$

You are correct that regression model is

$$ y_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i $$

that can be written differently as

$$ \mu_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n \\ y_i = \mu_i + \epsilon_i $$

or saying it differently

$$ y_i \sim \mathcal{N}(\mu_i, \sigma^2) $$

i.e. we predict conditional mean of $Y$ given $X$ and assume residuals to be normally distributed around it.

and when using generalized linear regression I'd expect it to be something like this

$$ g\left(y_i\right) = \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n + \epsilon_i, $$

$g$ being the link function.

But, instead I came across this formula

$$ g\left(E[\mathbf{y}]\right) = \mathbf{x'}\mathbf{\beta} $$

and I don't get, what's the expected value in the left doing there.

The above definition is correct. GLM's are generalizations of regression to distributions other then normal and link functions other then identity link. GLM is defined in terms of linear predictor

$$ \eta_i = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n $$

that is passed through inverse of the link function $g$ to obtain mean

$$ E(Y|X) = \mu_i = g^{-1}(\eta_i) $$

and $\mu_i$ is the conditional mean of $Y$ given $X$ of some probability distribution of choice (e.g. normal, Poisson, Bernoulli etc.).

$\endgroup$
  • $\begingroup$ Thank you very much. Just two little questions: 1) I suppose, that linear predictor in GLM should be $\mu$ and not $\eta$. Shouldn't it? 2) And is it possible to write it "my way" too or is it incorrect? $\endgroup$ – Eenoku Mar 31 '17 at 9:01
  • 2
    $\begingroup$ @Eenoku linear predictor is $\eta$ you transform it using inverse link to mean $\mu$. It is not possible to write GLM in terms of mean + noise since in cases other then normal (linear regression) it would be hard to define the "noise", e.g. in logistic regression you predict mean of Bernoulli distribution, so mean is real number in unit interval and $Y$ is 0's and 1's -- the "noise" would have nothing to do with Bernoulli distribution etc. Actually defining the model in terms of noise is pretty confusing and it is better to think of it in terms of predicting mean. $\endgroup$ – Tim Mar 31 '17 at 9:59

Not the answer you're looking for? Browse other questions tagged or ask your own question.