1
$\begingroup$

Is a GLM with a log link function the same as estimating:

$$y_i = \exp(\beta_0 + \beta_1 x_i)\ ?$$

A GLM is the following:

$$g(\mu_i) = \eta_i = b_0 + b_1 X_i$$

where $g(x)$ is the link function.

I don't understand $\log(\mu_i)=b_0+b_1 X_i$. Are you transforming the estimated $\mu_i$? Is $\mu_i$ estimated based on:

$$\mu_i = \exp(\beta_0 + \beta_1 X_i)\ ?$$

If $\mu_i$ is estimated based on $\exp(\beta_0 + \beta_1 X_i)$, then $\mu_i = y_i$ because we haven't changed its value in the dataset at all. So is a GLM just a transpose of the predictor variables and model parameters?

I am very confused by the GLM because it looks like they're estimating parameters based on a transformed estimated result ($\mu_i$) which has been transposed but how can you do that if it is already estimated by the model?

$\endgroup$
  • $\begingroup$ I edited the appearance of your post so the formulas read easier. Please check that the meaning remains the same. $\endgroup$ – usεr11852 Oct 4 '15 at 3:49
  • $\begingroup$ Obviously it cannot be that $y_i = \exp(\beta_0 + \beta_1 x_i)\ $ - that each data value is actually equal to some function of the population parameters. If it were, two points would give us an equation we could solve exactly for the $\beta$s, no uncertainty involved. You need to think more precisely about the model, & much of the mystery drops away immediately. What you mean for a GLM is actually more like $E(Y_i|x_i) = \exp(\beta_0 + \beta_1 x_i)\ $ (which when combined with the particular distributional family, and the assumption of independence allows you to form the likelihood function) $\endgroup$ – Glen_b Oct 4 '15 at 6:48
2
$\begingroup$

We have to be clearer on what it means to "estimat[e] $y_i = \exp(\beta_0 + \beta_1 x_i)$". Although that looks like a completely formed thought, it isn't quite. What's missing is the specification of the response distribution and/or error term. (Since you are using the log link, I'll assume you are using the Poisson GLiM as your example, since the log is the canonical link for the Poisson GLiM.) Now we have:
$$ \log(\lambda_i) = \beta_0 + \beta_1 x_i $$ Thus, we can find estimated betas by finding the values of the betas that will maximize the likelihood:
$$ L_i = \prod_{i = 1}^N \frac{\lambda_i^{y_i}}{y_i!}e^{-\lambda_i} $$ (or that minimize the deviance, $-2\times\ln(L_i)$, which is preferred for computational reasons, but yields the same estimated betas).


Since this is confusing, let's walk through it slowly.

Are you transforming the estimated $μ_i$?

Yes... or, sort of. We estimate $\hat\mu_i$ by back transforming the RHS of the equation.

Is $μ_i$ estimated based on: $μ_i=\exp(β_0+β_1X_i)$?

Yes.

...then [isn't] $μ_i=y_i$ because we haven't changed its value in the dataset at all[?]

No. The way this works is that we plug in some candidate values for the betas, run them through the RHS of the equation, and then exponentiate the result. That is the predicted mean of the data at that point in the covariate space (i.e., when $X=x_i$) given the stipulated candidate betas. For the Poisson distribution, the mean is $\lambda$, which is the parameter that governs the behavior of the distribution—once you know that, you know everything you need to know about that particular (conditional) Poisson distribution. For example, you can determine the relative likelihood of any observed datum $y_i$. So we are not assuming $y_i = \mu_i$; we use $\hat\mu_i = \hat\lambda_i$ to make it possible to determine $L(y_i|{\bf X}, \boldsymbol{\hat\beta})$.

Now, none of that implies that any given set of candidate beta values that we had stipulated are the best ones. We will have to search. But as we search, at each point we are now able to evaluate the fit / the likelihood of the stipulated beta values given the data.

For more on these topics, it may help you to read my answer here: Difference between logit and probit models.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.