0
$\begingroup$

In the generalized linear model, the distribution of a random variable $Y$ is assumed to be an exponential family distribution, and it is written in terms of an explanatory variable $X$ and parameter $\theta$. I was wondering if $X$ is viewed as a parameter of the distribution of $Y$, or a random variable and the distribution of $Y$ mentioned earlier is in fact a conditional one given $X$?

If $X$ can be viewed as a random variable, there seems to me a problem here. In the generalized linear model, the distribution of a random variable $Y$ is assumed to be an exponential family distribution. But since the distribution of $Y$ depends on $X$, it is actually that $P(Y|X=x)$ is an exponential family distribution. $X$ can have any distribution, and the unconditional distribution of $Y$ may not be an exponential family distribution. So how shall we understand the problem?

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Neither. The x's are assumed known (fixed) as for regression. The mean of the Y's is written as a function ($g$) of the linear predictor, $X\beta$, but the $X$'s aren't parameters in that, they're constants. $\endgroup$
    – Glen_b
    Jun 28, 2013 at 12:55

2 Answers 2

Reset to default
3
$\begingroup$

If X can be viewed as a random variable, there seems to me a problem here. In the generalized linear model, the distribution of a random variable Y is assumed to be an exponential family distribution.

No, it's not. The assumption holds for the conditional not the marginal distribution of Y.

But since the distribution of Y depends on X, it is actually that P(Y|X=x) is an exponential family distribution.

Exactly.

X can have any distribution, and the unconditional distribution of Y may not be an exponential family distribution. So how shall we understand the problem?

There is no problem. (Which is fortunate, since if I remember correctly X and Y must be jointly Normal for all the marginal and conditional distributions to also be Normal, so marginal and conditional form matches will be quite rare). In general, with arbitrary X, the marginal distribution won't be anything particularly familiar.

Perhaps a helpful way to think about motivating regression modelling is to consider the joint distribution P(X, Y) -- whatever that is -- and to note that whatever the distributional forms this can always be broken into P(X) and P(Y|X). If you are willing to entertain the assumption that one set of parameters specifies P(X) and that another separate and unrelated set of parameters specifies P(Y | X), e.g. with some kind of causal model in the background, then it is reasonable to ignore P(X) and its parameters and work just with a (regression) model of the conditional distribution P(Y | X).

This approach can also be more efficient, since it is usually easier to learn a conditional model directly than to learn a joint model and then condition yourself.

Improve this answer
$\endgroup$
2
$\begingroup$

As in any regression model the response is modelled conditional on observed/set values of the predictors.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thanks! So the distribution of response $Y$ may not be an exponential family distribution? $\endgroup$
    – Tim
    Jun 28, 2013 at 12:42
  • 1
    $\begingroup$ Unconditionally, if the predictors are random variables, it can be anything. Could be bimodal. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Jun 28, 2013 at 12:44
  • $\begingroup$ I saw in Wikipedia and other books, they defined generalized linear model, by first claiming the distribution of response $Y$ has an exponential family distribution before introducing the explanatory variable $X$. That make me think the unconditional distribution of $Y$ is an exponential family distribution, although it may not be true. $\endgroup$
    – Tim
    Jun 28, 2013 at 12:47
  • 1
    $\begingroup$ It is not true. The conditional distribution is exponential family $\endgroup$
    – Glen_b
    Jun 28, 2013 at 12:56
  • 3
    $\begingroup$ A metacomment here is likely to seem trite but possibly worth underlining to @Tim. Wikipedia is very, very patchy on many technical matters: sometimes utterly superb, sometimes dire. That's the nature of the beast. $\endgroup$
    – Nick Cox
    Jun 28, 2013 at 13:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.