Statistical Model from distributions I am aware of GLM. There the dependent variable $Y$ is assumed to be generated according to a certain distribution and for the mean it holds that
$$
 E[Y] = g^{-1}(X \beta).
$$
Is there a common theory for models of the form
$$
E[Y] = g^{-1}(X \beta)
$$
and we assume distributions for the components of $X$ (e.g. $X_1$ is Guassian, $X_2$ is Gamma)? I am thinking about an MLE approach in this context.
I guess this is related to GLM but maybe I am missing something. Thank you!
 A: If I understand you correctly you are asking about either the marginal distribution of $Y$ (or perhaps the joint distribution of $(X,Y)$?).  The analysis carried out for Generalized Linear Models are typically concerned with the conditional distribution of $Y|X$.  What you will get is that $Y$ has a mixture distribution, with its mean parameter following the sum distribution of the predictors.
This is probably easiest if we start with a linear link function, and two independent Gaussian predictors
Our model for the mean is
$E[Y] = \beta_1 X_1 + \beta_2 X_2$
but we want the distribution, so need the error term version of the model:
$Y = \beta_1 X_1 + \beta_2 + X_2 + \epsilon$
If we assume Gaussian noise (since we chose linear link), the clearly $Y \sim N(\mu_1+\mu_2, \sigma^2_1+\sigma^2_2+\sigma^2_\epsilon)$
This worked out quite nicely because a normal with a normal for the mean is another normal.  If $X$ come from a different distribution, then you will have a mixture distribution, where the mean of the normal itself follows whatever distribution you picked for the $X$.
If we move beyond Gaussian, the the same logic applies.  The GLM part gives you the mean of the response distribution, if you then view that mean in terms of the parameter of the distribution then you have a mixture distribution.
So for count data we have
$\ln(E[Y]) = \beta_1 X_1 + \beta_2 X_2$
Or
$Y \sim \text{Pois}(\exp(\beta_1 X_1 + \beta_2 X_2))$
We can't easily write a meaningful error term version this time like
$Y = \exp(\beta_1 X_1 + \beta_2 X_2) + \epsilon$
or maybe
$Y = \exp(\beta_1 X_1 + \beta_2 X_2 + \epsilon)$
because the distribution of $\epsilon$ would be bizarre.  This is what makes GLM so different from simple data transformations.  The version I just wrote, with the epsilon inside the link function is the model you get when you transform your data, then fit a linear model.  The one above that is the case where you transform your predictors and then fit a linear model.  GLM is different to both of these - it transforms the expectation, creating this "difficult" mixture distribution - but because we only normally work with the resulting conditional distribution, the difficulty doesn't bother us.
If the $X$ are normally distributed, then you have a Poisson distribution with a log-normal parameter (the Poisson-lognormal mixture: an overdispersion model).
It is difficult to find many examples where the distributions are all "nice", but the last example illustrates the key point.  The regression model gives you the conditional distribution, so the full marginal distribution of the response is the mixture of the predictor distributions and the response distribution.  This means that the marginal distribution is an over-dispersed version of the original distribution.
