Exponential distribution: How to avoid negative predictor of $\lambda$?

Question

I have a joint distribution $P(X, Y_{1}, Y_{2}, ....)$ which contains one univariate exponential distribution ($X$) and several univariate gaussian distributions ($Y_{1}, ...$). For details regarding the joint distribution see http://www.cs.utexas.edu/users/eunho/papers/mixedGM.pdf.

The exponential distribution conditioned on all other (gaussian) variables is:

$$ P(X|Y_{1}, Y_{2}, ....) \propto \exp \big\{ S^{x}(NP) \big\}$$

where $NP = \sum_{i} \theta_{i} S_{i}^{y}$ and $S_{x}$ is the sufficient statistic for the exponential distribution, $S_{i}^{y}$ are sufficient statistics of the gaussian distributions and $\theta_{i}$ is a parameter $\in \{0,1\}$ for the interaction between $X$ and $Y_{i}$.

So I predict the natural parameter $-\lambda$ by some linear combination of sufficient statistics $S_{i}^{y}$.

Now, the parameter $\lambda$ of the exponential distribution has to be $\lambda > 0$. This means that $NP = -\lambda$ and $- NP = \lambda$ and thus $NP$ has to be negative at all times otherwise we would have an impossible value for $\lambda$.

However, the Gaussian sufficient statistic $S_{i}^{y}$ can have both signs, thus it can happen that I have a linear combination $NP$ that is positive and I thus get a negative (impossible) $\lambda$.

Wikipedia writes about this problem:

In the cases of the exponential and gamma distributions, the domain of the canonical link function is not the same as the permitted range of the mean. In particular, the linear predictor may be negative, which would give an impossible negative mean. When maximizing the likelihood, precautions must be taken to avoid this. An alternative is to use a noncanonical link function.

(http://en.wikipedia.org/wiki/Generalized_linear_model#Link_function)

Is there any easy way around this problem that I do not see? If not, can somebody recommend some accessible reading on non-canonical link functions?

Any help is highly appreciated!

Answer 1

Good question. You say $S_i = Y_i$ or is some parametrized function thereof, and $\theta_i$ are your parameters?

I think the main issue is that you're using a Gaussians $S$ rather than some other distribution whose support is the negative reals.

Note that if you had exactly one $Y_i$ then the right thing to do is to model $S$ as a (negative) Gamma distribution, which is the conjugate prior of the exponential distribution with (negative) mean $\lambda$. The updates then become simple summation of observations into the shape parameter and summation of number of observations into the rate parameter.

I assume you wanted the Gaussians because the space of Gamma distributions is not closed under summation. My suggestion is to interpret the mean and second moment that you are summing in $S$ as the unique gamma distribution having that mean and second moment. You may need to prevent your parameter updates from letting any of the means become negative.