Exponential distribution: How to avoid negative predictor of $\lambda$?

I have a joint distribution $P(X, Y_{1}, Y_{2}, ....)$ which contains one univariate exponential distribution ($X$) and several univariate gaussian distributions ($Y_{1}, ...$). For details regarding the joint distribution see http://www.cs.utexas.edu/users/eunho/papers/mixedGM.pdf.

The exponential distribution conditioned on all other (gaussian) variables is:

$$P(X|Y_{1}, Y_{2}, ....) \propto \exp \big\{ S^{x}(NP) \big\}$$

where $NP = \sum_{i} \theta_{i} S_{i}^{y}$ and $S_{x}$ is the sufficient statistic for the exponential distribution, $S_{i}^{y}$ are sufficient statistics of the gaussian distributions and $\theta_{i}$ is a parameter $\in \{0,1\}$ for the interaction between $X$ and $Y_{i}$.

So I predict the natural parameter $-\lambda$ by some linear combination of sufficient statistics $S_{i}^{y}$.

Now, the parameter $\lambda$ of the exponential distribution has to be $\lambda > 0$. This means that $NP = -\lambda$ and $- NP = \lambda$ and thus $NP$ has to be negative at all times otherwise we would have an impossible value for $\lambda$.

However, the Gaussian sufficient statistic $S_{i}^{y}$ can have both signs, thus it can happen that I have a linear combination $NP$ that is positive and I thus get a negative (impossible) $\lambda$.

In the cases of the exponential and gamma distributions, the domain of the canonical link function is not the same as the permitted range of the mean. In particular, the linear predictor may be negative, which would give an impossible negative mean. When maximizing the likelihood, precautions must be taken to avoid this. An alternative is to use a noncanonical link function.

Is there any easy way around this problem that I do not see? If not, can somebody recommend some accessible reading on non-canonical link functions?

Any help is highly appreciated!

• Starting with your reference to "sufficient statistics" this question is very confusing. The only sufficient statistics that are relevant would be those for the exponential family, not for a Gaussian family. And what do you mean by "the Gaussian sufficient statistic $X_i$"?
– whuber
Commented Feb 10, 2015 at 16:05
• Thanks for your comment! Let's say we have a joint distribution containing one exponential and several gaussian univariate (with unit variance) distributions $P(X,Y)$. $P(X|...)$ is the distribution of the exponential variable conditioning on all the gaussians (For details see here: cs.utexas.edu/users/eunho/papers/mixedGM.pdf). The sufficient statistics of the gaussians do matter, because a linear combination of them define the natural parameter I am interested in. I hope I answered your question.
– jmb
Commented Feb 10, 2015 at 19:23
• You indicated the general nature of your question. However, none of this essential information actually appears in your post. Please edit it to explain what's really going on.
– whuber
Commented Feb 10, 2015 at 19:37
• I added the information in the post.
– jmb
Commented Feb 10, 2015 at 19:59

Good question. You say $S_i = Y_i$ or is some parametrized function thereof, and $\theta_i$ are your parameters?

I think the main issue is that you're using a Gaussians $S$ rather than some other distribution whose support is the negative reals.

Note that if you had exactly one $Y_i$ then the right thing to do is to model $S$ as a (negative) Gamma distribution, which is the conjugate prior of the exponential distribution with (negative) mean $\lambda$. The updates then become simple summation of observations into the shape parameter and summation of number of observations into the rate parameter.

I assume you wanted the Gaussians because the space of Gamma distributions is not closed under summation. My suggestion is to interpret the mean and second moment that you are summing in $S$ as the unique gamma distribution having that mean and second moment. You may need to prevent your parameter updates from letting any of the means become negative.

• thanks! $\theta_{i}$ are parameters for interactions between the univariate distribution $X$ and the univariate distributions $Y_{1}, Y_{2}, ...$. $S_{i}^{y}$ is the sufficient statistic for the univariate gaussian $Y_{i}$.
– jmb
Commented Feb 10, 2015 at 20:31
• Sorry for the ambiguity, I also edited above.
– jmb
Commented Feb 10, 2015 at 20:38
• @jmb: you should write NP as a function of $Y_i$. Commented Feb 11, 2015 at 1:23
• $S_{i}^{y}$ is a function of $Y_{i}$, so this is already the case, right?
– jmb
Commented Feb 11, 2015 at 8:23
• @jmb: So you have two sets of parameters, those relating S to Y and $\theta$. You should probably have written both equations, and I do think that having two sets of parameters that are working at the same goal is a fundamentally bad idea. Anyway, my answer still stands. Commented Feb 11, 2015 at 17:19