Mean and error bounds of log-transformed data using Gaussian process regression

To revive a past question and establish a definitive answer, how should the mean/mode and error intervals of log-transformed data be handled when applying Gaussian process regression?

For example, I obtain some original data, $$Z$$, from an experiment. Now suppose that $$Z = exp(Y)$$, where $$Y \sim N(\mu,\sigma^2)$$. Therefore $$Z$$ follows a log-normal distribution. It now appears natural to apply Gaussian process regression on $$Y = ln(Z)$$, which allows me to predict a future point $$Y^*$$, where $$Y^*\sim N(\mu^*,\sigma^{*2})$$.

But what is the appropriate method to convert $$Y^*$$ back into my original space, $$Z^*$$? For example, the 95% confidence interval in $$Y$$-space would be $$[\mu^* - 1.96\sigma^*,\mu^* + 1.96\sigma^*]$$. Is it appropriate to simply use a prediction interval of $$[e^{\mu^* - 1.96\sigma^*},e^{\mu^* + 1.96\sigma^*}]$$ when converting back into my original space? Are there established or recommended techniques when performing this back-transformation?

• Your GP should give you posterior credible intervals on the transformed scale. It is natural to apply the inverse transform, as you've noted, to the posterior credible intervals to obtain credible intervals on the natural scale. This is done in stats pretty frequently (see the variety of binomial confidence intervals). Apr 9, 2019 at 3:13
• Interesting, is it "valid" then to describe this interval in the original scale as a 95% confidence interval if it is in the transformed scale? Are there possible pitfalls when using this approach (e.g. evaluation of errors on the original data, confidence interval on the median as opposed to the mean)? Apr 9, 2019 at 14:16
• Yes, it is valud to describe the interval as a confidence interval. But in GP, since you are computing posteriors, they are credible intervals, not confidence intervals. Apr 9, 2019 at 14:18
• Hi @Mathews24, I am dealing with the same issue right now, I was wondering if you found a definitive answer to this? Apr 18, 2020 at 9:06

I just encountered the same issue, and the most definitive answer I can find is from this paper. The prediction in the original space, $$Z^*$$, is not normally distributed. Consequently it has a different median and mean. The median of $$Z^*$$ can be modelled as $$Z^\mathrm{med}=e^{\mu^*}$$ with confidence interval $$Z^{\mathrm{med}\pm1.96\sigma}=e^{\mu^*\pm1.96\sigma^*}$$.

However if you want the mean and variance in the original space, you'll have to integrate over the distribution in the transformed space:

$$\langle Z^*\rangle=\int_{-\infty}^\infty e^Y \mathcal{N}(\mu^*,{\sigma^*}^2)dY=e^{\mu^*+{\sigma^*}^2/2}$$

And similarly for the variance:

$$\mathrm{var}(Z^*) = \int_{-\infty}^\infty (e^Y-\langle Z^*\rangle)^2 \mathcal{N}(\mu^*,{\sigma^*}^2)dY = (e^{{\sigma^*}^2}-1)e^{2\mu^*+{\sigma^*}^2}$$

In this case this is the log-normal distribution. But the same method could be applied to other transformations.

In my own models I found the median to be a much better predictor - I'd need to do a bit more research into why that's the case though. The linked paper also uses the median rather than the mean but doesn't explain why.