Mean and error bounds of log-transformed data using Gaussian process regression

Question

To revive a past question and establish a definitive answer, how should the mean/mode and error intervals of log-transformed data be handled when applying Gaussian process regression?

For example, I obtain some original data, $Z$, from an experiment. Now suppose that $Z = exp(Y)$, where $Y \sim N(\mu,\sigma^2)$. Therefore $Z$ follows a log-normal distribution. It now appears natural to apply Gaussian process regression on $Y = ln(Z)$, which allows me to predict a future point $Y^*$, where $Y^*\sim N(\mu^*,\sigma^{*2})$.

But what is the appropriate method to convert $Y^*$ back into my original space, $Z^*$? For example, the 95% confidence interval in $Y$-space would be $[\mu^* - 1.96\sigma^*,\mu^* + 1.96\sigma^*]$. Is it appropriate to simply use a prediction interval of $[e^{\mu^* - 1.96\sigma^*},e^{\mu^* + 1.96\sigma^*}]$ when converting back into my original space? Are there established or recommended techniques when performing this back-transformation?

Answer 1

I just encountered the same issue, and the most definitive answer I can find is from this paper. The prediction in the original space, $Z^*$, is not normally distributed. Consequently it has a different median and mean. The median of $Z^*$ can be modelled as $Z^\mathrm{med}=e^{\mu^*}$ with confidence interval $Z^{\mathrm{med}\pm1.96\sigma}=e^{\mu^*\pm1.96\sigma^*}$.

However if you want the mean and variance in the original space, you'll have to integrate over the distribution in the transformed space:

$\langle Z^*\rangle=\int_{-\infty}^\infty e^Y \mathcal{N}(\mu^*,{\sigma^*}^2)dY=e^{\mu^*+{\sigma^*}^2/2}$

And similarly for the variance:

$\mathrm{var}(Z^*) = \int_{-\infty}^\infty (e^Y-\langle Z^*\rangle)^2 \mathcal{N}(\mu^*,{\sigma^*}^2)dY = (e^{{\sigma^*}^2}-1)e^{2\mu^*+{\sigma^*}^2}$

In this case this is the log-normal distribution. But the same method could be applied to other transformations.

In my own models I found the median to be a much better predictor - I'd need to do a bit more research into why that's the case though. The linked paper also uses the median rather than the mean but doesn't explain why.