What is the entropy of a composition of Gaussian Processes? I have a very basic understanding of Gaussian Processes.  From what I understand, a Guassian process for a set $X$, is the assignment of a Gaussian distribution to every element of the set.  This is meant to expand the idea of a function to the case where we don't have total information about a function.
I have a question about how we compute the entropy of a Gaussian Process over here.  I also asked about whether or not we can compose Gaussian Processes and that question is here.
This question combines those other ones.  Suppose we have entropies, $S_f$ and $S_g$, for two Gaussian Processes, $\mathcal{G}_f, \mathcal{G}_g$.  Next, suppose we can compose two Gaussian Processes, $\mathcal{G}_f \cdot \mathcal{G}_g$.  Is the entropy for the composition a simple function of the individual entropies? 
ie
$$S_{\mathcal{G}_f \cdot \mathcal{G}_g} = F(S_f, S_g)$$
 A: Based on my other two answers to your questions, I do not think that such an expression exists because the entropy of the composition $f \circ g$ will explicitly depend on values of the function $g$, which is random.
One way to estimate the entropy for a fixed set of indices $t_1, ..., t_m$ value is by simulating the $\log(p_{f \circ g}(t))$.
Note that:
$$ p_G(g) =  |2 \pi \Sigma_G(t_1, ..., t_m)|^{-1/2}exp(-0.5*g^T \Sigma_G^{-1}(t_1, ..., t_m)g)$$
$$ p_{f \circ g | g}(f) =  |2 \pi \Sigma_F(g_1, ..., g_m)|^{-1/2}exp(-0.5*f^T \Sigma_F^{-1}(g_1, ..., g_m)f)$$
... and so, the joint density can be obtained by multiplying these. If we simulate a bunch of $g$s and $f$s and plug them into the log joint density and obtain a sample mean, we can obtain an estimate for the entropy.
Side note: 'training' a GP usually means working out the posterior distribution given a set of observed values. This posterior has a different covariance matrix (see this link) than a GP prior, hence a different entropy.
