# Is it valid to perform Bayesian optimization and MCMC sampling with nested model parameters?

I am trying to think of a way to perform a somewhat complicated Bayesian optimization problem involving model parameters which are nested within other functions in the forward problem, and I am wondering if this is alid from a theoretical perspective or if I am violating one of the criterion for MCMC sampling methods.

To simplify the problem to it's most basic situation, I have some data vector, $$\mathbf{d}$$ (with error $$\mathbf{\sigma}$$ and length $$N$$) from which I want to ascertain some model, $$\mathbf{m}$$. I have a forward operator, $$F(\mathbf{m})$$ which takes a model and computes predicted data, $$\mathbf{d_p}$$. From this, I construct a log likelihood function using a chi-square misfit assuming Gaussian errors:

$$\chi^2 = (\mathbf{d}-F(\mathbf{m}))^T\mathbf{C_d}^{-1}(\mathbf{d}-F(\mathbf{m}))$$

where $$\mathbf{C_d}$$ is an $$N$$ x $$N$$ matrix with $$\mathbf{\sigma}$$ on the diagonal.

My question concerns the forward operator, $$F(\mathbf{m})$$.

In my scenario, the forward operator is complicated and contains many nested functions. For example, suppose I have 5 model parameters, $$m_1, m_2, ... m_5$$. In my situation I might have:

$$a = P(m_1)$$

$$b = Q(a, m_2, m_3)$$

$$c = R(a, b, m_4, m_5)$$

$$\mathbf{d_p} = S(a,b,c)$$

where $$P, Q, R,$$ and $$S$$ are some very complicated functions. In order to actually compute predicted data, $$\mathbf{d_p}$$, I need $$a,b$$, and $$c$$. But my model parameters of interest are in $$\mathbf{m}$$.

Is this nested-variables approach valid for sampling the posterior?

Note: I have tried implementing the problem as written using a known model and I run into various problems. First and foremost the MCMC sampler gets "stuck" in an area of low probability (even though I know that there is a better solution with higher probability since I am using a known model). This does not happen if I try to sample $$a, b$$ and $$c$$ as model parameters. So I am wondering if there is some theoretical violation of the method.

• (a) the $\chi^2$ expression is unlikely to be a likelihood function. Do you mean log- or negative log-likelihood? (b) I do not understand the distinction between $F(\mathcal m)$ and $\mathcal{d_p}$. (c) The model parameter is $\mathcal m$ and thus should be the one simulated, while $a,b,c$ are deterministic functions of $\mathcal m$. This seems unrelated with the use of MCMC or another simulation technique. Apr 16, 2021 at 16:57
• (a) Yes it should be log likelihood as you say. (b) $d_p = F(m)$ (i.e. in order to compute predicted data ($d_p$), I need to pass my model parameters through some function $F(m)$). (c) $m_1, m_2, m_3, ...$ are the unknown model parameters. As such $a, b,$ and $c$ are also unknown (and depend on $m$). I am trying to sample the posterior distribution (of e.g. $m$) given the likelihood (and some prior which at this point are just upper and lower limits on the modelled parameters). Apr 16, 2021 at 17:15

Considering a likelihood function $$\exp\{-(\mathbf d-F(\mathbf m))^2/\sigma^2\}$$ is equivalent to assume the observation is Normal. The posterior distribution is hence $$\pi(\mathbf m|\mathbf d) \propto \exp\{-(\mathbf d-F(\mathbf m))^2/\sigma^2\} \pi(\mathbf m)$$ which can be simulated by an MCMC algorithm, no matter how complex the transform $$F(\cdot)$$ is. Simulating directly and jointly the triplet $$(a,b,c)$$ is feasible, but this requires the projected prior $$\pi^\bot(a,b,c)$$ to be available in close form.