2
$\begingroup$

I am trying to think of a way to perform a somewhat complicated Bayesian optimization problem involving model parameters which are nested within other functions in the forward problem, and I am wondering if this is alid from a theoretical perspective or if I am violating one of the criterion for MCMC sampling methods.

To simplify the problem to it's most basic situation, I have some data vector, $\mathbf{d}$ (with error $\mathbf{\sigma}$ and length $N$) from which I want to ascertain some model, $\mathbf{m}$. I have a forward operator, $F(\mathbf{m})$ which takes a model and computes predicted data, $\mathbf{d_p}$. From this, I construct a log likelihood function using a chi-square misfit assuming Gaussian errors:

$$\chi^2 = (\mathbf{d}-F(\mathbf{m}))^T\mathbf{C_d}^{-1}(\mathbf{d}-F(\mathbf{m}))$$

where $\mathbf{C_d}$ is an $N$ x $N$ matrix with $\mathbf{\sigma}$ on the diagonal.

My question concerns the forward operator, $F(\mathbf{m})$.

In my scenario, the forward operator is complicated and contains many nested functions. For example, suppose I have 5 model parameters, $m_1, m_2, ... m_5$. In my situation I might have:

$a = P(m_1)$

$b = Q(a, m_2, m_3)$

$c = R(a, b, m_4, m_5)$

$\mathbf{d_p} = S(a,b,c)$

where $P, Q, R,$ and $S$ are some very complicated functions. In order to actually compute predicted data, $\mathbf{d_p}$, I need $a,b$, and $c$. But my model parameters of interest are in $\mathbf{m}$.

Is this nested-variables approach valid for sampling the posterior?

Note: I have tried implementing the problem as written using a known model and I run into various problems. First and foremost the MCMC sampler gets "stuck" in an area of low probability (even though I know that there is a better solution with higher probability since I am using a known model). This does not happen if I try to sample $a, b$ and $c$ as model parameters. So I am wondering if there is some theoretical violation of the method.

Improve this question
$\endgroup$
2
  • $\begingroup$ (a) the $\chi^2$ expression is unlikely to be a likelihood function. Do you mean log- or negative log-likelihood? (b) I do not understand the distinction between $F(\mathcal m)$ and $\mathcal{d_p}$. (c) The model parameter is $\mathcal m$ and thus should be the one simulated, while $a,b,c$ are deterministic functions of $\mathcal m$. This seems unrelated with the use of MCMC or another simulation technique. $\endgroup$
    – Xi'an
    Commented Apr 16, 2021 at 16:57
  • $\begingroup$ (a) Yes it should be log likelihood as you say. (b) $d_p = F(m)$ (i.e. in order to compute predicted data ($d_p$), I need to pass my model parameters through some function $F(m)$). (c) $m_1, m_2, m_3, ...$ are the unknown model parameters. As such $a, b, $ and $c$ are also unknown (and depend on $m$). I am trying to sample the posterior distribution (of e.g. $m$) given the likelihood (and some prior which at this point are just upper and lower limits on the modelled parameters). $\endgroup$
    – Darcy
    Commented Apr 16, 2021 at 17:15

1 Answer 1

Reset to default
1
$\begingroup$

Considering a likelihood function $$\exp\{-(\mathbf d-F(\mathbf m))^2/\sigma^2\}$$ is equivalent to assume the observation is Normal. The posterior distribution is hence $$\pi(\mathbf m|\mathbf d) \propto \exp\{-(\mathbf d-F(\mathbf m))^2/\sigma^2\} \pi(\mathbf m)$$ which can be simulated by an MCMC algorithm, no matter how complex the transform $F(\cdot)$ is. Simulating directly and jointly the triplet $(a,b,c)$ is feasible, but this requires the projected prior $\pi^\bot(a,b,c)$ to be available in close form.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.