Model-based Bayesian inference with unknown noise variance

I want to infer the conditional posterior distribution of the unknown system parameter $\theta_1$ given the empirical data $\boldsymbol{y}$, with $\theta_1 \in \mathbb{R}$ and $\boldsymbol{y} \in \mathbb{R}^3$. I have a numerical model $f(\theta_1)$ for the data. The model response differ from the data by some amount due to noise. This noise is assumed Gaussian with zero mean and unknown variance $\boldsymbol{\sigma^2}$, which leads to a Gaussian likelihood distribution of the data given the unknown parameters $\theta_1$, and $\theta_2=\boldsymbol{\sigma^2}$. I assume a noninformative uniform prior distribution for both parameters.

I have some difficulties regarding the implementation of the MCMC Metropolis-Hastings algorithm for the evaluation of the posterior distribution. I use truncated Gaussian proposals (the support of $\theta_1$ is bounded and $\theta_2$ must be positive) for both unknowns: $$q_1(\theta_{1}^{\ t}|\theta_{1}^{\ t-1}) = N(\theta_1^{\ t-1},\sigma_1^2)$$ $$q_2(\theta_{2}^{\ t}|\theta_{2}^{\ t-1}) = N(\theta_2^{\ t-1},\sigma_2^2)$$.

My problem is that I can not scale the proposals distributions such that the acceptance rate be between $0.3$ and $0.6$. In fact, convergence is never attained. However, if I simplify the problem and I adopt a value for the Gaussian noise variance, in order to estimate a single parameter ($\theta_1$), the posterior distribution converges and the proposal scale becomes easy to tune.

I would appreciate if someone can give me some insights regarding the application of the MH algorithm for the estimation of the system parameter $\theta_1$ with unknown noise variance $\boldsymbol{\sigma^2}$.

• If you have a single observation $\mathbf{y}$ then your posterior is improper. – Xi'an Jan 13 '17 at 17:42