Is it ok to widen a prior during an MCMC which did not converge yet? I am calibrating parameters of a process model. The runtime of the model is high and the calibration already ran for more than two weeks with many cores on a HPC.
After almost 150k iterations I realize that the uniform prior for one parameter was too narrow. Which means that the sampler is getting very close to the upper limit of this parameter.
I would prefer to not run everything again just to widen the prior of this parameter. So, the fastest (and most efficient) solution is to just continue the MCMC with adjusting the limits of the prior.
Is this ok?
More specifically, does this have any consequences for validity of the posterior or the calibration?
 A: Since the change of prior involves a change of support, the new prior can be written as
$$\pi(\theta)=\pi_0(\theta)+\pi_1(\theta)\tag{1}$$
where $\pi_0$ is the former prior and $\pi_1$ is the prior on the supplementary domain.$¹$ We thus have$$\pi_0(\theta)\pi_1(\theta)\equiv0$$ due to the exclusive supports. When moving to the posterior distribution,
$$\pi(\theta|x)\propto\pi(\theta)f(x|\theta)=\pi_0(\theta)f(x|\theta)+\pi_1(\theta)f(x|\theta)=\mathfrak e_0(x)\pi_0(\theta|x)+\mathfrak e_1(x)\pi_1(\theta|x)$$
where $\mathfrak e_i$ is the evidence attached to prior $\pi_i$
$$\mathfrak e_i=\int f(x|\theta)\pi_i(\theta)\,\text d\theta$$
This means that simulating from $\pi(\theta|x)$ can proceed by simulating from $\pi_0(\theta|x)$ [which is already done in the current setting] and from $\pi_1(\theta|x)$. Merging the two samples can be done in proportions $\mathfrak e_0(x)$ and $\mathfrak e_1(x)$.
Since computing the evidences is usually a challenge (or at least a chore), another MCMC algorithm could be run in parallel, directly targeting (1). Or, alternatively, an artificial RJMCMC step could be added to an MCMC algorithm targeting $\pi_1(\theta|x)$, proposing a pseudo-move to the support of $\pi_0(\theta|x)$ and averaging the probability of accepting this move, since it should be
$$\mathfrak e_0(x)\big/\mathfrak e_0(x)+\mathfrak e_1(x)$$

$¹$I intentionally do not renormalise the sum (1) as this representation covers the case when both components are differently weighted.
