MCMC using GIBBS sampling: can different burn-in be used for different parameters? I have run a stochastic volatility model with 4 parameters. I have used the Heidelberg and Welch convergence diagnostic. The result shows 3 out of 4 parameters have passed the stationary and half-width mean test but the 4th parameter failed the tests. I wanted to ask if I can use a different burn-in period for just the 4th parameter?
 A: Expanding on Xi'an's comment, your stationary distribution in four dimensions, and each sample drawn via Gibbs sampling is a four dimensional vector.
In fact, (unless you are using block Gibbs sampling in a particular way), each component of the Markov chain is not a Markov chain, and thus convergence cannot be assessed for each component separately. More formally,
Let $\pi(x)$ be the stationary distribution for the Markov chain $\{X_t\}_{t>0}$. Here is the state space is $\mathcal{X} \subseteq \mathbb{R}^4$. Then the draws are,
\begin{align*}
X_1 &= (X_{11}, X_{12}, X_{13}, X_{14}) \\
\vdots\\
X_n & = (X_{n1}, X_{n2}, X_{n3}, X_{n4}).
\end{align*}
Convergence is assessed on $X_1, X_2, \dots$ and not on any $i$th component $X_{1i}, X_{2i}, \dots$. Thus is the 4th component $X_{\cdot 4}$ has not converged to its marginal of $\pi(x)$, then that means that the whole Markov chain has not converged.
One way to address your more larger problem is to use a Random Scan Gibbs Sampler and setting the 4th component to be updated with larger probability. 
