# Question about paper on Bayesian Shrinkage Estimation

I am reading the paper Bayesian Shrinkage Estimation of the Relative Abundance of mRNA Transcripts Using SAGE, and I am trying to work out the calculations for the complete conditionals for the Gibbs scheme presented in Section 3.5 (the prior structure is described in Section 3.1).

I am confused about how to calculate the full joint probability model and how to derive $$p(\lambda \mid X, ...)$$, $$p(\pi^* \mid X, ...)$$, $$p(q_A \mid X, ...)$$, and $$p(q_S \mid X, ...)$$. Is the below expression for the full joint correct?

\begin{align*}p(\pi^*, q_A, q_S, \lambda, X) &\propto {(\pi^*)}^{\sum_{j=1}^{|A|} \lambda_j} (1-\pi^*)^{\sum_{j=|A| + 1}^{N} \lambda_j} \prod_{j=1}^{|A|} q_{A_j}^{\lambda_j} \prod_{j=|A|+1}^N q_{S_j}^{\lambda_j} \cdot \prod_{j = 1}^{|A|} q_{A_j}^{\theta_A - 1} \prod_{j = |A| + 1}^{N} q_{S_j}^{\theta_S - 1} \\ &\cdot \frac{1}{B(a_{\pi^*}, b_{\pi^*})} (\pi^*)^{a_{\pi^*} - 1}(1 - \pi^*)^{b_{\pi^*} - 1} \cdot P^{\lambda} (1 - P)^{1 - \lambda}\end{align*}

I'm not sure where to go from here to calculate $$p(\lambda \mid X, ...)$$, so I'd appreciate any help. Thanks in advance!

Edit: I was thinking of defining $$A_1 = \{j : \lambda_j = 1\}$$ and $$A_0 = \{j : \lambda_j = 0\}$$ with $$M_1 = |A_1|$$ and $$M_0 = N - M_1$$. Then we assume WLOG that $$\lambda_j = 1$$ for $$j = 1, ..., M_1$$ and $$\lambda_j = 0$$ for $$j = M_1 + 1, ..., M_1 + M_0$$. Then would the joint be

\begin{align*}p(\pi^*, q_A, q_S, \lambda, X) &\propto {(\pi^*)}^{\sum_{j=1}^{M_1} \lambda_j} (1-\pi^*)^{\sum_{j=M_1 + 1}^{N} \lambda_j} \prod_{j=1}^{M_1} q_{A_j}^{\lambda_j} \prod_{j=M_1+1}^N q_{S_j}^{\lambda_j} \cdot \prod_{j = 1}^{M_1} q_{A_j}^{\theta_A - 1} \prod_{j = M_1 + 1}^{N} q_{S_j}^{\theta_S - 1} \\ &\cdot \frac{1}{B(a_{\pi^*}, b_{\pi^*})} (\pi^*)^{a_{\pi^*} - 1}(1 - \pi^*)^{b_{\pi^*} - 1} \cdot P^{\lambda} (1 - P)^{1 - \lambda}?\end{align*}

Edit 2: I am still stuck on calculating the conditional posterior for $$\lambda$$ and $$\pi^*$$. Would it make sense to evaluate the joint at $$\lambda_j = 1$$ and $$\lambda_j = 0$$? I realize that there is a factor to consider in $$p(q_A) = Dir(q_A \mid a_\lambda,\ldots,a_\lambda)$$ for $$\lambda_j = \lambda$$.

• It looks like the author(s) of the paper used collapsed Gibbs sampling for $\lambda$. Your likelihood is incorrect; it appears to assume the 0s and 1s are ordered so that all 0s come before 1s. Instead, define index sets for $\lambda_j =0$ and $\lambda=1$. Commented Mar 16, 2022 at 3:33
• @DaeyoungLim I see; apart from that, is the likelihood correct?
– user310180
Commented Mar 16, 2022 at 4:00
• that is not correct. Use a more general notation and sum over the index set: $\sum_{j \in A_1} \lambda_j$. This reduces the relevant component to $(\pi^*)^{|A_1|}(1-\pi^*)^{N-|A_1|}$. I haven't read the paper thoroughly but your likelihood seems otherwise fine to me. Commented Mar 16, 2022 at 4:06
• @DaeyoungLim would you happen to have any ideas for the conditional posteriors $p(\lambda \mid ...)$ and $p(\pi^* \mid ...)$?
– user310180
Commented Mar 18, 2022 at 19:32
• They're conditionally conjugate priors, so $p(\lambda\mid -)$ should be a Bernoulli distribution and $p(\pi^*\mid -)$ should be a beta distribution. Commented Mar 18, 2022 at 19:34

Define the index set $$\mathcal{I}:=\{j: \lambda_j=1\}$$. Then, the posterior becomes \begin{align} p(\pi^*,\lambda,q_A,q_S\mid X)&\propto (\pi^*)^{|\mathcal{I}|}(1-\pi^*)^{N-|\mathcal{I}|}\prod_{j\in \mathcal{I}} q_{A_j}^{\lambda_j+\theta_A-1} \prod_{j \in \mathcal{I}^c} q_{S_j}^{1-\lambda_j +\theta_S-1} \\ &\times (\pi^*)^{a_\pi-1}(1-\pi^*)^{b_\pi-1} \,\prod_{j=1}^N P^{\lambda_j} (1-P)^{1-\lambda_j}, \end{align} where $$|\mathcal{I}|$$ indicates the cardinality of $$\mathcal{I}$$. The prior for $$\lambda_j$$ doesn't make sense without the subscript $$j$$ so I changed that part accordingly. When deriving a full conditional, ignore the components that don't have the parameter of interest. For example, $$p(\pi^*\mid -) \propto (\pi^*)^{|\mathcal{I}| + a_\pi-1}(1-\pi^*)^{N-|\mathcal{I}|+b_\pi-1} \sim \mathrm{Be}(a_\pi + |\mathcal{I}|, b_\pi + N - |\mathcal{I}|),$$ and $$p(\lambda_j\mid - ) \propto (q_{A_j} P)^{\lambda_j}\times \{q_{S_j}(1-P)\}^{(1-\lambda_j)} \sim \mathrm{Bernoulli}(p^*),$$ where $$p^* = \frac{q_{A_j}P}{q_{A_j}P+ q_{S_j}(1-P)}$$, since the components aren't guaranteed to sum to one.
• Ah, I completely missed combining $\prod_{j=1}^{M_1} q_{A_j}^{\lambda_j}$ and $\prod_{j = 1}^{M_1} q_{A_j}^{\theta_A - 1}$! Thanks for clearing it up for me. Would I need to marginalize the $p(\lambda_j \mid - )$ with respect to $q_{A_j}$ and $q_{S_j}$ to find the $p(\lambda_j \mid X, \pi^*, P)$ used in the collapsed Gibbs sampler for the $\lambda$?
• Yes, $\pi^*$ should be marginalized as well. Also, don't forget that when collapsed Gibbs sampling is used, sampling $\lambda_j$ must come before the parameters that have been marginalized. This is because sampling $(X,Y)$ jointly is equivalent to sampling $Y$ marginally and then sampling $X\mid Y$ (or vice versa). Collapsed Gibbs sampling is an application of this composition method. Commented Mar 18, 2022 at 20:28
• I see - how would I marginalize $p(\lambda_j\mid - ) \propto (q_{A_j} P)^{\lambda_j}\times \{q_{S_j}(1-P)\}^{(1-\lambda_j)}$?
• @cosmicflair you can’t marginalize the full conditional. Get $\lambda_j, \pi^*, q_{A_j}, q_{S_j}\mid -$ and integrate them out Commented Mar 18, 2022 at 21:43