0
$\begingroup$

I am reading the paper Bayesian Shrinkage Estimation of the Relative Abundance of mRNA Transcripts Using SAGE, and I am trying to work out the calculations for the complete conditionals for the Gibbs scheme presented in Section 3.5 (the prior structure is described in Section 3.1).

I am confused about how to calculate the full joint probability model and how to derive $p(\lambda \mid X, ...)$, $p(\pi^* \mid X, ...)$, $p(q_A \mid X, ...)$, and $p(q_S \mid X, ...)$. Is the below expression for the full joint correct?

\begin{align*}p(\pi^*, q_A, q_S, \lambda, X) &\propto {(\pi^*)}^{\sum_{j=1}^{|A|} \lambda_j} (1-\pi^*)^{\sum_{j=|A| + 1}^{N} \lambda_j} \prod_{j=1}^{|A|} q_{A_j}^{\lambda_j} \prod_{j=|A|+1}^N q_{S_j}^{\lambda_j} \cdot \prod_{j = 1}^{|A|} q_{A_j}^{\theta_A - 1} \prod_{j = |A| + 1}^{N} q_{S_j}^{\theta_S - 1} \\ &\cdot \frac{1}{B(a_{\pi^*}, b_{\pi^*})} (\pi^*)^{a_{\pi^*} - 1}(1 - \pi^*)^{b_{\pi^*} - 1} \cdot P^{\lambda} (1 - P)^{1 - \lambda}\end{align*}

I'm not sure where to go from here to calculate $p(\lambda \mid X, ...)$, so I'd appreciate any help. Thanks in advance!

Edit: I was thinking of defining $A_1 = \{j : \lambda_j = 1\}$ and $A_0 = \{j : \lambda_j = 0\}$ with $M_1 = |A_1|$ and $M_0 = N - M_1$. Then we assume WLOG that $\lambda_j = 1$ for $j = 1, ..., M_1$ and $\lambda_j = 0$ for $j = M_1 + 1, ..., M_1 + M_0$. Then would the joint be

\begin{align*}p(\pi^*, q_A, q_S, \lambda, X) &\propto {(\pi^*)}^{\sum_{j=1}^{M_1} \lambda_j} (1-\pi^*)^{\sum_{j=M_1 + 1}^{N} \lambda_j} \prod_{j=1}^{M_1} q_{A_j}^{\lambda_j} \prod_{j=M_1+1}^N q_{S_j}^{\lambda_j} \cdot \prod_{j = 1}^{M_1} q_{A_j}^{\theta_A - 1} \prod_{j = M_1 + 1}^{N} q_{S_j}^{\theta_S - 1} \\ &\cdot \frac{1}{B(a_{\pi^*}, b_{\pi^*})} (\pi^*)^{a_{\pi^*} - 1}(1 - \pi^*)^{b_{\pi^*} - 1} \cdot P^{\lambda} (1 - P)^{1 - \lambda}?\end{align*}

Edit 2: I am still stuck on calculating the conditional posterior for $\lambda$ and $\pi^*$. Would it make sense to evaluate the joint at $\lambda_j = 1$ and $\lambda_j = 0$? I realize that there is a factor to consider in $p(q_A) = Dir(q_A \mid a_\lambda,\ldots,a_\lambda)$ for $\lambda_j = \lambda$.

$\endgroup$
6
  • 1
    $\begingroup$ It looks like the author(s) of the paper used collapsed Gibbs sampling for $\lambda$. Your likelihood is incorrect; it appears to assume the 0s and 1s are ordered so that all 0s come before 1s. Instead, define index sets for $\lambda_j =0$ and $\lambda=1$. $\endgroup$
    – Daeyoung
    Commented Mar 16, 2022 at 3:33
  • $\begingroup$ @DaeyoungLim I see; apart from that, is the likelihood correct? $\endgroup$
    – user310180
    Commented Mar 16, 2022 at 4:00
  • 2
    $\begingroup$ that is not correct. Use a more general notation and sum over the index set: $\sum_{j \in A_1} \lambda_j$. This reduces the relevant component to $(\pi^*)^{|A_1|}(1-\pi^*)^{N-|A_1|}$. I haven't read the paper thoroughly but your likelihood seems otherwise fine to me. $\endgroup$
    – Daeyoung
    Commented Mar 16, 2022 at 4:06
  • $\begingroup$ @DaeyoungLim would you happen to have any ideas for the conditional posteriors $p(\lambda \mid ...)$ and $p(\pi^* \mid ...)$? $\endgroup$
    – user310180
    Commented Mar 18, 2022 at 19:32
  • $\begingroup$ They're conditionally conjugate priors, so $p(\lambda\mid -)$ should be a Bernoulli distribution and $p(\pi^*\mid -)$ should be a beta distribution. $\endgroup$
    – Daeyoung
    Commented Mar 18, 2022 at 19:34

1 Answer 1

0
$\begingroup$

Define the index set $\mathcal{I}:=\{j: \lambda_j=1\}$. Then, the posterior becomes $$ \begin{align} p(\pi^*,\lambda,q_A,q_S\mid X)&\propto (\pi^*)^{|\mathcal{I}|}(1-\pi^*)^{N-|\mathcal{I}|}\prod_{j\in \mathcal{I}} q_{A_j}^{\lambda_j+\theta_A-1} \prod_{j \in \mathcal{I}^c} q_{S_j}^{1-\lambda_j +\theta_S-1} \\ &\times (\pi^*)^{a_\pi-1}(1-\pi^*)^{b_\pi-1} \,\prod_{j=1}^N P^{\lambda_j} (1-P)^{1-\lambda_j}, \end{align} $$ where $|\mathcal{I}|$ indicates the cardinality of $\mathcal{I}$. The prior for $\lambda_j$ doesn't make sense without the subscript $j$ so I changed that part accordingly. When deriving a full conditional, ignore the components that don't have the parameter of interest. For example, $$ p(\pi^*\mid -) \propto (\pi^*)^{|\mathcal{I}| + a_\pi-1}(1-\pi^*)^{N-|\mathcal{I}|+b_\pi-1} \sim \mathrm{Be}(a_\pi + |\mathcal{I}|, b_\pi + N - |\mathcal{I}|), $$ and $$ p(\lambda_j\mid - ) \propto (q_{A_j} P)^{\lambda_j}\times \{q_{S_j}(1-P)\}^{(1-\lambda_j)} \sim \mathrm{Bernoulli}(p^*), $$ where $p^* = \frac{q_{A_j}P}{q_{A_j}P+ q_{S_j}(1-P)}$, since the components aren't guaranteed to sum to one.

$\endgroup$
6
  • $\begingroup$ Ah, I completely missed combining $\prod_{j=1}^{M_1} q_{A_j}^{\lambda_j}$ and $\prod_{j = 1}^{M_1} q_{A_j}^{\theta_A - 1}$! Thanks for clearing it up for me. Would I need to marginalize the $p(\lambda_j \mid - )$ with respect to $q_{A_j}$ and $q_{S_j}$ to find the $p(\lambda_j \mid X, \pi^*, P)$ used in the collapsed Gibbs sampler for the $\lambda$? $\endgroup$
    – user310180
    Commented Mar 18, 2022 at 20:20
  • $\begingroup$ Yes, $\pi^*$ should be marginalized as well. Also, don't forget that when collapsed Gibbs sampling is used, sampling $\lambda_j$ must come before the parameters that have been marginalized. This is because sampling $(X,Y)$ jointly is equivalent to sampling $Y$ marginally and then sampling $X\mid Y$ (or vice versa). Collapsed Gibbs sampling is an application of this composition method. $\endgroup$
    – Daeyoung
    Commented Mar 18, 2022 at 20:28
  • $\begingroup$ I see - how would I marginalize $p(\lambda_j\mid - ) \propto (q_{A_j} P)^{\lambda_j}\times \{q_{S_j}(1-P)\}^{(1-\lambda_j)}$? $\endgroup$
    – user310180
    Commented Mar 18, 2022 at 21:09
  • $\begingroup$ @cosmicflair you can’t marginalize the full conditional. Get $\lambda_j, \pi^*, q_{A_j}, q_{S_j}\mid -$ and integrate them out $\endgroup$
    – Daeyoung
    Commented Mar 18, 2022 at 21:43
  • 1
    $\begingroup$ @cosmicflair I saw your other post. I’ll come up with a complete answer tomorrow $\endgroup$
    – Daeyoung
    Commented Mar 19, 2022 at 4:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.