I have two interrelated response variables $A$ and $B$ over each observation $i$ in my data. I am trying to create an unsupervised model where observations could be explained by means of latent spaces(like clustering). I am trying to come up with a generative story that uses two overlapping latent spaces.

By that I first mean, each outcome $A_i$ depends on $\theta^A_i$ coming from a space $\mathcal{X}^A$, and each outcome $B_i$ depends on $\theta^B_i$ coming from a space $\mathcal{X}^B$.

In other words: $\theta^A_i \in \mathcal{X}^A$, and $\theta^B_i \in \mathcal{X}^B$.

However I want there to be a shared space $\mathcal{X}^S$ such that $\mathcal{X}^S = \mathcal{X}^A \cap \mathcal{X}^B \neq \varnothing$. For example, assuming that $\mathcal{X}^A$ is $K^A$ dimensional, $\mathcal{X}^B$ is $K^B$ dimensional space, and $\mathcal{X}^S$ is $K^S$ dimensional, $K^S \neq 0$ and $\mathcal{X}^A \cup \mathcal{X}^B$ is $K^A + K^B - 2K^S$ dimensional.

This suggests that each of the $\theta^A_i$'s are drawn from a space which contains $A$-specific and shared parts. and similarly $\theta^B_i$'s are drawn from a space that has $B$-specific and shared parts.

Let's say each of the $\theta^A_i$'s and $\theta^B_i$'s come from a Multivariate Gaussian, how can I specify their distributions so that I can formulate the specific and shared parts?

For example would it make sense to define the following?

$\begin{bmatrix} \theta^A_i \\ \theta^B_i \end{bmatrix}\sim MVN \begin{pmatrix} \begin{bmatrix} \mu^{A} \\ \mu^{B} \end{bmatrix} , \begin{bmatrix} \Sigma^{A} & \Sigma^{AB} \\ \Sigma^{BA} & \Sigma^{B} \end{bmatrix} \end{pmatrix} $


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.