In these two papers, mainly:
- Klys, Jack, Jake Snell, and Richard Zemel. "Learning latent subspaces in variational autoencoders." Advances in neural information processing systems 31 (2018). https://proceedings.neurips.cc/paper/2018/file/73e5080f0f3804cb9cf470a8ce895dac-Paper.pdf
ELBO: $\log p_{\theta, \gamma}(\mathbf{x}, \mathbf{y}) \geq \log \mathbb{E}_{q_\phi(\mathbf{z},\mathbf{w}|\mathbf{x},\mathbf{y})}\left[ p_{\theta}(\mathbf{x}|\mathbf{w},\mathbf{z}) \right] - D_{KL}\left( q_\phi(\mathbf{z}|\mathbf{x},\mathbf{y}) || p(\mathbf{z}) \right) +\log p(\mathbf{y}) - D_{KL}\left( q_\phi(\mathbf{w}|\mathbf{x},\mathbf{y})||p_\gamma(\mathbf{w}|\mathbf{y})) \right)$
- Ilse, Maximilian, et al. "Diva: Domain invariant variational autoencoders." Medical Imaging with Deep Learning. PMLR, 2020. http://proceedings.mlr.press/v121/ilse20a/ilse20a.pdf
ELBO: $\log p(\mathbf{x}, \mathbf{y}) \geq \mathbb{E}_{q_{\phi_x}(\mathbf{z}_x|\mathbf{x})q_{\phi_y}(\mathbf{z}_y|\mathbf{x})}\left[ \log p_\theta(\mathbf{x}|\mathbf{z}_x,\mathbf{z}_y) \right] - D_{KL}\left( q_{\phi_x}(\mathbf{z}_x|\mathbf{x})||p(\mathbf{z}_x) \right) - D_{KL}\left( q_{\phi_y} (\mathbf{z}_y | \mathbf{x}) || p_{\theta_y}(\mathbf{z}_y|y) \right) $
An auxiliary classifier is added to the VAE loss:
Given $\mathbf{x}$ = observed variables, $\mathbf{y}$ = labels corresponding to $\mathbf{x}$, $\mathbf{z}$ = latent variables irrelevant to $\mathbf{y}$, and $\mathbf{w}$ = latent variables related to $\mathbf{y}$, the additional maximization objective is: $\max_\delta \mathbb{E}_{q(\mathbf{z}|\mathbf{x})}\left[ q_\delta(\mathbf{y}|\mathbf{z}) \right]$
Similarly, the auxiliary classifier is: $\mathbb{E}_{q_{\phi_y}(\mathbf{z}_y|\mathbf{x})}\left[ \log q_{\omega_y}(y|\mathbf{z}_y) \right]$
But, in the two ELBOs above, I also notice there is KL-divergence term to on (1) $\mathbf{w}$ and (2) $\mathbf{z}_y$, respectively, which are the latent variables related to the label $\mathbf{y}$ encoded from the input $\mathbf{x}$:
$D_{KL}\left( q_\phi(\mathbf{w}|\mathbf{x},\mathbf{y}) || p_\gamma(\mathbf{w}|\mathbf{y}) \right)$
$D_{KL}\left( q_{\phi_y}(\mathbf{z}_y|\mathbf{x}) || p_{\theta_y}(\mathbf{z}_y|y) \right)$
Are these KL-div terms not enough to encourage encoding the information on $\mathbf{x}$ related to $\mathbf{y}$ into (1) $\mathbf{w}$ and (2) $\mathbf{z}_y$, respectively?
