I try to understand a statement in this paper:

http://papers.nips.cc/paper/4305-spike-and-slab-variational-inference-for-multi-task-and-multiple-kernel-learning.pdf

In particular, I am talking about the reparameterisation from (1c) to (3).

Given is a random variable $w_{qm}$, that is distributed according to:

(A) $p(w_{qm})=\Pi \mathcal{N}(w_{qm}|0,\sigma_w²)+(1-\Pi)\delta_0(w_{qm})$

Specifically, with probability $\Pi$, each $w_{qm}$ is zero, and with probability, it is drawn from a Gaussian.

Further, assume a Gaussian random variable $\tilde{w}_{qm}\sim \mathcal{N}(\tilde{w}_{qm}|0,\sigma_w²)$ and a Bernoulli random variable $s_{qm}\sim\Pi^{s_{qm}}(1-\Pi)^{1-s_{qm}}$. The new random variable $\tilde{w}_{qm}s_{qm}$ is assumed to follow (A), resulting in

(B) $p(w_{qm})=\mathcal{N}(w_{qm}|0,\sigma_w²)[\Pi^{s_{qm}}(1-\Pi)^{1-s_{qm}}]$

I don't really see how the authors did it ((A)→(B)). I am grateful for any help.

Thanx so much in advance!

Krn

I try to understand a statement in this paper:

http://papers.nips.cc/paper/4305-spike-and-slab-variational-inference-for-multi-task-and-multiple-kernel-learning.pdf

In particular, I am talking about the reparameterisation from (1c) to (3).

Given is a random variable $w_{qm}$, that is distributed according to:

(1c) $w_{qm} \sim \pi \mathcal{N}(w_{qm}|0,\sigma_w²)+(1-\pi)\delta_0(w_{qm})$

Specifically, with probability $\pi$, each $w_{qm}$ is zero, and with probability, it is drawn from a Gaussian.

Further, assume a Gaussian random variable $\tilde{w}_{qm}\sim \mathcal{N}(\tilde{w}_{qm}|0,\sigma_w²)$ and a Bernoulli random variable $s_{qm}\sim\pi^{s_{qm}}(1-\pi)^{1-s_{qm}}$. The new random variable $\tilde{w}_{qm}s_{qm}$ is assumed to follow (1c), resulting in

(3) $p(\tilde{w}_{qm},s_{qm})=\mathcal{N}(w_{qm}|0,\sigma_w²)[\pi^{s_{qm}}(1-\pi)^{1-s_{qm}}]$

I don't really see how the authors did it ((1c)→(3)). I am grateful for any help.

Thanx so much in advance!

Krn

I try to understand a statement in this paper:

http://papers.nips.cc/paper/4305-spike-and-slab-variational-inference-for-multi-task-and-multiple-kernel-learning.pdf

In particular, I am talking about the reparameterisation from (1c) to (3).

I don't really see how the authors did it. I am grateful for any help. Aslo, I attached some of my own guessing based thoughts.

Thanx so much in advance!

Krn

I guessed something like: Ok, the statement is $p(w_{qm})=p(s_{qm}\tilde{w}_{qm})$, $s_{qm}\in\{0,1\}$

(a) $p(w_{qm}) =\Pi \mathcal{N}(w_{qm}|0,\sigma_w²)+(1-\Pi)\delta_0(w_{qm}) $

(1) $\delta_0(w_{qm})=\delta_0(s_{qm})\delta_0(\tilde{w}_{qm})=\delta_0(s_{qm}) \mathcal{N}(w_{qm}|0,\sigma_w²)$

(1)→(a): $p(w_{qm}) =\mathcal{N}(w_{qm}|0,\sigma_w²)[\Pi + (1-\Pi)\delta_0(s_{qm}) ]= \mathcal{N}(w_{qm}|0,\sigma_w²)[\Pi^{s_{qm}}(1-\Pi)^{1-s_{qm}}$

I try to understand a statement in this paper:

http://papers.nips.cc/paper/4305-spike-and-slab-variational-inference-for-multi-task-and-multiple-kernel-learning.pdf

In particular, I am talking about the reparameterisation from (1c) to (3).

Given is a random variable $w_{qm}$, that is distributed according to:

(A) $p(w_{qm})=\Pi \mathcal{N}(w_{qm}|0,\sigma_w²)+(1-\Pi)\delta_0(w_{qm})$

Specifically, with probability $\Pi$, each $w_{qm}$ is zero, and with probability, it is drawn from a Gaussian.

Further, assume a Gaussian random variable $\tilde{w}_{qm}\sim \mathcal{N}(\tilde{w}_{qm}|0,\sigma_w²)$ and a Bernoulli random variable $s_{qm}\sim\Pi^{s_{qm}}(1-\Pi)^{1-s_{qm}}$. The new random variable $\tilde{w}_{qm}s_{qm}$ is assumed to follow (A), resulting in

(B) $p(w_{qm})=\mathcal{N}(w_{qm}|0,\sigma_w²)[\Pi^{s_{qm}}(1-\Pi)^{1-s_{qm}}]$

I don't really see how the authors did it ((A)→(B)). I am grateful for any help.

Thanx so much in advance!

Krn