Skip to main content
deleted 8 characters in body
Source Link
WeakLearner
  • 1.5k
  • 2
  • 16
  • 29

In this talk at 19:32, the following method of reducing a multivariate integral is discussed:

Let $\theta \in \mathbb{R}^p$ be the parameter vector, $\mathbf{x}_i \in \mathbb{R}^p$ be the $i$-th data vector, $y_i \in \{-1,1\}$, be the corresponding binary observation. Also let $\phi(\cdot)$ denote the logistic sigmoid function. We want to simplify the following integral (product of sigmoid and multivariate Gaussian terms)

\begin{align*} z & = \int_{\mathbb{R}^p} \phi(y_i \mathbf{x}_i^T \theta)~ \mathcal{N}(\theta|\mu, \Sigma) d \theta \\ \end{align*}

This can be expressed as an expectation with respect to $\theta$:

$$ \mathbb{E}_{\theta} [\phi(y_i \mathbf{x}_i^T \theta)] $$

and since $u := y_i \mathbf{x}_i^T \theta$ is just a constantsclar, we can write this as:

$$ \mathbb{E}_{u} [\phi(u)] $$

which is a one-dimensional expectation. I understand that $u$ is Gaussian, but I do not understand why the distribution the expectation is taken with respect to can be changed like that

In this talk at 19:32, the following method of reducing a multivariate integral is discussed:

Let $\theta \in \mathbb{R}^p$ be the parameter vector, $\mathbf{x}_i \in \mathbb{R}^p$ be the $i$-th data vector, $y_i \in \{-1,1\}$, be the corresponding binary observation. Also let $\phi(\cdot)$ denote the logistic sigmoid function. We want to simplify the following integral (product of sigmoid and multivariate Gaussian terms)

\begin{align*} z & = \int_{\mathbb{R}^p} \phi(y_i \mathbf{x}_i^T \theta)~ \mathcal{N}(\theta|\mu, \Sigma) d \theta \\ \end{align*}

This can be expressed as an expectation with respect to $\theta$:

$$ \mathbb{E}_{\theta} [\phi(y_i \mathbf{x}_i^T \theta)] $$

and since $u := y_i \mathbf{x}_i^T \theta$ is just a constant, we can write this as:

$$ \mathbb{E}_{u} [\phi(u)] $$

which is a one-dimensional expectation. I understand that $u$ is Gaussian, but I do not understand why the distribution the expectation is taken with respect to can be changed like that

In this talk at 19:32, the following method of reducing a multivariate integral is discussed:

Let $\theta \in \mathbb{R}^p$ be the parameter vector, $\mathbf{x}_i \in \mathbb{R}^p$ be the $i$-th data vector, $y_i \in \{-1,1\}$, be the corresponding binary observation. Also let $\phi(\cdot)$ denote the logistic sigmoid function. We want to simplify the following integral (product of sigmoid and multivariate Gaussian terms)

\begin{align*} z & = \int_{\mathbb{R}^p} \phi(y_i \mathbf{x}_i^T \theta)~ \mathcal{N}(\theta|\mu, \Sigma) d \theta \\ \end{align*}

This can be expressed as an expectation with respect to $\theta$:

$$ \mathbb{E}_{\theta} [\phi(y_i \mathbf{x}_i^T \theta)] $$

and since $u := y_i \mathbf{x}_i^T \theta$ is a sclar, we can write this as:

$$ \mathbb{E}_{u} [\phi(u)] $$

which is a one-dimensional expectation. I understand that $u$ is Gaussian, but I do not understand why the distribution the expectation is taken with respect to can be changed like that

Source Link
WeakLearner
  • 1.5k
  • 2
  • 16
  • 29

Linear subspace property of Gaussian integrals

In this talk at 19:32, the following method of reducing a multivariate integral is discussed:

Let $\theta \in \mathbb{R}^p$ be the parameter vector, $\mathbf{x}_i \in \mathbb{R}^p$ be the $i$-th data vector, $y_i \in \{-1,1\}$, be the corresponding binary observation. Also let $\phi(\cdot)$ denote the logistic sigmoid function. We want to simplify the following integral (product of sigmoid and multivariate Gaussian terms)

\begin{align*} z & = \int_{\mathbb{R}^p} \phi(y_i \mathbf{x}_i^T \theta)~ \mathcal{N}(\theta|\mu, \Sigma) d \theta \\ \end{align*}

This can be expressed as an expectation with respect to $\theta$:

$$ \mathbb{E}_{\theta} [\phi(y_i \mathbf{x}_i^T \theta)] $$

and since $u := y_i \mathbf{x}_i^T \theta$ is just a constant, we can write this as:

$$ \mathbb{E}_{u} [\phi(u)] $$

which is a one-dimensional expectation. I understand that $u$ is Gaussian, but I do not understand why the distribution the expectation is taken with respect to can be changed like that