Skip to main content
LaTeX the image and clarify
Source Link
Avraham
  • 3.9k
  • 26
  • 46

enter image description here The following quotation is from Machine Learning: a Probabilistic Perspective Chapter 7, page 234

In machine learning, we often care more about predictive accuracy than about interpreting the parameters. Using Equation 4.126, we can easily show that the posterior predictive distribution at a test point $\mathbf{x}$ is also Gaussian: $$ \begin{aligned} p(y|\mathbf{x}, \mathcal{D}, \sigma^2) &= \int\mathcal{N}\left(y|\mathbf{x}^T\mathbf{w}, \sigma^2\right)\mathcal{N}\left(\mathbf{w}|\mathbf{w}_N, \mathbf{V}_N\right)d\mathbf{w}\\ &=\mathcal{N}\left(y|\mathbf{w}^T_N\mathbf{x}, \sigma^2_N(\mathbf{x})\right)\\ \sigma^2_N(\mathbf{x}) &= \sigma^2 + \mathbf{x}^T\mathbf{V}_N\mathbf{x} \end{aligned} $$ The variance in this prediction, $\sigma^2_N(x)$, depends on two terms: the variance of the observation noise, $\sigma^2$, and the variance in the parameters, $\mathbf{V}_N$. The latter translates into variance about observations in a way which depends on how close $\mathbf{x}$ is to the training data $\mathcal{D}$.

InMy question is in which way "V_Ndoes "$\mathbf{V}_N$ translates into variance about observations observations in a way which depends on how close x$\mathbf{x}$ is to the training data D"$\mathcal{D}$"? I mean, OK intuitively, but how can i prove it in a formal way?

enter image description here

In which way "V_N translates into variance about observations in a way which depends on how close x is to the training data D"? I mean, OK intuitively, but how can i prove it in a formal way?

The following quotation is from Machine Learning: a Probabilistic Perspective Chapter 7, page 234

In machine learning, we often care more about predictive accuracy than about interpreting the parameters. Using Equation 4.126, we can easily show that the posterior predictive distribution at a test point $\mathbf{x}$ is also Gaussian: $$ \begin{aligned} p(y|\mathbf{x}, \mathcal{D}, \sigma^2) &= \int\mathcal{N}\left(y|\mathbf{x}^T\mathbf{w}, \sigma^2\right)\mathcal{N}\left(\mathbf{w}|\mathbf{w}_N, \mathbf{V}_N\right)d\mathbf{w}\\ &=\mathcal{N}\left(y|\mathbf{w}^T_N\mathbf{x}, \sigma^2_N(\mathbf{x})\right)\\ \sigma^2_N(\mathbf{x}) &= \sigma^2 + \mathbf{x}^T\mathbf{V}_N\mathbf{x} \end{aligned} $$ The variance in this prediction, $\sigma^2_N(x)$, depends on two terms: the variance of the observation noise, $\sigma^2$, and the variance in the parameters, $\mathbf{V}_N$. The latter translates into variance about observations in a way which depends on how close $\mathbf{x}$ is to the training data $\mathcal{D}$.

My question is in which way does "$\mathbf{V}_N$ translates into variance about observations in a way which depends on how close $\mathbf{x}$ is to the training data $\mathcal{D}$"? I mean, OK intuitively, but how can i prove it in a formal way?

Source Link

Posterior predictive distribution, linear regression

enter image description here

In which way "V_N translates into variance about observations in a way which depends on how close x is to the training data D"? I mean, OK intuitively, but how can i prove it in a formal way?