Predictive distribution derivation from Bishop "Pattern Recognition and Machine Learning " I have a question regarding one part in section 4.5.2. Specifically conclusions about equation (4.148):
$$p(a)=\int \delta(a-\mathbf{w}^T\phi)q(\mathbf{w})\mathrm{d}\mathbf{w}.$$
I think context is not that important, the only relevant thing is that $a$ is projection of $w$ on $\phi$. So the puzzling part is this

We can evaluate $p(a)$ by noting that the delta function imposes a linear constraint on $w$ and so forms a marginal distribution from the joint distribution $q(w)$ by integrating out all directions orthogonal to $\phi$.

I really do not see how linear constraint on $w$ imposes that all orthogonal directions to $\phi$ will be integrated out. Can someone elaborate?
 A: The Lebesgue substitution theorem tells us that for any map $A : \mathbb{R}^n \to \mathbb{R}^n$ that is 'nice enough' (diffeomorphism) we have
$$\int_{\mathbb{R}^n} f(x) dx = \int_{\mathbb{R}^n} f(A(x)) |\det \partial A(x)| dx$$
where $\partial A$ is the Jacobi matrix. For a linear map $A$ we have $\partial A(x) = M$ where $M$ is the representing matrix of $A$. Take a fixed nonzero vector $\phi_1 \in \mathbb{R}^n$ and complete it to any orthonormal basis $\phi_1, \phi_2, ..., \phi_n$ (for example, using the Gram Schmidt algorithm: https://en.wikipedia.org/wiki/Gram%E2%80%93Schmidt_process), i.e.
  $$\langle \phi_i, \phi_j \rangle = \mathbf{1}_{i=j}$$
i.e. is one iff. $i=j$ and zero otherwise.
Now we consider the function
  $$f(x) = \delta(a - x^T\phi_1) g(x)$$
for any other 'nice' (i.e. measurable so that the whole thing can still be integrated, your function is 'nice enough' :-)) function $g$. We consider the linear bijective map
  $$A : \mathbb{R}^n \to \mathbb{R}^n, ~~~ A(e_i) = \phi_1$$
where $e_i = (0, ..., 0, \underbrace{1}_{i-\text{th position}}, 0, ..., 0)^T$ is the reguar $i$-th standard vector. This matrix has the representing matrix $M$ where the vectors $\phi_1, ..., \phi_n$ w.r.t. the standard basis are the columns. Such a matrix must have determinant $\pm 1$: As the vectors are orthonormal to each other and $M^T*M$ contains the scalar product $\langle \phi_i, \phi_j \rangle$ as $i,j$-th entry,
  $$M^T*M=I$$
where $I$ is the identity matrix. Hence, $\det(M)^2 = \det(M^T*M) = \det(I) = 1$.
Using the substitution above we get
  $$\int_{\mathbb{R}^n} f(x) dx = \int_{\mathbb{R}^n} f(A(x)) |\det \partial A(x)| dx = \int_{\mathbb{R}^n} f(A(x)) |\pm 1| dx = \int_{\mathbb{R}^n} f(A(x)) dx$$
Now what happens when we write $A(x)$ into $f$? First of all we realize that 
  $$x^T\phi_1 = \langle x, \phi_1 \rangle$$
and then also
  $$A(x) = A(x_1, ..., x_n) = x_1\phi_1 + ... x_n \phi_n$$
so that
  $$A(x)^T \phi_1 = \langle x_1\phi_1 + ... x_n \phi_n, \phi_1 \rangle$$
but since all the $\phi_j$ are orthogonal to $\phi_1$ except for $\phi_1$ itself, we get
  $$A(x)^T \phi_1 = \langle x_1\phi_1 + ... x_n \phi_n, \phi_1 \rangle = x_1$$
and consequently
\begin{align*}
  \int_{\mathbb{R}^n} f(A(x)) dx 
    &= \int_{\mathbb{R}^n} \delta(a-x_1) g(x_1\phi_1 + ... + x_n \phi_n)dx\\
    &= \int_{\mathbb{R}} ... \int_{\mathbb{R}} \delta(a-x_1) g(x_1\phi_1 + ... + x_n \phi_n) dx_1 dx_2 ... dx_n \\
\end{align*}
Now here comes a catch: The author of the book you are referring to seems to think that
  $$\int_{\mathbb{R}} \mathbf{1}_{x_1 = a} \text{somefunction}(x_1) dx_1 = \text{somefunction}(a)$$
which is a physicians 'naive' view on the world (because a formal integral over a function that is zero almost everywhere is certainly zero!). So, repeated once again:
* FORMALLY, THIS IS NONSENSE, IT IS WRONG!!! *
Nevertheless, if interpreted correctly, this could work (not absolutely sure about this point though). Usually, people do not take the 'hard' delta function $\mathbf{1}_{a=w^T\phi}$ but some 'soft' kernel that allows values other than $a=w^T\phi$ but only close ones and the 'close' gets closer within a limit or so. In any case: If you want to do it formally correctly, then it is more complicated. 
Nevertheless, if we follow the 'wrong' way above then we obtain
\begin{align*}
  \int_{\mathbb{R}^n} f(A(x)) dx 
    &= \int_{\mathbb{R}^n} \delta(a-x_1) g(x_1\phi_1 + ... + x_n \phi_n)dx\\
    &= \int_{\mathbb{R}} ... \int_{\mathbb{R}} \delta(a-x_1) g(x_1\phi_1 + ... + x_n \phi_n) dx_1 dx_2 ... dx_n \\
    &= \int_{\mathbb{R}} ... \int_{\mathbb{R}} g(a\phi_1 + ... + x_n \phi_n) dx_2 ... dx_n
\end{align*}
you see: We take the function $g$ and integrate out all directions $\phi_2, \phi_3, ..., \phi_n$ that are orthogonal to $\phi_1$.
