Anyone know the Bishop's book in 2.53 they use Jacobian to convert covariance matrix x to y.

$$J_{ij}=\dfrac{\partial x_i}{\partial y_i}=U_{ji} \qquad{(2.53)}$$ $$\int_{\bf x} f({\bf x})d{\bf x} = \int_{\bf y} f({\bf y})|{\bf J}|d{\bf y}$$ $${\vert}{\bf J}{\vert}^2 = {\vert}{\bf U}^T{\vert}^2 = {\vert}{\bf U}^T{\vert}\;{\vert}{\bf U}{\vert} = {\vert}{\bf U}^T{\bf U}{\vert} = {\vert}{\bf I}{\vert} = 1 \qquad{(2.54)}$$ $$\left|\Sigma\right|^{\frac{1}{2}}=\prod_{j=1}^{D}\lambda_j^{\frac{1}{2}} \qquad{(2.55)}$$

$$p({\bf y}) = p(x)|{\bf J}| = \prod_{j=1}^{D}\dfrac{1}{(2\pi\lambda_j)^{1/2}}\exp\left\{-\dfrac{y_j^2}{2\lambda_j}\right\} \qquad{(2.56)}$$

1) why they need to change it with Jacobian ?(what is the reason) 2) what is the p(y) represent in 2.56 3) why they use jacobian for in machine learning covariance matrix?


1 Answer 1


1) why they need to change it with Jacobian: The Jacobian is used when you transform functions of multiple variables and integrate. The process is similar to u-substitution in single variable calculus but for multiple variables you have to include the Jacobian. Wikipedia will explain why better then me : wikipedia

2) what is the p(y) represent: If we perform the following linear transformation: $$y=U^{T}(x -u)$$

Where U is the eigenvectors of the covariance matrix. The new variable y will be zero mean and uncorrelated i.e: $$p(y) = N(0,diag(λ_1,..,λ_N)) $$

As the data is Gaussian uncorrelated means independent:

$$p(y_1,..,y_N)=p(y_1)p(y_2)..p(y_{N-1})p(y_N) $$

  • $\begingroup$ I would like to know if the Jacobean matrix is function domain is analogous to covariance matrix in probability theory domain? because the elements in covariance relate to how one variable changes while other changes while in jacobian its the change described in partial derivative. $\endgroup$ Mar 5, 2020 at 16:09
  • $\begingroup$ the Jacobian is a general consequence of changing variables and integrating in Multiple dimensions. You don't need to it explicitly with many operations with the Gaussian as. there are a bunch of tricks to avoid it. $\endgroup$ Jun 22, 2020 at 2:24
  • $\begingroup$ Allright, Also in your last statement, you mentioned that as the data is Gaussian, uncorrelated means independent. So in case if the data isn't gaussian will uncorrelated would also mean independent or its not necessary? $\endgroup$ Jun 22, 2020 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.