Questions tagged [jacobian]

For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.

Filter by
Sorted by
Tagged with
1 vote
1 answer
14 views

A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning

In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said Again, the implementation of such algorithms can be checked by using numerical ...
  • 67
0 votes
0 answers
31 views

How to use the Jacobian of the normal distribution's log-partition function to derive its moments?

The derivative of the log-partition function is equal to the expectation of the sufficient statistic vector (i.e. the distribution's marginals): $$ \begin{aligned} \partial_{\eta_i} \ln Z(\eta) &=\...
1 vote
1 answer
46 views

Mismatch between the dimensions of Jacobian matrixes when calculating derivatives during backprop?

I am trying to understand how back propagation works for a linear layer using minibatches by following this post: https://web.eecs.umich.edu/~justincj/teaching/eecs442/notes/linear-backprop.html. ...
  • 237
1 vote
0 answers
37 views

How does the full derivative of softmax + cross entropy have the correct dimensions?

The blog post the softmax function and its derivative explains the following: Imagine that each input has $N$ features / pixels / etc. Imagine each input can be classified into $C$ classes Let the ...
  • 237
1 vote
0 answers
9 views

How does the fixed point interation in invertible resnets work?

I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works. stated simply, the residual connection in a ...
  • 639
1 vote
1 answer
62 views

Is the Jacobian term needed if the prior is on the transformation parameter?

Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm. I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$...
  • 423
0 votes
1 answer
183 views

Jacobian Matrix of an Element wise operation on a Matrix

Is it right in saying that the Jacobian Matrix of a Matrix output of an elementwise operation to the same input is a diagonal matrix ? Context below. From ref 1 it is clear that when you have an ...
3 votes
2 answers
94 views

Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation

I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it ...
  • 33
0 votes
1 answer
244 views

Change of variables by doing a transformation with a Jacobian versus finding an inverse

I have been solving one problem and there is something unclear to me in the solutions. Namely, let's consider a probability density $p_x(x)$ defined over a continuous variable $x$, and suppose that we ...
  • 105
0 votes
0 answers
31 views

Name for a divergent PDF under a change of variables

Consider a particle free to move on the upper half of the circled defined by $x^2 + y^2 = 1$. The PDF of the particle in terms of the standard polar angle $\theta$ is $$\rho_\theta(\theta) = \frac{1}{\...
1 vote
1 answer
1k views

What is the Hessian of the Gaussian likelihood

I am trying to learn the fine differences between different methods of Kronecker factoring for approximate curvature (like [1], and [2]) which require taking the Hessian of the pre-activations of the ...
  • 639
3 votes
1 answer
46 views

How to know the number of dimensions of a Jacobian?

My question comes from a comment in this question Vector Jacobian product in automatic differentiation The question states... $$ t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times ...
  • 639
1 vote
1 answer
29 views

Integrating in log-space with a change of variable

I have a probability density function $P(f|\mu,\sigma) = \mathcal{N}(f|\mu,\sigma)$. I need to change the variable $f$ to $L = \log_{10}[f]$ so I can integrate it jointly with another PDF whose domain ...
4 votes
1 answer
188 views

Projecting a distribution onto the unit sphere in an arbitrary norm

So we represent the Dirichlet distribution as the projection of the $d$ independent gammas (on $R_+^d$ onto the unit simplex, and we arrive at that through the $L_1$ norm. That is, divide ${\bf x} \...
  • 215
0 votes
0 answers
30 views

Is $P(x|y) d x=P(z|y) d z$ correct? Otherwise, what is missing?

Given the conditional probability density $P(x)$, the following is correct for any change of variables $x\rightarrow z$: Eq.1: $P(x)dx=P(z)dz$ Eq.2: So $P(z)=P(x)[dx/dz]$ Is this also valid for the ...
0 votes
0 answers
40 views

Box Cox transformation in multiple regression using car package in R [duplicate]

Could you please confirm that my aproache is right. In my regression model: ...
0 votes
0 answers
24 views

Computing the Jacobian $J_F$ with $F = h \circ f$

Let $$ f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex] h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$ and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$ I want to compute ...
  • 185
1 vote
1 answer
109 views

Jacobian method

What is X1 and X2 both in terms of Y1 and Y2 if that's what I am supposed to find? I can't figure it out. How do I use the jacobian method to find the joint probability density?
  • 13
1 vote
1 answer
495 views

How do the derivatives of the loss function with respect to a layer's inputs form a Jacobian?

Suppose a multi-layer feed-forward neural network, e.g.: Using matrix form to account for all training samples $(i)$, the forward propagation can be written as follows: $Z^{[l]}=W^{[l]}A^{[l-1]}+\...
1 vote
0 answers
2k views

Vector Jacobian product in automatic differentiation

my questions is related to this post Higher Order of Vectorization in Backpropagation in Neural Network @shimao I don't really get the following claim (I know how the chain rule works and what is the ...
  • 107
1 vote
2 answers
1k views

Finding the pdf of Y from that of X, linear transformation

The question is Let $X$ be a continuous random variable with pdf $f_X(x) = 2(1 − x)$, $0 ≤ x ≤ 1$. If $Y = 2X − 1$, find the pdf of $Y$. I understand these steps$$F_Y(Y ≤ y) = P(2X-1 ≤ y) = P(X ≤ (y+...
  • 13
0 votes
0 answers
23 views

Given a function $\phi:X \to Y$ and a target distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ s.t. $\phi(x) \sim \pi_Y$?

More formally, given $X\subseteq\mathbb{R}^n$, $Y\subseteq\mathbb{R}^m$, $\phi: X \to Y$ and distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ such that $\phi(x) \sim \pi_Y$ ...
  • 1,019
3 votes
1 answer
530 views

How to derive Camera Jacobian

I'm dealing with a Kalman Filter situation, trying to track points in 3D using cameras, each of which can represent a 3D point as a 2D projection according to: $$ \begin{bmatrix} u \\ v \end{bmatrix} ...
3 votes
1 answer
552 views

Jacobian for transformation of discrete random variables (intuition)

I am reading Blitzstein's introduction to probability. He states that, while a transformation of continuous r.v.s needs a Jacobian (or derivative), a transformation of discrete r.v.s does not. Is ...
  • 263
0 votes
0 answers
37 views

Transformation of random normal variable $x/y^2$

If $x \sim N(\mu_x, \sigma_y)$ and $y \sim N(\mu_y, \sigma_y)$ what is the distribution of $x/y^2$ using Jacobian method?
6 votes
1 answer
462 views

Inverse Gaussian chi square connection

The inverse Gaussian distribution $IG(\mu,\lambda)$ is associated with the density $$f(x;\mu,\lambda) = \sqrt{\frac{\lambda}{2\pi x^3}}\,\exp\left\{-\frac{\lambda(x-\mu)^2}{2\mu^2x}\right\}\qquad \...
  • 94.6k
3 votes
1 answer
152 views

Jacobian of transformation

In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying: If we had instead constructed the normal approximation in terms of $p(\mu, \sigma^2)$, the ...
3 votes
1 answer
316 views

Vector-Jacobian Product Computational Cost

The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
  • 31
0 votes
0 answers
207 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
16 votes
3 answers
5k views

Generating random points uniformly on a disk [duplicate]

I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
0 votes
0 answers
172 views

Rescaling multivariable normal pdf and normalizing constant

I am trying to understand change of variables for a random variable and how it changes the pdf and the normalizing constant. Let $\mathbf{x}$ be $N$-dimensional normal variable and let $\mathbf{\...
  • 11
3 votes
1 answer
372 views

Change of variables in pdf

I have the joint pdf$$f(x_1,x_2)=x_1e^{-x_1(1+x_2)}I_{(0,\infty)}(x_1)I_{(0,\infty)}(x_2)$$and have to derive the joint pdf of $$Y_1=e^{-X_1}\qquad\text{ and }\quad Y_2=e^{-X_1X_2}$$ I set $x_1=-\ln(...
  • 31
0 votes
0 answers
143 views

Expectation (x/y) using jacobian

Let $X$ and $Y$ be two independent random variables with the density functions: $f(x) = 3 x^2$, for $0<x<1$, $0$ elsewhere $g(y) = 4y^3$, for $0 <y<1$, $0$ elsewhere Give $\mathbb E(x/...
1 vote
0 answers
23 views

How do you solve for inverses when doing the Jacobian transformation method?

Hopefully a very basic question. This is from a textbook exercise I'm doing to prepare for a class. Suppose I have two independent random variables, X and Y, and I am trying to find the joint ...
5 votes
2 answers
579 views

Are Jacobian adjustments necessary when the target parameter is a difference between two parameters in Stan?

[Note on cross-posting: This question has now been posted on the Stan Forums as well.] I want to model the index called Delta P (e.g., p.144 of this paper), which is basically a difference between two ...
2 votes
1 answer
215 views

How to estimate the PDF of the logarithm of a uniformly distributed random variable?

This is a question I have to solve and need help with. I know it's usual to give pointers and hints so the OP can follow from there. Thus, I'll appreciate all input that shows me the way to go. Let $...
  • 648
2 votes
2 answers
1k views

Is Covariance Matrix analogous to Jacobian Matrix?

In probability theory covariance matrix denote how each variable relates to other in a pairwise manner. So 1 would mean they are identical and 0 would mean they are independent and are not related. Is ...
2 votes
1 answer
254 views

Jacobian of Inverse Gaussian Transformation in Schwarz & Samanta (1991)

In the sample size $n=2$ case when transforming $\{x_1, x_2\}$ to $\{\bar{x}, s\}$ (where $X_1, X_2 \overset{iid}{\sim} IG(\mu, \lambda)$, $\bar{X}=\frac{\sum_i^2 X_i}{n}$, and $S=\sum_i^2 (\frac{1}{...
  • 43
2 votes
0 answers
91 views

Jacobian matrix X going from x to the y coordinate system [closed]

In Christoper Bishop's book Pattern Recognition and Machine learning, they use Jacobian to convert covariance matrix x to y. However, according to definitions on Wikipedia, the definition of Jacobian ...
3 votes
2 answers
92 views

Given X and Y are independent ~N(0,1), what is the distribution of $ Z=X^2 + Y^2 $

Our joint pdf is $f(x,y) = \frac{1}{\sqrt{2π}} e^\frac{x^2+y^2}{2}$ Now we let $ U = X^2 + Y^2 $ and $ V = Y$, we can then get our Jacobian as $ J = \frac{1}{\sqrt{u-v^2}} $ Since this ...
  • 133
0 votes
1 answer
73 views

Jacobian for function including cubic spline

I am trying to fit a measured spectrum with a linear combination of end-member spectra which are approximated by cubic spline functions ($f_1$ and $f_2$). I also need to incorporate terms that account ...
3 votes
2 answers
1k views

How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
11 votes
2 answers
3k views

How can I obtain a Cauchy distribution from two standard normal distributions?

I am interested in Let $X\sim N(0,1), Y \sim N(0,1)$ independently. Show $\frac{X}{X+Y}$ is a Cauchy random variable. My work: $f_{X,Y}(x,y)=\frac{1}{2\pi} e^{\frac{-1}{2}(x^2+y^2)}, -\infty&...
  • 1,615
3 votes
0 answers
192 views

Maximum Likelihood - Normal Errors - When is the Jacobian needed?

I am considering the following non-linear model $$h(z) - \lambda_0 - \lambda_1 z - \lambda_2x = v$$ where $v \sim \mathcal N(0,\sigma^2)$ unobserved error and where $\lambda_j$ are unknown ...
2 votes
0 answers
45 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
  • 18.9k
8 votes
1 answer
2k views

Reason for absolute value of Jacobian determinant in change-of-variable formula?

When we have a random variable $x$ with a probability density $p(x)$, and a function $y = f(x)$ that is differentiable and can be solved for $x = g(y)$, the change of variable formula leads us to a ...
  • 1,000
0 votes
0 answers
378 views

Transformation of random variables and Jacobian

When transforming 2+ continuous random variables, you use a Jacobian matrix and compute the determinant. Do you also compute the Jacobian for discrete random variables?
  • 1
1 vote
0 answers
29 views

Change of variables: 4-dimensional PDF to 2-dimensional PDF

I have a 4-dimensional joint-PDF between variables $X_1,X_2,X_3,X_4$ which are all Gaussian. I want to transform this into a 2-dimensional joint-PDF between new variables $Y_1=Y_1(X_1,X_2,X_3,X_4)$ ...
  • 143
0 votes
0 answers
146 views

Projection of multivariate distribution to lower dimensional subspace

Say that $X \in \mathbb{R}^n$ is a vector of $n$ r.v.'s with pdf $p(x_1,\ldots,x_n)$. Let's consider now the linear map $Y = A X$ where $Y \in \mathbb{R}^m$ with $m < n$. I am seeking $p(y_1,\ldots,...
1 vote
0 answers
349 views

How to understand Jacobian Matrix from the geometric perspective?

I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ...
  • 111