Questions tagged [jacobian]

For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.

Filter by
Sorted by
Tagged with
1 vote
1 answer
22 views

Is the Jacobian term needed if the prior is on the transformation parameter?

Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm. I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$...
user avatar
  • 423
0 votes
1 answer
53 views

Jacobian Matrix of an Element wise operation on a Matrix

Is it right in saying that the Jacobian Matrix of a Matrix output of an elementwise operation to the same input is a diagonal matrix ? Context below. From ref 1 it is clear that when you have an ...
user avatar
1 vote
0 answers
45 views

Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation

I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it ...
user avatar
  • 11
0 votes
1 answer
92 views

Change of variables by doing a transformation with a Jacobian versus finding an inverse

I have been solving one problem and there is something unclear to me in the solutions. Namely, let's consider a probability density $p_x(x)$ defined over a continuous variable $x$, and suppose that we ...
user avatar
  • 105
0 votes
0 answers
22 views

Name for a divergent PDF under a change of variables

Consider a particle free to move on the upper half of the circled defined by $x^2 + y^2 = 1$. The PDF of the particle in terms of the standard polar angle $\theta$ is $$\rho_\theta(\theta) = \frac{1}{\...
user avatar
1 vote
1 answer
877 views

What is the Hessian of the Gaussian likelihood

I am trying to learn the fine differences between different methods of Kronecker factoring for approximate curvature (like [1], and [2]) which require taking the Hessian of the pre-activations of the ...
user avatar
  • 619
3 votes
1 answer
31 views

How to know the number of dimensions of a Jacobian?

My question comes from a comment in this question Vector Jacobian product in automatic differentiation The question states... $$ t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times ...
user avatar
  • 619
1 vote
1 answer
26 views

Integrating in log-space with a change of variable

I have a probability density function $P(f|\mu,\sigma) = \mathcal{N}(f|\mu,\sigma)$. I need to change the variable $f$ to $L = \log_{10}[f]$ so I can integrate it jointly with another PDF whose domain ...
user avatar
0 votes
0 answers
12 views

Normalizing a custom Weight Shifted or Spiked Gaussian distribution

I have a custom weight shifted bivariate gaussian distribution that I wish to normalize. W is the weighted symmetric matrix that shifts the entire distribution and the λ below is the diagonal matrix ...
user avatar
0 votes
0 answers
28 views

shape of the Jacobian tensor of the output of the layer w.r.t. the input 𝑿

Suppose we have a linear (i.e. fully-connected) layer, defined with in_features=1024 and out_features=2048. We apply this layer to an input tensor 𝑿 containing a batch of N=128 samples. What would ...
user avatar
  • 1
0 votes
0 answers
89 views

Maximum likelihood versus minimum distance estimator

Let's assume that I have data $X_1,\ldots X_n$ and a parametrized distribution $p(x,\theta)$. I can get an estimate of $\theta$ through maximum likelihood estimation $\theta^*_{ML}$. I can also get an ...
user avatar
  • 101
3 votes
1 answer
160 views

Projecting a distribution onto the unit sphere in an arbitrary norm

So we represent the Dirichlet distribution as the projection of the $d$ independent gammas (on $R_+^d$ onto the unit simplex, and we arrive at that through the $L_1$ norm. That is, divide ${\bf x} \...
user avatar
  • 171
0 votes
0 answers
30 views

Is $P(x|y) d x=P(z|y) d z$ correct? Otherwise, what is missing?

Given the conditional probability density $P(x)$, the following is correct for any change of variables $x\rightarrow z$: Eq.1: $P(x)dx=P(z)dz$ Eq.2: So $P(z)=P(x)[dx/dz]$ Is this also valid for the ...
user avatar
0 votes
0 answers
40 views

Box Cox transformation in multiple regression using car package in R [duplicate]

Could you please confirm that my aproache is right. In my regression model: ...
user avatar
0 votes
0 answers
24 views

Computing the Jacobian $J_F$ with $F = h \circ f$

Let $$ f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex] h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$ and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$ I want to compute ...
user avatar
  • 143
1 vote
1 answer
66 views

Jacobian method

What is X1 and X2 both in terms of Y1 and Y2 if that's what I am supposed to find? I can't figure it out. How do I use the jacobian method to find the joint probability density?
user avatar
  • 13
1 vote
1 answer
325 views

How do the derivatives of the loss function with respect to a layer's inputs form a Jacobian?

Suppose a multi-layer feed-forward neural network, e.g.: Using matrix form to account for all training samples $(i)$, the forward propagation can be written as follows: $Z^{[l]}=W^{[l]}A^{[l-1]}+\...
user avatar
0 votes
0 answers
1k views

Vector Jacobian product in automatic differentiation

my questions is related to this post Higher Order of Vectorization in Backpropagation in Neural Network @shimao I don't really get the following claim (I know how the chain rule works and what is the ...
user avatar
  • 97
1 vote
2 answers
1k views

Finding the pdf of Y from that of X, linear transformation

The question is Let $X$ be a continuous random variable with pdf $f_X(x) = 2(1 − x)$, $0 ≤ x ≤ 1$. If $Y = 2X − 1$, find the pdf of $Y$. I understand these steps$$F_Y(Y ≤ y) = P(2X-1 ≤ y) = P(X ≤ (y+...
user avatar
  • 13
0 votes
0 answers
20 views

Given a function $\phi:X \to Y$ and a target distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ s.t. $\phi(x) \sim \pi_Y$?

More formally, given $X\subseteq\mathbb{R}^n$, $Y\subseteq\mathbb{R}^m$, $\phi: X \to Y$ and distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ such that $\phi(x) \sim \pi_Y$ ...
user avatar
2 votes
1 answer
441 views

How to derive Camera Jacobian

I'm dealing with a Kalman Filter situation, trying to track points in 3D using cameras, each of which can represent a 3D point as a 2D projection according to: $$ \begin{bmatrix} u \\ v \end{bmatrix} ...
user avatar
2 votes
1 answer
463 views

Jacobian for transformation of discrete random variables (intuition)

I am reading Blitzstein's introduction to probability. He states that, while a transformation of continuous r.v.s needs a Jacobian (or derivative), a transformation of discrete r.v.s does not. Is ...
user avatar
  • 243
0 votes
0 answers
37 views

Transformation of random normal variable $x/y^2$

If $x \sim N(\mu_x, \sigma_y)$ and $y \sim N(\mu_y, \sigma_y)$ what is the distribution of $x/y^2$ using Jacobian method?
user avatar
6 votes
1 answer
395 views

Inverse Gaussian chi square connection

The inverse Gaussian distribution $IG(\mu,\lambda)$ is associated with the density $$f(x;\mu,\lambda) = \sqrt{\frac{\lambda}{2\pi x^3}}\,\exp\left\{-\frac{\lambda(x-\mu)^2}{2\mu^2x}\right\}\qquad \...
user avatar
  • 91.7k
3 votes
1 answer
130 views

Jacobian of transformation

In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying: If we had instead constructed the normal approximation in terms of $p(\mu, \sigma^2)$, the ...
user avatar
3 votes
1 answer
209 views

Vector-Jacobian Product Computational Cost

The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
user avatar
  • 31
0 votes
0 answers
157 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
user avatar
16 votes
3 answers
4k views

Generating random points uniformly on a disk [duplicate]

I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
user avatar
0 votes
0 answers
150 views

Rescaling multivariable normal pdf and normalizing constant

I am trying to understand change of variables for a random variable and how it changes the pdf and the normalizing constant. Let $\mathbf{x}$ be $N$-dimensional normal variable and let $\mathbf{\...
user avatar
  • 11
3 votes
1 answer
283 views

Change of variables in pdf

I have the joint pdf$$f(x_1,x_2)=x_1e^{-x_1(1+x_2)}I_{(0,\infty)}(x_1)I_{(0,\infty)}(x_2)$$and have to derive the joint pdf of $$Y_1=e^{-X_1}\qquad\text{ and }\quad Y_2=e^{-X_1X_2}$$ I set $x_1=-\ln(...
user avatar
  • 31
0 votes
0 answers
121 views

Expectation (x/y) using jacobian

Let $X$ and $Y$ be two independent random variables with the density functions: $f(x) = 3 x^2$, for $0<x<1$, $0$ elsewhere $g(y) = 4y^3$, for $0 <y<1$, $0$ elsewhere Give $\mathbb E(x/...
user avatar
1 vote
0 answers
23 views

How do you solve for inverses when doing the Jacobian transformation method?

Hopefully a very basic question. This is from a textbook exercise I'm doing to prepare for a class. Suppose I have two independent random variables, X and Y, and I am trying to find the joint ...
user avatar
5 votes
2 answers
505 views

Are Jacobian adjustments necessary when the target parameter is a difference between two parameters in Stan?

[Note on cross-posting: This question has now been posted on the Stan Forums as well.] I want to model the index called Delta P (e.g., p.144 of this paper), which is basically a difference between two ...
user avatar
2 votes
1 answer
165 views

How to estimate the PDF of the logarithm of a uniformly distributed random variable?

This is a question I have to solve and need help with. I know it's usual to give pointers and hints so the OP can follow from there. Thus, I'll appreciate all input that shows me the way to go. Let $...
user avatar
  • 599
2 votes
2 answers
873 views

Is Covariance Matrix analogous to Jacobian Matrix?

In probability theory covariance matrix denote how each variable relates to other in a pairwise manner. So 1 would mean they are identical and 0 would mean they are independent and are not related. Is ...
user avatar
2 votes
1 answer
216 views

Jacobian of Inverse Gaussian Transformation in Schwarz & Samanta (1991)

In the sample size $n=2$ case when transforming $\{x_1, x_2\}$ to $\{\bar{x}, s\}$ (where $X_1, X_2 \overset{iid}{\sim} IG(\mu, \lambda)$, $\bar{X}=\frac{\sum_i^2 X_i}{n}$, and $S=\sum_i^2 (\frac{1}{...
user avatar
  • 43
2 votes
0 answers
66 views

Jacobian matrix X going from x to the y coordinate system [closed]

In Christoper Bishop's book Pattern Recognition and Machine learning, they use Jacobian to convert covariance matrix x to y. However, according to definitions on Wikipedia, the definition of Jacobian ...
user avatar
3 votes
2 answers
82 views

Given X and Y are independent ~N(0,1), what is the distribution of $ Z=X^2 + Y^2 $

Our joint pdf is $f(x,y) = \frac{1}{\sqrt{2π}} e^\frac{x^2+y^2}{2}$ Now we let $ U = X^2 + Y^2 $ and $ V = Y$, we can then get our Jacobian as $ J = \frac{1}{\sqrt{u-v^2}} $ Since this ...
user avatar
  • 133
0 votes
1 answer
56 views

Jacobian for function including cubic spline

I am trying to fit a measured spectrum with a linear combination of end-member spectra which are approximated by cubic spline functions ($f_1$ and $f_2$). I also need to incorporate terms that account ...
user avatar
3 votes
2 answers
961 views

How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
user avatar
11 votes
2 answers
2k views

How can I obtain a Cauchy distribution from two standard normal distributions?

I am interested in Let $X\sim N(0,1), Y \sim N(0,1)$ independently. Show $\frac{X}{X+Y}$ is a Cauchy random variable. My work: $f_{X,Y}(x,y)=\frac{1}{2\pi} e^{\frac{-1}{2}(x^2+y^2)}, -\infty&...
user avatar
  • 1,595
3 votes
0 answers
173 views

Maximum Likelihood - Normal Errors - When is the Jacobian needed?

I am considering the following non-linear model $$h(z) - \lambda_0 - \lambda_1 z - \lambda_2x = v$$ where $v \sim \mathcal N(0,\sigma^2)$ unobserved error and where $\lambda_j$ are unknown ...
user avatar
1 vote
0 answers
42 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
user avatar
  • 18.6k
8 votes
1 answer
1k views

Reason for absolute value of Jacobian determinant in change-of-variable formula?

When we have a random variable $x$ with a probability density $p(x)$, and a function $y = f(x)$ that is differentiable and can be solved for $x = g(y)$, the change of variable formula leads us to a ...
user avatar
  • 870
0 votes
0 answers
345 views

Transformation of random variables and Jacobian

When transforming 2+ continuous random variables, you use a Jacobian matrix and compute the determinant. Do you also compute the Jacobian for discrete random variables?
user avatar
  • 1
1 vote
0 answers
29 views

Change of variables: 4-dimensional PDF to 2-dimensional PDF

I have a 4-dimensional joint-PDF between variables $X_1,X_2,X_3,X_4$ which are all Gaussian. I want to transform this into a 2-dimensional joint-PDF between new variables $Y_1=Y_1(X_1,X_2,X_3,X_4)$ ...
user avatar
  • 143
0 votes
0 answers
107 views

Projection of multivariate distribution to lower dimensional subspace

Say that $X \in \mathbb{R}^n$ is a vector of $n$ r.v.'s with pdf $p(x_1,\ldots,x_n)$. Let's consider now the linear map $Y = A X$ where $Y \in \mathbb{R}^m$ with $m < n$. I am seeking $p(y_1,\ldots,...
user avatar
1 vote
0 answers
329 views

How to understand Jacobian Matrix from the geometric perspective?

I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ...
user avatar
  • 111
1 vote
0 answers
15 views

Is it possible to find the joint distribution of a random vector if only the distribution of scalar many-to-one transformation is known? [duplicate]

Theoretical Exercise: I'd like to derive the joint distribution $p_{\boldsymbol{X}}$ of a random vector $\boldsymbol{X} \in \mathbb{R}^K$ if only the distribution of a scalar many-to-one ...
user avatar
6 votes
2 answers
1k views

Higher Order of Vectorization in Backpropagation in Neural Network

I have a question about the dimensions of a Jacobian during backpropagation. The network looks like: The forward propagation can be defined as: where g is the activation function. The dimensions of ...
user avatar
  • 61