Questions tagged [jacobian]

For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.

Filter by
Sorted by
Tagged with
0
votes
0answers
10 views

shape of the Jacobian tensor of the output of the layer w.r.t. the input 𝑿

Suppose we have a linear (i.e. fully-connected) layer, defined with in_features=1024 and out_features=2048. We apply this layer to an input tensor 𝑿 containing a batch of N=128 samples. What would ...
3
votes
1answer
123 views

Projecting a distribution onto the unit sphere in an arbitrary norm

So we represent the Dirichlet distribution as the projection of the $d$ independent gammas (on $R_+^d$ onto the unit simplex, and we arrive at that through the $L_1$ norm. That is, divide ${\bf x} \...
0
votes
0answers
27 views

Is $P(x|y) d x=P(z|y) d z$ correct? Otherwise, what is missing?

Given the conditional probability density $P(x)$, the following is correct for any change of variables $x\rightarrow z$: Eq.1: $P(x)dx=P(z)dz$ Eq.2: So $P(z)=P(x)[dx/dz]$ Is this also valid for the ...
0
votes
0answers
39 views

Box Cox transformation in multiple regression using car package in R [duplicate]

Could you please confirm that my aproache is right. In my regression model: ...
0
votes
0answers
23 views

Computing the Jacobian $J_F$ with $F = h \circ f$

Let $$ f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex] h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$ and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$ I want to compute ...
1
vote
1answer
28 views

Jacobian method

What is X1 and X2 both in terms of Y1 and Y2 if that's what I am supposed to find? I can't figure it out. How do I use the jacobian method to find the joint probability density?
1
vote
1answer
51 views

How do the derivatives of the loss function with respect to a layer's inputs form a Jacobian?

Suppose a multi-layer feed-forward neural network, e.g.: Using matrix form to account for all training samples $(i)$, the forward propagation can be written as follows: $Z^{[l]}=W^{[l]}A^{[l-1]}+\...
0
votes
0answers
251 views

Vector Jacobian product in automatic differentiation

my questions is related to this post Higher Order of Vectorization in Backpropagation in Neural Network @shimao I don't really get the following claim (I know how the chain rule works and what is the ...
0
votes
2answers
97 views

Finding the pdf of Y from that of X, linear transformation

The question is Let $X$ be a continuous random variable with pdf $f_X(x) = 2(1 − x)$, $0 ≤ x ≤ 1$. If $Y = 2X − 1$, find the pdf of $Y$. I understand these steps$$F_Y(Y ≤ y) = P(2X-1 ≤ y) = P(X ≤ (y+...
0
votes
0answers
20 views

Given a function $\phi:X \to Y$ and a target distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ s.t. $\phi(x) \sim \pi_Y$?

More formally, given $X\subseteq\mathbb{R}^n$, $Y\subseteq\mathbb{R}^m$, $\phi: X \to Y$ and distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ such that $\phi(x) \sim \pi_Y$ ...
2
votes
1answer
177 views

How to derive Camera Jacobian

I'm dealing with a Kalman Filter situation, trying to track points in 3D using cameras, each of which can represent a 3D point as a 2D projection according to: $$ \begin{bmatrix} u \\ v \end{bmatrix} ...
1
vote
1answer
172 views

Jacobian for transformation of discrete random variables (intuition)

I am reading Blitzstein's introduction to probability. He states that, while a transformation of continuous r.v.s needs a Jacobian (or derivative), a transformation of discrete r.v.s does not. Is ...
0
votes
0answers
34 views

Transformation of random normal variable $x/y^2$

If $x \sim N(\mu_x, \sigma_y)$ and $y \sim N(\mu_y, \sigma_y)$ what is the distribution of $x/y^2$ using Jacobian method?
6
votes
1answer
257 views

Inverse Gaussian chi square connection

The inverse Gaussian distribution $IG(\mu,\lambda)$ is associated with the density $$f(x;\mu,\lambda) = \sqrt{\frac{\lambda}{2\pi x^3}}\,\exp\left\{-\frac{\lambda(x-\mu)^2}{2\mu^2x}\right\}\qquad \...
3
votes
1answer
101 views

Jacobian of transformation

In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying: If we had instead constructed the normal approximation in terms of $p(\mu, \sigma^2)$, the ...
0
votes
0answers
35 views

Vectorized backpropagation for Neural network?

I understand back-propagation for scalars, but vectorized examples are bit tricky. Here, the image above shows a specific computational graph example. We are calculating gradients with respect to ...
2
votes
1answer
66 views

Vector-Jacobian Product Computational Cost

The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
0
votes
0answers
92 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
16
votes
3answers
2k views

Generating random points uniformly on a disk [duplicate]

I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
0
votes
0answers
79 views

Rescaling multivariable normal pdf and normalizing constant

I am trying to understand change of variables for a random variable and how it changes the pdf and the normalizing constant. Let $\mathbf{x}$ be $N$-dimensional normal variable and let $\mathbf{\...
0
votes
0answers
18 views

Finding the log Jacobian of a transformation from lower to high dimensions, specifically in the case of normal cdf to Dirichlet?

I have random variables $\mu$ and $\sigma$ that I have transformed and I am interested in finding their joint distribution given the following information I have. Particularly, I need help finding the ...
3
votes
1answer
159 views

Change of variables in pdf

I have the joint pdf$$f(x_1,x_2)=x_1e^{-x_1(1+x_2)}I_{(0,\infty)}(x_1)I_{(0,\infty)}(x_2)$$and have to derive the joint pdf of $$Y_1=e^{-X_1}\qquad\text{ and }\quad Y_2=e^{-X_1X_2}$$ I set $x_1=-\ln(...
0
votes
0answers
79 views

Expectation (x/y) using jacobian

Let $X$ and $Y$ be two independent random variables with the density functions: $f(x) = 3 x^2$, for $0<x<1$, $0$ elsewhere $g(y) = 4y^3$, for $0 <y<1$, $0$ elsewhere Give $\mathbb E(x/...
1
vote
0answers
22 views

How do you solve for inverses when doing the Jacobian transformation method?

Hopefully a very basic question. This is from a textbook exercise I'm doing to prepare for a class. Suppose I have two independent random variables, X and Y, and I am trying to find the joint ...
5
votes
2answers
263 views

Are Jacobian adjustments necessary when the target parameter is a difference between two parameters in Stan?

[Note on cross-posting: This question has now been posted on the Stan Forums as well.] I want to model the index called Delta P (e.g., p.144 of this paper), which is basically a difference between two ...
2
votes
1answer
60 views

How to estimate the PDF of the logarithm of a uniformly distributed random variable?

This is a question I have to solve and need help with. I know it's usual to give pointers and hints so the OP can follow from there. Thus, I'll appreciate all input that shows me the way to go. Let $...
1
vote
2answers
431 views

Is Covariance Matrix analogous to Jacobian Matrix?

In probability theory covariance matrix denote how each variable relates to other in a pairwise manner. So 1 would mean they are identical and 0 would mean they are independent and are not related. Is ...
2
votes
1answer
153 views

Jacobian of Inverse Gaussian Transformation in Schwarz & Samanta (1991)

In the sample size $n=2$ case when transforming $\{x_1, x_2\}$ to $\{\bar{x}, s\}$ (where $X_1, X_2 \overset{iid}{\sim} IG(\mu, \lambda)$, $\bar{X}=\frac{\sum_i^2 X_i}{n}$, and $S=\sum_i^2 (\frac{1}{...
2
votes
0answers
33 views

Jacobian matrix X going from x to the y coordinate system [closed]

In Christoper Bishop's book Pattern Recognition and Machine learning, they use Jacobian to convert covariance matrix x to y. However, according to definitions on Wikipedia, the definition of Jacobian ...
3
votes
2answers
60 views

Given X and Y are independent ~N(0,1), what is the distribution of $ Z=X^2 + Y^2 $

Our joint pdf is $f(x,y) = \frac{1}{\sqrt{2π}} e^\frac{x^2+y^2}{2}$ Now we let $ U = X^2 + Y^2 $ and $ V = Y$, we can then get our Jacobian as $ J = \frac{1}{\sqrt{u-v^2}} $ Since this ...
0
votes
1answer
40 views

Jacobian for function including cubic spline

I am trying to fit a measured spectrum with a linear combination of end-member spectra which are approximated by cubic spline functions ($f_1$ and $f_2$). I also need to incorporate terms that account ...
3
votes
2answers
677 views

How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
11
votes
2answers
1k views

How can I obtain a Cauchy distribution from two standard normal distributions?

I am interested in Let $X\sim N(0,1), Y \sim N(0,1)$ independently. Show $\frac{X}{X+Y}$ is a Cauchy random variable. My work: $f_{X,Y}(x,y)=\frac{1}{2\pi} e^{\frac{-1}{2}(x^2+y^2)}, -\infty&...
3
votes
0answers
123 views

Maximum Likelihood - Normal Errors - When is the Jacobian needed?

I am considering the following non-linear model $$h(z) - \lambda_0 - \lambda_1 z - \lambda_2x = v$$ where $v \sim \mathcal N(0,\sigma^2)$ unobserved error and where $\lambda_j$ are unknown ...
1
vote
0answers
38 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
5
votes
1answer
858 views

Reason for absolute value of Jacobian determinant in change-of-variable formula?

When we have a random variable $x$ with a probability density $p(x)$, and a function $y = f(x)$ that is differentiable and can be solved for $x = g(y)$, the change of variable formula leads us to a ...
0
votes
0answers
270 views

Transformation of random variables and Jacobian

When transforming 2+ continuous random variables, you use a Jacobian matrix and compute the determinant. Do you also compute the Jacobian for discrete random variables?
1
vote
0answers
23 views

Change of variables: 4-dimensional PDF to 2-dimensional PDF

I have a 4-dimensional joint-PDF between variables $X_1,X_2,X_3,X_4$ which are all Gaussian. I want to transform this into a 2-dimensional joint-PDF between new variables $Y_1=Y_1(X_1,X_2,X_3,X_4)$ ...
0
votes
0answers
74 views

Projection of multivariate distribution to lower dimensional subspace

Say that $X \in \mathbb{R}^n$ is a vector of $n$ r.v.'s with pdf $p(x_1,\ldots,x_n)$. Let's consider now the linear map $Y = A X$ where $Y \in \mathbb{R}^m$ with $m < n$. I am seeking $p(y_1,\ldots,...
1
vote
0answers
277 views

How to understand Jacobian Matrix from the geometric perspective?

I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ...
1
vote
0answers
15 views

Is it possible to find the joint distribution of a random vector if only the distribution of scalar many-to-one transformation is known? [duplicate]

Theoretical Exercise: I'd like to derive the joint distribution $p_{\boldsymbol{X}}$ of a random vector $\boldsymbol{X} \in \mathbb{R}^K$ if only the distribution of a scalar many-to-one ...
5
votes
2answers
729 views

Higher Order of Vectorization in Backpropagation in Neural Network

I have a question about the dimensions of a Jacobian during backpropagation. The network looks like: The forward propagation can be defined as: where g is the activation function. The dimensions of ...
0
votes
0answers
31 views

Highest posterior interval and monotone changes of variables

Suppose $X$ is distributed with a unimodal pdf $f(x)$ and let $Y = g(X)$ for some strictly monotone function $g$. Hence $g$ is invertible. Is there an analytically tractable relationship between the ...
1
vote
3answers
191 views

Computing directly the pdf of $Y=X^2$ for the pdf $f_X(x) = \frac{2}{9}(x+1)$

In the book Statistical inference by Casella and Berger, given that the pdf of X is $f_X(x) = \frac{2}{9}(x+1)$ for $-1 \le x \le 2$, we want to find the pdf of $Y=X^2$. But it says we are not able to ...
0
votes
1answer
266 views

For a posterior $p(\theta |y)$, if I specify a one-to-one transformation $\phi = g(\theta)$, how can I apply the transformation? [duplicate]

Suppose I have a posterior distribution, $p(\theta \mid y)$, where $y$ was my data and $\theta$ is a random variable with some prior distribution. If I specify a one-to-one transformation $\phi = g(\...
1
vote
0answers
67 views

Wrong vector calculus in lecture note 5 of cs224n, Stanford

I am studying NLP via cs224n from Stanford. I am reading this lecture note now. When you refer to the 5th page, they want to derive the gradient with respect to W for RNN, to show the mathematical ...
3
votes
1answer
396 views

Shouldn't it be Jacobian Descent?

Whenever we have a multiclass prediction the classifier generates a vector output. Per the definition of a Jacobian we are actually taking Jacobian steps towards a local minimum - so should it ...
1
vote
1answer
292 views

Prediction Interval for Regression Models with Weights

Courtesy Prediction Interval for Neural Net With Hessian :: nnet in R I would like to understand how to manually derive a point-wise prediction confidence interval applicable to neural models. ...
3
votes
1answer
429 views

Weighted Least Squares weights not changing Jacobian matrix

I currently have 4 data points and the following Jacobian matrix $A$ and cost vector $b$ $ A = \begin{bmatrix} -0.7867 & 0.0464& -0.6155 & 1.0000 \\ -0.3751 & 0.4299 &...
5
votes
2answers
7k views

What is the jacobian for a neural network

I know that a Jacobian is a matrix holding all of the first-order derivatives for a vector-valued-function. What is the Jacobian of a neural network, though? What are the inputs and what are the ...