Skip to main content

Questions tagged [jacobian]

For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.

Filter by
Sorted by
Tagged with
1 vote
1 answer
38 views

Transformation of a Random Variable

I am working on this problem for class, where the setup is the following: Let X be a single observation from the $beta(\theta,1)$ pdf. (a) Let $Y=-(logX)^{-1}$. Evaluate the confidence coefficient of ...
Harry Lofi's user avatar
2 votes
1 answer
68 views

Reversible-jump MCMC and Poisson processes

Suppose we have a time interval $t \in [0, T]$ in which events occur as a Poisson process with some arbitrary time-dependent rate $\lambda(t)$. These events occur at times $Y=(Y_1, Y_2, \dotso, Y_M)$ ...
Jordan's user avatar
  • 31
0 votes
0 answers
25 views

Computing the Jacobian in the Extended Kalman Filter with Non-additive noise

I have the following problem. I have the following Kalman filter: $ \boldsymbol{x}_k=\boldsymbol{x}_{k-1} + \boldsymbol{w}_k$ $ \boldsymbol{y}_k=h(\boldsymbol{x}_{k}, \boldsymbol{v}_k)$ where $\...
user262278's user avatar
1 vote
0 answers
41 views

A confusion about computing transformation of random variables

Let $(X,Y)$ be a pair of random variables with joint pdf $f_{XY}$. Let $(U,V)$ be two random variables obtained from $(X,Y)$ by $U = u(X,Y)$ and $V = v(X,Y)$ where $u$ and $v$ are, say, nice ...
DeepC's user avatar
  • 11
1 vote
0 answers
32 views

Monotonicity of softmax (considering updates from all variables)

There's a relevant question here that doesn't quite answer my question, but I'm unable to comment. Define softmax to be $$a_i = \text{softmax}(u_i)= \frac{e^{u_i}}{\sum_j{e^{u_j}}}$$ As the linked ...
gcpx100's user avatar
  • 11
-1 votes
1 answer
73 views

What is the relation between the coefficients of linear models and the Jacobian matrix?

What is the relation between the coefficients of linear models and the Jacobian matrix? Should the matrix of coefficients of a (generalized) linear model be thought about as the Jacobian?
Peter Jordanson's user avatar
1 vote
1 answer
27 views

A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning

In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said Again, the implementation of such algorithms can be checked by using numerical ...
zzzhhh's user avatar
  • 333
2 votes
1 answer
314 views

Mismatch between the dimensions of Jacobian matrixes when calculating derivatives during backprop?

I am trying to understand how back propagation works for a linear layer using minibatches by following this post: https://web.eecs.umich.edu/~justincj/teaching/eecs442/notes/linear-backprop.html. ...
Foobar's user avatar
  • 359
1 vote
0 answers
231 views

How does the full derivative of softmax + cross entropy have the correct dimensions?

The blog post the softmax function and its derivative explains the following: Imagine that each input has $N$ features / pixels / etc. Imagine each input can be classified into $C$ classes Let the ...
Foobar's user avatar
  • 359
3 votes
1 answer
76 views

How does the fixed point interation in invertible resnets work?

I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works. stated simply, the residual connection in a ...
Joff's user avatar
  • 942
2 votes
1 answer
318 views

Is the Jacobian term needed if the prior is on the transformation parameter?

Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm. I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$...
Ding Li's user avatar
  • 453
0 votes
1 answer
953 views

Jacobian Matrix of an Element wise operation on a Matrix

Is it right in saying that the Jacobian Matrix of a Matrix output of an elementwise operation to the same input is a diagonal matrix ? Context below. From ref 1 it is clear that when you have an ...
Alex Punnen's user avatar
3 votes
2 answers
220 views

Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation

I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it ...
James's user avatar
  • 33
0 votes
1 answer
1k views

Change of variables by doing a transformation with a Jacobian versus finding an inverse

I have been solving one problem and there is something unclear to me in the solutions. Namely, let's consider a probability density $p_x(x)$ defined over a continuous variable $x$, and suppose that we ...
Relax295's user avatar
  • 117
1 vote
1 answer
4k views

What is the Hessian of the Gaussian likelihood

I am trying to learn the fine differences between different methods of Kronecker factoring for approximate curvature (like [1], and [2]) which require taking the Hessian of the pre-activations of the ...
Joff's user avatar
  • 942
3 votes
1 answer
101 views

How to know the number of dimensions of a Jacobian?

My question comes from a comment in this question Vector Jacobian product in automatic differentiation The question states... $$ t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times ...
Joff's user avatar
  • 942
1 vote
1 answer
80 views

Integrating in log-space with a change of variable

I have a probability density function $P(f|\mu,\sigma) = \mathcal{N}(f|\mu,\sigma)$. I need to change the variable $f$ to $L = \log_{10}[f]$ so I can integrate it jointly with another PDF whose domain ...
Lucidnonsense's user avatar
4 votes
1 answer
348 views

Projecting a distribution onto the unit sphere in an arbitrary norm

So we represent the Dirichlet distribution as the projection of the $d$ independent gammas (on $R_+^d$ onto the unit simplex, and we arrive at that through the $L_1$ norm. That is, divide ${\bf x} \...
Faydey's user avatar
  • 225
0 votes
0 answers
38 views

Is $P(x|y) d x=P(z|y) d z$ correct? Otherwise, what is missing?

Given the conditional probability density $P(x)$, the following is correct for any change of variables $x\rightarrow z$: Eq.1: $P(x)dx=P(z)dz$ Eq.2: So $P(z)=P(x)[dx/dz]$ Is this also valid for the ...
user1611107's user avatar
0 votes
0 answers
43 views

Box Cox transformation in multiple regression using car package in R [duplicate]

Could you please confirm that my aproache is right. In my regression model: ...
David Janda's user avatar
0 votes
0 answers
27 views

Computing the Jacobian $J_F$ with $F = h \circ f$

Let $$ f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex] h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$ and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$ I want to compute ...
lalaland's user avatar
  • 247
1 vote
1 answer
396 views

Jacobian method

What is X1 and X2 both in terms of Y1 and Y2 if that's what I am supposed to find? I can't figure it out. How do I use the jacobian method to find the joint probability density?
Yas  's user avatar
  • 13
1 vote
1 answer
1k views

How do the derivatives of the loss function with respect to a layer's inputs form a Jacobian?

Suppose a multi-layer feed-forward neural network, e.g.: Using matrix form to account for all training samples $(i)$, the forward propagation can be written as follows: $Z^{[l]}=W^{[l]}A^{[l-1]}+\...
AlwaysLearning's user avatar
4 votes
1 answer
7k views

Vector Jacobian product in automatic differentiation

my questions is related to this post Higher Order of Vectorization in Backpropagation in Neural Network @shimao I don't really get the following claim (I know how the chain rule works and what is the ...
jdeJuan's user avatar
  • 147
2 votes
2 answers
5k views

Finding the pdf of Y from that of X, linear transformation

The question is Let $X$ be a continuous random variable with pdf $f_X(x) = 2(1 − x)$, $0 ≤ x ≤ 1$. If $Y = 2X − 1$, find the pdf of $Y$. I understand these steps$$F_Y(Y ≤ y) = P(2X-1 ≤ y) = P(X ≤ (y+...
katie's user avatar
  • 23
0 votes
0 answers
24 views

Given a function $\phi:X \to Y$ and a target distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ s.t. $\phi(x) \sim \pi_Y$?

More formally, given $X\subseteq\mathbb{R}^n$, $Y\subseteq\mathbb{R}^m$, $\phi: X \to Y$ and distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ such that $\phi(x) \sim \pi_Y$ ...
deasmhumnha's user avatar
  • 1,069
4 votes
1 answer
1k views

How to derive Camera Jacobian

I'm dealing with a Kalman Filter situation, trying to track points in 3D using cameras, each of which can represent a 3D point as a 2D projection according to: $$ \begin{bmatrix} u \\ v \end{bmatrix} ...
Pavel Komarov's user avatar
4 votes
1 answer
1k views

Jacobian for transformation of discrete random variables (intuition)

I am reading Blitzstein's introduction to probability. He states that, while a transformation of continuous r.v.s needs a Jacobian (or derivative), a transformation of discrete r.v.s does not. Is ...
mikabozu's user avatar
  • 313
0 votes
0 answers
42 views

Transformation of random normal variable $x/y^2$

If $x \sim N(\mu_x, \sigma_y)$ and $y \sim N(\mu_y, \sigma_y)$ what is the distribution of $x/y^2$ using Jacobian method?
mehrdad's user avatar
8 votes
1 answer
803 views

Inverse Gaussian chi square connection

The inverse Gaussian distribution $IG(\mu,\lambda)$ is associated with the density $$f(x;\mu,\lambda) = \sqrt{\frac{\lambda}{2\pi x^3}}\,\exp\left\{-\frac{\lambda(x-\mu)^2}{2\mu^2x}\right\}\qquad \...
Xi'an's user avatar
  • 107k
3 votes
1 answer
353 views

Jacobian of transformation

In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying: If we had instead constructed the normal approximation in terms of $p(\mu, \sigma^2)$, the ...
Arnold Davidson's user avatar
4 votes
1 answer
791 views

Vector-Jacobian Product Computational Cost

The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
Lashoun's user avatar
  • 43
1 vote
0 answers
408 views

Jacobian term for Metropolis Hastings algorithm?

Suppose an acceptance ratio of the MH sampler without parameter transformation is like this: \begin{gather} r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
Keith Lau's user avatar
  • 115
18 votes
3 answers
11k views

Generating random points uniformly on a disk [duplicate]

I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
Ghosal_C's user avatar
  • 357
0 votes
0 answers
288 views

Rescaling multivariable normal pdf and normalizing constant

I am trying to understand change of variables for a random variable and how it changes the pdf and the normalizing constant. Let $\mathbf{x}$ be $N$-dimensional normal variable and let $\mathbf{\...
sk1995's user avatar
  • 11
3 votes
1 answer
544 views

Change of variables in pdf

I have the joint pdf$$f(x_1,x_2)=x_1e^{-x_1(1+x_2)}I_{(0,\infty)}(x_1)I_{(0,\infty)}(x_2)$$and have to derive the joint pdf of $$Y_1=e^{-X_1}\qquad\text{ and }\quad Y_2=e^{-X_1X_2}$$ I set $x_1=-\ln(...
Niklas's user avatar
  • 31
0 votes
0 answers
257 views

Expectation (x/y) using jacobian

Let $X$ and $Y$ be two independent random variables with the density functions: $f(x) = 3 x^2$, for $0<x<1$, $0$ elsewhere $g(y) = 4y^3$, for $0 <y<1$, $0$ elsewhere Give $\mathbb E(x/...
Sundaresh Subramanian's user avatar
3 votes
2 answers
315 views

How to compute the joint distribution of transformed variables using the Jacobian?

$X$ and $Y$ are independent continuous random variables and have the same distribution $F_x(t) = 1 - (1/t)$ for $t \gt 1$. We define two new variables $W$ and $Z,$ where $W = \min(X,Y)$ and $Z = \...
Sundaresh Subramanian's user avatar
1 vote
0 answers
27 views

How do you solve for inverses when doing the Jacobian transformation method?

Hopefully a very basic question. This is from a textbook exercise I'm doing to prepare for a class. Suppose I have two independent random variables, X and Y, and I am trying to find the joint ...
DoctorShemp's user avatar
5 votes
2 answers
827 views

Are Jacobian adjustments necessary when the target parameter is a difference between two parameters in Stan?

[Note on cross-posting: This question has now been posted on the Stan Forums as well.] I want to model the index called Delta P (e.g., p.144 of this paper), which is basically a difference between two ...
Akira Murakami's user avatar
2 votes
1 answer
535 views

How to estimate the PDF of the logarithm of a uniformly distributed random variable?

This is a question I have to solve and need help with. I know it's usual to give pointers and hints so the OP can follow from there. Thus, I'll appreciate all input that shows me the way to go. Let $...
Jxson99's user avatar
  • 664
3 votes
2 answers
2k views

Is Covariance Matrix analogous to Jacobian Matrix?

In probability theory covariance matrix denote how each variable relates to other in a pairwise manner. So 1 would mean they are identical and 0 would mean they are independent and are not related. Is ...
GENIVI-LEARNER's user avatar
2 votes
1 answer
401 views

Jacobian of Inverse Gaussian Transformation in Schwarz & Samanta (1991)

In the sample size $n=2$ case when transforming $\{x_1, x_2\}$ to $\{\bar{x}, s\}$ (where $X_1, X_2 \overset{iid}{\sim} IG(\mu, \lambda)$, $\bar{X}=\frac{\sum_i^2 X_i}{n}$, and $S=\sum_i^2 (\frac{1}{...
mdawgig's user avatar
  • 43
2 votes
0 answers
156 views

Jacobian matrix X going from x to the y coordinate system [closed]

In Christoper Bishop's book Pattern Recognition and Machine learning, they use Jacobian to convert covariance matrix x to y. However, according to definitions on Wikipedia, the definition of Jacobian ...
Cici Yang's user avatar
3 votes
2 answers
541 views

Given X and Y are independent ~N(0,1), what is the distribution of $ Z=X^2 + Y^2 $

Our joint pdf is $f(x,y) = \frac{1}{\sqrt{2π}} e^\frac{x^2+y^2}{2}$ Now we let $ U = X^2 + Y^2 $ and $ V = Y$, we can then get our Jacobian as $ J = \frac{1}{\sqrt{u-v^2}} $ Since this ...
Kevin's user avatar
  • 133
0 votes
1 answer
113 views

Jacobian for function including cubic spline

I am trying to fit a measured spectrum with a linear combination of end-member spectra which are approximated by cubic spline functions ($f_1$ and $f_2$). I also need to incorporate terms that account ...
oscarbranson's user avatar
4 votes
2 answers
2k views

How to calculate the Jacobian of the transformation ( for covariance matrix)

I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
Dedula33's user avatar
12 votes
2 answers
5k views

How can I obtain a Cauchy distribution from two standard normal distributions?

I am interested in Let $X\sim N(0,1), Y \sim N(0,1)$ independently. Show $\frac{X}{X+Y}$ is a Cauchy random variable. My work: $f_{X,Y}(x,y)=\frac{1}{2\pi} e^{\frac{-1}{2}(x^2+y^2)}, -\infty&...
Ron Snow's user avatar
  • 2,103
3 votes
0 answers
306 views

Maximum Likelihood - Normal Errors - When is the Jacobian needed?

I am considering the following non-linear model $$h(z) - \lambda_0 - \lambda_1 z - \lambda_2x = v$$ where $v \sim \mathcal N(0,\sigma^2)$ unobserved error and where $\lambda_j$ are unknown ...
Jesper for President's user avatar
1 vote
0 answers
62 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
Taylor's user avatar
  • 21.2k