Questions tagged [jacobian]
For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.
85
questions
0
votes
0
answers
19
views
Computing the Jacobian in the Extended Kalman Filter with Non-additive noise
I have the following problem. I have the following Kalman filter:
$ \boldsymbol{x}_k=\boldsymbol{x}_{k-1} + \boldsymbol{w}_k$
$ \boldsymbol{y}_k=h(\boldsymbol{x}_{k}, \boldsymbol{v}_k)$
where $\...
1
vote
0
answers
36
views
A confusion about computing transformation of random variables
Let $(X,Y)$ be a pair of random variables with joint pdf $f_{XY}$. Let $(U,V)$ be two random variables obtained from $(X,Y)$ by $U = u(X,Y)$ and $V = v(X,Y)$ where $u$ and $v$ are, say, nice ...
- 11
1
vote
0
answers
19
views
Monotonicity of softmax (considering updates from all variables)
There's a relevant question here that doesn't quite answer my question, but I'm unable to comment.
Define softmax to be $$a_i = \text{softmax}(u_i)= \frac{e^{u_i}}{\sum_j{e^{u_j}}}$$
As the linked ...
- 11
0
votes
0
answers
171
views
Meaning of vector Jacobian Product with matrix inputs
From my understanding of reverse-mode auto-differentiation, after the forward-pass computation graph has been constructed, gradients are passed from the loss down through the tree to the leaves. ...
- 11
0
votes
0
answers
23
views
Statistical Inference by Casella Exercise 4.51 [duplicate]
I am self-studying statistical inference by Casella and Berger and having difficulty solving exercise 4.51:
let $X, Y, Z \sim U(0,1)$ and they are independent. Find $P(X/Y \leq t)$ and $P(XY \leq t)$. ...
- 1
-1
votes
1
answer
58
views
What is the relation between the coefficients of linear models and the Jacobian matrix?
What is the relation between the coefficients of linear models and the Jacobian matrix? Should the matrix of coefficients of a (generalized) linear model be thought about as the Jacobian?
1
vote
1
answer
25
views
A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning
In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said
Again, the implementation of such algorithms can be checked by using
numerical ...
- 303
1
vote
1
answer
248
views
Mismatch between the dimensions of Jacobian matrixes when calculating derivatives during backprop?
I am trying to understand how back propagation works for a linear layer using minibatches by following this post: https://web.eecs.umich.edu/~justincj/teaching/eecs442/notes/linear-backprop.html.
...
- 247
1
vote
0
answers
220
views
How does the full derivative of softmax + cross entropy have the correct dimensions?
The blog post the softmax function and its derivative explains the following:
Imagine that each input has $N$ features / pixels / etc.
Imagine each input can be classified into $C$ classes
Let the ...
- 247
3
votes
1
answer
52
views
How does the fixed point interation in invertible resnets work?
I feel like I am missing some easy point about this invertible resnet paper which is making it hard for me to grasp how the fixed point iteration works.
stated simply, the residual connection in a ...
- 812
2
votes
1
answer
228
views
Is the Jacobian term needed if the prior is on the transformation parameter?
Suppose I have a strictly positive parameter $\sigma$ and I need to estimate it using the random walk Metropolis-Hasting algorithm.
I know that I can do a parameter transform, i.e., $\beta=log(\sigma)$...
- 453
0
votes
1
answer
708
views
Jacobian Matrix of an Element wise operation on a Matrix
Is it right in saying that the Jacobian Matrix of a Matrix output of an elementwise operation to the same input is a diagonal matrix ?
Context below.
From ref 1 it is clear that when you have an ...
- 211
3
votes
2
answers
195
views
Derivation of ELBO in ADVI Paper, Jacobian of Elliptical Transformation
I've been following the ELBO derivations in the paper Automatic Differentiation Variational Inference and have a few questions. With the model $p(x,\theta)$, they first transform $\theta$ so that it ...
- 33
0
votes
1
answer
1k
views
Change of variables by doing a transformation with a Jacobian versus finding an inverse
I have been solving one problem and there is something unclear to me in the solutions.
Namely, let's consider a probability density $p_x(x)$ defined over a continuous variable $x$, and suppose that we ...
- 107
1
vote
1
answer
3k
views
What is the Hessian of the Gaussian likelihood
I am trying to learn the fine differences between different methods of Kronecker factoring for approximate curvature (like [1], and [2]) which require taking the Hessian of the pre-activations of the ...
- 812
3
votes
1
answer
86
views
How to know the number of dimensions of a Jacobian?
My question comes from a comment in this question Vector Jacobian product in automatic differentiation
The question states...
$$
t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times ...
- 812
1
vote
1
answer
56
views
Integrating in log-space with a change of variable
I have a probability density function $P(f|\mu,\sigma) = \mathcal{N}(f|\mu,\sigma)$.
I need to change the variable $f$ to $L = \log_{10}[f]$ so I can integrate it jointly with another PDF whose domain ...
- 479
4
votes
1
answer
300
views
Projecting a distribution onto the unit sphere in an arbitrary norm
So we represent the Dirichlet distribution as the projection of the $d$ independent gammas (on $R_+^d$ onto the unit simplex, and we arrive at that through the $L_1$ norm. That is, divide ${\bf x} \...
- 245
0
votes
0
answers
36
views
Is $P(x|y) d x=P(z|y) d z$ correct? Otherwise, what is missing?
Given the conditional probability density $P(x)$, the following is correct for any change of variables $x\rightarrow z$:
Eq.1: $P(x)dx=P(z)dz$
Eq.2: So $P(z)=P(x)[dx/dz]$
Is this also valid for the ...
- 101
0
votes
0
answers
43
views
Box Cox transformation in multiple regression using car package in R [duplicate]
Could you please confirm that my aproache is right.
In my regression model:
...
- 31
0
votes
0
answers
25
views
Computing the Jacobian $J_F$ with $F = h \circ f$
Let
$$
f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex]
h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$
and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$
I want to compute ...
- 227
1
vote
1
answer
330
views
Jacobian method
What is X1 and X2 both in terms of Y1 and Y2 if that's what I am supposed to find? I can't figure it out. How do I use the jacobian method to find the joint probability density?
- 13
1
vote
1
answer
946
views
How do the derivatives of the loss function with respect to a layer's inputs form a Jacobian?
Suppose a multi-layer feed-forward neural network, e.g.:
Using matrix form to account for all training samples $(i)$, the forward propagation can be written as follows:
$Z^{[l]}=W^{[l]}A^{[l-1]}+\...
- 133
3
votes
0
answers
6k
views
Vector Jacobian product in automatic differentiation
my questions is related to this post Higher Order of Vectorization in Backpropagation in Neural Network @shimao
I don't really get the following claim (I know how the chain rule works and what is the ...
- 127
1
vote
2
answers
4k
views
Finding the pdf of Y from that of X, linear transformation
The question is
Let $X$ be a continuous random variable with pdf $f_X(x) = 2(1 − x)$, $0 ≤ x ≤ 1$. If $Y = 2X − 1$, find the pdf of $Y$.
I understand these steps$$F_Y(Y ≤ y) = P(2X-1 ≤ y) = P(X ≤ (y+...
- 13
0
votes
0
answers
24
views
Given a function $\phi:X \to Y$ and a target distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ s.t. $\phi(x) \sim \pi_Y$?
More formally, given $X\subseteq\mathbb{R}^n$, $Y\subseteq\mathbb{R}^m$, $\phi: X \to Y$ and distribution $\pi_Y$ on $Y$, does there exist a distribution $\pi_X$ on $X$ such that $\phi(x) \sim \pi_Y$ ...
- 1,039
3
votes
1
answer
1k
views
How to derive Camera Jacobian
I'm dealing with a Kalman Filter situation, trying to track points in 3D using cameras, each of which can represent a 3D point as a 2D projection according to:
$$ \begin{bmatrix} u \\ v \end{bmatrix} ...
- 1,237
4
votes
1
answer
1k
views
Jacobian for transformation of discrete random variables (intuition)
I am reading Blitzstein's introduction to probability. He states that, while a transformation of continuous r.v.s needs a Jacobian (or derivative), a transformation of discrete r.v.s does not.
Is ...
- 293
0
votes
0
answers
41
views
Transformation of random normal variable $x/y^2$
If $x \sim N(\mu_x, \sigma_y)$ and $y \sim N(\mu_y, \sigma_y)$ what is the distribution of $x/y^2$ using Jacobian method?
- 1
6
votes
1
answer
680
views
Inverse Gaussian chi square connection
The inverse Gaussian distribution $IG(\mu,\lambda)$ is associated with the density
$$f(x;\mu,\lambda) = \sqrt{\frac{\lambda}{2\pi x^3}}\,\exp\left\{-\frac{\lambda(x-\mu)^2}{2\mu^2x}\right\}\qquad \...
- 103k
3
votes
1
answer
279
views
Jacobian of transformation
In Bayesian Data Analysis, PDF freely available, section 4.1 (page 84, bottom) there is a comment saying:
If we had instead constructed the normal approximation in terms of
$p(\mu, \sigma^2)$, the ...
- 105
3
votes
1
answer
666
views
Vector-Jacobian Product Computational Cost
The paper FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models presents a continuous-time flow as a generative model which uses Hutchinson's trace estimator to give an ...
- 33
1
vote
0
answers
359
views
Jacobian term for Metropolis Hastings algorithm?
Suppose an acceptance ratio of the MH sampler without parameter transformation is like this:
\begin{gather}
r=\frac{f( \theta ',\gamma '|y,m) q( \theta ',\gamma '|\theta ,\gamma )}{f( \theta ,\gamma |...
- 115
17
votes
3
answers
9k
views
Generating random points uniformly on a disk [duplicate]
I have to randomly generate 1000 points over a unit disk such that are uniformly distributed on this disk. Now, for that, I select a radius $r$ and angular orientation $\alpha$ such that the radius $r$...
- 337
0
votes
0
answers
257
views
Rescaling multivariable normal pdf and normalizing constant
I am trying to understand change of variables for a random variable and how it changes the pdf and the normalizing constant.
Let $\mathbf{x}$ be $N$-dimensional normal variable and let $\mathbf{\...
- 11
3
votes
1
answer
511
views
Change of variables in pdf
I have the joint pdf$$f(x_1,x_2)=x_1e^{-x_1(1+x_2)}I_{(0,\infty)}(x_1)I_{(0,\infty)}(x_2)$$and have to derive the joint pdf of $$Y_1=e^{-X_1}\qquad\text{ and }\quad Y_2=e^{-X_1X_2}$$ I set $x_1=-\ln(...
- 31
0
votes
0
answers
232
views
Expectation (x/y) using jacobian
Let $X$ and $Y$ be two independent random variables with the density
functions:
$f(x) = 3 x^2$, for $0<x<1$, $0$ elsewhere
$g(y) = 4y^3$, for $0 <y<1$, $0$ elsewhere Give $\mathbb E(x/...
1
vote
0
answers
27
views
How do you solve for inverses when doing the Jacobian transformation method?
Hopefully a very basic question. This is from a textbook exercise I'm doing to prepare for a class. Suppose I have two independent random variables, X and Y, and I am trying to find the joint ...
- 11
5
votes
2
answers
771
views
Are Jacobian adjustments necessary when the target parameter is a difference between two parameters in Stan?
[Note on cross-posting: This question has now been posted on the Stan Forums as well.]
I want to model the index called Delta P (e.g., p.144 of this paper), which is basically a difference between two ...
- 263
2
votes
1
answer
432
views
How to estimate the PDF of the logarithm of a uniformly distributed random variable?
This is a question I have to solve and need help with. I know it's usual to give pointers and hints so the OP can follow from there. Thus, I'll appreciate all input that shows me the way to go.
Let $...
- 654
2
votes
2
answers
2k
views
Is Covariance Matrix analogous to Jacobian Matrix?
In probability theory covariance matrix denote how each variable relates to other in a pairwise manner. So 1 would mean they are identical and 0 would mean they are independent and are not related. Is ...
- 830
2
votes
1
answer
329
views
Jacobian of Inverse Gaussian Transformation in Schwarz & Samanta (1991)
In the sample size $n=2$ case when transforming $\{x_1, x_2\}$ to $\{\bar{x}, s\}$ (where $X_1, X_2 \overset{iid}{\sim} IG(\mu, \lambda)$, $\bar{X}=\frac{\sum_i^2 X_i}{n}$, and $S=\sum_i^2 (\frac{1}{...
- 43
2
votes
0
answers
139
views
Jacobian matrix X going from x to the y coordinate system [closed]
In Christoper Bishop's book Pattern Recognition and Machine learning, they use Jacobian to convert covariance matrix x to y.
However, according to definitions on Wikipedia, the definition of Jacobian ...
- 43
3
votes
2
answers
442
views
Given X and Y are independent ~N(0,1), what is the distribution of $ Z=X^2 + Y^2 $
Our joint pdf is $f(x,y) = \frac{1}{\sqrt{2π}} e^\frac{x^2+y^2}{2}$
Now we let $ U = X^2 + Y^2 $ and $ V = Y$, we can then get our Jacobian as $ J = \frac{1}{\sqrt{u-v^2}} $
Since this ...
- 133
0
votes
1
answer
99
views
Jacobian for function including cubic spline
I am trying to fit a measured spectrum with a linear combination of end-member spectra which are approximated by cubic spline functions ($f_1$ and $f_2$). I also need to incorporate terms that account ...
- 105
4
votes
2
answers
1k
views
How to calculate the Jacobian of the transformation ( for covariance matrix)
I'm reading this Paper about a separation strategy for modeling covariance matrices with focus on Bayesian analysis. Direct decomposition of covariance matrix is as follows: $\Sigma = \text{diag}(S)\,...
- 61
11
votes
2
answers
4k
views
How can I obtain a Cauchy distribution from two standard normal distributions?
I am interested in
Let $X\sim N(0,1), Y \sim N(0,1)$ independently. Show $\frac{X}{X+Y}$
is a Cauchy random variable.
My work:
$f_{X,Y}(x,y)=\frac{1}{2\pi} e^{\frac{-1}{2}(x^2+y^2)}, -\infty&...
- 1,829
3
votes
0
answers
294
views
Maximum Likelihood - Normal Errors - When is the Jacobian needed?
I am considering the following non-linear model
$$h(z) - \lambda_0 - \lambda_1 z - \lambda_2x = v$$
where $v \sim \mathcal N(0,\sigma^2)$ unobserved error and where $\lambda_j$ are unknown ...
- 5,301
1
vote
0
answers
55
views
Showing a useful result for Wisharts and Multivariate Beta random matrices
Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define
$$
\mathbf{S} = \...
- 20.2k
10
votes
1
answer
2k
views
Reason for absolute value of Jacobian determinant in change-of-variable formula?
When we have a random variable $x$ with a probability density $p(x)$, and a function $y = f(x)$ that is differentiable and can be solved for $x = g(y)$, the change of variable formula leads us to a ...
- 939