Questions tagged [jacobian]

For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.

Filter by
Sorted by
Tagged with
0
votes
0answers
13 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
3
votes
1answer
42 views

Reason for absolute value of Jacobian determinant in change-of-variable formula?

When we have a random variable $x$ with a probability density $p(x)$, and a function $y = f(x)$ that is differentiable and can be solved for $x = g(y)$, the change of variable formula leads us to a ...
0
votes
0answers
15 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
0
votes
0answers
14 views

CCA on feature maps: Gradient w.r.t to Jacobian

Assume I have two neural networks, abstracted as two feature maps, parametrized by $\theta_x,\theta_y$ respectively. $\phi_x(x;\theta_x) \in \mathbb{R}^{h_1}$, $\phi_y(x;\theta_y) \in \mathbb{R}^{h_2}$...
0
votes
0answers
41 views

Transformation of random variables and Jacobian

When transforming 2+ continuous random variables, you use a Jacobian matrix and compute the determinant. Do you also compute the Jacobian for discrete random variables?
1
vote
0answers
19 views

Change of variables: 4-dimensional PDF to 2-dimensional PDF

I have a 4-dimensional joint-PDF between variables $X_1,X_2,X_3,X_4$ which are all Gaussian. I want to transform this into a 2-dimensional joint-PDF between new variables $Y_1=Y_1(X_1,X_2,X_3,X_4)$ ...
0
votes
0answers
26 views

Projection of multivariate distribution to lower dimensional subspace

Say that $X \in \mathbb{R}^n$ is a vector of $n$ r.v.'s with pdf $p(x_1,\ldots,x_n)$. Let's consider now the linear map $Y = A X$ where $Y \in \mathbb{R}^m$ with $m < n$. I am seeking $p(y_1,\ldots,...
1
vote
0answers
152 views

How to understand Jacobian Matrix from the geometric perspective?

I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ...
1
vote
0answers
15 views

Is it possible to find the joint distribution of a random vector if only the distribution of scalar many-to-one transformation is known? [duplicate]

Theoretical Exercise: I'd like to derive the joint distribution $p_{\boldsymbol{X}}$ of a random vector $\boldsymbol{X} \in \mathbb{R}^K$ if only the distribution of a scalar many-to-one ...
3
votes
1answer
88 views

Higher Order of Vectorization in Backpropagation in Neural Network

I am learning a machine learning class online from Stanford, namely CS 229. There is one section about deep learning and back-propagation in deep learning. The network looks like: The forward ...
0
votes
0answers
21 views

Highest posterior interval and monotone changes of variables

Suppose $X$ is distributed with a unimodal pdf $f(x)$ and let $Y = g(X)$ for some strictly monotone function $g$. Hence $g$ is invertible. Is there an analytically tractable relationship between the ...
1
vote
3answers
155 views

Computing directly the pdf of $Y=X^2$ for the pdf $f_X(x) = \frac{2}{9}(x+1)$

In the book Statistical inference by Casella and Berger, given that the pdf of X is $f_X(x) = \frac{2}{9}(x+1)$ for $-1 \le x \le 2$, we want to find the pdf of $Y=X^2$. But it says we are not able to ...
0
votes
1answer
85 views

For a posterior $p(\theta |y)$, if I specify a one-to-one transformation $\phi = g(\theta)$, how can I apply the transformation? [duplicate]

Suppose I have a posterior distribution, $p(\theta \mid y)$, where $y$ was my data and $\theta$ is a random variable with some prior distribution. If I specify a one-to-one transformation $\phi = g(\...
1
vote
0answers
57 views

Wrong vector calculus in lecture note 5 of cs224n, Stanford

I am studying NLP via cs224n from Stanford. I am reading this lecture note now. When you refer to the 5th page, they want to derive the gradient with respect to W for RNN, to show the mathematical ...
2
votes
1answer
301 views

Shouldn't it be Jacobian Descent?

Whenever we have a multiclass prediction the classifier generates a vector output. Per the definition of a Jacobian we are actually taking Jacobian steps towards a local minimum - so should it ...
0
votes
1answer
174 views

Prediction Interval for Regression Models with Weights

Courtesy Prediction Interval for Neural Net With Hessian :: nnet in R I would like to understand how to manually derive a point-wise prediction confidence interval applicable to neural models. ...
3
votes
1answer
277 views

Weighted Least Squares weights not changing Jacobian matrix

I currently have 4 data points and the following Jacobian matrix $A$ and cost vector $b$ $ A = \begin{bmatrix} -0.7867 & 0.0464& -0.6155 & 1.0000 \\ -0.3751 & 0.4299 &...
3
votes
2answers
4k views

What is the jacobian for a neural network

I know that a Jacobian is a matrix holding all of the first-order derivatives for a vector-valued-function. What is the Jacobian of a neural network, though? What are the inputs and what are the ...
2
votes
0answers
117 views

Express the density of a function of two random variables using the Gradient and the joint density

I would like to know if it is possible to express the density $f_Z(z)$ of a function $Z = g(X,Y)$ of two continuous "nice" random variables $X$ and $Y$ only using the joint density $f_{XY}(x,y)$ and ...
18
votes
2answers
411 views

Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, 1)$

What is the easiest way to see that the following statement is true? Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, ...
1
vote
0answers
511 views

Meaning of Jacobian of the transformation for pdf of function of random vectors

I am studying multivariate statistics and I don't understand the meaning of Jacobian of the transformation for pdf of function of random vectors. If I have a random vector, let's say bivariate, (X,Y)...
1
vote
0answers
46 views

What are some interesting parameterizations of $4 \times 4$ correlation matrices, and also perhaps their associated jacobians?

I am studying (mainly using Mathematica) some constrained integration problems in which the six-dimensional convex set of $4 \times 4$ correlation matrices plays a central role. In light of this, I ...
1
vote
0answers
70 views

Normalization constant for many to one mapping (Laplace distribution)

Suppose $\alpha=U\beta$ where $U$ is $N\times K$ with $N > K$. What is the probability density function (PDF) of $\beta$, $p(\beta)$, given that we know that it is proportional to $q$, the PDF of $\...
2
votes
0answers
1k views

Calculating the Jacobian of a neural network

I'm trying to calculate confidence intervals for a neural network (rather than prediction intervals). I'm following this paper, which treats them in the same framework as any parametric (parameter-...
6
votes
2answers
1k views

Sample random variables conditional on their sum

Let $(X_1, \dots, X_n)$ be an iid sample of random variables with a known continuous distribution. I would like to simulate such a sample, conditional on the value of its sum, that is: $$ X_1, \dots, ...
2
votes
1answer
148 views

IS $\int_{-\infty}^\infty e^{-\beta\cdot g(x)}g(x)^{\alpha-1}\text{d}x={\Gamma(\alpha)\over \beta^\alpha}\ \ ?$ [closed]

Is the following statement true: Let $g(x)$ be some non negative continuous function of $x$.We know that$$\int_{0}^\infty e^{-\beta x}x^{\alpha-1}dx={\Gamma(\alpha)\over \beta^\alpha}$$ Does ...
15
votes
1answer
5k views

Derivation of change of variables of a probability density function?

In the book pattern recognition and machine learning (formula 1.27), it gives $$p_y(y)=p_x(x) \left | \frac{d x}{d y} \right |=p_x(g(y)) | g'(y) |$$ where $x=g(y)$, $p_x(x)$ is the pdf that ...
1
vote
1answer
769 views

Jacobian and covariance matrix

Anyone know the Bishop's book in 2.53 they use Jacobian to convert covariance matrix x to y. $$J_{ij}=\dfrac{\partial x_i}{\partial y_i}=U_{ji} \qquad{(2.53)}$$ $$\int_{\bf x} f({\bf x})d{\bf x} = \...
4
votes
1answer
6k views

Relation between Covariance matrix and Jacobian in Nonlinear Least Squares

I saw that CovB = inv(J'*J)*MSE in a MATLAB documentation here at http://www.mathworks.com/help/stats/nlinfit.html However, I cant find any sources for the ...
1
vote
0answers
170 views

Delta method with mix of continuous and discrete variables

This is my first question on Cross Validated so please bear with me if my question is lagging in any dimension. My question regards how to evaluate a Jacobian matrix when one variable is binary. I ...
3
votes
1answer
292 views

Change of Variable technique for two variables?

If, $\theta_1 = \ln \frac p{1-p}$ $\theta_2 = \ln \frac q{1-q}$ $\theta_2|\theta_1 \sim N(\theta_1, \sigma^2)$ which means $f(\theta_1,\theta_2) \propto e^{\frac{-(\theta_1-\theta_2)^2}{2\...
2
votes
0answers
130 views

About deriving PDFs from CDFs

Suppose I have some continuous random variable $X$. Further, suppose I am interested about a transformed random variable $Y = g(X)$ where $g$ is some increasing function. If I know the CDF of $X$, I ...
1
vote
1answer
101 views

Sampling from an (almost) multivariate normal over matrices

Consider $n$ points in the euclidean plane, $p_i = (x_i,y_i)_{1\leq i \leq n}$. Now consider a $2 \times 2$ matrix $M = \left(\begin{array}{cc}a & b\\c& d\end{array}\right)$ a vector $r = \...
9
votes
1answer
336 views

If $X_1,X_2$ are independent beta then show $\sqrt{X_1X_2}$ is also beta

Here is a problem that came in a semester exam in our university few years back which I am struggling to solve. If $X_1,X_2$ are independent $\beta$ random variables with densities $\beta(n_1,n_2)$ ...
4
votes
1answer
1k views

conditional probability, change of variable and Jacobian

I have a question, and I am guessing that the question arises due to my lack of good understanding in the change of variable technique. I would like to evaluate $f_X(x)$. When $f_Y(y)$ exists, I can ...
4
votes
2answers
2k views

Transformation of variables (Metropolis Hastings)

Say I have a bunch of data from a Poisson distribution and I want to find out my posterior i.e. I'm data fitting: $p(\lambda | X) \sim p(X|\lambda)p(\lambda)$ where $p(X|\lambda) = \frac{\exp(-\...
1
vote
0answers
682 views

Jacobian matrix in neural network

How do you calculate the Jacobian matrix using the results (weights and biases) of a neural network after training? I am working in MATLAB, if anyone has any code suggestions, that would be helpful as ...
3
votes
1answer
643 views

Sufficient statistics for a continuous distribution

Below is a discussion of sufficient statistics for a continuous distribution, taken from the third edition of Lehmann's Testing Statistical Hypotheses. I understand the discussion until the underlined ...