We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [jacobian]

For statistical questions involving the Jacobian matrix (or determinant) of first partial derivatives. For purely mathematical questions about the Jacobian it is better to ask at math SE https://math.stackexchange.com/.

Filter by
Sorted by
Tagged with
18
votes
2answers
418 views

Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, 1)$

What is the easiest way to see that the following statement is true? Suppose $Y_1, \dots, Y_n \overset{\text{iid}}{\sim} \text{Exp}(1)$. Show $\sum_{i=1}^{n}(Y_i - Y_{(1)}) \sim \text{Gamma}(n-1, ...
15
votes
1answer
6k views

Derivation of change of variables of a probability density function?

In the book pattern recognition and machine learning (formula 1.27), it gives $$p_y(y)=p_x(x) \left | \frac{d x}{d y} \right |=p_x(g(y)) | g'(y) |$$ where $x=g(y)$, $p_x(x)$ is the pdf that ...
9
votes
1answer
336 views

If $X_1,X_2$ are independent beta then show $\sqrt{X_1X_2}$ is also beta

Here is a problem that came in a semester exam in our university few years back which I am struggling to solve. If $X_1,X_2$ are independent $\beta$ random variables with densities $\beta(n_1,n_2)$ ...
6
votes
2answers
1k views

Sample random variables conditional on their sum

Let $(X_1, \dots, X_n)$ be an iid sample of random variables with a known continuous distribution. I would like to simulate such a sample, conditional on the value of its sum, that is: $$ X_1, \dots, ...
6
votes
1answer
7k views

Relation between Covariance matrix and Jacobian in Nonlinear Least Squares

I saw that CovB = inv(J'*J)*MSE in a MATLAB documentation here at http://www.mathworks.com/help/stats/nlinfit.html However, I cant find any sources for the ...
4
votes
2answers
2k views

Transformation of variables (Metropolis Hastings)

Say I have a bunch of data from a Poisson distribution and I want to find out my posterior i.e. I'm data fitting: $p(\lambda | X) \sim p(X|\lambda)p(\lambda)$ where $p(X|\lambda) = \frac{\exp(-\...
4
votes
1answer
1k views

conditional probability, change of variable and Jacobian

I have a question, and I am guessing that the question arises due to my lack of good understanding in the change of variable technique. I would like to evaluate $f_X(x)$. When $f_Y(y)$ exists, I can ...
3
votes
2answers
4k views

What is the jacobian for a neural network

I know that a Jacobian is a matrix holding all of the first-order derivatives for a vector-valued-function. What is the Jacobian of a neural network, though? What are the inputs and what are the ...
3
votes
1answer
94 views

Reason for absolute value of Jacobian determinant in change-of-variable formula?

When we have a random variable $x$ with a probability density $p(x)$, and a function $y = f(x)$ that is differentiable and can be solved for $x = g(y)$, the change of variable formula leads us to a ...
3
votes
1answer
291 views

Weighted Least Squares weights not changing Jacobian matrix

I currently have 4 data points and the following Jacobian matrix $A$ and cost vector $b$ $ A = \begin{bmatrix} -0.7867 & 0.0464& -0.6155 & 1.0000 \\ -0.3751 & 0.4299 &...
3
votes
1answer
320 views

Change of Variable technique for two variables?

If, $\theta_1 = \ln \frac p{1-p}$ $\theta_2 = \ln \frac q{1-q}$ $\theta_2|\theta_1 \sim N(\theta_1, \sigma^2)$ which means $f(\theta_1,\theta_2) \propto e^{\frac{-(\theta_1-\theta_2)^2}{2\...
3
votes
1answer
671 views

Sufficient statistics for a continuous distribution

Below is a discussion of sufficient statistics for a continuous distribution, taken from the third edition of Lehmann's Testing Statistical Hypotheses. I understand the discussion until the underlined ...
3
votes
1answer
119 views

Higher Order of Vectorization in Backpropagation in Neural Network

I am learning a machine learning class online from Stanford, namely CS 229. There is one section about deep learning and back-propagation in deep learning. The network looks like: The forward ...
2
votes
1answer
154 views

IS $\int_{-\infty}^\infty e^{-\beta\cdot g(x)}g(x)^{\alpha-1}\text{d}x={\Gamma(\alpha)\over \beta^\alpha}\ \ ?$ [closed]

Is the following statement true: Let $g(x)$ be some non negative continuous function of $x$.We know that$$\int_{0}^\infty e^{-\beta x}x^{\alpha-1}dx={\Gamma(\alpha)\over \beta^\alpha}$$ Does ...
2
votes
1answer
310 views

Shouldn't it be Jacobian Descent?

Whenever we have a multiclass prediction the classifier generates a vector output. Per the definition of a Jacobian we are actually taking Jacobian steps towards a local minimum - so should it ...
2
votes
0answers
122 views

Express the density of a function of two random variables using the Gradient and the joint density

I would like to know if it is possible to express the density $f_Z(z)$ of a function $Z = g(X,Y)$ of two continuous "nice" random variables $X$ and $Y$ only using the joint density $f_{XY}(x,y)$ and ...
2
votes
0answers
1k views

Calculating the Jacobian of a neural network

I'm trying to calculate confidence intervals for a neural network (rather than prediction intervals). I'm following this paper, which treats them in the same framework as any parametric (parameter-...
2
votes
0answers
131 views

About deriving PDFs from CDFs

Suppose I have some continuous random variable $X$. Further, suppose I am interested about a transformed random variable $Y = g(X)$ where $g$ is some increasing function. If I know the CDF of $X$, I ...
1
vote
3answers
158 views

Computing directly the pdf of $Y=X^2$ for the pdf $f_X(x) = \frac{2}{9}(x+1)$

In the book Statistical inference by Casella and Berger, given that the pdf of X is $f_X(x) = \frac{2}{9}(x+1)$ for $-1 \le x \le 2$, we want to find the pdf of $Y=X^2$. But it says we are not able to ...
1
vote
1answer
829 views

Jacobian and covariance matrix

Anyone know the Bishop's book in 2.53 they use Jacobian to convert covariance matrix x to y. $$J_{ij}=\dfrac{\partial x_i}{\partial y_i}=U_{ji} \qquad{(2.53)}$$ $$\int_{\bf x} f({\bf x})d{\bf x} = \...
1
vote
1answer
102 views

Sampling from an (almost) multivariate normal over matrices

Consider $n$ points in the euclidean plane, $p_i = (x_i,y_i)_{1\leq i \leq n}$. Now consider a $2 \times 2$ matrix $M = \left(\begin{array}{cc}a & b\\c& d\end{array}\right)$ a vector $r = \...
1
vote
0answers
19 views

Change of variables: 4-dimensional PDF to 2-dimensional PDF

I have a 4-dimensional joint-PDF between variables $X_1,X_2,X_3,X_4$ which are all Gaussian. I want to transform this into a 2-dimensional joint-PDF between new variables $Y_1=Y_1(X_1,X_2,X_3,X_4)$ ...
1
vote
0answers
177 views

How to understand Jacobian Matrix from the geometric perspective?

I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ...
1
vote
0answers
15 views

Is it possible to find the joint distribution of a random vector if only the distribution of scalar many-to-one transformation is known? [duplicate]

Theoretical Exercise: I'd like to derive the joint distribution $p_{\boldsymbol{X}}$ of a random vector $\boldsymbol{X} \in \mathbb{R}^K$ if only the distribution of a scalar many-to-one ...
1
vote
0answers
59 views

Wrong vector calculus in lecture note 5 of cs224n, Stanford

I am studying NLP via cs224n from Stanford. I am reading this lecture note now. When you refer to the 5th page, they want to derive the gradient with respect to W for RNN, to show the mathematical ...
1
vote
1answer
189 views

Prediction Interval for Regression Models with Weights

Courtesy Prediction Interval for Neural Net With Hessian :: nnet in R I would like to understand how to manually derive a point-wise prediction confidence interval applicable to neural models. ...
1
vote
0answers
546 views

Meaning of Jacobian of the transformation for pdf of function of random vectors

I am studying multivariate statistics and I don't understand the meaning of Jacobian of the transformation for pdf of function of random vectors. If I have a random vector, let's say bivariate, (X,Y)...
1
vote
0answers
47 views

What are some interesting parameterizations of $4 \times 4$ correlation matrices, and also perhaps their associated jacobians?

I am studying (mainly using Mathematica) some constrained integration problems in which the six-dimensional convex set of $4 \times 4$ correlation matrices plays a central role. In light of this, I ...
1
vote
0answers
90 views

Normalization constant for many to one mapping (Laplace distribution)

Suppose $\alpha=U\beta$ where $U$ is $N\times K$ with $N > K$. What is the probability density function (PDF) of $\beta$, $p(\beta)$, given that we know that it is proportional to $q$, the PDF of $\...
1
vote
0answers
193 views

Delta method with mix of continuous and discrete variables

This is my first question on Cross Validated so please bear with me if my question is lagging in any dimension. My question regards how to evaluate a Jacobian matrix when one variable is binary. I ...
1
vote
0answers
686 views

Jacobian matrix in neural network

How do you calculate the Jacobian matrix using the results (weights and biases) of a neural network after training? I am working in MATLAB, if anyone has any code suggestions, that would be helpful as ...
0
votes
1answer
109 views

For a posterior $p(\theta |y)$, if I specify a one-to-one transformation $\phi = g(\theta)$, how can I apply the transformation? [duplicate]

Suppose I have a posterior distribution, $p(\theta \mid y)$, where $y$ was my data and $\theta$ is a random variable with some prior distribution. If I specify a one-to-one transformation $\phi = g(\...
0
votes
0answers
28 views

Showing a useful result for Wisharts and Multivariate Beta random matrices

Let $\mathbf{A} \sim \text{Wishart}_m\left(k_a,\mathbf{V} \right)$ and $\mathbf{B} \sim \text{Wishart}_m\left(k_b,\mathbf{V} \right)$ be two full rank Wishart random matrices. Define $$ \mathbf{S} = \...
0
votes
0answers
24 views

Does conjugate prior for natural exponential family needs jacobian to transform natural parameter back to original parameter?

From bayesian theory, we have that if $f(x|\eta) \propto \exp(\eta \cdot T(x)- A(\eta))$ - a natural exponential family, then the prior conjugate of $\eta$ is $\pi^*(\eta | \mu, \lambda) \propto \exp(\...
0
votes
0answers
14 views

CCA on feature maps: Gradient w.r.t to Jacobian

Assume I have two neural networks, abstracted as two feature maps, parametrized by $\theta_x,\theta_y$ respectively. $\phi_x(x;\theta_x) \in \mathbb{R}^{h_1}$, $\phi_y(x;\theta_y) \in \mathbb{R}^{h_2}$...
0
votes
0answers
82 views

Transformation of random variables and Jacobian

When transforming 2+ continuous random variables, you use a Jacobian matrix and compute the determinant. Do you also compute the Jacobian for discrete random variables?
0
votes
0answers
32 views

Projection of multivariate distribution to lower dimensional subspace

Say that $X \in \mathbb{R}^n$ is a vector of $n$ r.v.'s with pdf $p(x_1,\ldots,x_n)$. Let's consider now the linear map $Y = A X$ where $Y \in \mathbb{R}^m$ with $m < n$. I am seeking $p(y_1,\ldots,...
0
votes
0answers
24 views

Highest posterior interval and monotone changes of variables

Suppose $X$ is distributed with a unimodal pdf $f(x)$ and let $Y = g(X)$ for some strictly monotone function $g$. Hence $g$ is invertible. Is there an analytically tractable relationship between the ...