# Questions tagged [matrix-calculus]

Matrix calculus deals with the problems of differentiating (possibly matrix-valued) functions of matrices

57 questions
Filter by
Sorted by
Tagged with
13 views

### Deriving Optimizer of Quadratic Loss for Classification

I'm currently considering a binary classification problem where we have data points $X_1,\dots,X_n\in\mathbb{R}^d$ and labels $y_i=\pm1$. I'm using a simple linear model to model $y_i$, and it has the ...
• 157
24 views

• 157
22 views

### How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]

How to get transpose part when derive mathematically $$\frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T$$
1 vote
25 views

### Optimize Log Likelihood Model based on Gaussian Process involving Matrix Calculus

Given $$\text{temperature(t, y)} = a_0 + a_1t + X(t)$$ where temperature(t, year) is the dataset temperature at day $t$ in year $y$. $a_0, a_1 \in \mathbb{R}$, and $X(t)$ ...
• 13
521 views

• 131
999 views

• 21
145 views

I got this problem while I was reading the book "Machine Learning: A Probabilistic Perspective" by Kevin Murphy. It is in section 7.6.1 of the book. Assume the likelihood is given by $$\begin{... • 33 1 vote 0 answers 329 views ### How to understand Jacobian Matrix from the geometric perspective? I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ... • 111 2 votes 1 answer 68 views ### Minimizing a least square [duplicate] I'm a bit confused with matrix calculus. Given is$$f(x) = \frac{1}{2}||Ax-b||^2_2$$and the derivate of it is in my book$$\nabla_xf(x) = A^T(Ax-b). $$I don't see how this works. My plan was to ... • 591 2 votes 3 answers 182 views ### Hessian of Log of Matrix-t distribution I am trying to calculate the hessian of the log of the matrix-t distribution. I know that the log of the matrix-t distribution can be written:$$\log T_{N\times P}(X| \nu, M, \Sigma, \Omega) \propto -\...
• 878
1 vote
425 views

I am in the process of implementing back propagation into my image classification neural net. I am using this cost function with a sigmoid output layer and ReLU hidden layers. The neural net has 3 ...
• 157
1k views

• 91
235 views

### Dimensions in single layer NN gradient

Given a neural network with one hidden sigmoid layer and softmax output layer, I want to derive the gradient of the cross entropy loss with respect to the first weight matrix. This is equivalent to ...
• 201
954 views

### Derivative of a quadratic form wrt a parameter in the matrix

I want to compute the derivative of: $\frac{\partial y^T C^{-1}(\theta)y}{\partial \theta_{k}}$, (Note that C is a covariance matrix that depends on a set of parameters $\theta$) for which I used ...
• 61
8k views

### What if do not use any activation function in the neural network? [duplicate]

or, for example, is it good to use activation function only for a last layer? As I know, if there are no activation functions in neural network, feedforward will be like simple matrix multiplication, ...
163 views

### What is the derivative of $\|X^T-S^TAX^T\|_F^2$ w.r.t $A$?

What is the derivative of $F = \|X^T-S^TAX^T\|_F^2$ w.r.t $A$, where $X \in\mathbb R^{d \times N}$, $S \in\mathbb R^{k \times N}$, and $A \in \mathbb{R}^{k \times N}$? I have tried, and it is as ...
• 51
40 views

### Derivative of matrix w.r.t vector [duplicate]

I'm quite out of my element trying to do some matrix calculus. I would like to know what the derivative of $z^{T}y$ w.r.t $z$ is, where z, y are n length vectors. Can anyone suggest good resources ...
1 vote
86 views

• 307
118 views

### Handling linear algebraic differentiation in OLS parameter estimation

Could someone please explain why: $$\frac{\partial (Y-\beta^T X)^T (Y-\beta^T X)}{\partial \beta}=2X^T(Y-\beta^T X)$$ and why: \frac{\partial \lambda \...