A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

# Questions tagged [matrix-calculus]

Matrix calculus deals with the problems of differentiating (possibly matrix-valued) functions of matrices

39 questions
Filter by
Sorted by
Tagged with
22 views

### Optimize Log Likelihood Model based on Gaussian Process involving Matrix Calculus

Given \begin{equation} \text{temperature(t, y)} = a_0 + a_1t + X(t) \end{equation} where temperature(t, year) is the dataset temperature at day $t$ in year $y$. $a_0, a_1 \in \mathbb{R}$, and $X(t)$ ...
34 views

151 views

74 views

I got this problem while I was reading the book "Machine Learning: A Probabilistic Perspective" by Kevin Murphy. It is in section 7.6.1 of the book. Assume the likelihood is given by $$\begin{... 0answers 190 views ### How to understand Jacobian Matrix from the geometric perspective? I found a good lecture about Jacobian Matrix which was part of a statistics course. However, it was published 20 years ago and lack of explanation. As a beginner of statistics, I'm not able to find ... 1answer 48 views ### Minimizing a least square [duplicate] I'm a bit confused with matrix calculus. Given is$$f(x) = \frac{1}{2}||Ax-b||^2_2$$and the derivate of it is in my book$$\nabla_xf(x) = A^T(Ax-b). $$I don't see how this works. My plan was to ... 3answers 119 views ### Hessian of Log of Matrix-t distribution I am trying to calculate the hessian of the log of the matrix-t distribution. I know that the log of the matrix-t distribution can be written:$$\log T_{N\times P}(X| \nu, M, \Sigma, \Omega) \propto -\...
264 views

I am in the process of implementing back propagation into my image classification neural net. I am using this cost function with a sigmoid output layer and ReLU hidden layers. The neural net has 3 ...
452 views

122 views

### Dimensions in single layer NN gradient

Given a neural network with one hidden sigmoid layer and softmax output layer, I want to derive the gradient of the cross entropy loss with respect to the first weight matrix. This is equivalent to ...
411 views

### Derivative of a quadratic form wrt a parameter in the matrix

I want to compute the derivative of: $\frac{\partial y^T C^{-1}(\theta)y}{\partial \theta_{k}}$, (Note that C is a covariance matrix that depends on a set of parameters $\theta$) for which I used ...
2k views

### What if do not use any activation function in the neural network? [duplicate]

or, for example, is it good to use activation function only for a last layer? As I know, if there are no activation functions in neural network, feedforward will be like simple matrix multiplication, ...
92 views

### What is the derivative of $\|X^T-S^TAX^T\|_F^2$ w.r.t $A$?

What is the derivative of $F = \|X^T-S^TAX^T\|_F^2$ w.r.t $A$, where $X \in\mathbb R^{d \times N}$, $S \in\mathbb R^{k \times N}$, and $A \in \mathbb{R}^{k \times N}$? I have tried, and it is as ...
38 views

### Derivative of matrix w.r.t vector [duplicate]

I'm quite out of my element trying to do some matrix calculus. I would like to know what the derivative of $z^{T}y$ w.r.t $z$ is, where z, y are n length vectors. Can anyone suggest good resources ...
76 views

100 views

### Handling linear algebraic differentiation in OLS parameter estimation

Could someone please explain why: \begin{equation} \frac{\partial (Y-\beta^T X)^T (Y-\beta^T X)}{\partial \beta}=2X^T(Y-\beta^T X) \end{equation} and why: \begin{equation} \frac{\partial \lambda \...
349 views

84 views

### Back-prop question: can this gradient be decomposed?

So, I was going over the lectures for the Oxford 2015 deep learning course, and in the lectures, they introduce back-propagation as a recursive procedure which involves two key formulas: The ...
4k views

969 views

728 views

### What are the 2nd derivatives of the log multivariate normal density?

I develop open-source statistical software (http://openmx.psyc.virginia.edu/), but matrix calculus is not my strong point. I need the 1st and 2nd derivatives of the log multivariate normal density. I ...