# Questions tagged [matrix-calculus]

Matrix calculus deals with the problems of differentiating (possibly matrix-valued) functions of matrices

58 questions
Filter by
Sorted by
Tagged with
414 views

### Derivative of vector Y=XW with respect to matrix W

In the first derivation of dL/dW, I use the rule for the derivative of a constant with respect to a matrix and then apply the chain rule. \begin{gather*} Y\ =\ XW\ +\ B\\ X=\begin{bmatrix} x_{0} & ...
36 views

1 vote
183 views

### Matrix Derivation for Neural Network Formula

I am learning some insights of Neural network but I have some problem with the derivation of matrix for backpropagation. On an assumption that the formula for calculating for one node in a neural ...
1k views

### matrix-calculus - Understanding numerator/denominator layouts

Also see this question for more external references! Consider the following machine-learning model: Here, $J = \frac{1}{m} \sum_{i = 1}^{m} L(\hat{y}^{(i)}, y^{(i)})$, and $m$ is the number of ...
45 views

### Confusion surrounding differentiation of parameter vector ridge regression

Let $\mathbf{\beta}$ be the parameter vector of a ridge regression. Now we can say that: \begin{equation} \frac{\partial \lambda \beta^T \beta}{\partial \beta}=2\lambda\beta. \end{equation} Why is ...
82 views

857 views

### How do I calculate the optimal hyperplane of SVMs by hand?

I am trying to get a better understanding of SVMs and their optimization process. I understand that we have a constrained optimization problem that we have to solve with Lagrange multipliers. The ...
25 views

### Computing the Jacobian $J_F$ with $F = h \circ f$

Let $$f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex] h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$ and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$ I want to compute ...
173 views

261 views

### What is the first order derivative of linear regression's cost function using matrix calculus?

For linear regression's cost function J(b), where X is a n*m matrix, b is a m*1 vector and y is n*1 vector: First order derivative with respect to vector b (coefficients) is shown to be Using the ...
388 views

49 views

188 views

### What reparametrization of vector parameters makes the Jeffreys prior correspond to the uniform prior?

What reparametrization of vector of parameters $\theta$ makes the Jeffreys prior $$\sqrt{\det I(\theta)}$$ correspond to the uniform prior? A change of parametrization from $\theta$ to $\eta$ changes ...
1 vote
I'm trying to use a Matérn 5/2 kernel for GP regression, so my kernel function is $K(x,x')\triangleq\theta_0(1+\sqrt{5r(x,x')}+5/3r)\exp(-\sqrt{5r}),$ where \$r(x,x')\triangleq\sum_{d=1}^D (x_d-x'_d)^...