# Questions tagged [matrix-calculus]

Matrix calculus deals with the problems of differentiating (possibly matrix-valued) functions of matrices

59 questions
Filter by
Sorted by
Tagged with
103 views

### Kalman Filter to minimize weighted errors on the states: what's wrong with my derivation

I am thinking about how to implement a "weighted Kalman Filter". Note that the weights here are on the states. Basically the classical KF minimizes $\sum (x_i - \hat{x_i} )^2$ but I want to ...
• 256
28 views

### Simple matrix calculus but I am struggling to understand [duplicate]

Here is my problem: We have $\mathbf{D} \in \Re^{m n}$, $\mathbf{W} \in \Re^{m q}$, and $\mathbf{X} \in \Re^{q n}$. Furthermore, $\mathbf{D} = \mathbf{W}\mathbf{X}$. (NOT an element wise ...
• 195
774 views

### Derivative of vector Y=XW with respect to matrix W

In the first derivation of dL/dW, I use the rule for the derivative of a constant with respect to a matrix and then apply the chain rule. \begin{gather*} Y\ =\ XW\ +\ B\\ X=\begin{bmatrix} x_{0} & ...
• 141
36 views

279 views

### Matrix Derivation for Neural Network Formula

I am learning some insights of Neural network but I have some problem with the derivation of matrix for backpropagation. On an assumption that the formula for calculating for one node in a neural ...
• 141
2k views

### matrix-calculus - Understanding numerator/denominator layouts

Also see this question for more external references! Consider the following machine-learning model: Here, $J = \frac{1}{m} \sum_{i = 1}^{m} L(\hat{y}^{(i)}, y^{(i)})$, and $m$ is the number of ...
• 240
49 views

### Confusion surrounding differentiation of parameter vector ridge regression

Let $\mathbf{\beta}$ be the parameter vector of a ridge regression. Now we can say that: $$\frac{\partial \lambda \beta^T \beta}{\partial \beta}=2\lambda\beta.$$ Why is ...
83 views

• 842
986 views

### How do I calculate the optimal hyperplane of SVMs by hand?

I am trying to get a better understanding of SVMs and their optimization process. I understand that we have a constrained optimization problem that we have to solve with Lagrange multipliers. The ...
• 343
27 views

### Computing the Jacobian $J_F$ with $F = h \circ f$

Let $$f: \mathbb{R}^l \rightarrow{} \mathbb{R}^m\\[.7ex] h: \mathbb{R}^m \rightarrow{} \mathbb{R}^o$$ and let $$F = h \circ f \quad (F : \mathbb{R}^l \rightarrow{} \mathbb{R}^o)$$ I want to compute ...
• 247
190 views

• 216
280 views

### What is the first order derivative of linear regression's cost function using matrix calculus?

For linear regression's cost function J(b), where X is a n*m matrix, b is a m*1 vector and y is n*1 vector: First order derivative with respect to vector b (coefficients) is shown to be Using the ...
• 333
406 views

• 165
51 views

• 899
192 views

### What reparametrization of vector parameters makes the Jeffreys prior correspond to the uniform prior?

What reparametrization of vector of parameters $\theta$ makes the Jeffreys prior $$\sqrt{\det I(\theta)}$$ correspond to the uniform prior? A change of parametrization from $\theta$ to $\eta$ changes ...
• 15.4k
1 vote