Linked Questions

0
votes
0answers
39 views

Derivative of matrix w.r.t vector [duplicate]

I'm quite out of my element trying to do some matrix calculus. I would like to know what the derivative of $z^{T}y$ w.r.t $z$ is, where z, y are n length vectors. Can anyone suggest good resources ...
0
votes
0answers
17 views

Derivation of matrix derivative [duplicate]

In the ML lecture notes of cs229, I am not able to derive the equation 3 in matrix derivatives. I applied the equation 1 to derive it, considering the product after A as another matrix, I am not ...
31
votes
4answers
13k views

Independent variable = Random variable?

I'm slightly confused if an independent variable (also called predictor or feature) in a statistical model, for example the $X$ in linear regression $Y=\beta_0+\beta_1 X$, is a random variable ?
7
votes
2answers
5k views

gradient versus partial derivatives

how exactly is partial derivative different from gradient of a function? In both the case, we are computing the rate of change of a function with respect to some independent variable. While I was ...
4
votes
1answer
114 views

Differentiation step in OLS

In deriving the parameter estimate in OLS, we differentiate the following (in matrix form) $$y^T y - 2\beta^T X^T y + \beta^T X^T X \beta$$ The part of the differentiation I don't understand is why $$\...
3
votes
2answers
144 views

What is the derivative of $\|X^T-S^TAX^T\|_F^2$ w.r.t $A$?

What is the derivative of $F = \|X^T-S^TAX^T\|_F^2$ w.r.t $A$, where $X \in\mathbb R^{d \times N}$, $S \in\mathbb R^{k \times N}$, and $A \in \mathbb{R}^{k \times N}$? I have tried, and it is as ...
3
votes
0answers
111 views

computing matrix function derivatives [closed]

I was able to use the following equation to find derivatives of matrix function: $f(X+h) = f(X) + Ah + o(|h|) \quad \cdots (1)$ where $h$ is small displacement and $A$ is the jacobian matrix. I ...
2
votes
0answers
75 views

How is a vector differentiation by a matrix defined?

I am a beginner of the machine learning. And at the same time, this is the first time to ask a question in this site, so I might have a problem with this question in terms of understandability. And ...
1
vote
0answers
51 views

How do you find the derivative (gradient) of the general linear regression method?

I am very new to statistical learning (I'm a graduate student in experimental biology with very little exposure to math or statistics) and I'm working my way through Introduction to Statistical ...
1
vote
0answers
39 views

update step requires gradient of $\mathbb R:f^{d\times d'} \rightarrow\mathbb R$ function - how to calculate

I have a neural network with one hidden layer and an activation function for the output layer: $$f_k(w,u)(x) = \sum_{i = 1}^{m_1} w_{ki} \max\left(0,\sum_{j=1}^d u_{ij}x_j\right)$$ I am supposed to ...
0
votes
0answers
21 views

How to derive mathematically that derivative of |Ax-y|^2 with respect to A is 2|Ax-y| x^T [duplicate]

How to get transpose part when derive mathematically $$ \frac {\partial|Ax-y|^2}{\partial A} = 2|Ax-y|x^T $$