Timeline for Intuition behind computing gradient for a model
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Mar 12, 2019 at 6:53 | vote | accept | Andrzej Gis | ||
| Mar 12, 2019 at 2:19 | answer | added | Soroush | timeline score: 4 | |
| Mar 11, 2019 at 22:07 | comment | added | Sycorax♦ | @AndrzejGis Math typsetting is supported using Latex-like markup. More information: math.meta.stackexchange.com/questions/5020/… | |
| Mar 11, 2019 at 21:59 | history | edited | Andrzej Gis | CC BY-SA 4.0 |
edited body
|
| Mar 11, 2019 at 21:59 | comment | added | Andrzej Gis |
@MatthewGunn That starts to make sense, but you used MSE activation function, while I used simple y - score. Why does it still work in my case?
|
|
| Mar 11, 2019 at 20:25 | comment | added | Matthew Gunn | Yes if $\mathbf{x}_i$ is a scalar, but that's not quite correct if $\mathbf{x}_i$ is a vector. Writing the gradient as a column vector, the answer is $\frac{\partial f}{\partial \mathbf{x}} = -\frac{1}{n} \sum_{i=1}^n 2 (y_i - \mathbf{x}'_i \mathbf{b}) \mathbf{x}_i$. Define $\mathbf{x}'_i \mathbf{b} - y_i$ as the forecast error, and you see the code computes the gradient (except the code drops the 2). | |
| Mar 11, 2019 at 19:45 | comment | added | Andrzej Gis | @MatthewGunn I suppose it will be: 1/n * sum_i_n(2 * x_i^2 * b - 2 * x_i * y_i) | |
| Mar 10, 2019 at 20:36 | comment | added | Matthew Gunn | Let $y_i = \mathbf{x}_i ' \mathbf{b} + e_i$. Let $f(\mathbf{b}) = \frac{1}{n} \sum_{i=1}^n e_i^2 = \frac{1}{n} \sum_{i=1}^n \left( y_i - \mathbf{x}_i' \mathbf{b} \right)^2$. What's the gradient of $f$ (with respect to $\mathbf{b}$)? | |
| Mar 10, 2019 at 19:58 | history | asked | Andrzej Gis | CC BY-SA 4.0 |