# Questions tagged [derivative]

For on-topic questions involving the mathematical concept of a derivative, i.e. $\frac{d}{dx} f(x)$. For purely mathematical questions about the derivative it is better to ask on math SE https://math.stackexchange.com/

192 questions
Filter by
Sorted by
Tagged with
97k views

### Backpropagation with Softmax / Cross Entropy

I'm trying to understand how backpropagation works for a softmax/cross-entropy output layer. The cross entropy error function is $$E(t,o)=-\sum_j t_j \log o_j$$ with $t$ and $o$ as the target and ...
39k views

### How is the cost function from Logistic Regression derivated

I am doing the Machine Learning Stanford course on Coursera. In the chapter on Logistic Regression, the cost function is this: Then, it is derivated here: I tried getting the derivative of the cost ...
12k views

### Step-by-step example of reverse-mode automatic differentiation

Not sure if this question belongs here, but it's closely related to gradient methods in optimization, which seems to be on-topic here. Anyway, feel free to migrate if you think some other community ...
428 views

### How can I fit a spline to data that contains values and 1st/2nd derivatives?

I have a dataset that contains, let's say, some measurements for position, speed and acceleration. All come from the same "run". I could construct a linear system and fit a polynomial to all of those ...
6k views

### Derivative of a Gaussian Process

I believe that the derivative of a Gaussian process (GP) is a another GP, and so I would like to know if there are closed form equations for the prediction equations of the derivative of a GP? In ...
1k views

### Interpretation of Radon-Nikodym derivative between probability measures?

I have seen at some points the use of the Radon-Nikodym derivative of one probability measure with respect to another, most notably in the Kullback-Leibler divergence, where it is the derivative of ...
860 views

### Second order approximation of the loss function (Deep learning book, 7.33)

In Goodfellow's (2016) book on deep learning, he talked about equivalence of early stopping to L2 regularisation (https://www.deeplearningbook.org/contents/regularization.html page 247). Quadratic ...
2k views

### Can a neural network learn a functional, and its functional derivative?

I understand that neural networks (NNs) can be considered universal approximators to both functions and their derivatives, under certain assumptions (on both the network and the function to ...
564 views

494 views

### Derivative of a quadratic form wrt a parameter in the matrix

I want to compute the derivative of: $\frac{\partial y^T C^{-1}(\theta)y}{\partial \theta_{k}}$, (Note that C is a covariance matrix that depends on a set of parameters $\theta$) for which I used ...
238 views

### If we have auto differentiate tool, do we also need EM algorithm?

I my opinion, EM algorithm is used to estimate the parameters of some complex log likelihood function. Because sometimes, it's hard to get the derivative, we can use EM algorithm. But if we have some ...
577 views

### If $f(x)$ is a unimodal probability density function, how can I show that its mode is at $f'(x)=0$?

If the function $f(x)$ is continuous and a probability density function (PDF), how can I show that its mode is at $f'(x)=0$?
2k views

### Computing gradients via Gaussian Process Regression

I have a set of noisy data that I am fitting using Gaussian Process Regression via Python's sklearn package. The posterior mean of the GP is essentially my output with an associated error. Based on ...
1k views

### Calculating the expression for the derivative of a Gaussian process

I know based on the answers to this question Derivative of a Gaussian Process that the derivative of a Gaussian process is another Gaussian process, but I was wondering if someone could tell (or show) ...
7k views

### Mean Absolute Error (MAE) derivative

$MAE=|y_{pred} - y_{true}|$ $\dfrac{dMAE}{dy_{pred}} = ?$ I'm trying to understand how MAE works as a loss function in neural networks using backpropogation. I know it can be used directly in some ...
127 views

### Slope of Curve with Unknown Functional Form

I have a monotonically-increasing curve whose functional form is not known a priori and would like to compute the curve's slope at the rightmost endpoint. Typically, when the functional form is known, ...
732 views

Referring to David Blei's notes on variational inference, I wonder how to get the derivative of $q(z_k)$, the distribution of $z_k$, in eq. 23 from eq. 22. Namely, eq. 22 is $L_k = \int q(z_k) \... 1answer 242 views ### Second directional derivate and Hessian matrix I am reading the following from the book Deep Learning, and I have the following questions. I don't quite understand second directional derivatives. The first directional derivative of a function$f:\...
207 views

The following is from Section 2.2 of the Auto-Encoding Variational Bayes paper, It says the gradient of the lower bound w.r.t $\phi$ is a bit problematic because the Monte Carlo estimator exhibits ...
619 views

### Marginal Effects of Discrete Variables in Quantile Regression

I find myself puzzled by a passage about marginal effects of discrete variables in quantile regression. On p. 217 of Cameron and Trivedi's MUS book, the authors write: For the $j$th (continuous) ...
40 views

### Normal distribution “stable” under derivative?

Suppose that $\theta(t)\sim\mathcal N(\mu(t),\Sigma(t))$ where $t$ is some parameter. Then it holds that $$\theta(t) = \mu(t) + \Sigma(t)^{0.5}\xi$$ for $\xi\sim\mathcal N(0, I)$. I am interested in ...
2k views

497 views

### Derivative of bivariate normal CDF with common mean parameters

I am trying to calculate a derivative of the form $\frac{d}{dz}\Phi_2(\mu_1(z),\mu_2(z),\rho)$, where $\Phi_2$ is the standard bivariate normal CDF. I am thinking it might be an application of a ...
436 views

### What is the second derivative of a B-spline?

A B-spline of degree $j$ is defined at knots $\vec k$ by the Cox-de Boor recursion formula \begin{align} B_{i,1}(x) &= \left\{ \begin{matrix} 1 & \mathrm{if} \quad ...
76 views

### When using OLS on $\ln(y) = \beta_1 \ln(x) + \epsilon$, is $\beta_1$ the elasticity of $E[y\vert x]$, or the $y$ in the data (or both)?

Specifically, suppose we are estimating $$\ln(y)=\beta_1\ln(x) + \epsilon$$ I understand that $\beta_1 = \frac{\partial \ln(y)}{\partial \ln(x)}$ which is the elasticity of $y$ with respect to $x$ ...
51 views

### Correlation between the j-feature and partial residual

I am reading a book for the lasso-regression. However, I think that this is a more general question regarding the derivative of the OLS-term: Can someone explain me why this $c_j$ term is ...
59 views

### Derivation of score vector

Can anyone explain the process of this derivation, step by step? This derivation is from Joint Models for Longitudinal and Time-to Event Data by Dimitris Rizopoulos. \begin{equation} \begin{aligned} ...