# Questions tagged [derivative]

For on-topic questions involving the mathematical concept of a derivative, i.e. $\frac{d}{dx} f(x)$. For purely mathematical questions about the derivative it is better to ask on math SE https://math.stackexchange.com/

192 questions
Filter by
Sorted by
Tagged with
12 views

187 views

### Where does the logistic function come from?

I first learned the logistic function in machine learning course, where it is just a function that map a real number to 0 to 1. We can use calculus to get its derivative and use the derivative for ...
25 views

### Derivative of all the parameters in Logistic Regression

$\mathcal{L}$ is the loss function, $\mathcal{L} = y_i \text{log} \sigma(z) + (1-y_i) \text{log} (1-\sigma(z))$, where $z = \sum_i w_ix_i$, with $w_i$ representing the weights and $x_i$ the features. ...
19 views

### n'th cumulant (of a CGF) for exponential family / exponential dispersion model

The n'th cumulant is defined to be the n'th derivative of the CGF (cumulant generating function). $$\kappa_n = \frac{d^n K(t)}{dt^n} |_{t=0}$$ But I'm reading in a book (p.215, chapter5, eq. 5.8) ...
268 views

### For back propagation in neural networks , how do we calculate vector by matrix derivative?

I am following the course deep learning ai by Andrew NG. In course1 week4, 04-06-Forward and Backward Propagation, he calculates backward propagation for layer $l$ in neural networks as follows (a ...
155 views

### Can we calculate the predictive process of the derivative of a stochastic process from the data?

I asked this question, Calculating the expression for the derivative of a Gaussian process, some time ago, and now I am interested in an extension to the question. So originally I wanted to know the ...
12k views

### Step-by-step example of reverse-mode automatic differentiation

Not sure if this question belongs here, but it's closely related to gradient methods in optimization, which seems to be on-topic here. Anyway, feel free to migrate if you think some other community ...
97k views

### Backpropagation with Softmax / Cross Entropy

I'm trying to understand how backpropagation works for a softmax/cross-entropy output layer. The cross entropy error function is $$E(t,o)=-\sum_j t_j \log o_j$$ with $t$ and $o$ as the target and ...
40 views

### Posterior distribution for a Gaussian Process with a transformation in a gaussian likelihood

Suppose we are modelling observations y as follows. Our likelihood is normal $y \sim \mathcal{N}(g(f(x)), \mathcal{I}\sigma^2)$, where $\mathcal{I}$ is the identity matrix and $g$ is some function ...
21 views

### Relation between dyi/dei and its leverage (hii)

Althought I have tried in different ways, I have not been able to show that $$\frac{\mathrm{d}Y_i}{\mathrm{d}e_i} = \frac{1}{1 - l_{ii}}$$ $e_i$ is equal to $l_{ii}$ is the $i-th$ element of the ...
20 views

### Is there a statistical test to determine correlation between the first derivative of two time based series?

I have two time series. They are normally distributed, parametric, exactly the same size and evenly spaced. I am seeking to determine if the rate of change of these two time series are correlated ...
13 views