# Questions tagged [derivative]

For on-topic questions involving the mathematical concept of a derivative, i.e. $\frac{d}{dx} f(x)$. For purely mathematical questions about the derivative it is better to ask on math SE https://math.stackexchange.com/

270 questions
Filter by
Sorted by
Tagged with
1 vote
14 views

### A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning

In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said Again, the implementation of such algorithms can be checked by using numerical ...
• 67
18 views

25 views

### Derivative of quadratic form of vector-valued function

This seems like a trivial question but I am currently stuck and cannot see what I am doing wrong. So let us consider a function $f(x) : \mathbb{R}^d \rightarrow \mathbb{R}^d$. I want to compute the ...
1 vote
24 views

### Best Error Function for Areas with Larger Slopes

I have some nonlinear data (let's say x's and y's) that I would like to perform regression on, and I would like to focus on having the error lower on regions where the graph is more sloped, rather ...
1 vote
36 views

1 vote
77 views

### XGBoost Objective Derivation Problem

This is the loss function of XGBoost. This is the Second-order approximation of the loss function. Note: \begin{equation} L^{(t)} \text{: cross entropy loss function.} \end{equation} \begin{equation}...
• 13
89 views

### How to write the derivative of the inverse gamma function?

I have recently been writing an R program on the inverse of the gamma function and the derivative of the inverse function. Now there is some confusion I would like to ask for advice. I have written ...
• 43
51 views

### Derivatives of output w.r.t input on a neural network trained with standardized data

I'm using a neural network to model an unknown function for which I would also like to know the derivatives. The nn has four inputs and four outputs, and the training data is preprocessed using scikit-...
• 25
25 views

### Difference between analytic and numeric gradient in Matlab

I am using matlab to solve a optimization problem. When I check the anlaytic and numeric gradient reported by matlab, they are quite different. So I want to ask if there is a mistake in the analytic ...
• 105
138 views

### Difference between forward-mode and reverse-mode automatic differentiation?

I have difficulty grasping the difference between forward and reverse mode automatic differentiation. To understand this problem I have created a simple equation and broken this equation into small ...
• 2,041
29 views

### Fisher Information of Weight in Mixture distribution

Let's assume $x$ follows a mixture of two arbitrary continuous probability distributions with probability density functions $p_1(x)$ and $p_2(x)$, respectively. The probability density function of $x$ ...
• 812
20 views

### How is derivative calculated for Grad-CAM if the final output is multidimensional?

For Grad-CAM, the derivative of the final output is found with respect to the elements of the channel considered Selvaraju et al. 2019. But if the output is a multidimensional matrix how is the ...
377 views

### Proper regression for determining correlations between derivatives of functions

Say we have a permanent-magnet DC motor that roughly obeys the system equation $\ddot{x}(t) = \alpha \dot{x}(t) + \beta u(t) + \gamma$, where $x(t)$ is the displacement of the rotor, and $u(t)$ the ...
• 624
1 vote
289 views

### Why do the derivatives of a function lead towards the extremum of the function?

Is there some theorem in mathematics that formalizes the idea that "for some function, at a given point, moving in the negative direction of the gradient leads you to some (local) extremum point&...
• 6,068
1 vote
74 views

### Derivative of a probability

If $Y$ is a discrete random variable, and I define $F(x)=P(Y \leq x),$ where $x \in \mathbb{R},$ can I differentiate $F(x)$?
• 21
36 views

### Deriving initial weights for IRLS in DESeq2's GLM model

DeSEQ2 is a frequently-used R package for researchers studying differential gene expression via changes in molecular markers such as Poly(A) RNA. Understanding how ...
• 4,792
1 vote
14 views

### What does the SVD of the numerical derivative matrix of paired instances tell us about our data? [closed]

Let's say we have real-valued random variables $X$ and $Y$, and that under simple random sampling we obtain paired values $\{(x_1,y_1), \cdots, (x_i,y_i), \cdots, (x_m,y_m) \}$. From this sample we ...
• 4,792
616 views

### matrix-calculus - Understanding numerator/denominator layouts

Consider the following machine-learning model: Here, $J = \frac{1}{m} \sum_{i = 1}^{m} L(\hat{y}^{(i)}, y^{(i)})$, and $m$ is the number of training-examples. While performing reverse-mode ...
• 206
1 vote
350 views

### reverse sigmoid and its derivative

I wonder, if someone could please check/help me with this simple code: ...
• 1,346
1 vote
50 views

### Minimizing Expectation [closed]

I am not entirely sure how the derivative follows from the preceding line in this example. $f(x)$ is a PDF. You are supposed to set the derivative to 0 as the expectation needs to be minimised. ...
• 11
110 views

### derivative of the error w.r.t parameters

let's assume my function is as simple as $y = xW + b$ We define an error function as $E = {\frac{(t - y)}{2}}^2$ I wonder if you can help me to write the derivative of the error w.r.t parameters (W ...
• 1,108
1 vote
42 views

### Inverse of a noisy derivative

I have a series of samples (x(t), y(t)), where both are noisy and with (assumed) iid errors (sx(t), sy(t)). I need to measure a ...
• 31
1 vote
789 views

### How to calculate derivative of cross entropy loss function?

I have a cross entropy loss function. $$L = -{1 \over N} \sum_i {y_i \cdot \log {1 \over {1+e^{-\vec x \cdot \vec w}}} + (1-y_i) \cdot \log (1-{1 \over {1+e^{-\vec x \cdot \vec w}}})}$$ I want to ...
• 113
20 views

### Are numerical solutions appropriate for inference (eg, estimating variance for confidence intervals)?

For a nicely differentiable objective function, we traditionally always derive the gradients to use for e.g. estimating the variance. (1) Is it common nowadays to use numerical rather than analytical ...
• 5,356
39 views

### Interpretation of regression coefficient of logged variable (log X)

I am struggling to see why a one percent change in $X$ is associated with a $\frac{\beta_1}{100}$ change in $Y$ in the following model: $Y = \beta_0 + \beta_1 \ln X + \beta_2 W + ... + u$. It is clear ...
• 383
1 vote
67 views

### Derivative of $\nabla_{\theta} f(x, \theta) f(x, \theta)$ (the gradient of the function times the function itself)

I am having troubles computing the derivative of $\nabla_{\theta}f(x, \theta)f(x, \theta)$ (the gradient of the function $f(x, \theta)$ times the function itself) that is \begin{align} D(\nabla_{\...
18 views

### While deriving Least Squares Estimators, how to find the derivate of a summation operate?

I'm calculating the Least Squares Estimators. There was one step here: $\frac{d}{d\hat\alpha}{\sum(y_i-\hat\alpha-\hat\beta x_i)}^2=0$ --> $-2{\sum(y_i-\hat\alpha-\hat\beta x_i)}=0$ I know it is ...
• 21
1 vote
241 views

### Deriving the PDF of the kth order statistic from the CDF

I am trying to understand how to get from the CDF to the PDF of the kth order statistic and I am following this article. I understand that I have to take the derivative of F to get f. I also ...
• 352
1 vote
57 views

• 111
1 vote
48 views

### Given a mean, what is the range of variance values that make for possible Beta distribution parameters

The beta distribution can have its parameter estimated via method of moments, which I will be doing. \hat\alpha = \bigg(\dfrac{\bar x (1-\bar x)}{var(X)} - 1\bigg)\bar{x}\\ \hat\beta = \bigg(\dfrac{\...
• 35.7k