Questions tagged [derivative]

For on-topic questions involving the mathematical concept of a derivative, i.e. $\frac{d}{dx} f(x)$. For purely mathematical questions about the derivative it is better to ask on math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
2
votes
1answer
21 views

Deriving Logit Maximum Likelihood Estimator

According to Verbeek, we can obtain the logit model by simplifying the first order condition of the log-likelihood function. Where,  $$logL(\beta) = \Sigma^N_{i=1} y_i logF(x^{'}_i\beta)+ \Sigma^N_{i=...
0
votes
1answer
19 views

Total differential of a linear model with interaction terms

Suppose I have the following function, representing a linear model with an interaction term: $$ f(x, y) = \beta_{1} x + \beta_{2} y + \beta_{3} xy. $$ Now I want to see how the function changes if ...
1
vote
1answer
36 views

Help with an application of Leibnitz's Rule

I'm trying hard to understand and solve the following: $$f_Y(y)=\frac{d}{dy}F_Y(y)=\frac{d}{dy}\int_{-\sqrt{y}}^{\sqrt{y}}{f_X(x)}dx=?$$ The background information is that $f_X(x)$ is the pdf of ...
0
votes
0answers
18 views

How to find derivative of log of determinant of a matrix? [migrated]

I have an equation of which i want to find second order derivative with respect to $A$. The equation is $$-N\text{ln}\pi+N_{d}\text{ln}(\text{det}(A))-\sum_{n=1}^{N_d}\textbf{y}_n^{H}A\textbf{y}_n.$$ ...
0
votes
0answers
26 views

How to get this bound?

I read the following part in a paper, it is trying to show that the difference between $g(x,\gamma)$ and its linearized version is small. Here $g(z,\gamma)$ depends on two generic functions $\gamma=(\...
0
votes
0answers
17 views

Vectorized backpropagation for Neural network?

I understand back-propagation for scalars, but vectorized examples are bit tricky. Here, the image above shows a specific computational graph example. We are calculating gradients with respect to ...
0
votes
0answers
35 views

Can you take derivatives of Confidence Intervals?

I have how defect density in a crystal change over time. 6 data points I want to regress this with a 4th order polynomial. I can do this and obtain 95% non-simultaneous, functional prediction ...
0
votes
0answers
21 views

How to interpret this kernel regression results?

I tried to do some kernel estimate of second order derivative. In my experiment, $y_i=z_{1i}+z_{1i}z_{2i}^3+e_i$, where $E(y_i|z_{1i},z_{2i})=z_{1i}+z_{1i}z_{2i}^3$. I'm interested in estimating $\...
1
vote
1answer
24 views

ML for Bernoulli trials

When reading up this page, I couldn't follow how the log of $L(p;x)$ for Bernoulli trials would be maximised at $\hat{p}$ = $\sum_{i=1}^nx_i/n$. Could you please explain the steps, particularly the ...
0
votes
0answers
7 views

How to find robustness of a matrix with respect to another matrix

I have a matrix $D_{s}$ that has been computed using another matrix $X_{s}$ by solving an optimization problem (in the optimization problem, $X_{s}$ was a data matrix and $D_{s}$ was estimated by the ...
0
votes
0answers
22 views

How can I use transformation properties to obtain the distribution of $h(\mathbf{s})$?

Let that $\mathbf{s}=(s_1,s_2) \sim Unif(S)$, where $S$ is some spatial area. Suppose $y=h(\mathbf{s})=1-[exp(exp(\beta_0+\beta_1(\mathbf{s}-\mathbf{x})^T(\mathbf{s}-\mathbf{x})))]^{-1}$. We have that ...
2
votes
1answer
45 views

What's wrong in this derivation of back-propagation errors?

I'm trying to find a rigorous derivation for the backpropagation algorithm, and I've gotten myself into something of a confusion. The confusion comes from when and why people transpose the weight ...
0
votes
0answers
59 views

Derivative of ReLu at 0 in practice

I used ReLu for hidden layer for the simple neural net XOR problem. My first attempts failed, because as derivative I used: ...
0
votes
0answers
19 views

How to estimate the right derivative of a function that is smooth except for a finite number of kinks?

Suppose $Y_i=g(X_i)+e_i$, where $g(\cdot)$ is a function unknown to the researcher, and $E(e_i|X_i)=0$. Suppose $X_i$ is a random variable in $[-1,2]$ with a density that is everywhere positive, and ...
1
vote
1answer
23 views

What happens to kernel regression (Nadaraya–Watson estimator) at a kink point?

Suppose $Y_i=g(X_i)+e_i$, where $g(\cdot)$ is a function unknown to the researcher, and $E(e_i|X_i)=0$. Suppose $X_i$ is a random variable in $[-1,1]$ with a density that is everywhere positive, and ...
1
vote
0answers
18 views

Matrix form of elementwise derivations

The elementwise derivations w.r.t e of $$ J = \frac{1}{2}[\Sigma_{r,s=1}^{R}a_{rt}a_{st}k(e_r,e_s) - 2\Sigma_{r=1}^{R}a_{rt}k(e_r, x_t)]$$ can be given by: $$ \frac{\partial J}{\partial e_r} = \Sigma_{...
2
votes
1answer
54 views

Covariance of derivative of Gaussian Process Regression

There are a quite a few questions and answers which discuss how to calculate the gradients/derivatives of the posterior of Gaussian Process Regression (see here, here). These include the equations for ...
1
vote
1answer
31 views

Derivative of a Function $ln(1+\exp(-y.w^T.\phi_W(x_i) ))$

I want to take derivative of $ln(1+\exp(-y.w^T.\phi_{W}(x_i) ))$ with respect to $w$. So far what i have done is Let $u=1+\exp(-y.w^T.\phi_{W}(x_i) )$, The above expression will become $ln(u)$ $\frac{...
0
votes
0answers
39 views

What are the derivatives of Squared Exponential kernel function w.r.t. characteristic length scale (Gauss Process)

I'm writing a matlab code to implement Gaussian process. In the book: Gaussian Process for machine learning by Carl Edward Rasmussen and Christopher K. I. Williams, the authors define the squared ...
0
votes
0answers
17 views

Derivation of matrix derivative [duplicate]

In the ML lecture notes of cs229, I am not able to derive the equation 3 in matrix derivatives. I applied the equation 1 to derive it, considering the product after A as another matrix, I am not ...
0
votes
0answers
45 views

Is my step by step derivation of quadratic cost function's (Neural Networks) partial derivative with respect to some weights matrix correct?

I am trying to revise the details of a Multi-layer Perceptron with a set of weight matrices $\mathcal W$ and a set of bias vectors $\mathbf b$. Here is the quadratic cost function I am using, $$C(\...
0
votes
1answer
29 views

Derivative of all the parameters in Logistic Regression

$\mathcal{L}$ is the loss function, $\mathcal{L} = y_i \text{log} \sigma(z) + (1-y_i) \text{log} (1-\sigma(z))$, where $z = \sum_i w_ix_i$, with $w_i$ representing the weights and $x_i$ the features. ...
1
vote
1answer
32 views

n'th cumulant (of a CGF) for exponential family / exponential dispersion model

The n'th cumulant is defined to be the n'th derivative of the CGF (cumulant generating function). $$\kappa_n = \frac{d^n K(t)}{dt^n} |_{t=0} $$ But I'm reading in a book (p.215, chapter5, eq. 5.8) ...
1
vote
0answers
48 views

Posterior distribution for a Gaussian Process with a transformation in a gaussian likelihood

Suppose we are modelling observations y as follows. Our likelihood is normal $ y \sim \mathcal{N}(g(f(x)), \mathcal{I}\sigma^2)$, where $\mathcal{I}$ is the identity matrix and $g$ is some function ...
0
votes
0answers
21 views

Relation between dyi/dei and its leverage (hii)

Althought I have tried in different ways, I have not been able to show that $$\frac{\mathrm{d}Y_i}{\mathrm{d}e_i} = \frac{1}{1 - l_{ii}}$$ $e_i$ is equal to $l_{ii}$ is the $i-th$ element of the ...
1
vote
0answers
21 views

Is there a statistical test to determine correlation between the first derivative of two time based series?

I have two time series. They are normally distributed, parametric, exactly the same size and evenly spaced. I am seeking to determine if the rate of change of these two time series are correlated ...
3
votes
0answers
15 views

Meaning of a notation regarding mean square derivative

I'm reading a paper (On Differentiable Functionals, Van der Vaart, 1991, Annals of Statistics), and I've got a question regarding a notation in the following part: My Question: Does $dP^{1/2}$ mean $...
9
votes
4answers
224 views

Where does the logistic function come from?

I first learned the logistic function in machine learning course, where it is just a function that map a real number to 0 to 1. We can use calculus to get its derivative and use the derivative for ...
1
vote
1answer
50 views

Partial derivative of a linear regression with correlated predictors

Let's set up the situation of having some $Y$ that I think depends on a linear combination of $X_1$ and $X_2$. I could fit a regression model: $$y_i = \beta_0 + \beta_1x_{i1} + \beta_2x_{i2}$$ We ...
4
votes
1answer
43 views

Normal distribution “stable” under derivative?

Suppose that $\theta(t)\sim\mathcal N(\mu(t),\Sigma(t))$ where $t$ is some parameter. Then it holds that $$\theta(t) = \mu(t) + \Sigma(t)^{0.5}\xi$$ for $\xi\sim\mathcal N(0, I)$. I am interested in ...
0
votes
1answer
52 views

What do I miss in this derivation?

The school is closed due to the ongoing pandemic. And I am interested in the application of the Bayes Theorem in COVID-19. Here is what I thought. The total population in U.S. is approximately 327,...
1
vote
0answers
13 views

error propagation for derivatives

I have the following problem: I have some data of a function f(x) with a set of 300 values of it associated to the same number of values of x including corresponding standard deviation σ(f) for each ...
0
votes
0answers
44 views

Finding Derivative of a spline

I have received data values for a spline (which was already fit to some ndvi data). I just have only the data points of the spline and do not know the function that the spline follows. My goal is to ...
0
votes
1answer
37 views

How to derive the gradient of the reparameterized score function estimator?

In the paper Evolution Strategies as a Scalable Alternative to Reinforcement Learning, the authors derive the following gradient of the score function estimator $$ \begin{align} \nabla_\psi\mathbb E_{...
1
vote
0answers
11 views

rate of convergence for cross derivative estimation in local linear regression

Suppose $Y_{i}=m(X_{1i},X_{2i})+\epsilon_{i}$, with $E(Y_{i}|X_{1i},X_{2i})=m(X_{1i},X_{2i})$ where $m(\cdot,\cdot)$ is an unknown smooth function. If the estimator $\widehat{m}(x_{1},x_{2})$ is ...
0
votes
0answers
64 views

Finding the gradient $\nabla$ of the logistic regression cost function

I want to use vector calculus to derive the gradient $\nabla_wJ(w)$ of the logistic regression cost function $J(w) = -\textbf{y}\cdot ln\textbf{ s} - (\mathbf{1} - \textbf{y}) \cdot ln( \mathbf{1} - \...
3
votes
2answers
110 views

When using OLS on $\ln(y) = \beta_1 \ln(x) + \epsilon$, is $\beta_1$ the elasticity of $E[y\vert x]$, or the $y$ in the data (or both)?

Specifically, suppose we are estimating $$ \ln(y)=\beta_1\ln(x) + \epsilon $$ I understand that $\beta_1 = \frac{\partial \ln(y)}{\partial \ln(x)}$ which is the elasticity of $y$ with respect to $x$ ...
0
votes
0answers
52 views

Back-propagation through cross entropy or logistic loss function

I have neural network which ends with softmax function and I want to minimize cross-entropy cost function which takes output of this network and one-hot labels as arguments. To calculate partial ...
1
vote
1answer
59 views

Expression for Derivative of Hyperparameter of Kernel with respect to New Data

I would like to determine how the hyperparameter will change when a new data is observed and the GP is updated with this new data. Considering the following predictive distribution of the GP: $$\mu(...
0
votes
0answers
16 views

Computing custom gradient for LSTM equations [closed]

Consider an LSTM that takes in as input a sequence of N words $X_1,\cdots,X_N$. Each word is a vector $\in R^D$. The dimension of the LSTM neuron is $H$. Suppose we are doing sentiment classification ...
0
votes
1answer
71 views

Derivation of gradient-bandit algorithm, Why is the sum of the derivatives is zero?

https://www.cs.mcgill.ca/~dprecup/courses/RL/Lectures/2-bandits-2019.pdf In above pdf document, page 19, they explain by formula: $$\sum _{ b }^{ }{ \frac { \nabla { \Pi }_{ t }(b) }{ \nabla { H }_{ ...
0
votes
0answers
106 views

Gradient of multivariate normal distribution function?

Let $X\sim\mathcal{N}_J(\mu,\Sigma)$ be a multivariate normal with PDF $f_X$ and CDF $F_X$. Taking derivatives of $f_X$ wrt $X$, $\mu$ and $\Sigma$ is easy as shown here. However, I am interested in ...
1
vote
0answers
57 views

Relating Two Derivatives (and Elasticities) of a Log-Log Regression

Consider a standard "log-log" linear regression model like this: $\log(y_i) = \log(a_i + b_i)\delta + \epsilon_i$, where $y$ is the dependent variable, $a$ and $b$ are two independent variables, and ...
3
votes
1answer
314 views

Derivative of Gaussian Process (continued)

This is to extend the discussion of the derivative of the GP. The formulation provided in the previous post describes the gradient of GP as derivative of kernel function as follows with respect to $(x^...
0
votes
1answer
63 views

is it a good idea to take the derivative or integral of some features and add them as new features in machine learning?

I'm learning how to do feature Engineering and come across some ideas in my head that's why I want to ask if I had some dataset with some features let's say 2 features and I have a timestamp column ...
1
vote
1answer
73 views

Why does gradient descent HAVE to find the minimum as oppose to a change in the opposite direction

I have a question about the gradient descent step in neural networks. I fully understand the derivative step and taking the steps required to move in the direction that reduces the loss (finding the ...
3
votes
1answer
62 views

Derivation of score vector

Can anyone explain the process of this derivation, step by step? This derivation is from Joint Models for Longitudinal and Time-to Event Data by Dimitris Rizopoulos. \begin{equation} \begin{aligned} ...
0
votes
2answers
61 views

Source of vanishing/exploding gradients in RNN

Problem I am trying to understand the source of vanishing/exploding gradients in vanilla RNN. The update rule of vanilla RNN is $$ \begin{aligned} &\mathbf{a}^{\left<t\right>}=\...
0
votes
0answers
287 views

Deriving Gradient from negative log-likelihood function

I have been having some difficulty deriving a gradient of an equation. I have a Negative log likelihood function, from which i have to derive its gradient function. Negative log likelihood function ...
0
votes
0answers
30 views

Show that $\int^b_a\phi'''(z)dz$ lies between $\pm[\phi(0)+2\phi(\sqrt3)]$ for every $a<b$

Show that $\int^b_a\phi'''(z)dz$ lies between $\pm[\phi(0)+2\phi(\sqrt3)]$ for every $a<b$. $$\phi(z)=\frac{1}{\sqrt{2\pi}}e^{-\frac{1}{2}z^2}$$ I've shown that: $$\phi''(z)=(z^2-1)\phi(z)$$ $$\...

1
2 3 4 5