Skip to main content

Questions tagged [derivative]

For on-topic questions involving the mathematical concept of a derivative, i.e. $\frac{d}{dx} f(x)$. For purely mathematical questions about the derivative it is better to ask on math SE https://math.stackexchange.com/

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

Information coefficient as loss function of XGBoost

I am trying to train an XGBoost regressor for stock price prediction. I want to customize the objective function to be Information Coefficient (IC). The definition of IC is the Pearson correlation ...
atlantic0cean's user avatar
1 vote
1 answer
27 views

How can I prove monotonicity of slope MLE in EIV regression model?

I'm trying to figure out Casella and Berger Exercise 12.4(c), regarding monotonicity of the maximum likelihood estimator of the slope of an errors-in-variables regression model. The goal is to show ...
narwahl's user avatar
  • 11
0 votes
0 answers
43 views

Derivative of the multivariate normal cumulative distribution function (CDF) with reparameterisation [duplicate]

I would like to learn how to calculate the derivatives of a multivariate normal cumulative distribution function (MVN CDF) w.r.t. certain elements by using the derivatives of the same MVN CDF w.r.t. ...
Kirin G.'s user avatar
0 votes
0 answers
23 views

How to determine statistical significance for a time series and forecasts?

With a simple example of mortality rates, and a basic three-year mean baseline: ...
electronix384128's user avatar
0 votes
0 answers
10 views

Decomposing model volatility with respect to factor contributions

Consider a linear model $\textbf{y} = \textbf{x}\pmb{\beta} + \pmb{\varepsilon}$ with $\textbf{y}$ a $T \times 1$ vector of random variables, $\pmb{\beta}$ a $K \times 1$ vector and $\textbf{x}$ a $T \...
user9875321__'s user avatar
0 votes
0 answers
4 views

Custom Model For Approximating Sin Function Using Backpropagation [duplicate]

I have very simple custom model which I am doing experiment with, I have model which takes one input and produce one output. the model equation is: y = sin(ax + b). (a) and (b) are single learnable ...
mohammad's user avatar
0 votes
0 answers
20 views

Optimizing parameters for a non-standard probability density function

We have a non-standard multivariate probability density function, P(x | q), where x is a vector, and q are the parameters of the density. I get events ...
Niteya Shah's user avatar
0 votes
0 answers
16 views

Derivatives in Computation graph

In Advanced Learning Algorithms by Andrew Ng, Coursera: One thing that makes backprop efficient is you notice that when we do the right-to-left calculation, we had to compute this term, the ...
derivative's user avatar
3 votes
1 answer
46 views

Smoothness of a neural network (specifically second-order)

If we use ReLU activations, then the function which our neural network represents is piecewise linear. It is not smooth and the first derivative doesn't exist everywhere. However, if we use sigmoid or ...
Blahblahblacksheep's user avatar
0 votes
0 answers
21 views

Optimal Conditional Distribution for Minimising Information-Theoretic Expression

Consider two countable sets $\mathcal{X}$ and $\mathcal{Y}$. I aim to find the conditional distribution $P_{Y|X}$ that minimizes the following expression for any $x \in \mathcal{X}$ $$\sum_y P_{Y|X}(y|...
pmoi's user avatar
  • 1
1 vote
0 answers
27 views

Independence of 2D gaussian process derivatives

Suppose I have a gaussian process which takes 2D inputs x and y and gives a 1D output z. I understand based on Calculating the expression for the derivative of a Gaussian process that each of the ...
MymanPJ's user avatar
  • 11
5 votes
1 answer
116 views

Order of derivative for MGCV splines and low EDF

I'm following up on the informative discussion here concerning choice of m (order of derivative) for MGCV splines. Using the default options in MGCV (thin plate spline, REML for optimization, k=10, ...
dean's user avatar
  • 455
0 votes
0 answers
37 views

How does the chain-rule look for the gradient of a loss function?

When we are computing the gradient of the loss function, $L$, of a Word2Vec model, for the context word-embedding, $w_i$, and the target word-embedding, $t$. Where the loss function, $L$, looks like: $...
ZenPyro's user avatar
0 votes
0 answers
28 views

Partial derivative of interaction term with endogenous variable

I am trying to estimate the price elasticity based on a 2sls regression. The price in logarithmic form is treated as an endogenous variable. The price appears in an interaction term with income $\...
octopus161's user avatar
0 votes
1 answer
76 views

In a linear model, why do we have $-2X^T \vec{y} + 2X^T X \vec{\beta}=0$? [duplicate]

When we derive the estimates of $\vec{\beta}$ such that they minimize the sum of squared error ($SSE$) we begin with $\sum_{i=1}^{n} (y_i - (\beta_0 + \beta_1x_1 + ... + \beta_kx_k))^2$. This is ...
AdmiralMunson's user avatar
1 vote
0 answers
46 views

Backpropagation in LSTM network [closed]

as we have Vanishing Gradient in Vanilla RNN and LSTM is the solution , according to some sources LSTM has Vanishing Gradient too but it doesnt cause any problem in the context of LSTM network cause ...
Kasra's user avatar
  • 11
2 votes
1 answer
243 views

2nd derivative of spline

This question has some similarities to a previous related question. How can I fit a spline to data that contains values and 1st/2nd derivatives? I went through the suggestions mentioned in that post, ...
Sundown Brownbear's user avatar
0 votes
0 answers
22 views

Neural network learns to mimic distribution of classes in dataset instead of using signal from input

I'm trying to implement example from a classic AI paper named "Learning representations by back-propagating errors" by Hinton et al. Example aims at training network able to predict third ...
Jan Grzybek's user avatar
1 vote
1 answer
28 views

Why does the score test work for values longer in the tail that have a small log-likelihood derivative?

The score test says that we take the derivative of the log-likelihood at $H_0$ and divide it by the fisher information at $H_0$. $U(\theta )={\frac {\partial \log L(\theta \mid x)}{\partial \theta }}.$...
Estimate the estimators's user avatar
6 votes
0 answers
98 views

Derivative-based effect size for Gaussian GAMs

It is often the realistic advice I have seen here that Gaussian GAMs are not regressions with which you can easily approximate an effect size for, as the effects are of course non-linear. However, we ...
Shawn Hemelstrand's user avatar
0 votes
0 answers
121 views

Understanding relation between axis of least and maximum second moment

I was going through computer vision lecture video. You can find the pdf of this lecture here. I was trying to understand how orientation of object corresponds to axis of least second moment aka ...
Mahesha999's user avatar
0 votes
0 answers
173 views

Is the equation of the derivative of a loss function relative to the input to the Sigmoid (z) the same whether computed backward or forward?

I am referring to the derivative of the binary cross-entropy loss function for logistic regression. Using back-propagation, the derivative of the loss function L ...
Joachim Rives's user avatar
1 vote
0 answers
31 views

Monotonicity of softmax (considering updates from all variables)

There's a relevant question here that doesn't quite answer my question, but I'm unable to comment. Define softmax to be $$a_i = \text{softmax}(u_i)= \frac{e^{u_i}}{\sum_j{e^{u_j}}}$$ As the linked ...
gcpx100's user avatar
  • 11
1 vote
0 answers
181 views

Understanding Backpropagation with Softmax and Quadratic Error

I'm trying to understand how to compute the derivative of the Softmax activation function in order to compute the gradient of the quadratic cost function w.r.t. the weights of the last layer. ...
Dario Ranieri's user avatar
0 votes
0 answers
33 views

Problem for a math formula in Weight Uncertainty in Neural Network

I am studying the paper "Weight Uncertainty in Neural Networks" by Blundell et al (2015, on arXiv), and there is a formula I don't get page 4, namely formula (3) in step 5: I don't ...
jacob89's user avatar
4 votes
1 answer
234 views

Rewriting the expectation of f(x) by means of its derivative

I have a question regarding this proposition. Let $f: \mathbb{R} \rightarrow \mathbb{R}$ be an a.e. differentiable function so that $\int \frac{\left|f^{\prime}(x)\right|}{(1+|x|)^s} d x<\infty$ ($...
Eryna's user avatar
  • 319
1 vote
2 answers
127 views

First derivative of multivariate normal density with exchangeable correlation structure

As part of a proof, I need to take the first derivative of the log of the following multivariate normal density: $(2\pi)^{-k/2} |\Sigma|^{-1/2} \exp\left(\frac{-1}{2} x'\Sigma^{-1}x\right)$. In this ...
bob's user avatar
  • 725
0 votes
0 answers
35 views

Gradient vs differences to remove non-stationarity in time series?

When dealing with non-stationary time series (for instance, in auto-correlation analysis), differencing (computing absolute differences between consecutive samples/observations) is often regarded as ...
joaocandre's user avatar
1 vote
0 answers
30 views

Elasticity estimates for zero-truncated negative binomial part in the hurdle model

I estimated a hurdle negative binomial regression model with zero-truncated negative binomial model as the count component in R using the pscl package. I wish to present elasticities for the count ...
Subid's user avatar
  • 11
0 votes
1 answer
40 views

Useful value for rate of change

I want to create a simple representation of "rate of change" for a number of different metrics, which aren't really comparable between each-other. To illustrate my problem, let's say I have ...
JHH's user avatar
  • 121
1 vote
0 answers
39 views

Propagation of uncertainty in a derivative of a function [duplicate]

I've performed an ordinary least squares on a data set with one variable. For simplicity, let's say I've fitted a polynomial function $$f(x)=a+bx+cx^2+dx^3.$$ I obtain the best fit and the standard ...
Bert's user avatar
  • 11
2 votes
2 answers
188 views

Probability Generating Function for The Difference of Two Binomially Distributed Random Variables?

Suppose I have 2 random variables: $X\sim \textrm{Bin}(m,p_1)$ and $Y\sim \textrm{Bin}(n,p_2).$ I want to find the distribution of $S=X-Y$ using the probability generating function ($PGF$) treating $S$...
Wind Fish's user avatar
1 vote
0 answers
158 views

Taking derivative of a function containing random variable wrt the variance of that variable [closed]

Say, I have a function containing a random variable such as $ f(X)$, where $X $ is the random variable that comes from a family of random variables that differ only in the first and second moments (e....
user383555's user avatar
2 votes
1 answer
145 views

When are $f(X)$ and $f'(X)$ independent?

I want to know the effect of differentiation on the independence of random variables. For a random variable $X$, when are $f^{(n)}(X)$ and $f^{(n+k)}(X)$ independent?, $\forall n\geq0\;, k\geq 1$.
new's user avatar
  • 21
2 votes
1 answer
139 views

How to express backpropagation dE/dV using matrix

I'm new in NN and my math is not that good. I try to do manual calculation using NN model. I already know and try to calculate the feedforward and backward one by one using the formula. but when I try ...
user382927's user avatar
1 vote
1 answer
93 views

The confusing derivation in the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy

In the section 15.5 of the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy, it discusses the Gaussian Process Latent Variable Model. The log-likelihood objective function is ...
dlheirmit's user avatar
0 votes
1 answer
55 views

Derivative conditional moments binary random variable

Let $e$ be a continuously distributed RV with pdf $f$ and let $q( x )$ be a binary RV that depends on the former through the relation $q ( x ) = 1[h( x ) \geq e ]$, where $h$ is a well-behaved ...
Seb's user avatar
  • 70
1 vote
1 answer
74 views

What is the derivative of a set or a string? [closed]

Neural networks operate on numbers, and it's well-known what the derivative of numeric functions are, as well as what the derivative of matrix functions are. What about functions that operate on maps ...
Wilhelm's user avatar
  • 111
0 votes
1 answer
256 views

Is the Inverse Mills Ratio Strictly Decreasing?

As far as I know, the Inverse Mills ratio, $\lambda(x)=\phi(x)/\Phi(x)$, is decreasing in $x$. Thus, I am curious now whether $\lambda(x)$ is in fact strictly decreasing in $x$. To see this, I derived ...
MinChul Park's user avatar
1 vote
1 answer
119 views

Derivative of an integral of a random normal variable

Let $x$ be a random normal variable with pdf $h(x)$ and CDF $H(x)$. Also let $\alpha$ be a constant, and $x^\star$ a variable. I am trying to take the following derivative: $$\frac{d}{d x^\star} \bigg ...
phdstudent's user avatar
0 votes
0 answers
36 views

Is it possible to find an explicit equation for this maximum likelihood for a particular variable of mean function

I have a log-likelihood equation that involves multivariate normal. Let's say, $le = \sum_{i=1}^n logf(y_i)$ and $f(y)=(2\pi)^{-\dfrac{n}{2}}|\sigma^2I_n|^{-\dfrac{1}{2}}exp[-\dfrac{1}{2}(y-x(t))^T|\...
user378967's user avatar
0 votes
0 answers
129 views

derivatives and distribution of a 3-dimensional copula in R

I am looking for a way to calculate in the R software, the distribution, the density and the derivatives (of order 1, 2) partial of a Gaussian copula of dimension 3. Indeed, I have three variables (u1,...
Sessi's user avatar
  • 1
0 votes
1 answer
154 views

How did we derive the least square estimator using OLS?

How does multiplying a matrix with its transpose equal "minimizing" it? When calculating the partial derivative, where does the X' come from? Why setting the value of third equation to 0 is ...
Shamim's user avatar
  • 1
2 votes
1 answer
372 views

Derivative error with respect to bias in binary cross entropy

I will do research using NN with 1 hidden layer. To calculate loss using binary cross entropy and for the activation function using sigmoid. I found the derivative formula from Sadowski, 2016 (link: ...
Andryan's user avatar
  • 47
2 votes
1 answer
147 views

What happens if we follow the gradient of a softmax activation

Given a softmax output layer, what does it mean to "follow the gradient"? Usually that would consist in "increasing the output" but obviously the softmax has no notion of "...
Alberto's user avatar
  • 1,217
0 votes
0 answers
119 views

four-point forward-difference formula using Newton's form for first order derivative [closed]

We know that ${f'(x) \approx \frac{f(x+h)- f(x)}{h}}$. If we have three points ${x_0 = x-h}$, ${x_1 = x}$, ${x_2 = x + h}$, we can compute the 3-point centered-difference formula using the Newton's ...
Ele975's user avatar
  • 217
0 votes
0 answers
27 views

Derivative of multivariate normal cdf with respect to it’s arguments [duplicate]

I'm using a result from the dissertation of Poddar(2016, link) and he states the following in his appendix A1: We will use the well known property, stated here for completeness, of the multivariate ...
Kurt Z.'s user avatar
  • 65
1 vote
0 answers
73 views

Please help me to understand the Taylor’s theorem when transiting from Gradient Boost to XGboost

I am reading this article, which explains how the algorithm replaces the actual loss function with so-called 2nd order Taylor expansion. I can understand til Step 4, and can't understand step 5. I ...
yts61's user avatar
  • 109
1 vote
1 answer
27 views

A question on computational complexity of a numerical differentiation (equation (5.77)) in Bishop's Pattern Recognition and Machine Learning

In page 249 of Christopher M. Bishop's book "Pattern Recognition and Machine Learning", it is said Again, the implementation of such algorithms can be checked by using numerical ...
zzzhhh's user avatar
  • 333
0 votes
0 answers
60 views

Derivation of (5.76) in "Pattern Recognition and Machine Learning"

The book "Pattern Recognition and Machine Learning" by Christopher M. Bishop says in page 248 ... for softmax outputs we have: $$\frac{\partial y_k}{\partial a_l}=\delta_{kl}y_k-y_ky_l.\tag{...
zzzhhh's user avatar
  • 333

1
2 3 4 5
7