Vector pointing in the direction where a function is growing fastest; its components are partial derivatives of this function. For questions about gradients in ecology, please use the [ecology] tag instead.

229 questions
Filter by
Sorted by
Tagged with
15 views

• 1
47 views

• 731
1 vote
45 views

### Is there anyway to calculate the integral of a trace? [closed]

I would like to calculate the integral of a scalar function as follows: $$f(x)=\mathrm{tr}((\mathbf{A}x+\mathbf{B})^{-1}\mathbf{B}),$$ where $\mathbf{A}$ and $\mathbf{B}$ are two $n\times n$ positive-...
• 21
1 vote
37 views

### How to merge 2 losses in a reasonable ratio

my question is pretty basic, but I can't find many resources online about this. Say we have two losses for a model, for example a pix2pix GAN, which for those who are not familiar with it, for the ...
• 1,207
1 vote
91 views

### Making sense of the Gain term in Gradient tree boosting

In the XGBoost Documentation they specify the Gain term as Gain=\frac{1}{2} \left[ \frac{G_L^2}{H_L+\lambda} + \frac{G_R^2}{H_R+\lambda}- \frac{(G_L+G_R)^2}{H_L+H_R+\lambda}\right]-\...
• 125
269 views

### Why do we regularize large gradients corresponding to large errors?

While reviewing some scientific blogs, I found them recommending using gradient clipping for large error gradients. However, intuitively one would think that when model predictions are completely off, ...
1 vote
112 views

### Lowering the weight of particular features in a neural network?

Given sample data $x$, we hypothesize that some features (i.e. dimensions) of $x$ will generalize well, while others will generalize poorly. For example, when predicting medical diagnosis, age and ...
• 121
376 views

### Need for reparameterization trick in RL (and others)?

This is a multi-fold question that has a number of closely related questions; that is why I will pose them all here, instead of separate questions. In RL you have a parameterized policy that dictates ...
• 145
1 vote
9 views

### Is backprop computed for every element in the minibatch and then averaged for every weight?

I'm trying to fully understand backpropagation by computing it by hand. Often is cited that is just an alternation of the derivative of the preactivation and the derivative of the activation, however, ...
• 1,207
1k views

### Can Poisson deviance be used to evaluate models that use loss functions other than Poisson? (Such as MSE)

I am currently doing a a study on emergency department utilization rates at various geography levels. Especially of interest, are tree-based approaches to this analysis - namely random forest and GBMs....
• 143
1 vote
230 views

### How does the full derivative of softmax + cross entropy have the correct dimensions?

The blog post the softmax function and its derivative explains the following: Imagine that each input has $N$ features / pixels / etc. Imagine each input can be classified into $C$ classes Let the ...
• 349
1 vote
128 views

### Explanation of the derivation of the analytical gradient for a SVM?

I'm trying to understand how to derive the analytical gradient for a SVM. I know that in a SVM, the loss function is defined as follows: From this blogpost, I know the full loss for each element in ...
• 349
369 views

### How does Generalized Random Forest calculate the gradient of the score function?

The reference is GENERALIZED RANDOM FORESTS by ATHEY, TIBSHIRANI and WAGER (2019). They construct a general algorithm to grow trees and forest for estimation of target parameters that are conditional ...
552 views

### Why Reparameterization Trick does not work with discrete latent variables?

I came to know from the Youtube Video here (Timestamp 1:03:55) that Reparameterization trick only works for continuous latent variable. But, I am not clear as to why it does not work for discrete ...
• 421
113 views

### Likelihood-ratio gradient estimator in linear dynamical system in python (Jax)

TL;DR I am trying to implement the likelihood-ratio gradient estimator in a linear dynamical system (LDS) with Gaussian transition noise and Gaussian observation noise I am currently using python and ...
• 21
1 vote