# Questions tagged [gradient]

Vector pointing in the direction where a function is growing fastest; its components are partial derivatives of this function. For questions about gradients in ecology, please use the [ecology] tag instead.

116 questions
Filter by
Sorted by
Tagged with
1answer
26 views

### can we use any learners in gradient boosting instead of trees?

As we are simply trying to predict residuals from weak learners and aggregating them, can we use any weak learners in gradient boosting machines instead of trees ? If so, why are the all the gbm ...
0answers
9 views

### Why cache gradients for params between training examples?

I was going through Karpathy's guide here where he defines an simple multiplication gate's forward and backward passes like so ...
1answer
25 views

### svm loss function gradient

I was taking Stanford's cs231n class and was unable to understand the gradient calculated using the SVM loss function. You should go here to check the notes which I am talking about. This is the SVM ...
1answer
28 views

### Backpropagation gradient of the average

In the Pytorch Udacity course, the following is said at one point: To calculate the gradients, you need to run the .backward method on a Variable, ...
0answers
30 views

### What is state of the art in gradient free neural network learning, esp. for images?

I've been recently looking into gradient free learning of neural networks. However, most of the techniques I've found seem to be only applied to toy problems, which I assume means they're infeasible ...
0answers
71 views

### Multiclass hinge loss gradient

I am trying to compute the gradient of multi class hinge loss function but i am kinda confused. First things first, I have a W matrix [10xD] (10 classes) that contains the weights. The loss ...
1answer
40 views

### Interpreting gradient descent as a constrained optimization problem- Reinforcement learning

I' m studying the lectures of Sergey Levine in reinforcement learning, specifically the TRPO algorithm, during his explanation we claims that gradient descent is the same as doing this. He does ...
1answer
31 views

### Optimisation by using directional derivative

So I’ve seen the code of an R package where a two dimensional optimisation (actually MLE, finding the minimum of the negative log likelihood) is performed with the optim function and also two optimise ...
0answers
49 views

2answers
2k views

### Gradient descent on non-convex functions

What situations do we know of where gradient descent can be shown to converge (either to a critical point or to a local/global minima) for non-convex functions? For SGD on non-convex functions, one ...
2answers
259 views

3answers
1k views

### what is vanishing gradient?

I have seen the word "vanishing gradient" many times in deep learning literature. what is that? gradient respect to what variable? input variable or hidden units? Does that mean the gradient vector ...