# Questions tagged [gradient-descent]

Gradient descent is a first-order iterative optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point. For stochastic gradient descent there is also the [sgd] tag.

844 questions
Filter by
Sorted by
Tagged with
1answer
21 views

0answers
22 views

### Does gradient decent happen during the back propagation of a layer or after back propagation is done for all layers?

I'm currently learning the back propagation algorithm for neural network and I need to clear some confusions: From what I understood, during the back propagation algorithm we would calculate the ...
0answers
7 views

### How do I test my recommender system? [duplicate]

I have created a recommender system based on collaborative filtering with gradient descent. I have completed the training. Now how do I test my recommender system for new users or new items? Here is ...
0answers
22 views

### What can cause a GNN to diverge?

I'm using A GIN (https://arxiv.org/abs/1810.00826) with a TopK pooling (https://arxiv.org/abs/1905.02850) and an Adam optimizer with some of my own data. Back-propagating works well as the loss ...
0answers
73 views

### Does the rate of convergence of optimizers matter in deep learning?

In classical optimization, an enormous amount of effort is taken to characterize the rate of convergence of optimization algorithms and designing fast gradient algorithms. You can find tables upon ...
1answer
93 views

### clarification on back-propagation calculations for a fully connected neural network

I am currently taking Andrew Ng's Deep Learning Course on coursera and I couldn't get my head around how actually back-propagation in calculated. Let's say my fully connected neural network looks like ...
0answers
69 views

### How to improve Levenberg-Marquardt's method for polynomial curve fitting?

Some weeks ago I started coding the Levenberg-Marquardt algorithm from scratch in Matlab. I'm interested in the polynomial fitting of the data but I haven't been able to achieve the level of accuracy ...
1answer
28 views

### automatic diffentiation (autograd): when the explicit definition of the gradient function is needed?

In Pytorch and similar machine learning software, the Autograd module computes the gradient of a function without needing to explicit declare the derivative of each single function which composes the ...
1answer
36 views

1answer
44 views

### How does having different scales on features make an elliptical contour plot?

I have been taking Andrew Ng's Machine Learning course, and in the lesson on feature scaling's effect on gradient descent, I just can't understand how because of the different scales on the features ...
1answer
79 views

### Why am I not getting the correct output from my gradient descent algorithm? [closed]

I have started taking online ML classes, and i was introduced to the topic of Gradient Descent, the Prof, himself hadnt shown us himself how to implement it in a programming language, so for fun, i ...
0answers
70 views

### Normalized steepest descent with nuclear/frobenius norm

In steepest gradient descent, we try to find a local minima to a loss function $f(\cdot)$ by the rule: $x_{t} = x - \alpha \triangledown_{x}f(x)$. I've found in textbooks that often we want to ...
0answers
188 views

### How to set the tolerance in Gradient descent?

I understand that one solution of setting the number of iterations, is to set it to a large number and then interrupt it when the gradient vector becomes tiny, so tiny that it is smaller than a ...