Gradient descent is a first-order iterative optimization algorithm. To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient (or of the approximate gradient) of the function at the current point. For stochastic gradient descent there is also the [sgd] tag.

788 questions
Filter by
Sorted by
Tagged with
25 views

### How to choose initial $\theta$ (intercept and $\beta$s) in simple linear regression? [closed]

I have the sales of items from January 2013 to October 2015. I just want to predict the total sales for the next month. Just for the sake of learning, I would like to transform it into a multiple ...
26 views

### What does vanishing gradient problem exactly mean? [duplicate]

I'm currently doing some stuff with ML, and I created a LSTM model to recognize activities such as walking or running. I have read that LSTM has advantages over traditional RNNs due to addressing the ...
22 views

36 views

### Mini-Batch Gradient Descent - Why does sampling with replacement work?

When sampling the data, either one at a time (as in online learning), or in mini-batches, there exist gradient descent methods which sample with replacement and without replacement. For Mini-Batch ...
71 views

### What is wrong with my approach on a custom way of creating Gabor-filter convolution kernels?

Disclosure: I am not a prominent mathematician (current bachelor student) like others on this website and my approach has been mostly pragmatic. Please do tell me if I can improve the formulation of ...
### Proximal gradient descent for “projective” $l1$ term
Proximal gradient descent is employed when you want to find $\min f$ where $f(v)=g(v)+h(v)$, $g$ is smooth and $h$ is not smooth. When $h$ is a lasso term of the form $h(v)=\lambda ||v||_1$, the ...