# Linked Questions

150 votes
6 answers
170k views

• 688
7 votes
1 answer
22k views

### Gradient Descent (GD) vs Stochastic Gradient Descent (SGD)

I know this question is redundant and has been answered here but I still want to understand it from my point of view to make sure if my terms are correct. My understanding of the difference between ...
• 489
5 votes
1 answer
4k views

### Why one epoch for stochastic gradient descent (SGD) is much better than one iteration for gradient decent (GD)?

Calculating gradient needs to sum over all the data points. So, SGD can be viewed as "using one data point to weakly approximate the gradient" to save time. Intuitively, I would think One epoch for ...
• 36.1k
4 votes
1 answer
3k views

### Stochastic Gradient Descent, Mini-Batch and Batch Gradient Descent

I was learning the optimization part in deep learning. Let's take linear regression as a simple example. Let $m$ be the total number of data points in the training set $(X,y)$ and $n$ is the number ...
• 6,819
0 votes
1 answer
3k views

### How to identify support vectors in SGD svm?

I am using SGD svm from scikit learn. I find that unlike SVC who has support_ as a member of the model to store the index of the support vectors, SGDClassifier only gives me the weight of the decision ...
1 vote
1 answer
218 views

### Performing Linear Regression using Stochastic Gradient Descent, by batches

I am presented with a data set, where I am supposed to perform linear regression on this using SGD. My first instinct would be to train each data point there is until I reach the last one. Only then ...
• 8,777
1 vote
1 answer
168 views

### Backpropagation not incremental

Why is backpropagation not incremental? The definition of incremental would be to be able to update the weights with every single new data point. However, in stochastic gradient descent with a ...
• 1,043
1 vote
2 answers
97 views

### Whats the horizontal and vertical axis denoting in the below SGD contour

What is that oscillation in the y axis in gradient descent contour.
1 vote
2 answers
67 views

### on-line regression with 1 output [closed]

I have 12 input variables from sensor (IMU) to predict 1 output (Speed of a boat) variable. Is it possible to use regression (or something else?) in this case where it is a continuous data stream from ...
• 29