Questions tagged [stochastic-gradient-descent]

Filter by
Sorted by
Tagged with
0
votes
1answer
860 views

How do I choose the initial features vectors for a Stochastic Gradient Descent trained SVD++ algorithm?

I'm reading the SVD++ Netflix Recommender Systems paper because I want to be able to properly assess this approach to building a recommender system. How should I choose the initial values of $q_i$ ...
1
vote
0answers
278 views

Repeated training examples in Gradient Descent [duplicate]

I am new to machine learning and trying to understand stochastic gradient descent. I understand in stochastic gradient descent, in each epoch, randomly an example is picked and given to the model. So ...
1
vote
1answer
2k views

Does memory ever really matter for mini-batch size selection?

I'm new to machine learning, and am confused about some aspects of stochastic gradient decent. I've read in several places that, when using vectorized code, the reason that mini-batching in ...
8
votes
2answers
3k views

Dealing with small batch size in SGD training

I am trying to train a large model (deep net using caffe) using stochastic gradient descent (SGD). The problem is I am constraint by my GPU memory capacity and thus cannot process large mini-batches ...
7
votes
2answers
4k views

comparison of SGD and ALS in collaborative filtering

Matrix factorization is widely applied in collaborative filtering, and briefly speaking, it tries to learn the following parameters: $$\min_{q_u,p_i}\sum_{\{u,i\}}(r_{ui} - q_u^Tp_i)^2$$ And we could ...
4
votes
1answer
694 views

SGD for Cox proportional hazards model

I am wondering if there's any implementation or theory about using stochastic gradient descent for estimation parameters in Cox proportional hazards model. Normally in that model the partial-log-...
2
votes
0answers
376 views

How to get SGD to reach global optimal point in logistic regression?

I am trying to write a tool which involves implementing logistic regression. With the batch gradient descent method, the convergence is guaranteed as it is a convex problem. However, I find that with ...
134
votes
4answers
131k views

Batch gradient descent versus stochastic gradient descent

Suppose we have some training set $(x_{(i)}, y_{(i)})$ for $i = 1, \dots, m$. Also suppose we run some type of supervised learning algorithm on the training set. Hypotheses are represented as $h_{\...
4
votes
3answers
3k views

Issues with stochastic gradient descent

I am using stochastic gradient descent to learn a model. Here is the plot of the objective function for the iterations. I am trying to maximize the function value. Taking the average of 500 ...

1 2 3
4