0
$\begingroup$

I'm wondering if there is a good procedure people are using for gradient descent that is pretty well validated--something like a package for R or Python, or generic code many people adapt. After taking Andrew Ng's machine learning course in Coursera I was able to implement gradient descent in Octave, but I'm hoping to work with it in R or Python, with which I am more familiar. I'm also hoping for something more standardized than code I would write itself, to give it more credibility with my place of employment or other researchers.

Most of what I've found on this site is questions about writing code for gradient descent, like this post (https://stats.stackexchange.com/questions/115425/multiplicative-gradient-descent ) and this one (https://stats.stackexchange.com/questions/142257/procedure-for-gradient-descent). And I've found examples of code from other sites, like this http://spin.atomicobject.com/2014/06/24/gradient-descent-linear-regression/.

So I think there is no gradient descent package for R. But I was wondering if people had any thoughts for a good way to start implementing it in R or Python that doesn't rely on trust in the individual coder (e.g. me). Is there a blog post or tutorial with good generic code that can easily be used to standardize different gradient descent implementations?

$\endgroup$
3
  • 2
    $\begingroup$ Are you sure what you want is a gradient descent implementation? GD is often taught for pedagogical reasons as a general-purpose optimization routine, but other general-purpose optimizers are preferred over GD for reasons of numerical stability, precision, and not requiring any manual tuning. Some examples of these routines are implemented in R via optim. On CV.SE, @MarkLStone has made a hobby of trying to persuade people not to use GD; refer to his answers for more information. stats.stackexchange.com/users/78964/mark-l-stone $\endgroup$
    – Sycorax
    Aug 24, 2015 at 14:50
  • $\begingroup$ Since GD is indeed pedagogical, there's this course in which they implement it for python. I don't think it is polite if I just post their code, but you can access the course materials for free. GD is implemented in lesson 3/Problem set 3. $\endgroup$
    – lrnzcig
    Aug 25, 2015 at 10:11
  • $\begingroup$ thanks. I will check out @MarkLStone posts and optim, and think about alternatives to GD. $\endgroup$
    – PSR
    Aug 26, 2015 at 17:21

2 Answers 2

0
$\begingroup$

What about this blog? It seems to be based on the same training course. Plus, if you share the Octave code I might be able to translate it to R and/or Python for you.

$\endgroup$
1
  • $\begingroup$ thanks, I'll check that out. It seems like GD might not be the best way to proceed, so I may redirect my efforts. $\endgroup$
    – PSR
    Aug 26, 2015 at 18:07
0
$\begingroup$

I just wanted to update this. Scikit-learn in Python has a stochastic gradient descent module- http://scikit-learn.org/stable/modules/sgd.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.