# Logistic regression algorithm in Ruby

I have been using R to calculate logistic regression with many independent variables for a Ruby on Rails web application. However, I can no longer import data from the database to R using RPostgreSQL. The web host has stopped allowing insecure connections to the database. The point is, I either need to get a new web host, or write my own logistic regression algorithm in Ruby. Ruby probably isn't the best programming language for that kind of thing, but I don't really have a choice. Is there an easy to implement algorithm for multiple logistic regression?

• Some of the responses here might be informative: stackoverflow.com/questions/13673769 – smillig Apr 24 '13 at 9:59
• @smillig That's an interesting reference. Gradient descent seems like a poor strategy for solving the ML equations, though: it's just too easy for it to lock into a local maximum that is not a global one. – whuber Apr 24 '13 at 13:34
• I actually contacted that guy about his Ruby algorithm months ago. I don't remember what came of it, but I didn't come out with anything I could use. He had an interesting website though. – Eric Apr 24 '13 at 21:20
• @whuber the negative conditional log likelihood is convex, so every local maxima is a global maxima. See for example this short writeup. – alto Apr 25 '13 at 0:25
• @user603 I was mainly commenting on the global minima local minima. Thanks for the additional info though. I likely haven't encountered these issues as the only time I've rolled my own logistic/multinomial regression, I just used SGD and some appropriate learning rate schedule tweaking. – alto Apr 26 '13 at 0:13

At any rate, the underlying likelihood surface can be nearly flat so you have to be careful about the small prints of the implementation and test it on many corner cases (these are situations where the $X$ are highly correlated or when the two groups are nearly perfectly separable).
A possible (quick and dirty) alternative is to rescale all your $X$'s to be in $[0,1]$ --for example by using the inverse logit function on each of them individually (after they have been standardized first to have mean 0 and unit variance)-- and estimate a fit by OLS (this approach is called the linear probability model). It will not be the same model and the coefficients won't be comparable but the results will be better than doing OLS on the raw data. The advantage here is that implementing OLS is trivial, assuming you have access to a good ruby linear algebra library (googling around I have found quix/linalg)