1
$\begingroup$

I am experimenting with stochastic gradient descent and observing very strange output.

In a toy problem, the $X\beta$ for stochastic gradient descent is always larger than $0$, which will be predicting $1$ all the time. But the shape is similar to glm.

enter image description here

What could be wrong?


My code:

library(sgd)
set.seed(0)
d=ggplot2::diamonds
d$price=ifelse(d$price>2500,1,0)
noise_idx=sample(nrow(d),nrow(d)/5)
d$price[noise_idx]= ifelse(d$price[noise_idx]==1,0,1)

glm.fit=glm(price~.-carat,d,family=binomial)

sgd.fit <- sgd(price ~ .-carat, data=d,
                 model="glm", model.control=binomial(link="logit"))

glm.p=model.matrix(price~.-carat,d) %*% glm.fit$coefficients
sgd.p=model.matrix(price~.-carat,d) %*% sgd.fit$coefficients

par(mfrow=c(1,2))
hist(glm.p,30)
hist(sgd.p,30)
$\endgroup$
1
  • $\begingroup$ Does the same thing happen with different seeds? What about the coefficients, are they also consistently offset? $\endgroup$ Commented May 31, 2017 at 14:47

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.