Recently, I work on a linear regression model of my project. I have 200 samples, each of which has only one feature, to train my model. When I try to apply
gradient descent algorithm I cannot reach optimum parameter value even if I choose small step rate, which is equal to 0.001.
When I choose this step value, computed parameter values always cross the other side. I tried to plot how gradient descent algorithm behaves like.
\ / w1<- \*<---*/ -> w0 \ / \__/
I thought the reason a little bit, and I concluded that main reason is the fact that feature values are very big. Feature values and ground truth labels are around 7000-8000.
Is there a solution to such a problem ? The feature values are from
cm^3 unit and ground truth labels are from
gram unit. I think that I can convert them into
kg, but if I do that, I think that I may cause domain shift problem.