2
$\begingroup$

I'm going to be using a logistic regression model and using SGD to determine the feature weights. Is it OK for me to use a mix of binary and real features, without doing anything like scaling or normalization, and just leave it to SGD to give me a model with weights that will work?

Thanks

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Binary features have a natural scale, so there is no need to rescale them. The real valued features need to be normalized by subtracting the mean and dividing by the range.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Is it necessary, though, in logistic regression? Wouldn't gradient descent find suitable coefficients regardless of whether I scaled the parameters or not? $\endgroup$
    – Walrus the Cat
    Aug 19, 2014 at 19:54
  • 1
    $\begingroup$ Rescaling has the effect of making the surface of the gradient of your error less elliptical, allowing you to tune the step-size for an epoch more sensibly. Cross-validate with and without normalization. If you don't have too much data (<500,000 observations), compare the results with bfgs. $\endgroup$
    – Jessica Collins
    Aug 20, 2014 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.