0
$\begingroup$

I am fitting a model where I estimate my Dependent Variable based on about 20 Binary Variables (0/1), and one continuous variable. I've read about the importance of normalizing that continuous variable, but have a question about the details.

I frequently see mean=0 and st dev=1 recommended as the distribution to normalize continuous variables to. And that makes sense to me if I'm normalizing a bunch of continuous variables to compare to each other. However...

Question 1: Given that this continuous variable will be a covariate alongside several binary covariates of value 0 or 1, should I normalize my continuous variable to fall entirely in the range (0,1)? Dividing it by it's maximum value achieves that, which makes sense to me as a "normalization" method. Which leads to...

Question 2: When I divide each value by the maximum value to get a range of (0,1), I have a skewed distribution with most "normalized" values falling in the range (0.5,0.8). Is that a problem? Should I transform/normalize the data further to achieve mean=0 and mean +/- three standard deviations to get to 1 / 0?

$\endgroup$
2
  • $\begingroup$ What sort of model are you using? Sometimes this sort of normalization is necessary, sometimes it helps with interpretation or comparisons, but often it is superfluous. $\endgroup$
    – dimitriy
    Apr 29, 2015 at 23:58
  • $\begingroup$ Good question. I have a logistic regression model, which I've read means normalizing variables is important. $\endgroup$
    – Max Power
    Apr 30, 2015 at 1:16

1 Answer 1

1
$\begingroup$

With plain vanilla logistic regression, you don't really need to normalize. There are also no assumptions made about the distribution of the covariates, only about the error, so you don't need to worry about covariate skew either.

Sometimes when there are enormous scale differences across the covariates, changing the units to something sensible will help with convergence.

$\endgroup$
3
  • $\begingroup$ Thanks for responding Dimitriy. Are you saying there's no need to normalize because I only have one continuous variable? Or that in "plain vanilla logistic regression" normalization is not important even for many continuous variables? If the latter, do you have a source, or a brief explanation of why? That seems to go against what I've been reading, but I trust you understand this better than I do. $\endgroup$
    – Max Power
    May 2, 2015 at 15:44
  • 1
    $\begingroup$ I have not seen this requirement mentioned in any statistics or econometrics books that I own, so I cannot quote any particular source. Standardisation is useful when doing regularization, or to facilitate interpretation (for instance, to make the intercept meaningful or for comparing the coefficients of covariates expressed in different units). More broadly, other transformations (like logs or roots) can achieve a linear relationships with the logit of the dependent variable. $\endgroup$
    – dimitriy
    May 2, 2015 at 19:10
  • $\begingroup$ Occasionally, having all the variables on a similar scale can helps with convergence, though that is not a statistical reason. The number of covariates does not alter this in any way. You should cite/quote what you have been reading in the body of your question so that I and others can offer a rebuttal. $\endgroup$
    – dimitriy
    May 2, 2015 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.