11
$\begingroup$

My data consists of several continuous measurements and some dummy variables representing the years the measurements have been made. Now, I want to learn a neural network with the data. Therefore, I am zScore-normalizing all variables, including the dummy variables. However, I wonder if this is a reasonable approach, because normalizing the dummy variables alters their ranges, which I guess makes them less comparable if their distributions differ. On the other hand, not normalizing the dummy variables might also be questionable, because without normalization their influence on the networks output might be suboptimal.

What is the best approach to deal with dummy variables, normalizing them (zScore) or just leaving them as they are?

Improve this question
$\endgroup$
1

3 Answers 3

Reset to default
10
$\begingroup$

Normalization would be required if you are doing some form a similarity measurement.

Dummy variables by its nature acts as a binary switch. Coding it as (0,1) or (-.5,.5) should have no impact on the relationships it exhibits to a dependent variable, if what you are trying to do is some form or regression or classification.

It would matter if you are performing clustering because it would be scale dependent.

Improve this answer
$\endgroup$
4
$\begingroup$

Normalizing dummy variables makes no sense. Usually, normalization is used when the variables are measured on different scales such that a proper comparison is not possible. With dummy variables, however, one puts just a binary information in the model and if it is normalized the information of the impact of e. g. one year is lost.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So, according to the question, how do we deal with loss function? $\endgroup$
    – Afshin Amiri
    Commented Jul 19, 2018 at 2:45
  • $\begingroup$ What if I want to compare variable importance between say a logistic regression and a tree-based model that doesn't have coefficients? $\endgroup$
    – qwr
    Commented May 15 at 15:55
  • $\begingroup$ You could compare the difference of the prediction performance of both models with and without the variable. $\endgroup$
    – random_guy
    Commented May 28 at 8:45
0
$\begingroup$

Scaling regression inputs by dividing by two standard deviations by Andrew Gelman suggests scaling the numeric inputs by dividing 2 standard deviations so that they can be directly compared to untransformed binary variables. The idea is that a binary variable with equal probabilities has mean 0.5 and standard deviation 0.5, so comparing x=0 and x=1 (unscaled) is a difference of two standard deviations.

The caveat is that this standard deviation estimate works for binary variables that are roughly symmetric, since a highly skewed variable such as 10%/90% variable has two standard deviations only 0.6. However, this is still more reasonable than scaling by one standard deviation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.