2
$\begingroup$

I am dealing with data with different units of measurement for NYC neighborhoods and I am trying to build a composite score with it. For example, I have total population by neighborhood, mean income and children as a percentage of the population. Currently, I am just subtracting the mean for each variable, and then dividing it by its standard deviation. These gives me all the data in units of standard deviation. Additionally, one I have standardized the data as described, I subtract to each value in each variable its minimum value, and then I divide it by the different between its maximum and minimum value (max - min value per variable) so that all values run from 0 to 1.

What are good papers or books to understand if what I did is correct?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ Standardization is so straightforward that I doubt there is a book about it. What you did seems fine, but the 2nd normalization seems unnecessary. $\endgroup$
    – gung - Reinstate Monica
    Dec 10, 2014 at 1:18
  • 1
    $\begingroup$ Are you planning to add the three [0,1] variables into a composite index? What is you goal in doing that? $\endgroup$
    – dimitriy
    Dec 10, 2014 at 2:12
  • $\begingroup$ Hi Dimitriy, yes, I am planning to add each of the variables that would run from 0 to 1 into a composite score. I am doing so to facilitate interpretation: I presume that because all data is in a scale 0-1, the final score will also be in 0-1 scale. Thank you! $\endgroup$
    – Manuel Q
    Dec 10, 2014 at 18:36
  • $\begingroup$ @ManuelQ I think that approach imposes some strong assumptions. Do you want to compare neighborhoods to each other using a single number or do you have an outcome that you want to tie this index to? $\endgroup$
    – dimitriy
    Dec 10, 2014 at 21:27
  • $\begingroup$ @DimitriyV.Masterov I am expecting to build a score that will rank all neighborhoods. I think having everything scaled to 0-1 would be convenient for the interpretation of the overall score. Again, I am presuming that if each element of the score (which is a simple average) is scaled 0-1 the score itself will also be 0-1. But this might not be true (for example, if each of the score components is weighted differently). $\endgroup$
    – Manuel Q
    Dec 10, 2014 at 23:02

3 Answers 3

Reset to default
2
$\begingroup$

Chapter 4 of Gelman & Hill (Data Analysis Using Regression and Multilevel/Hierarchical Models) covers this topic with respect to linear models. Linear transformations won't change the quality of your fit, but may help you interpret model coefficients (especially if you have interactions!).

A log transformation is worth considering for variables like total population and income. As log is nonlinear, it will change both the fit and the interpretation.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Many thanks, this is a wonderful book! Do you have any other similar recommendations? $\endgroup$
    – Manuel Q
    Dec 10, 2014 at 18:37
2
$\begingroup$

Nick Cox has a very nice Intro To Transformations, that also contains some references to the literature.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Nice! I need to submit my project soon, but I will delight myself reading this! $\endgroup$
    – Manuel Q
    Dec 10, 2014 at 18:37
-1
$\begingroup$

You are mostly on the right track here. Standardization is generally a good practice because lots of simple mainstream approaches you would use (like linear regression) assumes you have a normal distribution in your data.Here is a write up that talks about it with some references to books towards the end. Hope that helps

Improve this answer
$\endgroup$
1
  • $\begingroup$ (-1) You seem to be confusing standardization (mere centring & scaling) with transformations intended to make the data's distribution more like that of the normal distribution (e.g. the Box-Cox transformation mentioned in the link). $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Dec 10, 2014 at 9:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.