4
$\begingroup$

I'm trying to figure out the best way of creating groups in a dataset with many dimensions. I have 1000 measurements, and each measurement has 40 dimensions. The measurements are of neighborhoods with physical and socio-economic data. For example I have measures like the number of people in each area, the population density, the number of buildings, average household size etc, average income.

I'm quite new to this field, so I'm not sure what is the best way of going about this. Two thoughts I had were using CART (specifically the rpart package in R), principal component analysis, and k-means clustering. Ideally I would do all of this analysis in R.

I would like to end up with 6 to 8 "typologies" or representative groupings of the data. They do not necessarily have to include all 40 dimensions.

I would appreciate any advice on the best way of doing this.

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ Can you please be more specific about what kind of data set do you have? Answers may vary accordingly. rpart may be Ok - but may be something better is available. $\endgroup$
    – suncoolsu
    Commented Feb 8, 2011 at 3:28
  • 2
    $\begingroup$ Before you do any of this stuff, perform univariate exploration of each of your 40 attributes. Many of them should be re-expressed as square roots or started logarithms before you go any further. In practice this will improve subsequent results immensely. You also want to identify outliers and make a plan for assessing their influence in your analyses, at least if you're going to use least-squares techniques (including PCA, which can be extremely sensitive to geometric [multivariate] outliers). $\endgroup$
    – whuber ♦
    Commented Feb 8, 2011 at 16:25
  • $\begingroup$ My preliminary visual survey of the data was plotting all parameters against each other, so I have a plot of every relationship, and can observe a few patterns and outliers. Should I apply the same transformations to every variable, and include them, or should I examine what the most likely transformations should be first. $\endgroup$
    – celenius
    Commented Feb 8, 2011 at 16:53

1 Answer 1

Reset to default
1
$\begingroup$

You may want to start using a dimensionality reduction,maybe using PCA or MDS, which may give a clearer picture. I would also try clustering, whether k-means or a more sophisticated method.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.