2
$\begingroup$

I have 17 variables catching different properties of neighbourhoods (geometrical and topological). The correlation matrix shows little correlation among them; as I understand, that's okay because it means my variables are actually catching different features of the individuals (300 neighbourhoods).

I ran PCA using FactoMineR, I generated the factor map, and then cut the dendrogram using HCPC. The dendrogram comes out with a line that cuts the it at 5, which, I assume, is the optimal number of clusters.

Here's my questions:

  1. The factor map indicates that my two first principal components explain 48% of the variance in the data. I've read that this might be due to that the correlation among variables is not very significant. Is this really important? My goal is mostly to group the individuals by type and claim that there's x types of neighbourhoods in my city.

  2. The dendrogram seems to indicate 5 as an optimal number of clusters but when I cut it at 8 it seems closer to what I'm looking for. Would it be statistically wrong if I decided to go for 8 and omit the grouping of 5? I know that 8 sounds arbitrary and that I maybe should have gone with a simple k-means clustering but I tried that and it didn't work. The groups k-means generated didn't make any sense.

enter image description here

enter image description here

$\endgroup$
2
  • $\begingroup$ The estimation of the number of clusters is a very difficult problem and will normally require some user input and decisions. I assume the question concerns the function HCPC in FactoMineR. This uses a heuristic criterion that, as far as I see from the documentation, doesn't come with any reference or systematic quality evaluation let alone theory. There is no reason to trust this. $\endgroup$ Commented Mar 9 at 11:23
  • $\begingroup$ Furthermore, in most cases there is no reason whatsoever to believe that running a PCA before clustering improves clustering results. PCA removes information, and there is no reason to believe that the lost information is useless or bad for clustering. The aim of PCA is not to find dimensions that are particularly meaningful for clustering, and therefore it will not normally do that, and clustering the data without PCA will in most cases be better. $\endgroup$ Commented Mar 9 at 11:25

2 Answers 2

0
$\begingroup$

If the first two components would explain almost 100%, then this would essentially say that most of your attributes are meaningless. So no, you do not want this value to be too high.

At the same time, if you would need all 17 components, you will likely already see the curse of dimensionality where similarity becomes hard to decide.

$\endgroup$
1
  • $\begingroup$ Also take a look at sparse principal components analysis which combines clustering and PCA. $\endgroup$ Commented Nov 8, 2023 at 12:43
0
$\begingroup$

1) The factor map indicates that my two first principal components explain 48% of the variance in the data. I've read that this might be due to that the correlation among variables...

PCA does only make sense when correlation exist. It is a variance maximizing orthogonal projection. I.e. no correlation no fun.

2) The dendrogram seems to indicate 5 as an optimal number of clusters but when I cut it at 8 it seems closer to what I'm looking for. Would it be statistically wrong if I decided to go for 8 and omit the grouping of 5...

The dendogram is created with a distance matrix (and the distance measure is up to you). If it makes practical sense to go for 8 and you can explain those 8 in a meaningful way, then go for it.

What do you try to analyze? There might be better ways to achieve dimensionality reduction.

$\endgroup$
2
  • $\begingroup$ Thanks for your answer. That was pretty much what I ended up doing. I was almost 3 years ago. I was measuring properties of the street network for an urban morphology project. I wanted to find a way to determine types of neighbourhoods based on quantitative variables. $\endgroup$
    – 1buzz
    Commented Aug 4, 2019 at 22:17
  • $\begingroup$ Wups. Didn't see the date of you post. $\endgroup$ Commented Aug 5, 2019 at 8:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.