2
$\begingroup$

As stated in Hennig et al. 2016 Handbook of cluster analysis:

If for subject matter reasons some variables are more important than others regardless of the within-variable variation, one could reweight them by multiplying them with constants reflecting the relative importance after having standardized their data-driven impact.

I feel that this is related to my data which I want to cluster using K-medoids algorithm, but I don't know the exact relationship between variables, i.e. I know that $var2$ should be more important than $var1$, but it is unknown if $var2$ is twice much important as $var1$ or maybe threefold, or even fourfold. Is there any established method or a measure to assess what should be the value of this weight other than an eye-test?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

You can't statistically measure which variable is more important on an unsupervised setting.

What is more important: shoe size, or income? Probably income, but statistics cannot "prove" this, or quantify the weights. Maybe you are a shoe salesman for oversized shoes, and want to pick that district where most people have oversized feet. It's user and problem dependent, so it will be subjective and require experience.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. This may be due to my imperfect understanding but I was hoping for a way to adjust a techniques that measure clustering validitity. For example, maybe we could calculate average silhoutte width for clustering with different weights and adjust/standardise it with weights in order to make them comparable? A special case would be to compare clustering using subjective weights with clustering using original variables. What do you think? Is it valid? If so, how such standardisation should look like? If it's invalid, why? $\endgroup$
    – jakes
    Aug 1, 2018 at 5:34
  • 1
    $\begingroup$ You'll hit a catch-22. If you don't know the proper weighting yet, clustering will likely not be good, but then the Silhouette will not be informative. You'll likely get the "best" Silhouette with only a few features and extreme distortion - the contrary of what you want to achieve. $\endgroup$
    – Has QUIT--Anony-Mousse
    Aug 1, 2018 at 17:30
-4
$\begingroup$

@jakes, I disagree with @anony-mousse on the statement, "You can't statistically measure which variable is more important on an unsupervised setting." This is wrong! What do you think Principal Component Analysis (PCA) does. PCA is an unsupervised dimensionality reduction algorithm. It works by extracting the variables that contain the maximum variance in them by doing an orthogonal transformation of the original variables. Besides, there exist scores of unsupervised algorithms that can be used for detecting and proving relationships between variables in an unsupervised setting.

Again, "What is more important: shoe size, or income? Probably income, but statistics cannot "prove" this, or quantify the weights." Again, you are wrong. Statistics can prove this relationship provided how one has coded the variables and then apply an appropriate algorithm.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for your comment. Any examples of measures (other than percent of variance explained used in PCA) that would statistically tell which variable is more important? Any indications where should I look before matching one to my problem? $\endgroup$
    – jakes
    Aug 4, 2018 at 14:32
  • 2
    $\begingroup$ Variance does not measure importance. So PCA does not, either. $\endgroup$
    – Has QUIT--Anony-Mousse
    Aug 17, 2018 at 10:20
  • $\begingroup$ @Anony-Mousse I never said, PCA is meant for measuring importance of variables. I have stated it clearly, that PCA is a dimensionality reduction algorithm. It "EXTRACTS" and not measures "importance". There is a lot of difference between the two. Enlighten the audience with what exactly is a measure of "importance", if "variance" is not!! $\endgroup$
    – mnm
    Aug 17, 2018 at 10:46
  • 2
    $\begingroup$ It "extracts" variance, not importance. I already mentioned that you cannot just measure importance. Show me your rigour, where is it? $\endgroup$
    – Has QUIT--Anony-Mousse
    Aug 17, 2018 at 11:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.