# Overrepresented Features in Clustering

So I was thinking, if I have a set of features (let's say $$(X_1, X_2, X_3)$$) that basically describe the same overarching feature $$Y$$, and can somehow be mapped $$(X_1, X_2, X_3) \rightarrow Y$$. In addition, there exists a forth feature $$X_4$$ which is not part of this mapping but still contains meaningful data and is conceptually is on the same level as $$Y$$, meaning $$X_4 \rightarrow X_4$$.

To relate this to a more tangible example, $$(Height, Muscle Density, Weight) \rightarrow Gender$$ and the additional independent feature $$HairColor$$. Furthermore, let's assume that $$HairColor$$ can somehow be meaningful represented as a continuous variable, all features are normalized properly and I don't want to apply factor analysis.

Now, I want to explore this dataset with 4 features $$(X_1, X_2, X_3, X_4)$$, meaning I want to cluster. The majority of cluster algorithms (if not even all) intrinsically rely on distance metrics. This leads me to my first question:

1) The features mapping to $$Y$$ will account for the majority of distances between observations within my dataset and the distances of $$X_4$$ between observations will become negligible. This probably results in two clusters (or in our contemporary age maybe more), namely male and female. Information about the hair color will get lost. Is this interpretation/claim (at least in some circumstances) correct?

2) And if so, how would one express/prove this in a formal way?

If you want to study this more formally, begin with a model where the differences in each variable is composed of signal $$w_i$$ and error $$\epsilon_i$$ and then study how much effect each variable has. Use Manhattan distance, as this is easiest to handle.
• Isn´t that the reason why you standardize your variables? My question was targeting every $L^q$ distance. My idea currently is to use cosine similarity instead of $L^q$ metrics. – Raphael Prager Mar 21 '19 at 13:11