In some machine learning problem, we have objects represented as feature vectors $X$. We want to cluster them.
We want some features of $X$ to have no influence on the clustering. Call $X_1$ such a feature. Of course we can just remove it but it is most often strongly correlated to other unknown features, so that removing $X_1$ is far from removing its influence. Actually we willingly introduce $X_1$ as a feature in order to remove its influence.
My main focus are bags of words in natural language processing using cosine distance, but I'm open to more general ideas.
- documents are texts from several authors: Features are words and the author. We want to remove the clustering tendency caused by an author using a certain vocabulary over an over again
- documents are articles from generalist web sites: Features are words and the website. We want to remove the clustering tendency caused by unwanted text referring to the site itself (like "please subscribe to stackexchange.com" in CV) or authors style
- remove the ghost presence of the source language in translated texts (with low quality automated translation)