I have a basic doubt in dimension reduction for text dataset eg. 20Newsgroup, rcv1 etc. Initially I extract the number of word occurrence in each document, i.e word x document matrix would be $n \times d$ where $n$ is the number of documents and $d$ is the dimension.
I would like to reduce the dimension, say $d_1 << d$. What is the standard technique of reducing the dimension?
- Choose the top $d_1$ feature from the original word occurrence matrix$( n \times d)$ and then calculate TF-IDF for the reduced matrix $(n \times d_1)$, or
- Calculate the TF-IDF matrix for $n \times d$ matrix and then select top $d_1$ features.
Also, it is mentioned in many literature that top feature are selected. I wanted to know what is selecting the top features means? How do they define it?