I am using decomposition.SVD on a TF-IDF features and I would like to know how I can improve my dimension selection?
I know that scikit-learn advice to use 100 for LSA but I would like to be sure that's the new dimension fit well my dataset.
Desired dimensionality of output data. Must be strictly less than the number of features. The default value is useful for visualisation. For LSA, a value of 100 is recommended.
I don't have annotated dataset. Should I apply LSA many times to see how behave the sum(explained_variance) and choose the dimension that seem to tend toward the asymptote?