Lets say we have a data with 100 features of which only I have 10 features of uncorrelated data and rest 90 features are correlated to one of these 90 features(My problem is similar to these assumptions). To select the best data and to apply machine learning I need to get rid of features which are correlated. I know PCA can do this but data transformation is not suited because it transforms features into uninterpretable features i.e feature selection is preferred for my particular problem. When I run sequential feature selection, for different trials it selects different subset of 10 features each time I run. How do I measure similarity between the data of features selected for different trials to check whether my sequential feature selection giving me good results?
Simply calculate the correlation and drop correlated columns given a threshold (ideally tuned, but a heuristic might do).
This type of approach is usually called filter feature selection, and there are many other filters devised to tackle this issue, usually aiming at information entropy and related concepts such as mutual information (check this thread Mutual information versus correlation).