I have a matrix $x=N\times M$ of $N$ data points, where each one has $M$ features. Also, $y$ is the binary labels vector. In my case, $N$ is much smaller than $M$, so before running a classifier like SVM I used PCA to reduce the dimensionality. I picked the first 100 components which explained 85% of the variance. SVM with those 100 features gave me a poor discrimination of AUC ~0.6. Recently I thought on another method for dimensionality reduction, which I'm not sure about its validity: Each feature is a vector of $N$ values, which can be separated into 2 different vectors according to the labels. So for each feature, I can calculate how well it can differentiate the two classes by its own, using AUC, or even just the p-value of a t-test (given the histograms are normal). The problem with this method, is the dependency of the features vectors. Does anyone know how can I pick the best $k$ features vectors, which together differentiate the classes in the best way?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.