Feature selection without target variable

Question

Let's assume I have a NxD matrix X with the N rows being observations and the D columns being features.

I would now like to know which are the most "interesting" features of this dataset. I.e. which features depend on each other, which are redundant etc.

At the end, I would like to have a dataset of dimensionality k < D, because I could dismiss (D-k) features.

My first idea was using PCA to get an approximation to the "intrinsic" dimensionality of my dataset. However, PCA will not directly tell me which features are the most interesting ones, it will only give me a number of principal components and their "strengths" (eigenvalues of the covariance matrix of X).

So I thought about using a classical feature selection method like stepwise regression. However, stepwise regression requires a target vector y (since it is regression, of course) which I don't have. I only have the dataset X.

I only have basic machine learning skills, so I would like to know what is the appropriate method to select the most interesting features of my dataset X without having a target vector y.

Answer 1

Have you considered some kind of generative model?

The idea here is, if you have a model that knows how to make things that look like the training data (e.g. with similar covariance structure), it must have captured some of the important structure in your data.

If you can access this Science paper, it claims to have a method that works really well. If you don't have access, let me know and I'll send it to you privately.