Let's assume I have a NxD matrix X with the N rows being observations and the D columns being features.
I would now like to know which are the most "interesting" features of this dataset. I.e. which features depend on each other, which are redundant etc.
At the end, I would like to have a dataset of dimensionality k < D, because I could dismiss (D-k) features.
My first idea was using PCA to get an approximation to the "intrinsic" dimensionality of my dataset. However, PCA will not directly tell me which features are the most interesting ones, it will only give me a number of principal components and their "strengths" (eigenvalues of the covariance matrix of X).
So I thought about using a classical feature selection method like stepwise regression. However, stepwise regression requires a target vector y (since it is regression, of course) which I don't have. I only have the dataset X.
I only have basic machine learning skills, so I would like to know what is the appropriate method to select the most interesting features of my dataset X without having a target vector y.