Unsupervised learning with missing features Assume I have a set of N vectors with M features each. If I want to create a latent space to project these vectors into, there are a variety of techniques available to me, such as Principle Component Analysis (PCA).
However, if I were using PCA and I decide to only consider x of M features for a latent space, I would then have to carry out PCA again for the N vectors with x features each.
Is there an unsupervised learning technique which allows for training with N vectors of M features, but allows for selection of a subset of M features with completely recomputing the space? Is my only option to carry out PCA for all possible subsets of M which might be considered and cache the results? Is there a different term for what I'm trying to do?
 A: You need to take a time out and think about the $M$ features you already have, since your OP basically starts with shoving a lot of stuff in an oven and expecting a pizza to come out when done.  For example, when performing micro-surgery on your features -- before you do anything with them -- you need to determine whether:


*

*the features were already selected from a class or function approximation prediction analysis

*the features represent everything known about a system, i.e., all possible measurements including signals and noise, and PCA would be used to learn about groups of correlated features because I don't know anything about the data.


Firstly, what's the scale and range of all the features, and what's the correlation between the features?  If the data are merely all noise, then the correlation would be degenerate (but not zero, orthogonality for noise data only occurs as $n \rightarrow  \infty$).  PCA is not intended for all possible subsets or random sets of features to understand the spectral information of a set of data.  You could always develop something that would analyze the data ad nauseam via random resampling, but the result would be like hyperemesis $\rightarrow$ projectile vomiting.  
The beauty of PCA stems from eigendecomposition, and there have been many theoretical aspects of eigendecomposition that are still unexplored in applied numerical analysis.  
A: You might be looking for something more like independent component analysis (ICA). The goal there is to find the statistically independent components in a latent space that generated the observed features. In the standard, linear version of ICA there are just as many independent features found as features in each sample vector, but there are generalizations of ICA to the nonlinear case which act in a manner similar to an autoencoder and can produce latent factor spaces of lower dimension than the original feature space. This may be related to factor disentanglement or for a linear alternative to low-rank representations, depending on your overall goals and linearity requirements.
