I want to do dimensionality reduction on a data set $X_{ij}$. In this case, $i$ indexes samples and $j$ indexes a large number of variables (densities at different locations in space). The units of all elements are the same, but the uncertainties (which I can estimate) vary quite a lot, mostly across samples. Each $X_{ij}$ is Gaussian, with standard deviation $\sigma_{ij}$ varying across both samples and variables.
My problem is that doing standard PCA without taking into account the uncertainties will tend to produce results which are dominated by variance due to measurement error. What I would like to do is estimate some similar quantities to the principal components, but which will be more robust to outliers with large $\sigma_{ij}$. The variables $X_{ij}$ are drawn from $\mathcal{N}(\mu_{ij},\sigma_{ij})$. I have one such sample for each $i$ and $j$, and I would like to calculate a transformation involving $X_{ij}$ and $\sigma_{ij}$ which will estimate principle components of the matrix $\mu_{ij}$, for example.
It is difficult for me to make progress analytically because I cannot calculate how uncertainty in $X$ propagates to its principle components. I can generate surrogate data to confirm that uncertainty dominates the principle component analysis, and I can also do this to validate any alternative techniques. My main goal is to look for independent modes across the variables, so my concerns are with dimensionality reduction for presentation, description and simplification, rather than physical interpretation.
