3
$\begingroup$

PCA based filtering is used to identify and eliminate noise in data. This would basically involve computing the PCs and using the top k PCs to denoise the data. What if I know for sure that only the extremely small values in my matrix are noise? Now, a value may be small w.r.t the entire matrix but not small w.r.t a particular row/column. Can I achieve this with some transformation of the input matrix followed by a PCA based filter?

$\endgroup$
  • 3
    $\begingroup$ What exactly do you mean by denoise? Do you want to remove some data which satisfies certain criteria, or do you want to estimate the signal from your data? $\endgroup$ – mpiktas Jan 18 '11 at 17:18
  • $\begingroup$ Or, along those lines but slightly different, do you want to perform an especially selective PCA that only utilizes the most informative components? For example many people default to extracting components with eigenvalue >=1.0 when a stricter criterion might serve their needs better and reveal more replicable results. $\endgroup$ – rolando2 Jan 23 '11 at 18:47
3
$\begingroup$

PCA depends on the scaling of your columns. If you perform a PCA on matrix $X$, then rescale each column to be norm 1 (i.e. divide by the two norm of each column), then perform a PCA on the transformed $X$, you will get different answers. I believe this is part of what 'small w.r.t. a particular row/column' is referring to in the original question.

However, the small elements in the matrix should contribute very little to a PCA/downsample/reconstruction operation. Perhaps you would be better served by determining, perhaps in a semiautomatic way, how many PCs to take in the representation. For this purpose, you might want to look at the scaling of the eigenvalues you get from the SVD, and look for the 'knee' in the scree plot, or if you know something about the small noise, you can rely on the distribution of eigenvalues of the Gramian matrix. For normally distributed noise, with equal variances, they should follow a Marchenko-Pastur Distribution, up to scaling. This will give you the limits of the eigenvalues you expect to see in the pure noise situation. Basing your PC cutoff on that limit may be fruitful.

Sorry this is somewhat vague, I do not think I fully understand what technically is desired from the OP.

$\endgroup$
  • $\begingroup$ +1, even though I cannot fathom what the OP is looking for either. $\endgroup$ – whuber Jan 28 '11 at 4:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.