# How to use PCA to detect outliers?

A PCA will reduce the dimensionality of the original data and construct a subspace generated by eigenvectors of which each represents the (next) highest variance to explain the data.

Let's start at this subspace: A PCA has been performed and we have a look at the according subspace now:

Now let's assume there are outliers (however where exactly). How can they be detected now?

So far, I know there are two methods:

• Track the angle(s ?) between the PCs

• Check the number of PCs

I think both are not robust, because new or more data will probably change the angles without providing an outlier. The number of axes makes more sense but still I can construct situations in my head where new data might cause introducing a new axis without making all the data there outliers. I thought of

• using a distance/defined radius to scan for new outliers but I can hardly find according approaches? On

• Why is PCA sensitive to outliers? it is explained why it is sensitive to Outliers, this can probably used as a tool, as well.

In other words: How exactly is PCA used to detect outliers respectively how are they detected after performing the PCA?

• If the outlier is really an outlier, then the variability should win out variation in other dimensions. This should mean that you can check the outlier by checking the outlier in the first component?
– Art
Sep 17, 2019 at 9:56
• The PCs are mutually orthogonal (except for a situation with probability zero). There are always p-many PCs for p-variate data. Even if your methods are not robust, what did you have in mind for using these?
– Dave
Sep 17, 2019 at 10:05

One approach is to consider outliers those points that can not be well reconstructed using the principal vectors that you have selected.

The procedure goes like this:

1.Fix two positive numbers, a and b (see the next steps for there meaning an to understand how to select them; to be refined using cross-validation).

1. Compute PCA.

2. Keep the principal vectors that are associated with principal values greater than a, say $$v_1,v_2,..,v_k$$ (this are orthonormal vectors).

3. For each data point compute the reconstruction error using the principal vectors from step 3. For a data point x, the reconstruction error is: $$e = ||x-\sum_{i=1}^{k}w_iv_i||_2$$ , where $$w_i = v_i^Tx$$

4. Output as outliers those data points that have a reconstruction error greater than b.

Update: The procedure capture only "direction" outliers . Additionally, before the first step, a "norm" outliers detection step can be included. This consist of computing the norms of the data points and labeling as outliers those that have a too small or too big norm.

It depends on what an outlier is in your context.

• Thank you, do you also know if this is already implemented in e.g. python?
– Ben
Sep 17, 2019 at 10:11
• If you want to use a library , there are many good algorithms for outliers detection, in special in Python . For example - scikit-learn.org/stable/modules/outlier_detection.html . What I have described is just a heuristic which I don't know how good is it . But if you want to implement it is quite straight forward : PCA is available in sklearn ( scikit-learn.org/stable/modules/generated/… ) and the other steps can be easily implemented using Numpy (numpy.org) . Sep 17, 2019 at 11:13
• @Ben, PyOD has an implementation specifically for outlier detection, pyod.models.pca.PCA. find more at: pyod.readthedocs.io/en/latest/… other implementations: pyod.models.kpca.PyODKernelPCA, pyod.models.kpca.KPCA Apr 17, 2023 at 15:59