I want to conduct a principal component analysis (PCA) in SPSS. One assumption for PCA is that there are no significant outliers. How can I identify outliers in SPSS?
$\begingroup$
$\endgroup$
14
-
5$\begingroup$ I don't think there is any such "assumption". PCA is just a multivariate transformation. How well it works for your purposes may be affected by whether outliers are present, but that depends on your purposes as well as the data. But you are right that checking for outliers is sensible data analysis. I haven't used SPSS for some decades, so can't fill in myself. $\endgroup$– Nick CoxCommented Oct 4, 2013 at 9:28
-
4$\begingroup$ Outliers can completely screw a classical PCA analysis and make it meaningless (e.g. yield a model with an arbitrarily bad fit to the genuine part of the data). You can see an example here. There is an discussion of robust methods for PCA estimation here. I'm not aware of SPSS implementations, but many user friendly ones are available in R. I would not recommend doing serious statistical analysis in a closed source package such as SPSS. $\endgroup$– user603Commented Oct 4, 2013 at 9:59
-
2$\begingroup$ @user603: Does PCA mean anything & does it yield models? Genuine question - like Nick I'm used to thinking of it as just a multivariate transformation, but perhaps there's another view, PCA as something akin to factor analysis. $\endgroup$– Scortchi ♦Commented Oct 4, 2013 at 12:08
-
1$\begingroup$ @Scortchi: without pondering on this debate, i'm wondering how is someone to use a data analysis tool --for any purpose-- when its results are liable to being swayed by a few observations, regardless of the sample size --putting the fitted eigen-vectors so far away from what they would have been without these data points as to make classical diagnostic tool completely useless. It's not like we don't have many real data examples of this kind of phenomenon... $\endgroup$– user603Commented Oct 4, 2013 at 12:15
-
2$\begingroup$ @user603 As usual, I expect we (e.g. you, Scortchi and I) agree more closely than is apparent. Outliers can certainly make a PCA problematic or at least difficult to interpret, even though in principle, outliers can also be consistent with the correlation structure of the rest of the data. In practice, I might recommend PCA on transformed scales if outliers appeared to sway a PCA; robust estimation of PCA is certainly an alternative, but I'm guessing wildly a bit out of the OP's usual territory and unlikely to be available in SPSS (I guess wildly). $\endgroup$– Nick CoxCommented Oct 4, 2013 at 13:02
|
Show 9 more comments
1 Answer
$\begingroup$
$\endgroup$
5
Robust PCA is a very active research area, and identifying and removing outliers in a sound way is quite delicate. (I've written two papers in this field, so I do know a bit about it.) While I don't know SPSS, you may be able to implement the relatively simple Algorithm (1) here.
This algorithm (not mine) has rigorous guarantees but requires only some basic computations and a "while" loop. Assuming you are searching for $d$ principal components, the basic procedure is
- Compute PCA on your data,
- Project your data on to the top $d$ principal components,
- Throw away "at random" one of the data points whose projection is "too large", and
- Repeat this "a few" times.
Everything in quotation marks is a heuristic; you can find the details in the paper.
The idea behind this procedure is that vectors whose projection after PCA is large may have effected the estimate too much, and so you may want to throw them away. It turns out that choosing the ones to throw away "at random" is actually a reasonable thing to do.
If anyone actually wants to take the time to write the SPSS code for this, I'm sure @cathy would appreciate it.
-
1$\begingroup$ +1, and nice of you to share the article! It might be indeed worth writing the code. The article is quite mathematical though (not easy to understand by data analysts like me). Can you point me to the main places there, particularly corresponding to the 4 stages of the algo you describe? $\endgroup$– ttnphnsCommented Oct 4, 2013 at 20:19
-
1$\begingroup$ @ttnphns Algo 1 is at the top of p. 6. The steps above summarize step 2 in the algorithm. As for their notation, $\hat d$ is the number of principal components, $\bar{T}$ is the maximum number of loops (for safety, take $\bar{T}=n-1$, but smaller values usually won't hurt and you get the "best solution so far"), $\hat{t}$ is any lower bound on the number of good points (e.g., you can set $\hat t$ to be half the total number of points in most situations), $\bar{V}_{\hat{t}}(w)$ is the sum of the norms of the "smallest projections". $\endgroup$ Commented Oct 4, 2013 at 20:29
-
$\begingroup$ +1, but why not cite your own papers on the topic? I would be curious. $\endgroup$– amoebaCommented Jan 22, 2015 at 17:24
-
3$\begingroup$ @amoeba My papers (and a number of others) involve semidefinite programming, which can be rather laborious to explain. I wanted to give a simple-to-understand algorithm that is, in some sense, effective. $\endgroup$ Commented Jan 23, 2015 at 3:08
-
$\begingroup$ Do you know of a public implementation of this method? I've asked a separate question about it here. stats.stackexchange.com/questions/389204/… $\endgroup$ Commented Jan 25, 2019 at 22:11