The best tool to resolve (multi-) collinearity is in my view the Cholesky-decomposition of the correlation/covariance matrix. The following example discusses even the case of collinearity, where none of the bivariate correlations are "extreme", because we have rank-reduction only over sets of more variables than only two.
If the correlation-matrix, say **R**, is positive definite, then all entries on the diagonal of the cholesky-factor, say **L**, are non-zero (aka machine-epsilon). Btw, to use this tool for the collinearity-detection it must be implemented as to allow zero-eigenvalues, don't know, whether, for instance, you can use SPSS for this.
The number of on-zero entries in the diagonal indicate the actual rank of the correlation-matrix. And because of the triangular structure of the **L**-matrix the variables *above* the first occuring diagonal zero form a partial set of variables which is of reduced-rank. However, there may be some variables in that block, which do not belong to that set, so to find the crucial subset which contains **only** the multicollinearity you do several recomputations of the cholesky-decomposition, where you reorder the variables such that you find the smallest possible subset, which shows rank-reduction - so this is an iterative procedure.
(If needed, I'll show an example where I use my MatMate-program for the script, later).
<hr>
Here is an example using random-data on 5 variables, say $x_1$ to $x_5$ which I configured, such that the correlation matrix is positive semidefinite (up to machine precision) because I made $x_5 = 2 \cdot x_2 + \sqrt 2 \cdot x_4 $ (and after that normed to unit-variance) and thus that subset of three variables make a collinear subspace (more exactly: we should call it "co-planar" since they are linearly dependent only in a plane).
here is the correlation-matrix
;MatMate-Listing vom:06.03.2013 17:43:23
;============================================
C= x1 x2 x3 x4 x5
x1 1.0000 -0.7506 0.2298 -0.8666 0.0952
x2 -0.7506 1.0000 -0.2696 0.4569 0.5355
x3 0.2298 -0.2696 1.0000 0.1890 -0.4407
x4 -0.8666 0.4569 0.1890 1.0000 -0.5066
x5 0.0952 0.5355 -0.4407 -0.5066 1.0000
and here the cholesky-factor / loadingsmatrix:
[20] L = cholesky(C)
L= f1 f2 f3 f4 f5
x1 1.0000 . . . .
x2 -0.7506 0.6607 . . .
x3 0.2298 -0.1469 0.9621 . .
x4 -0.8666 -0.2930 0.3587 0.1856 .
x5 0.0952 0.9186 -0.3406 -0.1762 .
As we see, that only 4 of 5 diagonal elements are non-zero (above machine-epsilon) we know, that correlation matrix has rank 4 instead of 5 and we have collinearity. But we do not yet know, whether 4 variables are linnearly dependent or whether we have possibly a rank reduced subspace of smaller dimension. So we try iteratively the rotation to triangularity, where the order of the variables $x_1$ to $x_5$ is systematically altered to identify any possible smalles subset.
For instance, we make the last item "the first"
[22] l1=rot(L,"drei",5´1´2´3´4)
L1= f1 f2 f3 f4 f5
x1 0.0952 0.9955 . . .
x2 0.5355 -0.8053 0.2545 . .
x3 -0.4407 0.2730 0.7320 0.4421 .
x4 -0.5066 -0.8221 0.2598 . .
x5 1.0000 . . . .
and we see,. that rank-reduction is already occuring if we ignore variable 3, because the variables $x_1,x_2,x_4,x_5$ define already a 3-dimensional subspace (instead of a 4-dimensional one).
Now we proceed altering the order for the cholesky-decomposition (actually I do this by a column rotation with a "triangularity-criterion"):
[24] l1=rot(L,"drei",5´4´1´2´3)
L1= f1 f2 f3 f4 f5
x1 0.0952 -0.9492 0.3000 . .
x2 0.5355 0.8445 . . .
x3 -0.4407 -0.0397 0.7803 . -0.4421
x4 -0.5066 0.8622 . . .
x5 1.0000 . . . .
Now we've nearly done: the subset of $x_2,x_4,x_5$ forms a reduced subspace and to see more, we put them at "the top" of the cholesky-process:
[26] l1=rot(L,"drei",5´4´2´1´3)
L1= f1 f2 f3 f4 f5
x1 0.0952 -0.9492 . 0.3000 .
x2 0.5355 0.8445 . . .
x3 -0.4407 -0.0397 . 0.7803 0.4421
x4 -0.5066 0.8622 . . .
x5 1.0000 . . . .
and see, that $x_1$ has a component outside of that reduced space, and $x_3$ has a further component outside of the rank 3 space, and are thus partly independent of that 2-dimensional subspace (which can thus be give the term "co-planarity").
We can now decide, which of the three variables $x_2,x_4$ or $x_5$ can be removed to overcome the multi-collinearity problem.
If we would use some software which does not allow this flexible reordering "inside" the rotation-parameters/procedure, we would re-order the variables forming the correlation-matrix and would do the cholesky-decomposition to arrive at something like:
[26] l1=cholesky(...) // something in your favorite software...
L1= f1 f2 f3 f4 f5
------------------------------------------------------
x5 1.0000 . . . . Co-planar subset
x2 0.5355 0.8445 . . .
x4 -0.5066 0.8622 . . .
------------------------------------------------------
x1 0.0952 -0.9492 . 0.3000 . further linearly independent
x3 -0.4407 -0.0397 . 0.7803 0.4421 variables