# Testing for collinearity and parallelism with the axes

Cross-posted from math; let me know if you need me to wait there for a bit before posting here

I have a set of datapoints in $\mathbb{R}^3$ (integer or real values of $X$, $Y$ and $Z$, for, say. 30 points).

I need to do a least squares regression for linearity. If the data set is linear, I need to see if it is close to vertical or horizontal. How could I do this?

-
Do you mean your dependent variable is 3-dimensional? –  Macro May 11 '12 at 1:53
Crossposting is discouraged - I've flagged your post for migration. It seems to me to be more relevant here than on the math stackexchange. –  Macro May 11 '12 at 1:56
@Marco, no, I mean all my datapoints have 3 dimensions (x y z). If I understand you correctly thats not what you meant –  jamesson May 11 '12 at 2:14
Can you edit your question and write out the model you're trying to fit? –  Macro May 11 '12 at 2:15
@Macro, hows that? –  jamesson May 11 '12 at 2:24
show 4 more comments

## Collinearity

If all the points are collinear, we should be able to explain almost the variance in the sample by one vector. In other words, the leading principal value should be significantly greater than the other two (since we are working with a total of three dimensions). How much greater it should be is up to you to decide. I would say well above 95%, say 98% or 99% of the total variance, should be explained by the leading principal component.

## Alignment with the axes

Once you have calculated the leading principal component, it is a simple matter of taking its dot product of the unit vectors along all three axes to determine the alignment. The closer the absolute value of the dot product is to unity, the more aligned the principal component is to that axis.

## Mathematica Example

I'm going to generate points that are randomly dispersed around a randomly-generated quadratic curve. The greater the coefficient of the highest-order term, the tighter the curve, and the closer the points are to being collinear.

v = RandomReal[{-1, 1}, 3]; v /= Norm[v];
u = RandomReal[{-1, 1}, 3];
w = RandomReal[{-1, 1}, 3];
x = Table[RandomVariate[
NormalDistribution[0, 1], 3] + u + v t + w t^2, {t, -10, 10, 0.1}];
#/Total[#] & @ Eigenvalues@Covariance@x // First


This is what it looks like with 95% total variance (cumulative energy) in the leading principal component (vector in red):

If we were satisfied with the collinearity, we would proceed with alignment verification:

Abs[First@Eigenvectors@Covariance@x.#] & /@ {{1, 0, 0}, {0, 1, 0}, {0,
0, 1}}


The illustrated example returns {0.329372, 0.805966, 0.491867}, so we would conclude that the first principal component is not well-aligned with any of the axes.

-