Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

Cross-posted from math; let me know if you need me to wait there for a bit before posting here

I have a set of datapoints in $\mathbb{R}^3$ (integer or real values of $X$, $Y$ and $Z$, for, say. 30 points).

I need to do a least squares regression for linearity. If the data set is linear, I need to see if it is close to vertical or horizontal. How could I do this?

share|improve this question
Do you mean your dependent variable is 3-dimensional? – Macro May 11 '12 at 1:53
Crossposting is discouraged - I've flagged your post for migration. It seems to me to be more relevant here than on the math stackexchange. – Macro May 11 '12 at 1:56
@Marco, no, I mean all my datapoints have 3 dimensions (x y z). If I understand you correctly thats not what you meant – jamesson May 11 '12 at 2:14
Can you edit your question and write out the model you're trying to fit? – Macro May 11 '12 at 2:15
@Macro, hows that? – jamesson May 11 '12 at 2:24
show 4 more comments

2 Answers

up vote 1 down vote

Collinearity

If all the points are collinear, we should be able to explain almost the variance in the sample by one vector. In other words, the leading principal value should be significantly greater than the other two (since we are working with a total of three dimensions). How much greater it should be is up to you to decide. I would say well above 95%, say 98% or 99% of the total variance, should be explained by the leading principal component.

Alignment with the axes

Once you have calculated the leading principal component, it is a simple matter of taking its dot product of the unit vectors along all three axes to determine the alignment. The closer the absolute value of the dot product is to unity, the more aligned the principal component is to that axis.

Mathematica Example

I'm going to generate points that are randomly dispersed around a randomly-generated quadratic curve. The greater the coefficient of the highest-order term, the tighter the curve, and the closer the points are to being collinear.

v = RandomReal[{-1, 1}, 3]; v /= Norm[v]; 
u = RandomReal[{-1, 1}, 3]; 
w = RandomReal[{-1, 1}, 3];
x = Table[RandomVariate[
 NormalDistribution[0, 1], 3] + u + v t + w t^2, {t, -10, 10, 0.1}];
#/Total[#] & @ Eigenvalues@Covariance@x // First

This is what it looks like with 95% total variance (cumulative energy) in the leading principal component (vector in red):

Principal component with 95% total variance.

If we were satisfied with the collinearity, we would proceed with alignment verification:

Abs[First@Eigenvectors@Covariance@x.#] & /@ {{1, 0, 0}, {0, 1, 0}, {0,
0, 1}}

The illustrated example returns {0.329372, 0.805966, 0.491867}, so we would conclude that the first principal component is not well-aligned with any of the axes.

share|improve this answer
up vote 0 down vote

Although the model is not very clear it seems that you are saying that if a linear regression seems to fit then you are asking a question about the slope of the regression line. If the line id horizontal that would mean the slope is 0. If it were horizontal the slope is infinite. So test the null hypothesis than the slope is 0 versus the alternative that it is positive. If you reject you casn conclude that it is not horizontal. If you can't reject the data suggests that the slope might be horizontal. If you want to test that the slope is nearly infinite you could set the null hypothesis that the slope is a large posItIve value B versus a one-sided alternative that it is > than B.

share|improve this answer
I had imagined this, but how would I even compute slope in 3space? Moreover wouldn't anything with zero slope fall on the z plane regardless of whether or not they're parallel to the xz axis? Re model being unclear, what other info do you need? More importantly, how do I do the linear regression in the first place? – jamesson May 11 '12 at 2:35
Re model being unclear, what other info do you need? – jamesson May 11 '12 at 2:41
2  
What is the regression equation? Is it Y=aX +bZ +c? Or is one of the other variables the dependent variable? Are X, Y, Z each 1 dimensional or 3 dimensional? – Michael Chernick May 11 '12 at 2:59
X, Y, and Z are each one domensional. The regression equation is the one you gave. – jamesson May 11 '12 at 3:04
1  
In that case a and b are the slopes. This is ordinary linear regression. Are you trying to show that the slopes a and b are 0 or infinite or does this have to do with the plane determined by X and Y? If it is a and or b there are standard tests for whether or not these slopes are 0 or a fixed specified value with the corresponding t tests. You can't test that a slope is infinite but can test that the slope is some large value B versus the one-side alternative that it is larger than B. – Michael Chernick May 11 '12 at 4:33
show 1 more comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.