0
$\begingroup$

considering that $2$ vectors such as $x_2=\begin{bmatrix}1 & 1 \end{bmatrix}$ and $y_2=\begin{bmatrix} -1 & 1 \end{bmatrix}$ are orthogonal in $2D$ (i.e. their scalar product is $0$) however when we consider a third (in $3D$ ) non-zero component i.e. $x_3=\begin{bmatrix}1 & 1 & 1 \end{bmatrix}$ and $y_3=\begin{bmatrix} -1 & 1 & 1 \end{bmatrix}$, they are not anymore orthogonal in the sense of the scalar product because $=1$ this time and not $0$. However these vectors are indeed "at right angle in $3D$" so to speak. I would have the impression that they are just like in the $z=1$ plane... but maybe i m missing a point here ...? I hope i m not forgetting fundamental things. Please can someone help me clarify this, think about this...

(Also, it's linked to a question w.r.t. principal compenents. Given a $4 \times 4$ matrix $E$ of eigenvalues of a covariance matrix $C$, considering all eigenvectors (columns orthogonal) why can't we just plot the two $2D$ principal compononents as the $2$ columns of the truncated $2\times 2$ upper matrix of $E$? (i tried for fisher iris the $2\times 2$ submatrix of eigenvectors of the covmatrix, the $2 \times 1$ "sub vectors" are not orthogonal))

What about my question about the eigenvalues/Principal components?

Improve this question
$\endgroup$
7
  • $\begingroup$ I think the misunderstanding stems from the statement "these vectors are indeed 'at right angle in 3D' so to speak," which is simply false. Using the well known identity $\cos \theta = \frac{x^\top y}{||x||\cdot||y||}$ for $\theta$ the angle between $x$ and $y$, you can show this quite easily. $\endgroup$ – Sycorax♦ Jan 28 '18 at 18:04
  • $\begingroup$ @Machupicchu Take two pens. Put them on your desk so that they are orthogonal. Now slowly start increasing the angle of one of the pens toward the third dimension so that it points to $(1,1,1)$. The pens are still orthogonal. Now start increasing the angle of the second pen so that it points to $(-1,1,1)$. You can see that the angle between the two pens is decreasing! $\endgroup$ – Andreas G. Jan 28 '18 at 18:19
  • $\begingroup$ Thanks for your answers. But i think my problem is to know (when in 3D) with respect to which axis is the vector being rotated. for example: if i rotate 90degrees (pi/2 radians) along Z axis the vector [1 1 1] i get : [-1 1 1] which is at about 70° from [1 1 1] according to the formula $\cos \theta = \frac{x^\top y}{||x||\cdot||y||}$ I think i cannot visualize (anymore) what is happening when it comes to 3D ans its very frustrating... For me if i rotate along Z by 90 degrees the vectors should be at 90 degrees...? but again i must be missing something basic i guess? $\endgroup$ – Machupicchu Jan 28 '18 at 20:28
  • $\begingroup$ P.S: is it possible to send personnal messages to members of the forum? (i don't see an option to do so) $\endgroup$ – Machupicchu Jan 29 '18 at 0:37
  • $\begingroup$ @Machupicchu No I don't think it is possible to send PMs. So if you start with the pens in an orthogonal position on your desk, you should rotate one of the pens towards the direction that would eventually get the pen completely vertical. But do not rotate all the way, rotate for about 45 degrees. That should get you to $(1,1,1)$. The pens are still orthogonal because one of them is still on the desk. Now if you do the same with the other pen, you will see the angle becoming smaller $\endgroup$ – Andreas G. Jan 29 '18 at 13:08
1
$\begingroup$

After carefully reading your statements, I think the aspect you are missing is that the vectors are all with respect to the origin. So, you can think of your original vectors as representing the displacement between the points $(0,0,0)$ and $(1,1,0)$ and between $(0,0,0)$ and $(-1,1,0)$.

The new vectors $(1,1,1)$ and $(-1,1,1)$ would be orthogonal if you moved the origin to the point $(0,0,1)$. That is in the $z=1$ plane as you noted. But that's not how we think about them. Think about a pair of lines going from $(0,0,0)$ to the two points $(1,1,1)$ and $(-1,1,1)$ respectively. It should make more sense why they aren't orthogonal.

For some intuition about this, take your thumb and pointing finger and put them at a right angle. Close one eye so that it looks more 2D, now move your hand around, does the 2D picture always look like they are orthogonal? It doesn't always right? Yet you know they are orthogonal.

This addresses your question principal components as well. You can't necessarily make a new pair of orthogonal vectors by dropping some of the coordinates from known pair of orthogonal vectors.

Improve this answer
$\endgroup$
5
  • $\begingroup$ yes i see! indeed i started have this intuition about the imossibility to simply trucate an othrog high dim vector to get a 2d vect that would be orthog to others. However could you exmplai how i can get these both 2D vectors (PC1 PC2) in 2D for principal components ? (i have a hard time finding this information, i guess for many people in the field it must be obvious?) $\endgroup$ – Machupicchu Jan 29 '18 at 20:18
  • $\begingroup$ If you want to see the PCs in 2D, just drop all the other coordinates for the PCs like you were originally doing. The point is they won't necessarily be orthogonal in 2D just because they are orthogonal in higher-dimensions. $\endgroup$ – Kareem Carr Jan 30 '18 at 3:37
  • $\begingroup$ hum i see what you mean however that seems to conflict with the statement that all the axes must be mutually orthogonal in all dimensions? $\endgroup$ – Machupicchu Jan 30 '18 at 13:37
  • $\begingroup$ moreover think of the example of a cloud of points in 3D that looks like a "fish". the depth can be dropped and you remain with 2 PCs that are orthogonal in the plane (the most variance if "you draw a fish" is in the lenth and height"... do you see what i mean? $\endgroup$ – Machupicchu Jan 30 '18 at 13:40
  • $\begingroup$ no , am I not pointing something here or am I missing something? Do you see my problem? $\endgroup$ – Machupicchu Jan 31 '18 at 15:40
1
$\begingroup$

Reading your comments, I think your confusion comes from mixing up rotation of a vector along any axis, and creating a vector at some fixed angle to another vector.

Note that rotating a vector in 3D (or any dimension higher than that) with $\alpha$ degrees does not necessarily give you a vector which is $\alpha$ degrees from the original vector. To see this, simply assume a vector along the Z-Axis (e.g. (0,0,1)) and rotate this vector by any amount. You will always get the same vector no matter how much you rotate, so the angle will always be $0$. In mathematical terms, this is shown by the fact, that rotations in higher dimensions always must have fixed points, i.e. axis along which they do not change (Euler's rotation theorem).

This means, whenever you rotate a vector along any axis, and this vector has any "part" of this axis (i.e. it is a linear combination of this axis and another vector), then only the other part is rotated, whereas the part along the axis does not get rotated. Therefore, the angle is reduced.

So about your question along which axis to rotate a vector (in 3D) by $\alpha$ degrees to get a new vector at an angle of $alpha$, the answer is simply: Rotate along any axis, which is orthogonal to the original vector (this may not necessarily be a coordinate axis, as none of them may be orhtogonal). Since you can use any axis, this also means, there is a whole infinite number of vectors at the given angle. This only happens at spaces with at least 3 dimensions, as the orthogonal vector is unique up to multiplication with a scalar for 2D.

Improve this answer
$\endgroup$
2
  • $\begingroup$ thanks for your answer! it is indeed the kind of explanation i need. I find it quite hard to "think" in 3D. $\endgroup$ – Machupicchu Jan 29 '18 at 19:47
  • $\begingroup$ thanks for your answer! it is indeed the kind of explanation i need. I find it quite hard to "think" in 3D. Indeed you example about rotating [0 0 1] around Z axis is quite informative for me. I observe that i always get the same vector i.e. [0 0 1] ! So if i understand correctly, it's like if it is "stuck", "glued" around Z axis, right? $\endgroup$ – Machupicchu Jan 29 '18 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.