Prove that $XX^\top$ and $X^\top X$ have the same eigenvalues I'm not sure how to do the following exercise. Could someone help me by either suggesting the knowledge I should have to solve this, or pointing me out in the right direction?
I have tried to expand B and A by substituting X in the equations, and then trying to get the eigenvalues for each. But, for B, I got a 1x1 reduction, while for A, it's a N by N. 
When I tried to calculate the eigenvalues, A presented the greatest problem and that's where I got stuck.
But I think that I'm taking the difficult approach and the proof should be simpler.

 A: You will be surprised by how simple the problem is if you approach it from the right angle. Start from the definitions of eigenvalues and eigenvectors. It only takes a few manipulations afterwards to get where you want. Both problems can be solved almost on the spot.
To illustrate, for the matrix $\mathbf{A}^{\prime}\mathbf{A}$ we have by definition
$$\mathbf{A}^{\prime} \mathbf{A} \mathbf{x} = \lambda \mathbf{x} $$
Multiply both sides by $\mathbf{A}$ to get
$$\mathbf{A} \mathbf{A}^{\prime} \mathbf{A} \mathbf{x} = \lambda \mathbf{A} \mathbf{x}$$
but then we see that $\lambda$ is still an eigenvalue for $\mathbf{A} \mathbf{A}^{\prime}$ and the corresponding eigenvector is simply  $\mathbf{A} \mathbf{x}$. Your intuition was correct.
From this you can also get the relationship between the eigenvectors. Note that while we have equal eigenvalues, it would be unreasonable to expect equal eigenvectors as well. You can see the comment section below for a relevant discussion.
You can do the other case on your own.
