Are PCA solutions unique? When I run PCA on a certain data set, is the solution given to me unique? 
I.e., I obtain a set of 2d coordinates, based on interpoint distances. 
Is it possible to find at least one more arrangement of the points that would meet these constraints?
If the answer is yes, how can I find such different solution?
 A: No, the answer is not unique. There are many ways to show this. One possibility is to notice that spectral decomposition of a square $p$ by $p$ matrix $X$ is the solution to the maximization of a convex function of $w$. Consider the first eigen-vector/value:
$$\lambda_1=\underset{w\in\mathbb{R}^{p}:||w||=1}{\max} w'Xw$$
(where $\lambda_1$ is the first eigen-value and $w^*$ the first eigen-vector).
The solution to such problems (e.g. the values of $w$ attaining that maximum) are, in general, not unique. 
However the algorithms for computing these solutions are deterministic, meaning that  save for numerical corner cases, the solutions you get should be the same.
Example of such numerical corner cases: cases where several eigen-values are (numerically) the same, cases where the $X$ is rank-deficient...
A: Something that hasn't been noticed yet is that simply reversing the sign of a PC produces a different solution. That is, if $\mathbf{w}$ is the $n$th principal component, then $-\mathbf{w}$ is also a solution to the $n$th principal component. This has caused confusion before, especially when your computer outputs alternating PCs. See this question.
A: 
It depends.
If the eigenvalues of the covariance matrix are different, then the PCA is unique. Else not.


The fact that the variances of the principal components are given by λi has
an important implication for the uniqueness of PCA. If two of the eigenvalues
are equal, then the variance of those principal components are equal. Then,
the principal components are not well-defined anymore, because we can make a
rotation of those principal components without affecting their variances. This is
because if
p
zi and zi+1 have the same variance, then linear combinations such as
1/2zi +
p
1/2zi+1 and p
1/2zi −
p
1/2zi+1 have the same variance as well; all
the constraints (unit variance and orthogonality) are still fulfilled, so these are
equally valid principal components. In fact, in linear algebra, it is well-known
that the eigenvalue decomposition is uniquely defined only when the eigenvalues
are all distinct.

Source: Principal component analysis; Aapo Hyvärinen; Based on material from the book Natural Image Statistics,2009; https://www.mv.helsinki.fi/home/amoaning/movies/uml/pca_handout.pdf
Or

PCA is unique up to signs, if the eigenvalues of the covariance matrix are different from each other.


Is PCA unique or not, that is, is there only one PCA solution. Multiple solutions may fulfill the PCA criteria. We consider the decomposition
X = Y UT
where U is orthogonal, Y T Y = Dm with Dm as m-dimensional diagonal matrix, and the eigenvalues of Dm are sorted increasingly.

Source: Machine Learning: Unsupervised Techniques; 2014; Sepp Hochreiter; Institute of Bioinformatics Johannes Kepler University Linz
