PCA on Identity Matrix Just for fun, I did a PCA on Identity Matrix with Python and got the following output: 
Python 2.7.10 (default, Oct  6 2017, 22:29:07) 
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from sklearn.decomposition import PCA #PCA Package
>>> ls=[[1,0],[0,1]]
>>> pca=PCA()
>>> res=pca.fit(ls)
>>> res.explained_variance_
array([  1.00000000e+00,   2.81351049e-34])
>>> pca.explained_variance_ratio_
array([  1.00000000e+00,   2.81351049e-34])
>>> pca.components_ 
array([[-0.70710678,  0.70710678],
       [-0.70710678, -0.70710678]])

I understand that Identity Matrix has 1 as eigenvalue and any vector as eigenvectors. I just don't get why the python returned a eigenvalue of 2.81351049e-34 there with a eigenvector  [-0.70710678, -0.70710678]. What does it mean? 
 A: PCA is not solving the eigenproblem on the data itself, but rather on the correlation matrix of the data.
The correlation matrix of the identity matrix is
$
cor \Big( \begin{bmatrix}
1 & 0 \\
0 & 1 
\end{bmatrix} \Big)
=
\begin{bmatrix}
1 & -1 \\
-1 & 1 
\end{bmatrix}
$
But clearly there is only one component here: $[1 -1]^T$ because the second column is simply the first times $-1$. Normalize to get 
$\begin{bmatrix} 
  \frac{1}{\sqrt(2)} \\ 
  \frac{-1}{\sqrt(2)} 
\end{bmatrix}$
Which is the first basis vector of pca.components_ in your code. To get the second basis vector, we choose the only vector orthogonal to the first. 
Note that our original data, the identity matrix, actually exhibits perfect multiple colinearity. Because only the first basis vector is needed to represent our data, it gets a loading and an explained variance of 1, while the other gets 0. The fact that it is not exactly zero is simply due to numerical imprecision; it is simply not possible to represent $\sqrt{2}$ exactly as a floating point number so when we try we are off by an small amount. That's where the 2.8e-34 in your code comes from.
A: I see what is the reason now. Thanks to @Aksakal , @amoeba, and olooney. 
So sklearn actually first calculate the covariance matrix of the identity matrix, which is 
1/2, -1/2
-1/2, 1/2 

And then, it calculates the eigenvalues and eigenvectors of the covariance matrix, which is the PCA results. 
I mistakenly thought that the results that I feed into the PCA data is the covariance matrix. 
Also, as mentioned by @olooney, because of perfect multi-colinearity, the explained variance of the second term is almost 0. 
The result is mostly driven by rounding errors. 
However, I must point out that sklearn uses covariance matrix to calculate PCA results by default, no correlation matrix. 
