There appears to be some confusion over this matter, so I will provide some observations and a pointer to where an excellent answer can be found in the literature.
Firstly, PCA and Factor Analysis (FA) are related. In general, principal components are orthogonal by definition whereas factors - the analogous entity in FA - are not. Simply put, principal components span the factor space in an arbitrary but not necessarily useful way due to their being derived from pure eigenanalysis of the data. Factors on the other hand represent real-world entities which are only orthogonal (i.e. uncorrelated or independent) by coincidence.
Say we take s observations from each of l subjects. These can be arranged into a data matrix D having s rows and l columns. D can be decomposed into a score matrix S and a loading matrix L such that D = SL. S will have s rows, and L will have l columns, the second dimension of each being the number of factors n. The purpose of factor analysis is to decompose D in such a way as to reveal the underlying scores and factors. The loadings in L tell us the proportion of each score which make up the observations in D.
In PCA, L has the eigenvectors of the correlation or covariance matrix of D as its columns. These are conventionally arranged in descending order of the corresponding eigenvalues. The value of n - i.e. the number of significant principal components to retain in the analysis, and hence the number of rows of L - is typically determined through the use of a scree plot of the eigenvalues or one of numerous other methods to be found in the literature. The columns of S in PCA form the n abstract principal components themselves. The value of n is the underlying dimensionality of the data set.
The object of factor analysis is to transform the abstract components into meaningful factors through the use of a transformation matrix T such that D = STT-1L. (ST) is the transformed score matrix, and (T-1L) is the transformed loading matrix.
The above explanation roughly follows the notation of Edmund R. Malinowski from his excellent Factor Analysis in Chemistry. I highly recommend the opening chapters as an introduction to the subject.