Ian Jolliffe discusses this on p.viii of the 2002 second edition of his Principal Component Analysis (New York: Springer) -- which, as you can see immediately, jumps one way. He expresses a definite preference for that form principal component analysis as similar to say factor analysis or cluster analysis and cites evidence that it is more common any way. Fortuitously, but fortunately for this question, this material is visible on my local Amazon site, and perhaps on yours too.
I add that the form independent component analysis seems overwhelmingly preponderant for that approach, although whether this is, as it were, independent evidence might be in doubt.
It's not evident from the title but J.E. Jackson's A User's Guide to Principal Components (New York: John Wiley, 1991) has the same choice.
A grab sample of multivariate books from my shelves suggests a majority for the singular form.
An argument I would respect might be that in most cases the point is to calculate several principal components, but a similar point could be made for several factors or several clusters. I suggest that the variants factors analysis and clusters analysis, which I can't recall ever seeing in print, would typically be regarded as non-standard or typos by reviewers, copy-editors or editors.
I can't see that principal components analysis is wrong in any sense, statistically or linguistically, and it is certainly often seen, but I would suggest following leading authorities and using principal component analysis unless you have arguments to the contrary or consider your own taste paramount.
I write as a native (British) English speaker and have no idea on whether there are arguments the other way in any other language -- perhaps through grammatical rules, as the mathematics and statistics of PCA are universal. I hope for comments in that direction.
If in doubt, define PCA once and refer to that thereafter, and hope that anyone passionate for the form you don't use doesn't notice. Or write about empirical orthogonal functions.