1
$\begingroup$

I have got some data regarding some tv series episodes characteristics, like the number of characters or of cuts, and I have been adviced to try out PCA in order to find out the characteristics this movies have in common. Reading online, I have not understood any reason other than feature reduction, which I don't really think I need. However, after running PCA this are the results I get: PCA And these two plots say how the variance of each dimension is explained, I guess: first second Now, what I'd like to know is: how can this be useful in my case? Does this provide me with anything interesting I am not seeing? Thanks a lot in advance

$\endgroup$
3
  • 1
    $\begingroup$ Does this answer your question? Making sense of principal component analysis, eigenvectors & eigenvalues That shows how PCA can summarize multiple related characteristics into fewer groups to simplify. For example, in your case the first principal component mostly has to do with reviews and the second with characteristics of words and sentences. $\endgroup$
    – EdM
    Commented Jun 20, 2022 at 20:59
  • $\begingroup$ @EdM Thank you a lot for your interest. I understand what you mean with the interpretation of the dimensions, but at the end, what do I get from this? I mean yes, the second component has the characteristics of words and sentences, but how does that add some information I could not have got before PCA? $\endgroup$
    – Jonathan
    Commented Jun 20, 2022 at 21:17
  • 1
    $\begingroup$ What you get from PCA is a way for the data themselves to tell you which individual variables tend to vary together and thus can be effectively grouped together. In your case those groupings might not be too surprising, but that's not always the case. And you can't get the same information by just doing things like pairwise correlations--PCA is a multivariate technique that evaluates all the variables at once. $\endgroup$
    – EdM
    Commented Jun 21, 2022 at 14:27

1 Answer 1

4
$\begingroup$

Principal components are mutually orthogonal. For linear regression, this means that they are entirely independent of each other, and there will be no hint of collinearity and the standard error variance inflation that comes with that. Consequently, regression performed on PCs will have coefficients with the smallest standard errors, and that will result in the smallest confidence intervals for your response variable estimates.

The question is not "why should I use PCs?" Rather, it is, "why don't I use PCs for absolutely every regression problem I ever face?" -- which has a longer and more complicated answer. That is, there are reasons why you shouldn't use PCs for every problem. But you didn't ask that question, so I'll spare you. ;)

The reduction of feature space is not my favorite reason to use PCs. Finding, understanding and ultimately exploiting covariances between features is.

For example, an athletic director may sort student athletes by height and weight, and may suggest team sports appropriate for a height-weight classification system. But really, the students are better sorted by "size" (big/little) and "shape" (fire-plug/bean-pole) which could be the names of principal components based on height and weight. No dimensional reduction, but the rotation in feature space perhaps makes it easier to recommend team sports to students.

But then we wouldn't have a 5'6" Spud Webb playing basketball in the NBA or 6'9" Kyle Hudlin playing English Premier League soccer(center forward, not a goal keeper !?) That is, there are always more features to consider -- like passion, and know-how.

$\endgroup$
5
  • 1
    $\begingroup$ How can you justify the "consequently" in your first paragraph? You seem to be overlooking that standard errors depend on more than linear relationships among the explanatory variables: they also depend on how well those explanatory variables are linearly associated with the response. In particular, when you choose PCs that happen to be uncorrelated with the response, there is no reduction in residual error, leading to large standard errors of the coefficients, not smaller ones. $\endgroup$
    – whuber
    Commented Jun 20, 2022 at 22:33
  • $\begingroup$ @whuber, thank you for your comment. It is certainly true that uncorrelated "predictors" will not be improved by PCA. I should tighten/formalize my language on parameter standard error reduction to make plain that the improvement is greater than or equal to zero. $\endgroup$ Commented Jun 21, 2022 at 3:16
  • $\begingroup$ @PeterLeopold Thank you very much for your answer, it really is precious for me. You said that your favourite reason to use PCA is to exploit covariances between feature, but can't you do this without PCA too? Also, I think the athletic director example is great, and I am trying to understand if there is such an obvious relation between variables inside my components, but I am not sure. Someone stated in one comment that "For example, in your case the first principal component mostly has to do with reviews and the second with characteristics of words and sentences.". Do you agree with this? $\endgroup$
    – Jonathan
    Commented Jun 21, 2022 at 10:27
  • 1
    $\begingroup$ Yes. Now the challenge is to name your PCs and use the names meaningfully. You might call PC1 Awareness, and PC2 Verbosity. Now your "mission" is to make the regular measurements on a new sample, compute Awareness and Verbosity using the PC1 and PC2 recipes, then make a prediction based on your ML analysis of these values. In time your colleagues will start using the names, too. That's progress! It will be like inventing the notions of Color, Cut, Clarity, & Carat for diamond pricing based on engineering notions of occlusion density, optical dispersion, etc. $\endgroup$ Commented Jun 21, 2022 at 21:33
  • $\begingroup$ Thank you a lot @PeterLeopold $\endgroup$
    – Jonathan
    Commented Jun 22, 2022 at 10:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.