0
$\begingroup$

enter image description hereI have a dataset of 40 variables and 55 samples. I want to run classification algorithm. Is this possible that I do PCA and based on which variables are more important in each principle component, use about 4-5 of my original variables?

let's assume that I choose the first two components, and the biplot I get for my pca (given that I have 3 independent variables) is like the figure below (in answer 1). Then it means that my first variable is the most important aspect of PC1 (with the higarhest negative coefficient), and my second variable is the most important variable of PC2. then in order to make a model that is easier to interpret can I use variables 1 and 2 to make my classifier?

$\endgroup$
  • 1
    $\begingroup$ What's your current understanding of what PCA does? Why do you think this technique you're describing would be helpful? These questions will give me more insight into what your use case is. $\endgroup$ – tchainzzz Jul 5 at 19:33
  • $\begingroup$ let's assume that I choose the first two components, and the biplot I get for my pca (given that I have 3 independent variables) is like the figure above (in answer 1). Then it means that my first variable is the most important aspect of PC1 (with the higarhest negative coefficient), and my second variable is the most important variable of PC2. then in order to make a model that is easier to interpret can I use variables 1 and 2 to make my classifier? $\endgroup$ – Parnian Jul 5 at 19:51
  • $\begingroup$ If you do that using the entire dataset and then run a classification evaluation like cross-validation, it’s not correct. You can apply the procedure on the training set of each round of the cross-validation. $\endgroup$ – user289381 Jul 5 at 21:30
0
$\begingroup$

In my opinion, yes you can use the variables that are most significant in constructing the respective factor scores, especially if you data is data-centered (minus the respective mean).

The most obvious advantage is, unlike factor analysis produced constructs, your explanatory variable is readily understandable. However, the standard regression derived beta coefficient may not be intuitively explainable (as in wrong sign,...). There is also a question do you actually get a better model specification as you may be missing variables that are, at least partially captured, in say a Factor Analysis approach.

The main issue in PCA/Factor Analysis is that do the factor constructs may any sense?

However, a potential advantage of a factor analysis relates to a possible conceptual discovery of driving factors themselves. This may suggest other variables to improve the modeling process. Also, the impact of missing important variables may be mitigated in a factor analysis as witness in a better fit.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.