-1
$\begingroup$

I am trying to model credit risk for retail loans, with my dependent variable being a default flag. I have 15 independent factors, which I need to cut down, using principal component analysis. While I have conducted PCA, I am not really sure as to how do I identify the final variables based on the principal factors generated. The PCA output results in generation of 7 principal factors, as also component matrix containing the Coefficient of Principal Component Score of Variables. Can I group the variables having high correlation for each of the principal factors and use these for regression (say log regression)? If the preceding statement is correct, how do I identify the most relevant variable within each group that I can further shortlist for regression.enter image description here

enter image description here

Edit: Thanks for the prompt revert guys, really appreciate it. Let me state some more details I am actually working with a very large number of variables (92 to be precise) and hence using a correlation matrix may be very tedious. I essentially what to find out which variables have similar corelation to default and hence select one of these, from various groups of related variables. And in turn, use the most suitable set of original variables (not factors of variables) for log regression. Is there an alternative approach to achieve the same, if not PCA.

@amoeba: I not trying to use the original variables here, while the similar thread you shared does not really answer what I am asking.

Improve this question
$\endgroup$
3
  • $\begingroup$ Your question does not seem to tell us why you want to reduce the number of variables. If you goal is prediction would it not be better to include as much information as possible? $\endgroup$
    – mdewey
    Feb 28, 2018 at 13:43
  • $\begingroup$ There are other ways to approach this, some much simpler than what you seem to be trying. For example, use a correlation matrix to decide what you can leave out as variables highly correlated with others; use an initial regression to indicate what doesn't help (often frowned upon, but can be effective); a principled selection so that you don't include predictors with similar meaning. $\endgroup$
    – Nick Cox
    Feb 28, 2018 at 14:23
  • 1
    $\begingroup$ What you seem to be using is some flavour of factor analysis because otherwise why talk of selection, rotation or "principal factors". Note that many would not regard that as PCA at all. Prejudices apart, it is a very complicated technique to use if you don't understand it well. Don't assume that software defaults correspond to what you most need. $\endgroup$
    – Nick Cox
    Feb 28, 2018 at 14:26

1 Answer 1

Reset to default
0
$\begingroup$

When you use PCA, you are creating new variables which are linear combinations of your original variables. So, instead of using the original 15 variables, you use the 7 components. The rotated component matrix tells you how to create the new components using your original variables. For example, the first component is given by: $$ Y_1 = 0.724 V9 + 0.695 V6 + 0.619 V11 + 0.13 V1 + \cdots + 0.120 V5 + 0.294 V15 $$ You then use these components in your regression model.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ Yes, except that any software worth discussing here will have some means to do this calculation of PC scores for you. $\endgroup$
    – Nick Cox
    Feb 28, 2018 at 14:29

Not the answer you're looking for? Browse other questions tagged or ask your own question.