0
$\begingroup$

Consider a new variable that is linear combination of original variables (e.g. one of the principal components). How can we find out which original variables "product" the new variable?

$\endgroup$
5
  • 1
    $\begingroup$ Could you elaborate please ? Do you mean you have a set of variables, a linear combination of some of these variables, but you don't know which ones ? $\endgroup$ Commented Aug 14, 2014 at 8:58
  • $\begingroup$ yes, I have one variables that is combination of some other variables. now, we want to know which variables is used for construct new variable. $\endgroup$
    – sara
    Commented Aug 14, 2014 at 9:15
  • $\begingroup$ Could you edit your post to add that ? And what is the link with PCA ? If your linear combination comes from a PCA analysis, why do you not know what are the variables ? $\endgroup$ Commented Aug 14, 2014 at 9:25
  • $\begingroup$ my linear combination is production of sufficient reduction analysis. $\endgroup$
    – sara
    Commented Aug 14, 2014 at 9:39
  • $\begingroup$ in PCA analysis, how do you know what are the variables? $\endgroup$
    – sara
    Commented Aug 14, 2014 at 9:40

2 Answers 2

4
$\begingroup$

If you have a number of variables $\{x_1, x_2, \dots x_N\}$ and you consider a new variable that is a linear combination of the original ones: $$y=\alpha_1 x_1 + \alpha_2 x_2 + \dots + \alpha_N x_N = \sum \alpha_i x_i$$ Then, obviously, all original variables are "used to construct" the new variable (unless some of the coefficients $\alpha_i$ are equal to zero). However, if some coefficients are $\approx 0$, then you can say that the new variable "mostly" depends only on a subset of the original variables --- on those that have large coefficients.

When you perform PCA, your e.g. first principal component is a linear combination of the original variables. So what you want to do, is to look at the coefficients $\alpha_i$ and see which ones have large absolute values (and which ones are close to zero).

How to actually do it, depends on your software, programming language, etc.

$\endgroup$
7
  • $\begingroup$ Sufficient reduction analysis image predictors on subspace that is smaller than predictor's subspace. so in fact, we have new predictors that is combination of initial predictors. now what is your idea about this? can you help me for finding coefficients that are equal to zero? $\endgroup$
    – sara
    Commented Aug 14, 2014 at 9:59
  • $\begingroup$ I don't know what "sufficient reduction analysis" is; what is its relation to PCA? In any case, if your new variables are linear combinations of original variables, then you should be able to look at coefficients of the linear combination ("loadings" in PCA terminology). Then you can check what coefficients are zero, or close to zero. Is your problem that you cannot access these loadings? Then it becomes a question about your particular software / programming language. $\endgroup$
    – amoeba
    Commented Aug 14, 2014 at 10:05
  • $\begingroup$ @sara: You are welcome, but I am not sure if I clarified the issue for you... If not, please edit your question to provide additional information and specify your problem better (if it refers to particular software or language, you should mention that). If yes, consider "accepting" the answer by clicking on a green tick sign nearby. And welcome to CrossValidated! $\endgroup$
    – amoeba
    Commented Aug 14, 2014 at 10:20
  • $\begingroup$ Going back to our conversation, I'd like to discourage people to throw about with word "loading" everywhere. For me, loading is a coefficient in linear combination of scoring observed variables by latent variable(s), not vice versa (vice versa are "weights"). With PCA's unaltered eigenvectors, their elements are weights=loadings. In other settings and analyses weights and loadings usually differ, although they can be easily transformed one in another. My terminologic "purism" is not about maths, it is for didactics. $\endgroup$
    – ttnphns
    Commented Aug 14, 2014 at 11:26
  • 1
    $\begingroup$ @ttnphns, in fact I find the word "loading" only confusing, so I would rather refrain from using it whenever possible (hence I was happy to make the edit). My background is math and machine learning, and the word "loading" is not much used there, it comes more from the psychology literature on factor analysis. $\endgroup$
    – amoeba
    Commented Aug 14, 2014 at 11:54
-1
$\begingroup$

After having viewed the loadings in each factor, I use PCA as a tool to propose how to construct a new variable (that could be used elsewhere) that intuitively encompasses a shared attribute that have been outlined by a purely statistical technique. I do not recommend embracing a new factor without understanding why those variables could be possibly associated with the underlying factor. Even then, replication with new data, is needed to verify the new construct.

$\endgroup$
1
  • 1
    $\begingroup$ -1. How does this answer the original question? $\endgroup$
    – amoeba
    Commented Dec 17, 2014 at 0:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.