Often, a variable is considered to be significantly loaded on a PC if its loading value in the loading table is above a cut off value (suppose 0.4 or 0.5 in some published cases). Is there any statistical/mathematical method to check whether a variable is significantly loaded on a particular PC or not?

  • 3
    $\begingroup$ (One note: if you consider a "loading", it is how a component loads a variable, not vise versa). The problem here is that a component is a pure function of the variables, not something external modelled with error (as in regression). However, if you have population of correlated data and do samplings which you expose to PCA, sampling distributions for loadings can be obtained. $\endgroup$
    – ttnphns
    Nov 13 '13 at 17:36
  • $\begingroup$ Thanks ttnphns for the reply. I guess I could not convey the question well. Ok i try again. Usually, A variable is said to load well on a principal component if its correlation with it is above 0.40. Now (rather than using cut off value of 0.4) is there some statistical method to decide whether a variable loads well on PC or not? $\endgroup$
    – mzalikhan
    Nov 13 '13 at 18:13
  • 2
    $\begingroup$ I am not sure that it is meaningful to speak about significant loadings in a non-specific sense: before extraction all variable variance is fully distributed across factors, so what would it mean to be significantly loaded? If you choose fixed thresholds than these would have to be realtive to the number of factors. The more interesting question is: why is it useful in your view to have this threshold? What is your motive to classify variables in this way? $\endgroup$
    – jank
    Nov 13 '13 at 20:01
  • $\begingroup$ In other words, When the resultant PCs are discussed in the PCA report, it is said that PC1 represents temperature, humidity and wind speed because PC1 has significant loadings of these three variables (loading greater than 4 are considered as significant). (here temperature, humidity and wind speed are 3 of the original variables which were input to PCA). My question is, rather than concluding on the basis of cut off value of 0.4, is there some proper method to determine which of the original variables PC1 actually represent? Thanks. $\endgroup$
    – mzalikhan
    Nov 14 '13 at 6:46
  • 1
    $\begingroup$ You can test a variable's correlation with a variable that is the mean or weighted mean of certain other variables. But as @jank said, it's important to think about your purpose in doing so. Typically PCA is used to reduce many variables into a few, and exploratory factor analysis (EFA), to (arguably) do the same thing while also uncovering latent dimensions inherent in the data. PCA and EFA involve so many decision points that there is no truly standard procedure, for testing loadings' significance or for other aspects of the process. $\endgroup$
    – rolando2
    Nov 14 '13 at 20:48

Significance tests are not merely indicators of "how strong" or "how much." Significance tests serve to contrast a result with the sort of results that typically occur through some known random process. How large would a group difference be if the groups were merely assigned at random...How large would the correlation between two variables be if in the population those variables were unrelated to one another, and if their relationship in a sample were thus purely the result of random error.

Now consider your situation: variables load on some principal component that is formed through a particular procedure -- moreover, one that, as @ttnphns and @jank have said, explicitly "seeks" to form a component that summarizes individual variables. There is no random process with which to contrast this, no process that might be otherwise occurring that would furnish us with a null hypothesis. This is further reason why it's not meaningful to test the statistical significance of a loading on a PC.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.