1
$\begingroup$

I have data, when I normalize it and then performed PCA, I calculated the variance of PC components, I found that, the first component is 72% and seconed component is 8% (total 72+8=80%) and so on.

Now I used to the same data and performed some data cleaning function and then normalize it and then performed PCA, calculated variance of the PCA components I found that, first component gives 40% and second component gives 20% (40+20=60%).

What does this mean? Which one is better? I found that, after data cleaning operation, the classifier gives highest accuracy. I am unable to have a sense what is happening exactly and why? Would anyone explain?

$\endgroup$
  • $\begingroup$ That very likely means that your uncleaned data had some outliers and that your initial PCA solution was much driven by those outliers (they pulled the first component towards them; or, because you did z-standardizing which changes the shape of the data cloud, they might have pulled the component away from them). $\endgroup$ – ttnphns Jun 9 '18 at 7:43
0
$\begingroup$

This situation is typically conspicuous when you perform data cleaning, you must be clear about the data structure (highly balanced or unbalanced) before performing it. After all, it means that using the PCA before data cleaning tend to explain more variability of the data with less dimension.

To answer your question more precisely, could you tell me what classifier(s) are you using?

By the way, the formula of variance captured by component k is:

$$\frac{\lambda_k}{\sum_{i=1}^n \lambda_i}$$ ,where $\lambda_1 \ge \lambda_2 \ge......\ge \lambda_n \ge 0$ is the variance captured by each component.

$\endgroup$
  • $\begingroup$ I am using SVM classifier. $\endgroup$ – M Arjun Jun 9 '18 at 5:53
0
$\begingroup$

For classifiers the critical issue is the ratio of variance within groups to variance between group means. With SVM the shape of the groups can be more complex than a simple linear discriminant analysis but the same principle holds (SVM can be viewed as remapping a complex group shape into a simple one).

What this means is that overall variance across the dataset can't predict performance in a classifier. If the data cleanup has removed irrelevant variations then the classifier should work better. If the variations removed by the cleanup are relevant to the classier it will be worse.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.