I just started to read about PCA in machine learning , and got to know that the main goal to determine principal components is to maximize variance so that more information is retained.But, why does more variance imply more information ? According to me , considering something like 'coefficient of unalikeability' would make more sense. For example, consider a data set having 2 features F1 and F2 , where
F1 = (0.1,0.2,0.3,0.4,0.5,0.6) ,variance = 0.03
F2 = (0.1,0.1,0.1,0.9,0.9,0.9) ,variance = 0.16
Now, here according to me F1 has more information than F2 while variance of F2 is more than F1.
Is there a flaw in my understanding of PCA?