I have a classifier that I'm doing cross-validation on, along with a hundred or so features that I'm doing forward selection on to find optimal combinations of features. I also compare this against running the same experiments with PCA, where I take the potential features, apply SVD, transform the original signals onto the new coordinate space, and use the top $k$ features in my forward selection process.
My intuition was that PCA would improve the results, as the signals would be more "informative" than the original features. Is my naive understanding of PCA leading me into trouble? Can anyone suggest some of the common reasons why PCA may improve results in some situations, but worsen them in others?