The difference between Feature selection and PCA to improve accuracy

Question

I have a dataset of 21 features where each numeric feature has various scale something like the following:

budget  title_year  actor_2_facebook_likes  imdb_score  aspect_ratio    movie_facebook_likes
237000000   2009    936 7.9 1.78    33000
300000000   2007    5000    7.1 2.35    0
245000000   2015    393 6.8 2.35    85000
250000000   2012    23000   8.5 2.35    164000
263700000   2012    632 6.6 2.35    24000
258000000   2007    11000   6.2 2.35    0
260000000   2010    553 7.8 1.85    29000
250000000   2015    21000   7.5 2.35    118000
250000000   2009    11000   7.5 2.35    10000
250000000   2016    4000    6.9 2.35    197000
209000000   2006    10000   6.1 2.35    0
200000000   2008    412 6.7 2.35    0

I applied several classifiers such as SVM, KNN, Logistic Regression, and Random Forests. However, I am getting accuracy around 60s even after adjusting the parameters of each classifier. Thus, I decided to work more with the dataset and make further preprocessing (specifically, normalization and feature selection). Kindly, I have two questions:

1. Which one is the correct way and why: Performing feature selection after or before normalization?
2. I understand PCA is a dimensionality reduction but I don't understand what is the difference between PCA and feature selection as both will eventually rank the features for you.

Thank you very much.

Answer 1

You are framing this question as though feature selection and PCA are exclusive alternatives - pick one and you can't pick the other. This is not the case. Feature selection is critically examining your features and deciding that one or more don't contain useful information. PCA is an algorithm which can be used to transform two or more features that perform badly in your model due to possible mutual correlation into a single feature that performs more effectively. Twenty-one features is a large enough set of features that when you examine them you could find instances where each is appropriate. Hence it is more a case of understanding your features, how they relate to each other and the dependent variable before deciding the correct next step.