I have 37 features in my dataset. I used Boruta package in R and according to its analysis, all the features are "important" and should be retained. I examined this result of Boruta and found that if I exclude any feature from my dataset, then I am losing accuracy of my Random Forest classifier.
However, when I am plotting the scree plot of my dataset in R, then I am getting the "elbow" right after the first component. Does it mean only the first component is enough?
I am a newbie in data analysis and statistics. So, I would really appreciate if anyone can explain what is going on here.