Y - Dependent variable is a factor (with levels A, B, and C) Independent variables are all numerical variables. Important: I have only 70 data points.
End Goal: Building a classification model to classify a future entry as A, B or C.
- First I need to check multicollinearity among the independent variables and get rid of any variables to tackle this. I know we can have a look at VIF values or covariance among variables to decide this. But the fact that dependent is a factor with levels keeps me throwing me off.
- After dealing with multicollinearity what are my options to build a good classification model, that would give me a model at the end. So nothing like knn or random forest.
- I did a principal component analysis and when I plot PC1 vs PC2 I could see two clusters A separate and B&C together. Then when I plot PC1 vs PC5 I could see a vague separation between B and C as well. How can I utilize this information?
What I have done so far 1. I did a principal component analysis. I could see a separation between the levels but I do not know how to utilize this information as I mentioned above. 2. I divided the data set into 80/20 training and validation and did k nearest neighbor classification method.
To be honest I feel like everything is all over the place and would like some guidance on how to properly approach this question using R.