0
$\begingroup$

I have a data-set with a decent number of attributes, half of them nominal. I used a binary vectorizer to convert the nominal attributes to numerical, but now there are far too many of them. I'm not sure how to use PCA on the data-set for the best results. Here are the three approaches I thought of:

  1. Load the data, remove the target variable, then run PCA on the entire data-set. The problem is my data here is non-standardized because some of it is income, some of it is age etc. It's also not scaled. I could scale it to go from 0-1 but some of the values have extremely high ranges, from 0 to 99999.
  2. Load the data, remove the target variable, then run PCA only on the values that i converted from being Nominal. Re-add these PCA-generated columns to the original numeric data.
  3. Load the data, remove the target variable, then run PCA separately for each attribute that used to be nominal, so for example run it for education-College, education-High School, generate n number of principal components, re-add to the original columns and then run it again for another ex-nominal variable, keep repeating until done.

The second approach seems the most sensible to me, but I'm not too sure. If it helps I am using Python/Pandas/Scikit-Learn for all of this.

$\endgroup$
1
$\begingroup$

The problem is my data here is non-standardized because some of it is income, some of it is age etc. It's also not scaled. I could scale it to go from 0-1 but some of the values have extremely high ranges, from 0 to 99999.

That's exactly why someone would use standardization/scaling.

Since you mentioned you use scikit-learn - actually its PCA implementation does standardization and mean-centering as preprocessing by default.

  1. Load the data, remove the target variable, then run PCA only on the values that i converted from being Nominal. Re-add these PCA-generated columns to the original numeric data.

Seems like a valid approach - also you could also use some different dimensionality reduction method for this data, for example maybe some of these features don't influence prediction or are redundant (you can check out contingency tables with pandas.crosstab or try sklearn.feature_selection algorithms on your one-hot encoded data).

  1. Load the data, remove the target variable, then run PCA separately for each attribute that used to be nominal, so for example run it for education-College, education-High School, generate n number of principal components, re-add to the original columns and then run it again for another ex-nominal variable, keep repeating until done.

That's not particularly good idea, since PCA finds useful features as combinations of input features. For example if you have two perfectly correlated binary features PCA will find it out, but under 3rd approach it won't be possible.

$\endgroup$
  • $\begingroup$ Hi Jakub, thanks for the answer. Since writing this post I decided to look at the first approach. I used a min-max scaler to scale my numeric data from 0 to 1, so my entire dataset is now all within a 0-1 range. Am I alright to just feed this data-set into PCA? Also I'll try and take a look at the other dimensionability reduction stuff you mentioned too, but if I'm honest I am a bit pressed for time with this. $\endgroup$ – Mateusz J Jan 6 '18 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.