3
$\begingroup$

I want to apply PCA to a dataset $X_{train}\in R^{n\times m}$, where $X_{train} = [ x^{(1)}_{train} x^{(2)}_{train} x^{(3)}_{train} x^{(4)}_{train} ...x^{(m)}_{train} ] $ and $x^{(i)}_{train}\in R^{n}$, such that after applying PCA, $X_{train}$ reduces to $X_{train} \in R^{k,m}$. If I want to train this new dimension reduced dataset using Neural Networks i have to initialize my training model with input layer having nodes equal to to number of features i.e $k$ .Now it has been advised that we should not use PCA on the testing data. Since there will be a mismatch of dimensions how can I fit/predict testing data of dimensions $x_{test}^{(i)} \in R^{n}$ on a Neural Network model which has only $k$ input nodes without reducing its dimensions? Is it possible to implement NN along with PCA?

$\endgroup$
2
  • 1
    $\begingroup$ It's not difficult: fit PCA on training data. Apply its predict/transform method on test data and select first m components. Use these as new "test" set for evaluation of the nn. The logic is identical e.g. to scaling of input. $\endgroup$ – Michael M May 9 '20 at 10:29
  • $\begingroup$ @NishantaBoro it seems you removed the accept from my answer. Is there something you think wrong? $\endgroup$ – gunes May 24 '20 at 20:51
1
$\begingroup$

You can't enter $n$ dimensional vectors into a NN with input dimension $k$. Even if you could, it wouldn't make sense because the interpretations of dimensions changed. What is sensible here is to project your test set onto new axes defined by your training set, and use them for testing.

$\endgroup$
4
  • $\begingroup$ so basically there is no way around without applying PCA on the test data while modeling a NN ? $\endgroup$ – Nishanta Boro May 9 '20 at 10:26
  • 1
    $\begingroup$ Consider a black box machine (not even a NN) whose input dimension is $k$. What would you do to input a vector of size $n$? Note that while modelling, you won't use your test set. It's only for testing and test set will be projected onto axes defined by training set. You won't apply PCA to test data directly. Instead, you'll use the PCA model trained on the training data, and transform your test set. $\endgroup$ – gunes May 9 '20 at 10:28
  • $\begingroup$ we have to make a transform otherwise it is not possible.. Thanks!! $\endgroup$ – Nishanta Boro May 9 '20 at 10:33
  • 1
    $\begingroup$ Yes, and the transform/projection should be on the new axes defined by the training set. $\endgroup$ – gunes May 9 '20 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.