Skip to main content
Cross Validated

Return to Question

Post Closed as "Duplicate" by amoeba, Nick Cox, John, gung - Reinstate Monica, whuber♦
clarified title
Source Link
amoeba
amoeba
  • 107.3k
  • 36
  • 321
  • 346

Multi-class Classification using SVM with PCA on train and test datasets: should I run one PCA on train+test or two separate on train and on test?

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image to $n$ (say $n = 100$), and then use SVM to to learn and classify using the reduced feature space. I'm wondering which of these two paths should I follow?

(1) PCA on the training set $T$ and use SVM to learn on new $T$ with reduced dimensions. For prediction, PCA again on the test set $S$ and use the learned SVM parameters to classify.

(2) PCA on both $T$ and $S$ at the same time. That is, merge $T+S$ into a large matrix and perform PCA on them. Then split the reduced dimension matrix into $T$ and $S$ again. Learn the SVM on the reduced $T$ and then use the learned SVM to predict on the reduced $S$.

  1. PCA on the training set $T$ and use SVM to learn on new $T$ with reduced dimensions. For prediction, PCA again on the test set $S$ and use the learned SVM parameters to classify.

  2. PCA on both $T$ and $S$ at the same time. That is, merge $T+S$ into a large matrix and perform PCA on them. Then split the reduced dimension matrix into $T$ and $S$ again. Learn the SVM on the reduced $T$ and then use the learned SVM to predict on the reduced $S$.

Multi-class Classification using SVM with PCA

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image to $n$ (say $n = 100$), and then use SVM to to learn and classify using the reduced feature space. I'm wondering which of these two paths should I follow?

(1) PCA on the training set $T$ and use SVM to learn on new $T$ with reduced dimensions. For prediction, PCA again on the test set $S$ and use the learned SVM parameters to classify.

(2) PCA on both $T$ and $S$ at the same time. That is, merge $T+S$ into a large matrix and perform PCA on them. Then split the reduced dimension matrix into $T$ and $S$ again. Learn the SVM on the reduced $T$ and then use the learned SVM to predict on the reduced $S$.

PCA on train and test datasets: should I run one PCA on train+test or two separate on train and on test?

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image to $n$ (say $n = 100$), and then use SVM to to learn and classify using the reduced feature space. I'm wondering which of these two paths should I follow?

  1. PCA on the training set $T$ and use SVM to learn on new $T$ with reduced dimensions. For prediction, PCA again on the test set $S$ and use the learned SVM parameters to classify.

  2. PCA on both $T$ and $S$ at the same time. That is, merge $T+S$ into a large matrix and perform PCA on them. Then split the reduced dimension matrix into $T$ and $S$ again. Learn the SVM on the reduced $T$ and then use the learned SVM to predict on the reduced $S$.

Source Link
Joe
Joe
  • 403
  • 1
  • 7
  • 13

Multi-class Classification using SVM with PCA

I'm doing an image classification task and the number of features of each example image is pretty huge (3,072: # pixels in each image). I'm thinking of using PCA to reduce the # features of each image to $n$ (say $n = 100$), and then use SVM to to learn and classify using the reduced feature space. I'm wondering which of these two paths should I follow?

(1) PCA on the training set $T$ and use SVM to learn on new $T$ with reduced dimensions. For prediction, PCA again on the test set $S$ and use the learned SVM parameters to classify.

(2) PCA on both $T$ and $S$ at the same time. That is, merge $T+S$ into a large matrix and perform PCA on them. Then split the reduced dimension matrix into $T$ and $S$ again. Learn the SVM on the reduced $T$ and then use the learned SVM to predict on the reduced $S$.