Timeline for Is it realistic to achieve better results when doing PCA before neural network classification? [duplicate]
Current License: CC BY-SA 3.0
19 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 26, 2017 at 3:34 | history | closed |
amoeba Michael R. Chernick gung - Reinstate Monica whuber♦ |
Duplicate of How can top principal components retain the predictive power on a dependent variable (or even lead to better predictions)? | |
| Jan 26, 2017 at 0:12 | review | Close votes | |||
| Jan 26, 2017 at 3:34 | |||||
| Jan 25, 2017 at 23:54 | comment | added | amoeba |
Exactly, was always using bsxfun. I don't know if it has a big influence or not. What you are doing is not really PCA (because you compute eigenvectors of the scaled data but transform unscaled data), but as you are doing the same thing with train and test then it's okay. Regarding your main question of how it's possible that results are better, please read stats.stackexchange.com/questions/141864 and follow the links in the top answer. (Also, please include @amoeba in your comments, otherwise I am not notified of them.)
|
|
| Jan 25, 2017 at 20:42 | comment | added | SwingNoob | You´re welcome, and yes I am. U had to use some bsxfun before, hadnt u? But does this have a big influence, especially because I am using unscaled versions of both Data and Test_Data, so there is no difference in preparation of both. And PCA is working well. So in sum there is impmrovement potential by multiplying scaled data, but in general my solution is working? | |
| Jan 25, 2017 at 19:12 | comment | added | amoeba |
In any case, regarding the last two lines: you should multiply your Scaled with PCA coeff, not the Data. And for the test data, you need to scale it first with the mean/std of the training data.
|
|
| Jan 25, 2017 at 19:11 | comment | added | amoeba | Wow!! You are probably using 2016b version, aren't you? I googled and found that 2016b indeed introduced this functionality, so it is working since September last year. Amazing. I've been using Matlab for years and it never worked before and it used to be so annoying. Thanks for letting me know. | |
| Jan 25, 2017 at 17:25 | comment | added | SwingNoob | It indeed is matlab code. And Data - mean(Data) does subtract the column mean. Just tried that out. Also verified the dividing by std(Data). Works perfectly fine. Calculates standard derivation of each column and divides each colum element by the corresponding standard derivation of that column. | |
| Jan 25, 2017 at 16:32 | comment | added | amoeba |
Is this Scaled = (Data - mean(Data))./std(Data); a Matlab command or rather a pseudo-code? If Data is a 2D matrix, then in Matlab Data - mean(Data) will not subtract column means, and dividing by std(Data) like that won't work at all.
|
|
| Jan 25, 2017 at 10:44 | comment | added | SwingNoob | Or could anyone please tell me if and why the code is wrong? | |
| Jan 25, 2017 at 9:50 | comment | added | SwingNoob | Why are they not? | |
| Jan 25, 2017 at 9:48 | comment | added | Matthew Gunn |
The two lines starting with Data_Reduced = Data * coeff(:, 1:pca_size); are not correct.
|
|
| Jan 25, 2017 at 9:31 | comment | added | amoeba | See stats.stackexchange.com/questions/141864 and stats.stackexchange.com/questions/142557. It's all covered there. | |
| Jan 25, 2017 at 9:24 | history | edited | amoeba | CC BY-SA 3.0 |
edited tags
|
| Jan 25, 2017 at 9:22 | comment | added | SwingNoob | Added sample code how I´m doing PCA. Thank you so far. | |
| Jan 25, 2017 at 9:21 | history | edited | SwingNoob | CC BY-SA 3.0 |
added 338 characters in body
|
| Jan 25, 2017 at 9:20 | comment | added | Matthew Gunn | Reducing the dimensionality of your input reduces the complexity of your estimated neural network. It's quite plausible that if high variance components obtained from PCA are mostly what's important for classification, the reduction in variance (overfitting) due to a simpler model may dominate the increase in bias. | |
| Jan 25, 2017 at 9:05 | comment | added | Riff | PCA manages to keep most (if your dataset is structured) of the information in your dataset but condenses it in far less variables. Because of the curse of dimensionality your original dataset is not suited for classification, and because PCA condense that into few variables you end up with better results. Seems good to me. | |
| Jan 25, 2017 at 8:42 | review | First posts | |||
| Jan 25, 2017 at 9:31 | |||||
| Jan 25, 2017 at 8:41 | history | asked | SwingNoob | CC BY-SA 3.0 |