Timeline for What does "Mean of each pixel over all images" mean?
Current License: CC BY-SA 3.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 8, 2019 at 6:34 | comment | added | CMCDragonkai | If an image is RGB, should we have a stddev for each channel over all pixels over all images? | |
| Apr 17, 2018 at 9:13 | comment | added | Soltius | Great. I'll leave it as comments as it does not directly answer the question which is more of a "how is normalization implemented ?" rather than "why is it implemented ?" (I came here indeed looking for the former). Cheers ! | |
| Apr 17, 2018 at 8:28 | comment | added | Nick Cox | All fine by me. Complementary answers and comments and correcting what is wrong or incomplete are all key to how the site works. If you want to bring your comments together into another answer I will happily upvote it. | |
| Apr 17, 2018 at 7:23 | comment | added | Soltius | Anyway, sorry for necromancing such an old answer, I didn't want to point out any lack of precision on your behalf, just add some info (which probably emerged to the ML comunity somewhere between your answer and now) in case someone comes here today ! | |
| Apr 17, 2018 at 7:21 | comment | added | Soltius | of the current state of knowledge, including right here on Cross-Validated eg stats.stackexchange.com/questions/7757/… or on SO stackoverflow.com/questions/4674623/…. | |
| Apr 17, 2018 at 7:16 | comment | added | Soltius | It is known to have many interesting/necessary effects such as : a/ feature scaling (maybe less relevant in the case of images), b/ avoid saturation of hidden units (can also be expressed as "inputs fall in the useful range of the non-linearities), c/ data standardization which helps your network to generalize to new unseen data since variability is reduced... The "go to" paper is usually LeCun's one yann.lecun.com/exdb/publis/pdf/lecun-98b.pdf (section 4.3) although it does not talk about std specifically. But simply googling "why do we normalize in neural networks" gives a good idea | |
| Apr 16, 2018 at 16:55 | comment | added | Nick Cox | @Soltius I can happily believe you. Thanks! Is this documented or just well known? | |
| Apr 16, 2018 at 14:23 | comment | added | Soltius | Regarding your last sentence : In the case of neural networks (such as the autoencoders mentionned in the question), the division by a constant may in fact have quite an effect regarding numerical issues and performance. | |
| Oct 12, 2013 at 12:19 | vote | accept | pinkpanther | ||
| Oct 12, 2013 at 12:16 | history | answered | Nick Cox | CC BY-SA 3.0 |