Timeline for Why the Gaussian Distribution in ML/DL?
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 14, 2020 at 19:10 | vote | accept | Kamal Raydan | ||
| Nov 24, 2020 at 0:24 | history | became hot network question | |||
| Nov 23, 2020 at 19:33 | comment | added | Kamal Raydan | Agreed. I might as well perform some more research on the topic and come up with another question (if I haven’t already answered it myself that is) | |
| Nov 23, 2020 at 19:12 | comment | added | Sycorax♦ | It seems that the remaining question is too vague to be reasonably answered. Asking why some unnamed authors talk about unit gaussians in some unnamed resources doesn't seem sufficiently specific to be answered, aside from whuber's general observation that this kind of misinformation is common. | |
| Nov 23, 2020 at 18:27 | comment | added | whuber♦ | There is a huge amount of misinformation out there concerning the desirability of Gaussian distributions. In most cases what one is interested in is achieving some approximately symmetric distribution, preferably without very long tails. That is far weaker than being approximately Gaussian, yet it simplifies the description, interpretation, and analysis of the data. Few statistical procedures require approximate Gaussian distributions of data and many are robust to substantial departures from the Gaussian shape, provided the symmetry/no long tail conditions are met. | |
| Nov 23, 2020 at 17:30 | comment | added | Kamal Raydan | Still though, would you care to explain maybe in what context, surrounding DL, would you require or advise to have the data distribution looking like a gaussian distribution, and/or its usefulness? | |
| Nov 23, 2020 at 17:29 | comment | added | Kamal Raydan | Interesting. Well now that we got that misunderstanding, or just out right false statement, out of the way, the question as to why is the Gauss. Dist. mentioned a lot in Deep learning still remains a mystery to me. It seems like i attributed the likes of batch normalization to a changing of the original distribution which i now know is false... | |
| Nov 23, 2020 at 17:26 | history | edited | Kamal Raydan | CC BY-SA 4.0 |
added 255 characters in body
|
| Nov 23, 2020 at 16:46 | comment | added | Sycorax♦ | Centering and scaling the data is not the same as transforming to a Gaussian distribution. The benefits of centering and scaling the data for machine learning are discussed in a number of places on stats.SE, such as stats.stackexchange.com/questions/421927/… It's not required or even recommended to have zero-mean/unit-variance data in all contexts; alternative centering and scaling methods can work well for idiosyncratic reasons. | |
| Nov 23, 2020 at 16:43 | answer | added | Dave | timeline score: 4 | |
| Nov 23, 2020 at 16:28 | comment | added | whuber♦ | Re "which has the effect of transforming the input feature distributions into a unit gaussian:" Certainly not. It only has the effect of changing their units of measurement. It does not change the shapes of their distributions. As such, your question is predicated on something that just isn't true -- not even approximately. In light of that, would you perhaps like to edit and refine it? | |
| Nov 23, 2020 at 16:20 | history | asked | Kamal Raydan | CC BY-SA 4.0 |