Skip to main content
16 events
when toggle format what by license comment
Sep 21, 2020 at 19:01 history bumped CommunityBot This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
May 23, 2020 at 18:04 comment added Aksakal Wht's the question? you made statements, and but didnt pose a clear question
May 23, 2020 at 18:00 history bumped CommunityBot This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Jan 19, 2020 at 9:02 history bumped CommunityBot This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
Sep 17, 2019 at 19:56 history reopened Sycorax neural-networks
Sep 17, 2019 at 19:53 history edited RMurphy CC BY-SA 4.0
By request, clarified question. Previously it sounded like a duplicate.
Sep 17, 2019 at 19:45 review Reopen votes
Sep 17, 2019 at 20:00
Sep 17, 2019 at 19:42 comment added RMurphy @Sycorax, thank you for unduplicating it. I will edit it.
Sep 17, 2019 at 19:33 comment added Sycorax My advice is to use the edit button rewrite your question to clearly articulate what you know, what you would like to know, and where you are stuck. Right now, I can't make heads or tails of what you're trying to ask and what you would like to know.
Sep 17, 2019 at 19:28 history edited RMurphy CC BY-SA 4.0
added 94 characters in body
Sep 17, 2019 at 19:24 comment added RMurphy @Sycorax, please, it is not a duplicate question. I understand perfectly well why nonlinear functions are not used, in terms of the answer you have given me above. Also, I do not agree that computing a standard deviation is linear in its arguments. The x you have written will have weights from the previous layer, and we will evaluate a square root of those weights.
Sep 17, 2019 at 14:42 comment added Sycorax A $z$ score is just a linear transformation of the inputs; if this is unclear, note that you can re-write $\frac{x - \mu}{\sigma}$ as $\frac{1}{\sigma}x - \frac{\mu}{\sigma}$. The duplicate question addresses why neural networks use nonlinear activation functions instead of linear functions: linear functions are closed under composition, so a network of linear functions is simply a linear model.
Sep 17, 2019 at 14:40 history closed Sycorax neural-networks Duplicate of ReLu vs a linear activation function
Sep 17, 2019 at 13:59 comment added RMurphy @Sycorax. In some sense, that's the point of my question. I understand the original motivation behind batch norm, but couldn't it also double as a nonlinearity? I mean, why do we need to compose something like a sigmoid, a relu, an elu etc. etc. with a z-standardization or its glorified cousin batch norm?
Sep 16, 2019 at 20:04 comment added Sycorax I'm not sure what information you're seeking. My best guess is that your question is premised on a misunderstanding of how batch norm works. The point of batch norm is to use running mean and running standard deviation estimates; these estimates are used to compensate for the shifting means and standard deviations of inputs to the norm layer which occur because the network is training. Does this answer your question, or do you need clarification about a different component of batch norm? (What component do you wish to understand in more detail?)
Sep 16, 2019 at 19:58 history asked RMurphy CC BY-SA 4.0