Timeline for Why not just dump the neural networks and deep learning? [closed]
Current License: CC BY-SA 3.0
36 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 25, 2018 at 12:47 | history | made wiki | Post Made Community Wiki by whuber♦ | ||
| Nov 2, 2017 at 17:11 | history | closed |
Xi'an mdewey John jbowman Sycorax♦ |
Opinion-based | |
| Oct 31, 2017 at 15:44 | review | Close votes | |||
| Nov 2, 2017 at 17:13 | |||||
| Oct 31, 2017 at 15:00 | history | protected | Sycorax♦ | ||
| Oct 31, 2017 at 13:25 | answer | added | Aksakal | timeline score: 7 | |
| Oct 31, 2017 at 12:46 | answer | added | ROBERTO EDWINS | timeline score: -2 | |
| Aug 20, 2017 at 19:41 | answer | added | Gnattuha | timeline score: 0 | |
| Aug 16, 2017 at 0:27 | comment | added | SQLServerSteve | The bottom line is that we can't always apply the degree of mathematical rigor we'd like since we can't always know the underlying functions in advance. Neural nets approximate unknown functions that can be too ridiculously complex + of high dimensionality for us to analyze. We may also run into No Free Lunch problems with global maxima we can't pin down, for functions we can only approximate, till we've covered the entire search space. So there'll always be room for inexact deep learning techniques like SVMs, neural nets, genetic algorithms (or others we haven't thought of yet). | |
| Aug 15, 2017 at 7:41 | comment | added | Quickbeam2k1 | The same happens in theory and numerics of partial differential equations. Often the theory imposes strict assumptions for existence of solutions, but the numerics oft work for larger cases. Additionally, those findings in the numerics lever the proceedings in the theories and vice versa. Particularly, note that for some equations you can finde solutions numerically but the theory lacks behind. | |
| Aug 15, 2017 at 2:57 | answer | added | Sid | timeline score: -3 | |
| Aug 12, 2017 at 15:02 | comment | added | Mark Rosenblitt-Janssen | I think your ideas were tried in the 60's and 70's -- models that could be understood before being implemented. | |
| Aug 12, 2017 at 2:46 | answer | added | Rajesh D | timeline score: -5 | |
| Aug 11, 2017 at 23:57 | comment | added | Peter - Reinstate Monica | Are you a mathematician? Then you may not have noticed that In real life mathematical correctness is often not important. I know that hurts. | |
| Aug 11, 2017 at 21:00 | comment | added | shadowtalker | "something that is consistent with a set of mathematical equations" how does this not describe NNs? | |
| Aug 11, 2017 at 20:17 | comment | added | Fernando | Our brain is some kind of neural net... we just don't know the exact architecture. Why drop the best approximation? | |
| Aug 11, 2017 at 20:16 | comment | added | Wildcard | @josh, "e.g." means "exempli gratia," which is Latin for "for the sake of an example." "i.e." means "id est," which translates to "that is" and is used to add explanatory information or state something in different words. In your comment, the phrase "they work, so we continue to use them" is not an example of what precedes it, but a restatement. Hence, "i.e." would be appropriate, but "e.g." doesn't fit. | |
| Aug 11, 2017 at 17:47 | comment | added | Julian Habekost | Neural networks are inspired by the living brain. You say "so we never know we end up with a global or a local minimum". Funny, because I think this is the definition of life. | |
| Aug 11, 2017 at 16:22 | answer | added | Miguel | timeline score: 4 | |
| Aug 11, 2017 at 15:54 | comment | added | Mayou36 | @RajeshDachiraju, exactly. But your question was, why people still use neural networks and don't dumb them. And people use them to explain the real world. So why don't people dumb them: because they can use them. The question should then probably be: why are mathematicians still interested in neural networks (whereby the answer on that would be: because they may increase the performance of NNs that are applied in the real world. With no hope for a "purely mathematical" advance on the understanding, probably) | |
| Aug 11, 2017 at 15:40 | comment | added | Rajesh D | @Mayou36 : Math doesn't explain real world! All math does is explain itself. Its we who use math as a language to make our explanation of real world simple. | |
| Aug 11, 2017 at 15:24 | answer | added | DeltaIV | timeline score: 8 | |
| Aug 11, 2017 at 15:07 | answer | added | Lily Long | timeline score: 0 | |
| Aug 11, 2017 at 12:54 | comment | added | Mayou36 | @RajeshDachiraju, yes, there may are infinit solutions for a neural network, so are there for any physical problem. We just chose the most simple one. In case of neural networks, we just select one. Because, at least at the level of our current understanding, there is no better one. This may changes. And, AFAIK, the math of NN is not inconsistent, we just do fail to explain the behavior we observe with the math we use now, right? Seems to me like for example QM: before we knew about it, chemistry worked as well. But the math/physics failed completely to explain phenomena. | |
| Aug 11, 2017 at 12:30 | comment | added | Rajesh D | @Mayou36 : The mathematics of Classical mechanics and QM and relativity are all consistent, while the case of neural network is not. Just that in real world nothing is perfect, doesn't mean that you tie all theories to one tree with no regard to Mathematical consistency? | |
| Aug 11, 2017 at 12:30 | comment | added | Rajesh D | @josh and others : The mathematics of Neural networks is not consistent, while the mathematics of Linear and SVM models is consistent. Simple as that. How well they are useful in real world is a totally different thing. I hope you understand the difference! | |
| Aug 11, 2017 at 12:09 | comment | added | Mayou36 | You know what is wrong as well: Newtonian mechanics. Quantum Mechanics. Relativity. All the physics is wrong (there is not one single model describing everything, all have their flaws). Chemistry is completely wrong with so many things (describing an atom is always just a good approximation but never exact). The only exactly true thing in the world is math. Pure math. Everything else comes close to the right answer. Should we throw away the rest? (starting from your computer built with wrong laws?). No. Again: all models are wrong, but some are useful. | |
| Aug 11, 2017 at 11:47 | answer | added | Carrosive | timeline score: 14 | |
| Aug 11, 2017 at 10:40 | comment | added | josh | @RajeshDachiraju - it is a old idiom, but I was perhaps a bit vague. You asked why not throw away NNs because they are not perfect. My retort is that they are not perfect, but they are USEFUL. People use them to autodrive cars, translate foreign languages, tag videos, in conservation of whales and even to apply those rubbishy snapchat filters with dog ears to your photos! e.g. they work, so we continue to use them :) | |
| Aug 11, 2017 at 9:54 | comment | added | VisualMelon | @RajeshDachiraju trained Linear models and SVMs are only "right" if you have ideal or arbitrarily large amounts of data, and the system is linear or otherwise perfectly representable by your SVM model. A system doesn't need to be perfectly linear for a linear model to remain a useful model of it (a linear model can be wrong yet useful). There is also interest in NNs from a biological standpoint, as they (vaguely) map onto 'real-world' learning machines: we don't only study them because they work well, we study them simply because we want to better understand them. | |
| Aug 11, 2017 at 8:52 | history | tweeted | twitter.com/StackStats/status/895930978992082945 | ||
| Aug 11, 2017 at 8:15 | comment | added | Rajesh D | @josh : I don't know what you mean by wrong. May be you are talking a bit philosophical. "Linear" and "SVM" are correct models in the sense that they are mathematically consistent, only thing is that they are not very useful. Neural networks are very useful, but unfortunately not mathematically consistent (reasons mentioned in the OP). | |
| Aug 11, 2017 at 8:05 | comment | added | josh | "All models are wrong but some are useful" and nns are certainly useful. | |
| Aug 11, 2017 at 6:13 | comment | added | Kilian Foth | "Why not come up with...?" You wouldn't believe how many researchers are busy trying to do exactly that! They just haven't had success so far. | |
| Aug 11, 2017 at 4:22 | answer | added | shimao | timeline score: 49 | |
| Aug 11, 2017 at 3:08 | comment | added | Matthew Drury | If you find it, people will. | |
| Aug 11, 2017 at 2:30 | history | asked | Rajesh D | CC BY-SA 3.0 |