0
$\begingroup$

Say we are designing a neural network for self driving cars. This question came about after seeing a video in the Machine Learning class taught by Andrew Ng at Stanford. The video uses two neural networks, one is trained to drive on single lane roads, while the other is trained to drive on two lane roads. When the prediction of one is stronger than the other, the stronger neural network is used to drive the car. This allows the car to switch from single to two lane roads (one way and two way roads).

Would one neural network be able to do this? Basically use two "systems" internally to decide the action to take? Intuitively I think somehow the neural network can do this.

Another example of this question I can think of is, predicting if it will rain in the next four hour block. We want to divide the day into six four hour blocks, and take features at each block start and predict if it will rain in the next four hours. Would six neural networks (one that works best with a start of midnight, another of 4am, another of 8am, etc...) be better than a single neural network?

$\endgroup$
  • $\begingroup$ One neural network would be able to combine those two systems quite easily as a matter of fact. First train your two systems to work with the same kind of input. Then train a neural network that decides which 'sub-neuralnetworks' to choose. Then combine them into one, by combining outputs with inputs. Hard to explain like this, but you can basically map anything just like @Alex_P said in his answer. $\endgroup$ – Thomas W Apr 5 '17 at 11:18
0
$\begingroup$

Theoretically speaking, neural networks have been proven to be able to map any function, regardless of the number of inputs and/or outputs of that function.

In practice, i don't think there is a guaranteed answer to your question. There may be problems for which training a single "centralized" (= two "systems" internally) network is easier, in the sense of time and/or accuracy. In my opinion, you could merge the sub-problems and attack them with a single network if the process of solving each of them is the same or very similar.

In your example, I would definitely try to train one network for the rain forecasting problem, as you would be searching for certain, identical features in all hour blocks. For the car problem though, I would be more cautious. If the inputs and outputs are the same, perhaps with the exception of a few, I would try it out of curiosity. If the form (inputs-outputs) of the sub-problems was substantially different, I wouldn't try it.

The bottom line is, theoretically you can map any function with a neural network, so the answer would be yes, but, in practice, you might not benefit.

$\endgroup$
  • $\begingroup$ Thanks for the answers everyone. About neural networks being able to map any function, it doesn't make intuitive sense. If an feature is a number, and the system behaves slightly differently if that feature is odd or even. How can a neural network "learn" that? For the rain one, it might not be because the characteristic the "time" is trying to predict may vary as the day goes by in the same direction (as time goes up, Y goes "up" or "down", as opposed to if time is X, Y is down, then X+1, Y is up, then X+2, Y is down, then next 3 Y is up, etc). $\endgroup$ – Tony Tieger Apr 5 '17 at 18:19
  • $\begingroup$ @TonyTieger Your question is not clear, but if I infer correctly, you ask how it is possible for a neural net to map any function when that function can be extremely sensitive to small changes in the input. The answer to that question is the nature of neural networks themselves. If you look at the mathematical background and principles of these nets, you understand that they learn by examining how the output changes with respect to tiny changes in each parameter (weights) and thus the inputs themselves. This is the verbal description of the gradient of a function. $\endgroup$ – Alex P Apr 5 '17 at 18:34
  • $\begingroup$ If the gradient is very big along one parameter, it means that, for a very small region around the given parameter's value, the output changes a lot. By providing enough data points, the net will adapt to these changes and provide good generalization regardless of the complexity of the function (unless you overfit the network, don't use optimization methods or provide few datapoints). $\endgroup$ – Alex P Apr 5 '17 at 18:35
  • $\begingroup$ Hi Alex P, thanks, I think I get that. I did some more "research" (web searching), and I think I can better phrase my question. Assume a function outputs 1 if input feature is odd, 0 if it's even. Neural network can detect this because of the activation function (allowing "small change" detection like <100 is even, 101+ is odd with a trailing off below 99 and above 102). If it detects odd/even from 0-100, it might need say X nodes. To work for 0-1000, it'll need 10*X nodes, right? For humans, odd/even is easy, but there's no way for neural network without infinite nodes for any number, right? $\endgroup$ – Tony Tieger Apr 9 '17 at 1:33
  • $\begingroup$ @TonyTieger This is a very good question! First of all, the assumption that it would need 10*X nodes to work for a wider range of numbers is incorrect. Or, more precisely, not always true or mathematically sound. The reason is this (and can be extended to problem solving in general): how you present the problem is, more often than not, more important that how you approach solving it. Say for example that you code the numbers in binary and have n binary input nodes, the binary representation of the number. $\endgroup$ – Alex P Apr 9 '17 at 21:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.