4
$\begingroup$

A question similar to this was asked before but the question wasn't answered in enough detail to explain what I'm wondering about.

I've been reading introductory material to Artificial Neural Networks (ANN). The examples given for inputs of continuous values always use a single input neuron which takes the value of the variable. This is like the first diagram below.

To me this seems limiting. If a continuous variable X can take values between 1 and 10 then rather than having a single input neuron with a value x one could create two input neurons. If x is small (between 1 and 5) the first neuron is activated, if x is big (between 5 and 10) the second neuron is activated. This is like the second diagram below.

This input in the second diagram discretizes x so it loses some information about x, but I think this could be useful if the response to x should change depending on whether x is small or big. For example, if an ANN is to determine whether somebody is male or female given an image and the persons age. The ANN should look for different distinguishing features depending on whether the person has a low age or a high age.

Is this sort of discritization used in practice? Does it have great disadvantages and is it still useful in certain situations?

Continuous input Discrete input

$\endgroup$
  • $\begingroup$ Not sure I understand what the advantage would be. Basically you are applying a function before feeding in the input to the network already (the 0 or 1 decision for both inputs is the function I am talking about) - why would you not model this function as part of the network? Or am I misunderstanding something? $\endgroup$ – Pegah Sep 5 '15 at 21:45
  • $\begingroup$ @Pegah Perhaps this simple example will show the advantage. Suppose the task is to square a number 'x' between 1 and 10. If the neural network with few neurons has just 1 input with the value x it is going to have a hard time finding weights to suit cases for x=1 and x=10 because squaring is nonlinear. On the other hand if there were 2 input neurons which indicate if x lies between 1 and 5 or 5 and 10 then it can find weights which suit either the case where x is small or the case where x is large. I'm imagining it ending up like a piecewise linear function approximating the square function. $\endgroup$ – Hugh Sep 5 '15 at 22:59
2
$\begingroup$

You're touching on problem formulation, where the designer defines input variables and their characteristics. Looking at the structure you show, if you were trying to predict whether a liquid is frozen or solid, such a structure would be useful. If you were trying to predict a future value of the variable, then it would be much less useful. So your question is quite abstract, and so the answer is yes such a structure is surely used and has advantages in certain circumstances. But the challenge in machine learning is the inverse of the challenge you propose. Given a problem that needs a solution, what are the best variables and model to solve the problem.

$\endgroup$
  • $\begingroup$ Thank you Bob that's very helpful. I see now why there is so much information on training the network instead of how to present the data. $\endgroup$ – Hugh Sep 6 '15 at 10:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.