1
$\begingroup$

According to the answer here: How to choose the number of hidden layers and nodes in a feedforward neural network?

How many hidden layers? Well, if your data is linearly separable (which you often know by the time you begin coding a NN) then you don't need any hidden layers at all.
  1. Why this is true?

  2. If the data is linearly separable:

    2.1 Do we need only to use input and output layers?

    2.2 Does the activation function on the output layer will do the logic of the separation? (Is it enough)?

$\endgroup$

1 Answer 1

4
$\begingroup$

Yes, hidden layers are not needed for linearly separable data. Because, the output layer already calculates a linear combination of features and outputs a number that has discriminative power, i.e. $f(\sum w_ix_i + b)$, where $w_i$ are output neuron's weights, $b$ is bias and $f$ is the activation function. Linear separability means that there exist a hyperplane separating the classes in the feature space, i.e. $\sum w_ix_i+b$. Activation functions also matter, but as long as they translate to a decision rule of the form $\sum w_ix_i+b >\tau$, the form of activation is not important, e.g. tanh, sigmoid. So, you don't need hidden layers trying to discover a more complex decision boundary. It's already been discovered by using only the output layer. And, you have input layer as always, which just represents your features.

$\endgroup$
2
  • $\begingroup$ Thanks. How the activation function influence the output in this case ? $\endgroup$ Jul 5, 2021 at 12:32
  • $\begingroup$ For example, while learning, it helps you to define a proper loss function like log-loss, instead of mse, where the problem can turn out to be non-convex $\endgroup$
    – gunes
    Jul 5, 2021 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.