Skip to main content
Tweeted twitter.com/StackStats/status/1209896697746382849
Became Hot Network Question
edited title
Link
Thomas Moore
  • 1.7k
  • 15
  • 20

Neural Network - Can Hidden Layer Nodes be omitted from output equation

edited body
Source Link
Thomas Moore
  • 1.7k
  • 15
  • 20

I was considering a feedforward neural network, where the output can be written as:

$y_o = \sigma(z_h)$,

where $z_h$ is the logit from the hidden layer, say, $w^T_{h}. x_h$, where the $h$ subscript denotes the hidden layer.

My question is since the hidden layer inputs $x_h$ are just the outputs from the input layer, can one just not write this as:

$y_o = \sigma(z_h) = \sigma(w^T_{h} . x_h) = \sigma(x^T_{h} y_i) = \sigma(x^T_{h} \sigma(w^T_{i} x_{i}))$$y_o = \sigma(z_h) = \sigma(w^T_{h} . x_h) = \sigma(w^T_{h} y_i) = \sigma(w^T_{h} \sigma(w^T_{i} x_{i}))$, where this last equality explicitly shows the composition of functions (i denotes the input layer), and you don't have any $x_h$ terms in there?

Thanks.

I was considering a feedforward neural network, where the output can be written as:

$y_o = \sigma(z_h)$,

where $z_h$ is the logit from the hidden layer, say, $w^T_{h}. x_h$, where the $h$ subscript denotes the hidden layer.

My question is since the hidden layer inputs $x_h$ are just the outputs from the input layer, can one just not write this as:

$y_o = \sigma(z_h) = \sigma(w^T_{h} . x_h) = \sigma(x^T_{h} y_i) = \sigma(x^T_{h} \sigma(w^T_{i} x_{i}))$, where this last equality explicitly shows the composition of functions (i denotes the input layer), and you don't have any $x_h$ terms in there?

Thanks.

I was considering a feedforward neural network, where the output can be written as:

$y_o = \sigma(z_h)$,

where $z_h$ is the logit from the hidden layer, say, $w^T_{h}. x_h$, where the $h$ subscript denotes the hidden layer.

My question is since the hidden layer inputs $x_h$ are just the outputs from the input layer, can one just not write this as:

$y_o = \sigma(z_h) = \sigma(w^T_{h} . x_h) = \sigma(w^T_{h} y_i) = \sigma(w^T_{h} \sigma(w^T_{i} x_{i}))$, where this last equality explicitly shows the composition of functions (i denotes the input layer), and you don't have any $x_h$ terms in there?

Thanks.

Source Link
Thomas Moore
  • 1.7k
  • 15
  • 20

Neural Network - Can Hidden Layer Nodes be omitted

I was considering a feedforward neural network, where the output can be written as:

$y_o = \sigma(z_h)$,

where $z_h$ is the logit from the hidden layer, say, $w^T_{h}. x_h$, where the $h$ subscript denotes the hidden layer.

My question is since the hidden layer inputs $x_h$ are just the outputs from the input layer, can one just not write this as:

$y_o = \sigma(z_h) = \sigma(w^T_{h} . x_h) = \sigma(x^T_{h} y_i) = \sigma(x^T_{h} \sigma(w^T_{i} x_{i}))$, where this last equality explicitly shows the composition of functions (i denotes the input layer), and you don't have any $x_h$ terms in there?

Thanks.