1
$\begingroup$

I am trying to build an ANN model with 1 hidden layer ( 4 hidden units ) that is capable of learning non-linear regression.

Both X and Y in my training data are in 1D with 60 samples and the data is non-linear.

I am kind of confused in terms of the shape of the weights and the number of neurons I will have in my network.

So, my understanding is:

  • 1 input neuron (because my X is in 1D) with [60,1] shape

  • 4 hidden neuron (as stated above)

  • 1 output neuron with [60,1] shape

  • 4 weights in the first layer (weights from input to hidden)

  • 5 (including bias) weights in the second layer (weights from hidden to output)

Is this reasoning makes sense? If it does not, how should I think this through?

Thank you.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ If you have doubt, write out the matrix-vector multiplication for a single sample. Suppose the sample has $k$ features. What shapes do you need to have for weights and biases in order for the arithmetic to work out? (I'm not trying to be obscure or coy -- It's important to understand how network size is ultimately related to linear algebra.) $\endgroup$
    – Sycorax ♦
    Dec 6, 2020 at 17:33

1 Answer 1

Reset to default
1
$\begingroup$

Number of training samples doesn't determine your network shape (so, there is influence of 60 in your network). As you said, you have 4 hidden neurons, each have one weight (connected to your input layer) and one bias. This makes your the hidden layer's output four dimensional. In the output layer, you have one neuron, and the input to this neuron has four dimensions because it's hidden layer's output, which means there are four weights and one bias.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.