1
$\begingroup$

I am fairly new to machine learning and Neural Network. I was given a scenario where there is a 2-input single unit backpropagation Neural Network has 3 initial weights. The inputs are x and y. The weights are WO, Wa, Wb.

To my prior knowledge, each weight is connected to each input. Thus, a 2-input neural network often has x connected to Wa and y connected to Wb. This makes the net, Z = x * Wa + y * Wb + bias.

Can there be three weights (W0, Wa, Wb) with two inputs (x, y)? If there is, how would the network architecture and the mathematics behind it look like?

$\endgroup$
0

1 Answer 1

1
$\begingroup$

All dense layers follow the rules for matrix arithmetic. We can write a dense layer (using the identity activation function for simplicity) as $Ax+b$ where $A$ is a $k \times 2$ matrix, $x$ is your $2\times 1$ input, and $b$ is a $k \times 1$ vector. So for a dense layer of $k$ units, there are $2k +k=3k$ numbers.

If $k=1$, then a layer can have 3 numbers associated with it, if and only if two of those numbers are weights, and one of the numbers is a bias.

If you insist that none of the numbers are biases and all of the numbers are weights, then this is not possible: $A$ must be $k \times 2$ because you have 2 inputs. This means that $A$ always has an even number of elements, so 3 weights isn’t possible.

$\endgroup$
1
  • $\begingroup$ Thanks. I figured that bias also requires a weight, so a neural network with 2 inputs also include that for the bias. $\endgroup$
    – Tragend
    Commented Jan 1, 2023 at 11:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.