# How to represent the bias node in simple MLP?

I have a simple 2-2-1 fully connected network, which I suppose means it does not include bias nodes since you don't connect bias nodes at every layer. I have two sets of weights for each layer W2: (2 x 2) and W3:(1 x 2) and I only perform forward propagation using them. For backprop, I also only update these two sets of weights, like so:

 def backward(self,X,y):

n_examples = len(X)

yh,Z2,A2,Z3 =  self.forward(X,y)

delta3 =(yh-y)
de3 = np.dot(delta3,A2.T)
delta2 = (np.dot(self.W3.T,delta3)) * self.dsigmoid(Z2)
de2 = np.dot(delta2,X)

self.W3 =  self.W3 - self.lr * (de3 /n_examples)
self.W2 =  self.W2 - self.lr * (de2 /n_examples)

return de3/n_examples ,de2 /n_examples


I have tried including a third feature ($$x_{0}$$ as an array of 1s) and $$w_{0}$$ as the corresponding bias, but this messes up the dimensions. Specifically, Z2:(2 x N) where N is the number of samples, whose dimension does not match with the dimension of np.dot(self.W3.T,delta3), which is now (3 x N) and I can no longer take their Hadamard product without appending an artificial row of 1s for the bias node (shown below in red) to $$z_{2}$$, to make it (3 x N), which seems like a terrible way to me.

Is there a better way to include the bias? For instance, just adding it to $$z_{3}$$ during forward propagation like so:

 def forward(self,X,y):
...
z3 = self.params_l2.dot(a2) + b2
...


But then how would I update it during backprop?