# Representing bias term in simple linear neural network(linear regression) using analztical solution

Assume that output $$y$$ depends on input $$x$$ and some noise $$\epsilon \sim N(0,\sigma^2)$$. $$y = f(x) + \epsilon$$

Suppose we want to model relationship mentioned above using linear neural network:

$$\hat{y} = w * x + b$$

where $$w$$ is weight matrix and $$b$$ is a bias term of neural network.

We can calculate weights using classical analytical solution for OLS:

$$w = (X^TX)^{-1}X^TY$$

Question: how do we calculate (or maybe represent) bias term $$b$$ in neural network without using gradient descent?

In OLS, $$X_{m\times n}$$ matrix is the form $$\begin{bmatrix}\mathbf x_1 & \dots & \mathbf x_{n-1} & \mathbf 1\end{bmatrix}$$ where each element represents an $$m\times 1$$ vector. First $$n-1$$ are the feature vectors and the last one is all-1 vector that is to be multiplied with the bias. So, the formula for OLS solves for the bias already using the model:
$$y = X\begin{bmatrix}\mathbf w\\ b\end{bmatrix}+\epsilon$$