I'm really struggling to see the analogy between linear regression and a single layer perceptron. They are supposedly the same thing.
I completely understand the concept of the inputs to the neuron being the explanatory variables, and having weights applied to them which are the $\beta$ coefficients, as well as the "bias" term which is the intercept in normal regression terms.
Mostly my problem is that I don't understand 1) whether the data is "fed in" to the neural network one data point at a time or all at once, and 2) whether an iterative method is used to find the $\beta$ coefficients or if it's just least squares. If an iterative method is used, that introduces the whole gradient descent thing. But when people say a single layer neural network and linear regression are the same thing, are they getting into backpropagation at all? Or are they just saying you can "view" regression in that way, and just minimize the loss function in the normal OLS way?
I hope I'm making at least some sense. Any help is really appreciated.