I know for LASSO and elastic net regression, standardization is important, because coefficient penalization in regularization will be biased if the ranges of data are different.
Meanwhile, OLS regression does not run into this problem. We are just using stochastic gradient descent to find the best coefficients to minimize error. Maybe it would impact it if the step size or learning rate biases against larger ranged columns, such that you're not moving across as large of a range of the gradient for those columns? Maybe it's also that there's a closed-form solution for OLS, so even that issue with stochastic gradient descent doesn't matter?
So, given neural networks are essentially like multiple layers of regressions, why is data standardization recommended or more common practice? Is it because there's more layers of regressions, so standardizing just makes it more stochastic gradient descent more efficient, while efficiency to getting to a solution isn't really a problem in OLS?