What is the difference between linear perceptron regression and LS linear regression? Recently, a project I'm involved in made use of a linear perceptron for multiple (21 predictor) regression. It used stochastic GD. How is this different from OLS linear regression?
 A: scikit-learn's Perceptron class (equivalent to SGDClassifier(loss="perceptron", penalty=None, learning_rate="constant", eta0=1)) uses the following objective function:
$$\frac{1}{N} \sum_{i=1}^N \max(0, - y_i w^T x_i).$$
In this case, $y_i \in \{-1, 1\}$. If $w^T x_i$ has the right sign, it doesn't incur any loss; otherwise, it gives linear loss. The perceptron in particular uses a fixed learning rate which can lead to some optimization weirdness as well.
Least squares regression, by contrast, uses
$$\frac{1}{N} \sum_{i=1}^N (y_i - w^T x_i)^2.$$
Here $y_i$ can be any real; you can give it classification targets in $\{-1, 1\}$ if you want, but it's not going to give you a very good model.
You can optimize this with SGDRegressor(loss="squared_loss", penalty=None) if you'd like.
The two define fundamentally different models: the perceptron predicts a binary class label with $\mathrm{sign}(w^T x_i)$, whereas linear regression predicts a real value with $w^T x_i$. This answer talks some about why trying to solve a classification problem with a regression algorithm can be problematic.
