1
$\begingroup$

I know that logistic regression is for binary outputs and Softmax is for multiple classes. Would it be fair to say that Softmax regression is the same thing as multiclass Logistic regression?

$\endgroup$
0
$\begingroup$

Softmax regression is a generalization of logistic regression.

Remember in logistic regression labels and model parameters were: $$ y^{(i)} \in \{0,1\},\space \theta = \begin{bmatrix} \theta_1 \\ \theta_2 \\ \vdots \\ \theta_n \end{bmatrix} $$ Whereas in softmax regression labels and model parameters are: $$ y^{(i)} \in \{1, 2, \ldots, K\},\space \theta = \begin{bmatrix} \theta_1^1 & \theta_1^2 & \theta_1^k \\ \theta_2^1 & \theta_2^2 & \theta_2^k \\ \vdots & \vdots & \vdots\\ \theta_n^1 & \theta_n^2 & \theta_n^k \\ \end{bmatrix} $$ In that sence it is easy to see that logistic regression can be expressed as softmax with two classes. Of course, cost functions and hypothesis are a bit different.

Would it be fair to say that Softmax regression is the same thing as multiclass Logistic regression?

It's more about the naming convention: when talking about a two-class problem, we usually call it logistic regression.

Reference: http://ufldl.stanford.edu/tutorial/supervised/SoftmaxRegression/

$\endgroup$
  • 3
    $\begingroup$ It's true that softmax regression (aka multinomial logistic regression) is a multiclass generalization of logistic regression. But this answer doesn't show why. Maybe you could edit to add more detail. $\endgroup$ – user20160 Apr 17 '18 at 9:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.