2
$\begingroup$

Are the weight of a neural network updated differently due to back propagation for a classification network vs. a regression network, if so how?..

My concern comes due to the both network uses different cost function, hence the update must be different as well.

I am under the intuition that log regression often are used for classification, and linear regression is used for normal neural network for regression problems..

Hence must the update of the weight also be different.. If the statement is correct? how does one in a classification process update the weights differently?..

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ I don't know what you mean by "different". If you have two functions, $f(x)$ and $g(x)$ which are different (i.e. $f(x) \ne g(x)$) then is the process of calculating $\frac{d}{dx} f(x)$ different than calculating $\frac{d}{dx} g(x)?$ What ever you consider to be the answer to that question is exactly the answer to your question. $\endgroup$
    – Bridgeburners
    Commented Sep 22, 2017 at 21:00
  • $\begingroup$ I am under the impression that classification task usually uses cross entropy as cost function, and regression network uses the mean squared difference between target and output.. Hence would the way the weight being trained in the classification also be different, but how are the weights being updated? @Bridgeburners $\endgroup$
    – Bob Burt
    Commented Sep 22, 2017 at 21:10
  • 4
    $\begingroup$ Backpropagation simply means taking the derivative of the cost function with respect to the weights. (More formally, taking the gradient, which is really just a vector of the derivative with respect to each weight.) You already understand that regression NNs and classification NNs have different cost functions. So backpropagation is only different between the two in the sense that I described in my previous comment. Put another way: if you know calculus, and you know how to learn the weights of a regression NN, no new insight is required to learn the weights of a classification NN. $\endgroup$
    – Bridgeburners
    Commented Sep 22, 2017 at 21:15

1 Answer 1

Reset to default
1
$\begingroup$

This question is basically answered in Bridgeburners' comment.

The backprop algorithm is the same in both the classification and regression contexts. The only difference is that the loss functions are (often) different, so the derivatives of the loss are likewise different. But the principles are exactly the same -- if you know how to find a derivative and how to do backprop for regression, then you also know how to do backprop for classification (and vice versa).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.