1
$\begingroup$

is it true that:

After training a multilayer perceptron with one hidden layer using gradient descent, we always get the same decision boundary regardless of the initialization point.

$\endgroup$
  • $\begingroup$ The titular question does not match the text in the question itself. $\endgroup$ – Frans Rodenburg Oct 29 at 7:40
  • $\begingroup$ I thought that the two are equivalent? $\endgroup$ – user42493 Oct 29 at 12:56
  • $\begingroup$ Sorry, let me elaborate. A function can have a single global minimum, but that does not exclude the possibility of local minima or saddle points. Hence, having a global minimum does not mean your optimizer will reach it irrespective of the starting point. $\endgroup$ – Frans Rodenburg Oct 29 at 13:51
1
$\begingroup$

NO.

With one hidden layer, the objective function is not convex (even for squared loss in regression). Therefore, we may have many local minima instead of one global minima.

Using gradient decent and where to end, will depend on the initialization point.

$\endgroup$
  • $\begingroup$ why is the cost function not convex? $\endgroup$ – user42493 Oct 29 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.