0
$\begingroup$

I noticed when training MLP that weights of neurons can be larger than 1. Would this have negative effects on the outcome of the network? If yes, how to mitigate this problem?

$\endgroup$
1
$\begingroup$

Of course they can be larger than $1$, since MLP does not enforce any hard constraint on its weights. In general, negative effects are produced when weights become very large, not when larger than specifically $1$. To mitigate such issues, you can apply regularization or increase its degree if you're already using it.

If you use the MLP implementation in scikit-learn, the regularization is achieved by alpha parameter. Typically, it's set to some value other than $0$, so you probably have some degree of regularization in your model. This parameter increases/decreases the level of L2 regularization you apply to the weights, which means punishing square of $W_{ij}$ severely for it not to become too large.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.