Timeline for Impose a condition on neural network
Current License: CC BY-SA 4.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 1, 2022 at 14:40 | comment | added | Sycorax♦ | @RadioControlled More like gradient descent is an unconstrained optimizer, so if you want to enforce inequality or equality constraints, it's a non-trivial amount of work and not something that is supported out of the box. | |
| Feb 1, 2022 at 11:55 | comment | added | Radio Controlled | "constrained optimization is not something NN libraries are designed for" -- if I understood correctly, it would not be possible in general because a 0/1 condition is not differentiable? | |
| Jan 31, 2022 at 13:54 | vote | accept | Inevitable | ||
| Jan 30, 2022 at 14:09 | comment | added | Sycorax♦ | @Inevitable It's not trial and error -- one is just a rescaling of the other, so they are in a certain sense identical. The choice between them amounts to whether you want to worry about rescaling your learning rate when you change minibatch size. stats.stackexchange.com/questions/358786/… | |
| Jan 30, 2022 at 14:07 | comment | added | Inevitable | Yup I know the options but I thought some of theme has priority instead of try and error fashion of finding the good one. | |
| Jan 30, 2022 at 14:00 | comment | added | Sycorax♦ | @Inevitable Each example has loss something like $(\hat y - y)^2 + \lambda \text{ReLU}(i_3 - O)$. Some obvious options are $\sum_{j=1}^N \left[ (\hat y_j - y_j)^2 + \lambda \text{ReLU}(i_{3j} - O_j) \right]$ and $\frac 1 N\sum_{j=1}^N \left[ (\hat y_j - y_j)^2 + \lambda \text{ReLU}(i_{3j} - O_j) \right]$ | |
| Jan 30, 2022 at 11:40 | comment | added | Inevitable | I figured it out how to implement custom keras loss function. but the thing I'm thinking is how to deal with it when we have a array of outputs and inputs (rather than only one) is it a good idea to sum over all λReLU(i3−O) for each pair of input and output or you have better idea for it | |
| Jan 29, 2022 at 15:45 | vote | accept | Inevitable | ||
| Jan 31, 2022 at 13:54 | |||||
| Jan 29, 2022 at 15:03 | history | edited | Sycorax♦ | CC BY-SA 4.0 |
added 14 characters in body
|
| Jan 29, 2022 at 14:52 | comment | added | Sycorax♦ | This isn’t a code website, and I don’t use Keras/TF, but tensorflow implements addition, multiplication and ReLU. There’s no guarantee that this will always respect the inequality, but choosing a larger $\lambda$ will assign larger penalties to violations of the constraint. | |
| Jan 29, 2022 at 14:50 | history | edited | Sycorax♦ | CC BY-SA 4.0 |
deleted 1 character in body
|
| Jan 29, 2022 at 14:34 | comment | added | Inevitable | Thank you. I got the concept you explained. is this solution definite? I mean that if I implement it successfully the condition always is true or there will be the risk that sometimes I fall into the same issue. and for the last part would you mind append a snippet code for your answer in order to implement it in tensorflow (although I'm looking for the way to implement it now) It would be your kindness. Thank you again | |
| Jan 29, 2022 at 14:24 | history | answered | Sycorax♦ | CC BY-SA 4.0 |