Timeline for Loss function in machine learning - how to constrain?
Current License: CC BY-SA 4.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 16, 2020 at 16:25 | history | edited | Sycorax♦ | CC BY-SA 4.0 |
added 740 characters in body
|
| May 4, 2020 at 12:37 | comment | added | Sycorax♦ | @jkpate This is a good find! I can edit my answer to add this, but on the other hand, I think if you wrote an answer summarizing these findings, you'd get several upvotes. I know you'd get one from me. | |
| May 1, 2020 at 11:10 | comment | added | jkpate | Notice that if you incorporate a maximization over $\lambda$, then we do get exactly what the poster asked for because we now have a two-player formulation of the Lagrange dual to the original constrained optimization problem. Essentially, rather than setting $\lambda$ be fixed, we allow the penalty for a violation to grow. See Cotter et al. (2019) for the theory and github.com/google-research/tensorflow_constrained_optimization for a tensorflow implementation. | |
| May 1, 2020 at 4:20 | vote | accept | user570593 | ||
| Apr 30, 2020 at 20:47 | comment | added | Sycorax♦ | @MatthewDrury it might be worth adding your answer just because I’m interested in understanding why you prefer to square it. I’ve never used a loss like this so I’m just spitballing. | |
| Apr 30, 2020 at 19:42 | comment | added | Matthew Drury | This is pretty much what I would think of as well, though I was typing up $RELU(L_2 - L_1)^2$ as the penalty term. I've done this in practice for constrained minimization problems. It's a bit fussy, but can work out. | |
| Apr 30, 2020 at 18:30 | history | edited | Sycorax♦ | CC BY-SA 4.0 |
added 127 characters in body
|
| Apr 30, 2020 at 18:17 | history | answered | Sycorax♦ | CC BY-SA 4.0 |