3
$\begingroup$

Let's say I have different sensors in an engine, and I make a neural net which predicts the engine's temperature given different operating conditions measured by the sensors. I happen to know the engine temperature stays in a 195 to 220 F range. How can I impose this constraint on my network? Will imposing it make the predictions any more accurate than simply setting values below/above this range to 195/220 F?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

There exist multiple strategies:

  • You can add an extra term to the cost function. This term should be a function of the predicted target variable and should be equal to zero when the target is in the desired interval and positive when it is not. It should be also smooth (differentiable) for obvious reasons.

  • You can apply a bounded activation function to your output such as sigmoid. Then, you can isolate the output between 195 to 220F by multiplying it with $220-195=25$ and then adding $195$. This will bound your output but you should note that sigmoid is a nonlinear function which makes the mapping itself nonlinear.

  • Apart from all these, you might not need to do anything since your model will tend to follow the output values in your training set which are already within the desired interval. You can apply clipping for those predictions falling outside of the interval.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ By 'clipping', you mean setting all values above 220 to 220 F, and all values below 195 to 195 F, right? There is something called gradient clipping which seems to be unrelated. (For those interested in how to implement the cost function, here is a relevant example). $\endgroup$
    – KAE
    Commented May 24, 2019 at 11:35
  • $\begingroup$ Yes, that is exactly what I mean by clipping. Gradient clipping is something different, but not entirely unrelated. During network training, the error gradient might become very large (exploding gradient problem) which leads to an unstable learning process (and network). Gradient clipping method clips these gradients with intent to prevent them from taking very large values. $\endgroup$
    – Monotros
    Commented May 24, 2019 at 11:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.