2
$\begingroup$

I am trying to train a neural network that takes Height(X0) and Weight(X1) as input, output the corresponding prediction to calories(y).

First 4 columns below are part of the training set, the last column is the prediction by the model.

   Height(X0)  Weight(X1)  Calories(y)  Measurement(C)  Prediction(y_hat)
1      177.11       69.85       2800.0             0.0             2800.1
2      167.11       59.85       2600.0            -0.1             2800.2
3      197.11       79.85       3500.0             0.2             2800.3
...

Calories(y) is the target value in the dataset for training.

"Measurement" is some kind of measurement of Calories(y), whose absolute value is closer to 0, the better, which indicates how precisely the values of Calories(y) is. The negative and positive sign of C denote the direction, negative means less than, positive mean greater than.

For instance, first record means 2800.0 Calories is the exact amount a person needs whose Height(X0) = 177.11 and Weight(X1) = 69.85;

2nd record means 2600.0 is actually less than the exact amount a person needs whose Height(X0) = 167.11 and Weight(X1) = 59.85;

3rd record means 3500.0 is actually greater than the exact amount a person needs whose Height(X0) = 197.11 and Weight(X1) = 79.85;

Well, those numbers are just used to explain how does the system work, may be wrong to nutritionists.

As to the last column, the learning/training algorithm should keep the prediction for record 1 around 2800, so, 2800.1 is pretty good; prediction for 2nd record would be a bit greater than 2600.0; prediction for 3rd record would be a bit less than 3500.0.

The question is that:

How to put the preceding knowledge "Measurement" in training?

$\endgroup$

1 Answer 1

2
+50
$\begingroup$

I think the most sensible way to incorporate this information is by adding it to your loss function when training your model.

For example we could take the standard loss function of mean square error or mean absolute error and modify it.

You say it is another question to encode what value the model should predict given $B$ (which I'm going to refer to as $y$) and $C$. Lets just say there is a function $f$ that exists that encodes this. So the best prediction would be $y+f(C)$ instead of $y$ and in the simplest case might be something like $f(C)=C$.

You could then use the loss function something like

$loss = \frac{(\hat{y}-(y+f(C)))^2}{|\hat{y}-y|}$ if $sign(\hat{y}-y)\neq sign(C)$ else $V$

where $V$ is just some large value. This is shown in the code and plot below. This gives an equivalent of the mean absolute loss.

You could modify this to higher order (to mean square loss) using something like

$loss = \frac{(|\hat{y}-(y+f(C))|)^3}{|\hat{y}-y|}$

set order=2 below.

Hopefully this form is enough to get you started.

import numpy as np
import matplotlib.pyplot as plt

y0=0 # value of target

ys = y_hats=np.linspace(-3, 3, 10000) # potential predictions of y0, the target

def f(c):
    return c

def loss(y_hat, y, c, order=1, clipval=10):
    sign_penalty = (np.sign(y_hat-y)!= np.sign(c))*clipval
    value_penalty = np.abs(y_hat-(y + f(c)))**(order+1)  * np.abs(y_hat-y)**-1
    penalty = sign_penalty + value_penalty
    return np.clip(penalty, 0, clipval)

plt.plot(ys, loss(ys, y0, c=1, order=1), label='$c=1$')
plt.plot(ys, loss(ys, y0, c=-0.5, order=1), label='$c=-0.5$')
plt.xlabel('y_hat')
plt.ylabel('loss')
plt.legend()
plt.show()

enter image description here

$\endgroup$
6
  • $\begingroup$ Your answer is very informative. Thank you so much. Does y0 in code refer to the "B" or "C" in OP? The answer says y denotes "B" while the comments in code says y0 represents the measurement, which is actually "C" in OP. $\endgroup$
    – JJJohn
    Apr 3, 2020 at 22:53
  • $\begingroup$ Why does $loss = \frac{(\hat{y}-(y+f(C)))^2}{|\hat{y}-y|} = 0 \text{ or } =10$ could represent sign? In other words, what does the large value clipval(V) do? $\endgroup$
    – JJJohn
    Apr 4, 2020 at 3:54
  • $\begingroup$ I see you have edited the question since I answered, but now I think our notation lines up. y0 would be $y$ as you have it in your edited question. Our y_hats are the same. $\endgroup$ Apr 6, 2020 at 11:47
  • $\begingroup$ The reason for this clip value is just to set some upper (and lower) limit on what the loss could be, and also to remove the symmetry of the graph around zero. Otherwise if you predicted $\hat{y}-y$=0 then your loss would go to infinity, and $loss(-y)=loss(y)$ (if y0=0). I arbitrarily chose an upper limit of 10 because that made the graph look nicer (I set the lower limit of 0 but that does nothing as this loss will never be negative anyway). You can see what the clipval does by taking this code and playing around with it. $\endgroup$ Apr 6, 2020 at 11:52
  • $\begingroup$ Also, generally I just meant this as an example metric you could pick and wanted to set you up with some code to experiment with and modify to choose a metric. If C=-0.5 then the loss for $y_hat=y$ is the same as $y_hat=y-11$ and maybe $y_hat=y$ is actually a 'better' estimate? Also note that $y_hat=y$ gets the same penalty as any other value where $y_hat>y$ and maybe you might like to put a slope on this loss so that worse bad estimates are further penalised. $\endgroup$ Apr 6, 2020 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.