I'm building a neural network classifier, lets say for cats and dogs.

I have a dataset that include a measure of intensity for the training examples. Dog example A may thus have an intensity of 200, while Dog Training B may have an intensity of 300. The intensity indicates how much they fit into the classes, so Dog Example B is 100 better than Dog Training Example A of containing the features of a Dog.

How can I include these intensity levels into my neural network, so that the neural network knows which training example's features to give more impact?

Would it make sense to just repeat the examples for their intensity levels, i.e. train the network 300 times on Dog Training example B per epoch, or would that lead to overfitting?

  • $\begingroup$ "I have a dataset that include a measure of intensity for the training examples" - so just to be clear, this intensity will not be available during inference time? $\endgroup$
    – kbrose
    Sep 17 '19 at 2:29

I would use these "intensity" levels to modify the loss function. From your description of these intensity values, I would intuitively want my model to be more sure about the more intense examples.

So for examples with a smaller intensity, I would decrease the loss function. For examples with a larger intensity, I would increase the loss function. In essence, I would penalize the model more for mis-classifying Dog Example B (intensity 300) than Dog Example A (intensity 200).

E.g., say $intensity$ is a normalized value ($0 < intensity \leq 1$), and $L$ is the loss function. I would train with a new loss function

$$ L' = L * intensity $$

It's important to scale your intensity values, otherwise your default learning rate will likely be out of scale.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.