I understand that loss functions encode how we want our learning system to behave in a sense. For example, take a binary classification task. There are two types of error: False negatives, and false positives.
Claim 1: If our error function penalizes false negatives more than false positives, then after learning our system will be more likely to make the error of a false positive.
Is this claim true?
And if so, how does this encoding happen exactly? I feel that it's an automatic consequence of minimizing the loss function but what's going on behind the scenes?