Let's say I am trying to learn a classifier. I have a set of observations, with output $y_i$ and features $a_i$ and $b_i$. But for each observation, I know that if I have output $y_i$ for features $a_i$ and $b_i$, increasing $b_i$ while keeping $a_i$ fixed will also yield the same output $y_i$. I know this from domain knowledge, but it's rare to see two observations where $a$ is the same, so it may not be directly visible from observed data.
Example: imagine that our observations are stock buy and sell decisions. Let's say $y_i$ is 1 if the stock was bought, and 0 if it was sold, and we have two features - $a$ is some observation about the company, and $b$ is price. So if in our supervised learning example, we see the observation that the stock was bought at price $b_i$, that means that it would also have been bought at any price lower than $b_i$, all other features being equal. But we do not know if it should be bought or sold at prices higher than $b_i$. Likewise, if it was sold at some price, it would have also been sold at any higher price.
How should I encode that fact in my training data? Should I just generate "fake" observations with output $y_i$ and different values $b > b_i$? If so, how many and at what intervals? Or is it a bad idea, because the observation may be erroneous, and adding additional "fake" observations that way would give it more weight?
