I'm currently working on a DQN network and this question comes to me. As far as I know, neural networks are good at dealing with values that have never seen (generalisation). E.g. If a classification model classifies two samples with the same features except for feature A, the first sample has a value of 10.0 and the second sample has a value of 9.9. And they are both classified correctly to class C. Then when tested on a similar sample which has its feature A value equals 9.85, it is most likely to be classified to class C as well.

However, if the comparison is the key to classification/regression. For example, the 'hidden rule' is: if feature A > feature B, then class C, otherwise class D. Since the comparison relation is not known, we cannot feed in the binary comparison result as a feature. Can neural networks still handle this well? Or some particular structure/feature engineering/algorithm can help with this situation.

For example, if feature B is equal to 9.88 in the above example, the model has seen samples with feature A values of 10.0, 9.9, 9.95, 9.89, and they all fall into class C, and another group of samples with feature A value of 8.0,9.0,9.7, and they fall into class D, then this 9.85 sample come into test. How can the model classify it correctly?

  • $\begingroup$ Learning the relation $$ \hat{y}_i = \begin{cases} 1 & \text{if $x_1 >x_2$} \\ 0 &\text{otherwise} \end{cases} $$ seems straightforward. The expression $x_1 - x_2$ is linear in $x$, and the coefficients in $w^\top x$ can be estimated in the usual way. Where are you stuck? $\endgroup$ – Sycorax Aug 20 '18 at 4:18
  • $\begingroup$ @Sycorax Thanks. I am doing reinforcement learning, and I'm expecting the model to outperform simple relationship like you stated, yet I am far behind. So I come up with this question. Looks like my model is having a hard time converging to a 'local optimal' solution as simple as this. $\endgroup$ – Kevin. Fang Aug 20 '18 at 4:25
  • $\begingroup$ Both neural networks and RL are hard to use. I have a hunch that the problem is estimating Q/model divergence, since this relation is straightforward. $\endgroup$ – Sycorax Aug 20 '18 at 4:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.