1
$\begingroup$

How does a deep neural network, trained for regression with back propagation, deal with cases when different training pairs have the same input value, but different output values, i.e. the relationship between input and output is not smooth?

In other words, suppose I train a deep neural network for regression from $x$ to $y$, using back propagation. Two of my data pairs have the same $x$ ($x_1 = a$ and $x_2 = a$), but very different $y$'s ($y_1 = b$ and $y_2 = c$). After training the network, what would be the output $y_k$ for an input of $x_k = a$? Would it be $b$, or the average of $b$ and $c$?

$\endgroup$
1
$\begingroup$

Have you trained some network with the setting you have mentioned? I have not yet, but if we think in some function like $y=\sqrt{x}$ or $y=sin^{-1}(x); x\in [0,2\pi ]$, these are simple functions, however if you want to approximate them, complex values should be supported by the learned transformation $x\mapsto f(x) = y$, i.e. in order to approximate functions like the mentioned ones, the network's architecture must be endowed with complex valued basis funtions 1. Otherwise, the imaginary part of the image of $f(x)$ could be lost. It is seemed to the case of approximating an injective function, where more than a $x$ corresponds to a $y$, e.g. $y = f(x) = x^2$. Then try to approximate $f^{-1}(y) = x$.

I hope this helps.

Complex valued neural networks

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.