I'm replicating this paper for my PhD, which says that they are using deep learning to predict stock returns. So the inputs are (mostly) continuous variables that can be negative and positive. Outputs are stock returns which can be [-1, inf). The paper is rather vague in their methodology but mentioned that they are using ReLU for their hidden layers and linear activation for output. So far I haven't had any luck getting anywhere near reasonable results which led me to question every parameter in my network. So my question is, how does ReLU work when expected outputs can be negative?
$\begingroup$
$\endgroup$
3
-
$\begingroup$ Are you sure they're using ReLUs for the output layer, instead of only in the hidden layers? $\endgroup$– Sycorax ♦Commented Aug 28, 2019 at 12:22
-
$\begingroup$ They are using linear output layer. I guess this is indeed very similar to the other question. Thanks for pointing out. $\endgroup$– stevewCommented Aug 28, 2019 at 12:56
-
$\begingroup$ In any case, the answer turns on whether or not the universal approximation theorem applies to ReLUs. It turns out that the answer is "yes." arxiv.org/pdf/1708.02691.pdf If we accept that the UAT is applicable to your problem (a strong assertion), then the question becomes "Why is my network not working very well?" for which we have several threads: (1) stats.stackexchange.com/questions/352036/… and (2) stats.stackexchange.com/questions/365778/… $\endgroup$– Sycorax ♦Commented Aug 28, 2019 at 13:07
Add a comment
|