2
$\begingroup$

We all know about the success of ImageNet, AlphaGo etc which used deep neural networks in computer vision, or the use of RNNs in Google Translate. But why are we not seeing similar advances in other fields like finance?

$\endgroup$
  • 2
    $\begingroup$ There's plenty of deep learning in finance: sinews.siam.org/Details-Page/deep-learning-models-in-finance-2 $\endgroup$ – Alex R. Jul 31 '17 at 17:22
  • 1
    $\begingroup$ There is a lot of buzzword bllsht everywhere, including finance. Any reports of a company making billions because of deep learning! Not as far as I know. Overfitting looks great, but fails badly in reality. $\endgroup$ – Anony-Mousse Jul 31 '17 at 19:27
  • $\begingroup$ "We all know about the successes of Imagenet, AlphaGo, etc." -- some readers (like myself) may not be aware of the success. Can you provide a reference for this statement? $\endgroup$ – Jon Jul 31 '17 at 21:51
2
$\begingroup$

This is obviously a question for a detailed discussion, but here is the simplest answer I can come up with.

The reason that neural networks are a good model for complicated tasks like modelling high-level tasks like image understanding is that they are universal approximators; Put very blantly, you can approximate any function with a network with sufficient capacity.

However, this is only true in theory: In practice, the search space of the opimization problem is so huge, that actually getting to a good solution is very difficult (this is essentially the expected "no free lunch" result).

Most of the success of deep learning in the last few years have been around making the optimization easier to solve. Generally, there are two approaches to doing this:

The first approach is just to "hand craft" optimization procedures that work well for the types of functions we usually want to estimate in DL (typically, high-dimensional and very non-convex). There are numerous examples, from dropout to different variants of gradient descent methods.

The second approach is to incorproate some structure into the network's design. Essentially, translate some real-world knowledge you have about the problem you're trying to solve, in order to make the search space (hypothesis class) significantly smaller.

I claim that while the first approach is undoubtedly useful and important, it's not enough. Hence, the reason that most of the "success stories" in Deep Learning are in the field of computer vision and speech recognition is that in this field scientists have been able to impose the "correct" type of structure - namely, convolutional architectures. By "correct", I mean that convolutional models are significantly simpler models (much fewer parameters), but we do not lose anything in the sense that we can still express the sort of functions we see in natural images and natural audio.

I believe that the right way of using domain knowledge from other fields - economics, biology - to simplify the search space of neural networks is still an open question. Progress on that will definitely lead to some nice results like the ones we have in vision.

$\endgroup$
  • 1
    $\begingroup$ Isn't a lack of good datasets a big problem? For computer vision, at least, the big breakthroughs came shortly after ImageNet became available. For speech getting good datasets is reasonably easy. In finance or economics? It is extremely difficult as we do not even know which labeling would be "correct". In financial markets, merely making predictions about it can change the way it operates, which is not the case for images or sounds. There is way more noise and a lot less signal in the actual price data. We cannot generate new data like in games of Go or Chess or even poker. $\endgroup$ – rinspy Aug 1 '17 at 8:07
3
$\begingroup$

Deep neural networks need a LOT of data to power up. Not all the domain has huge amount of data like computer vision and natural language processing. (For example, in Internet, we can easily get millions of images about cat / dog. Or build a web crawler to get millions of articles about "computer network".).

On the other domains, for example, medical research, the data is very expensive and hard to get (think about controlled trails).

Without huge amount (if not "infinite data"), having a very complex model may lead over fitting. In addition, with a deep neural network, we lose the power of "explaining things / interpolation".

In addition, here is a slide of "basic recipe for machine learning" by Andrew Ng. Note, we need bigger network when the model is not able to handle existing training data. With relative limited training data in other fields, not-so-big network (if not classical linear methods) is sufficient.

enter image description here

$\endgroup$
  • $\begingroup$ How about finance, in particular high frequency trading, which would have massive amounts of data with microsecond timestamps over many many years? $\endgroup$ – housecat64 Jul 31 '17 at 17:24
  • 3
    $\begingroup$ The problem may be that the data generating process is constantly evolving in a rather unpredictable way, thus data from a year ago is not representative of data to be observed tomorrow. $\endgroup$ – Richard Hardy Jul 31 '17 at 17:27
  • 1
    $\begingroup$ @RichardHardy That itself is a problem deep learning could handle, don't you think? $\endgroup$ – horaceT Jul 31 '17 at 17:45
  • 2
    $\begingroup$ @horaceT, more importantly, I think the financial markets are inherently unpredictable in the following sense. Once someone finds a pattern according to which the market currently develops, one invests in the market and by that very action changes the supply-demand balance and thus changes the pattern. It is like some elementary particles in physics: once you measure them, they are changed forever. You cannot measure the particle without changing it. I think financial markets are similar. It is similar with macroeconomics, too, where macroeconomic policy affects the economy. $\endgroup$ – Richard Hardy Jul 31 '17 at 18:23
  • 1
    $\begingroup$ @horaceT, reinforcement learning requires you to explore the state space many times, to learn which states different actions from different states take you to, and the expected rewards of taking certain actions in certain states. The state space in finance is huge, but more importantly, we cannot simulate it effectively. Now, if we could develop good agent-based models... $\endgroup$ – rinspy Aug 1 '17 at 8:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.