6
$\begingroup$

This might be quite a broad question, but I was wondering if I could leverage the wisdom of the people on this site.

So I have recently started to learn about deep learning. I have written a simple feedforward network from scratch, learnt how to code up MLPs, and CNNs using various kinds of regularization techniques in Tensorflow.

I have started to read papers, and this was when it occurred to me that there is still so much to learn and I'm a bit overwhelmed by how much there is.

Furthermore, while studying RMBs I read that they are hardly used today for pre-training but rather batch normalization is used, so I don't want to be studying something that is already out of date!

As I am self-studying are there any suggestions as to what areas of deep learning that should take priority after MLPs, CNNs and RMBs? Or this question is too hard to answer, I guess the next best question is how would you study deep learning if you could do it again?

Improve this question
$\endgroup$
1
  • $\begingroup$ it should be learned deeply $\endgroup$
    – Aksakal
    Apr 3, 2019 at 13:47

2 Answers 2

Reset to default
2
$\begingroup$

Deep learning is quite of broad topic. I can not say that you can not learn everything ( human capabilities are unlimited ) but working in every field is little difficult.
As far as I can see you have grasped all the basic concepts so its time to choose a field where you want to apply your knowledge or do more research.
If you want to work choose a field where you want to apply your knowledge and learn with it. Natural Language Processing , Image processing , Object recognition, Audio signal Processing and many more fields are there. Choose a field and build something useful.
If you want to read more and do research, new experiments/research are being done daily. You can improve them or can develop your new concept. GAN ( Generative Adversarial Networks ), using Reinforcement learning with Deep learning, deep dream (Inceptionism: Going Deeper into Neural Networks ), Generative context aware encoder decoder and many more new things are there.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your answer what you said really makes sense! $\endgroup$
    – YellowPillow
    Feb 15, 2017 at 11:16
3
$\begingroup$

A good place to start is this book, you can download it online.

Quick recap and starting points:

  • If you're looking for image processing, CNN are a great choice and it seems you already played with it.
  • For speech recognition - you can take a look at RNN (recurrent neural networks). Basically you can use them for images and videos too. And (sic!) you can use CNN to process audio (especially if it's preprocessed with FFT to get a 2D spectrum)
  • NLP utilizes RNN as well. In order to extract features, you might take a look at Word2Vec. Additionally, encoder-decoder architecture is being used for machine translation.
  • (Active research, not for production) for statistical inference and unsupervised learning (anomaly detection etc.) - you might take a look at auto-encoders and RBMs. Autoencoders are particularly used as an alternative for Factor Analysis: Factor Analysis vs Autoencoders
  • Deep learning helps with reinforcement learning too: https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
  • Recommender Systems use deep-learning as well.

TensorFlow models (they are usually state-of-the-art): https://github.com/tensorflow/models

Small collection of Java-examples to complement those from TensorFlow: https://github.com/deeplearning4j/dl4j-examples

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.