Reference request - Computer Vision Book

Question

What are the best books for obtaining a strong understanding of computer vision? From what I understand based on my undergraduate class, almost all current state-of-the-art computer vision is just relying on deep learning, particularly convolutional neural networks.

• I've read through the Courville-Goodfellow-Bengio book, so I feel like I have a broad understanding of deep learning as a whole, but I don't feel like I have mastery over any of the specific topics that their book discusses since only the chapter about convolutional neural networks and a section about applications of convolutional neural networks really ever talk about computer vision.
• I've also read through Bishop's book on machine learning and Murphy's book on probabilistic machine learning, so I think I have a broad understanding of machine learning as a whole but am much less knowledgable about specific subdomains.
• I also see the Szeliski book on computer vision recommended to me, and I haven't read it yet. However, based on the contents, it doesn't seem like that book covers neural network usage in computer vision, and considering the recent rapid growth of the field using convolutional neural networks, I'm not sure if the material in this book would then be considered outdated. Is it still necessary to read this book (or a book of equivalent material) to be able to do state-of-the-art research in computer vision. If not necessary, would it still at least be valuable for me?

What are any other recommendations? While I would prefer more comprehensive texts, paper and survey paper recommendations are also good here too. I have a very good grasp of both pure mathematics and statistics, so references leaning on the more theoretical ends are fine too.

Answer 1

To the best of my knowledge there is not yet a comprehensive academic computer vision textbook (as of 2019) that has been written that incorporates the deep learning.

It's useful to separate the discussion and formulation of a problem from algorithms deployed to solve it. @shimao makes good point that often earlier methods are recycled with new deep learning components. Goodfellow et al., and Bishop's book are good books on deep learning and machine learning, respectively but they don't talk much about computer vision problems. By that I mean the tasks the occupy a lot of the computer vision research like noise filtering, 3D reconstruction, image registration, computational photography, structure from motion, etc.

To better understand CV, Szeliski's book is still quite good, although it is a high level survey. Topics which CNNs dominate such as segmentation and recognition are only two chapters in that book so there's lots of interesting material.

I think the following books are also useful and worth having a look at although they cover classical methods:

Prince, Simon JD. Computer vision: models, learning, and inference. Cambridge University Press, 2012.

-

Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003.

-

Gonzalez, Rafael and Woods, Richard. Digital Image Processing. Pearson Higher Ed, 2011.

If you are really interested in CV, I think it can also be interesting to learn about human visual perception and optics as well as related fields like robotics, computer graphics and medical/scientific imaging.

Answer 2

To add to @MachineEpsilon's answer. Deep learning has become the de-facto tool for tasks like segmentation and object detection. And is starting to take over in domains like 3D reconstruction. Nevertheless, it is still an ongoing process.

You still need to have a good knowledge of projective geometry (Hartley and Zimmerman's book) to solve tasks like metrology (performing accurate measurements on images). There are situations where you want to extract some features (like edges or contours). Also, matching and tracking planar objects with feature based methods are still a very good option, and easier to set up, compared to fine tuning a neural network.

This techniques are also at the core of many SLAM approaches.

My point is: there are still many relevant use cases where either deep learning does not provide a (satisfactory) solution, or where traditional approaches are easier to use. If only because there are well-proven software libraries like OpenCV.