I'm a software engineer working on Deep Learning, but mostly for NLP. I have a Raspberry Pi 3 and some cameras and wanted to play around a bit with target tracking (implementing models from scratch in TensorFlow, not using OpenCV or similar), i.e. actually understand the techniques. What are currently the most important papers in this area? I was unable to find any good reading list for this topic and there are enough papers that starting out it's hard to know which techniques to focus on.

  • $\begingroup$ SISR (sequential importance sampling resampling) technique may be a good starting point. You can check this one: "A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking". (2002 paper, quite old now). $\endgroup$
    – ahstat
    May 30 '17 at 14:59

Start with the best.

The results of the ILSVRC 2017 competition were released yesterday (July 17, 2017). The winner in the two tracking categories, Task 3c (Object detection/tracking from video with provided training data) and Task 3d (Object detection/tracking from video with additional training data), was this team:

Jiankang Deng(1), Yuxiang Zhou(1), Baosheng Yu(2), Zhe Chen(2), Stefanos Zafeiriou(1), Dacheng Tao(2), (1)Imperial College London, (2)University of Sydney

Here are their publications, source code, and a presentation:

[1] Deep Feature Flow for Video Recognition Xizhou Zhu, Yuwen Xiong, Jifeng Dai, Lu Yuan, and Yichen Wei, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

[2] Flow-Guided Feature Aggregation for Video Object Detection, Xizhou Zhu, Yujie Wang, Jifeng Dai, Lu Yuan, and Yichen Wei. Arxiv tech report, 2017.

Presentation https://www.youtube.com/watch?v=J0rMHE6ehGw

Source Code https://github.com/msracver/Deep-Feature-Flow

The code has the following prerequisites:

  • Python 3.2.0+
  • Microsoft's MXNet
  • Cython
  • OpenCV (Python bindings)

Their code requires a GPU with at least 6GB of memory.

Another option is ROLO. The author is Guanghan Ning and he uses You Only Look Once (YOLO) for detection and uses TensorFlow to implement LSTMs for tracking.

His published a paper: Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking, IEEE International Symposium on Circuits and Systems, 2017

His code is here: https://github.com/Guanghan/ROLO

Project page: http://guanghan.info/projects/ROLO/


  • Python 2.7 or 3.3+
  • TensorFlow
  • Scipy
  • OpenCV (Python bindings)

Some videos of his work:


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.