As it is well known, deep learning can help to learn the features. However, are there any ways for us to either understand or visualize these learned features? Moreover, how to explain the patterns based on these features?
For instance, I can build a deep-learning based classifier to categorize car and airplane. Assume we use CNN, and the first layer may show some edges; while the second layer may show wheels or wings, etc. But if I want to ask, based on this CNN model, what are the patterns to help us distinguish car from airplane? How can we answer it? Are there any paper address this kind of question?