What is known as "DAgger Problem" in imitation learning?

Question

I have seen in imitation learning papers that they have mentioned their algorithm suffered from "DAgger problem". Can someone please explain me this.

Is it about taking the same action by the agent again and again?

5. Conclusion
Although we have implemented a model that enables InfoGAIL to use visual input and intended to show that it increases the applicability of InfoGAIL, we instead discovered that InfoGAIL (and by extension GAIL) suffer from the “DAgger problem.” This is not through a fault of the original algorithms, but it is a fact of using distribution matching (like with a generative adversarial framework) with experts that take only one type of action or with environments that lend themselves to these kinds of skewed state-action distributions.

Source:

• Ivanovic, et al. Learning a Visual State Representation for Generative Adversarial Imitation Learning (pdf)

Answer 1

(Note that I don't have much expertise with this material.)

This seems to be a term they coined themselves in the same paper (my italics):

4.2. Model Generalization
In order to show that our image-based model generalizes to states that it has not seen before (even states that the expert has not seen before) we devised the following experiment on Game 1: Navigate to Object (“Go to Goal0”).
...
Very surprisingly, we do not see generalization at all. We actually see the agent taking more-or-less random actions in the unexplored states. We believe that this occurs not because of a fault in the GAIL or InfoGAIL algorithms, but because this “Grid World” environment and expert lend themselves to creating sharply-peaked state-action distributions. An example of how this could occur is shown in Fig. 5. Since the agent learns to match this state-action distribution, it will similarly always take actions to the right of the world. However, if it overshoots the goal (due to some noise in the agent’s output action), the agent will never make it back to the goal since its learned state-action distribution is completely skewed to taking actions to the right (to match the expert). Thus, it will never take actions to the left to correct its overshoot. We call this problem the “DAgger problem” as it is reminiscent of a previous RL method [15] that suffered from the inability to generalize because it cannot work on states that an expert has not been to.
...
[15] S. Ross, G. J. Gordon, and J. A. Bagnell. No-regret reductions for imitation learning and structured prediction. CoRR, abs/1011.0686, 2010.

I can't seem to find a copy of Ross et al. online, but the description seems clear enough to me. The algorithm learns to match an expert that has never been in a novel situation and the actions it has learned to mimic mindlessly¹ are inappropriate for the new situation.

_{1. This is editorializing on my part, but I think it's relevant here to remind ourselves here that strong AI doesn't exist yet and that's part of the explanation for why this fails in the way that it does.}