# Temporal Distance Intervals Classification Task

I was reading a paper Playing hard exploration games by watching YouTube https://arxiv.org/abs/1805.11592.

By my understanding the authors use convolutional neural networks to generate embeddings for the game state and the demonstration trajectory from a video recording (which is being learned from). They then use the embeddings for the game state and the video recording at some checkpoint to present an imitation reward to the learning agent (the only learning signal the agent receives). They use this signal to train a standard reinforcement learning agent.

However I do not understand what the authors are doing when they are predicting the Temporal Distance Intervals. Are they just predicting how different the embedding of the next frame should be? Or are they predicting what the agents next action should be?

I don't understand how predicting the Temporal Distance Intervals is related to generating the optimal policy for controlling the agent.

## 3.1 Temporal distance classification (TDC)

We first consider the unsupervised task of predicting the temporal distance ∆t between two frames of a single video sequence. This task requires an understanding of how visual features move and transform over time, thus encouraging an embedding that learns meaningful abstractions of environment dynamics conditioned on agent interactions. We cast this problem as a classification task, with K categories corresponding to temporal distance intervals,dk ∈{[0],1,[2],[3−4],[5−20],[21−200]}.Giventwoframesfromthesamevideo, v, w ∈ I, we learn to predict the interval dk s.t. ∆t ∈ dk. Specifically, we implement two functions: an embedding function φ : I → RN , and a classifier τtd : RN × RN → RK , both implemented as neural networks (see Section 5 for implementation details). We can then train τtd(φ(v), φ(w)) to predict the distribution over class labels, dk, using the following cross-entropy classification loss:

where yi and yˆi are the true and predicted label distributions respectively.