In what form is optical flow used an input to a neural network? [closed]

I have extracted the optical flow of images. Should I pass the x,y optical flow or should I pass the RGB visualization as the input. What is the difference between the two?

• what is the "RGB visualization" – shimao Apr 18 '18 at 6:45

You should pass in the optical flow field, and not a visualization of that field. The difference is that one is the optical flow, and the other is an arbitrary function of the flow $f: \mathbb R^{H \times W \times 2} \rightarrow \mathbb R^{H \times W \times 3}$.