I am working on a problems that consist of simple 2D scenes, a typical example being a top-down traffic situation encountered by an autonomous vehicle. These structured scenes have multiple actors that can have area, direction (and its first and second order temporal derivatives, i.e. velocity and acceleration), and position, as well as a representation of the "background" (think road layout, or paths).
I am wondering if anyone is aware of any prior work that allows a principled calculation of distance between such scenes? I was thinking along the lines of optimal transport? But I'm not aware of any work that allows comparison of different structured domains (i.e. different number of vehicles) that can't readily be represented as a graph.
I am not interested in metric learning, so scoring with a parameterized generative model is not a valid approach.