The scenario is like this:

I have 2 point clouds, representing the radar readings of a car at the moment t and t + 0.1 seconds. Each point is a solid point that you don't want to drive trough, it's probably part of a solid object like a road cone. Objects on the road are hence represented by little clouds of points.

The points have been grouped into clusters (using dbscan lets say) and had their convex hull computed, so now we have the polygons that represent each object the car sees.

The next part is tracking an object across multiple such readings taken at t, t+1, t+2, etc.

If there is a road cone let's say in front of the car at moment t, and the car drives towards it, then at moment t+1 the cone should be of similar shape/area but slightly closer to the car (the coordinates of the vertices of the polygon change). Not closer by a whole lot, just a bit closer.

We can algorithmically map the road cone at moment t to the one at moment t+1 by allowing some tolerances like "if its area is less than 10-15% different that the road cone at moment t and it is not farther than 3 meters from the road cone at moment t then it is probably the same road cone.".

However I am looking for a solution that involves some kind of ML algorithm. It would probably have to involve some kind of clever application.

The most I could come up with was classification, where every object from moment t represents a class, so we try to pair each object from t+1 to an object from t. But this has several design issues such as there needs to be at most 1 object classified in each class (so if a class receives more than 1 object, we need to be able to... send the 2nd object away to find a new class to fit into? I hope this makes sense) and there needs to be some way to have a class for objects that are newly detected and not close to any other object. So I think pure classification may not do the trick.

I am looking for a way to do this using ML. This would be a summer project for a student at my company, so the ideal solution would not require 2+ months of programmer time from an experienced programmer.


1 Answer 1


You could think of this as a distance learning task. You could train your model with triplet loss and the output of your model would be some vector in a latent space such that the same objects would be really close together regarding some standard distance measure, e.g. cosine similarity. See Wang, et al for an example that matches yours.

However, this approach might be prohibitive compute consuming for real-time applications in a car, depending on the hardware that would be available.

I supervised a student who did something similar from a different domain (cover song identification) for his bachelor thesis. If your intern has some experience with neural networks this could be done as a summer project. Exploring 3D data and getting the data right to feed it into the network can be time consuming. I think it would be crucial to provide the student with tools to visually explore the data and the data in such a format that the data wrangling does not take too much time (e.g. grouping and labelling the objects already done).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.