I'll give a concrete toy problem, then give some comments on what sorts of abstractions I care about.
Toy problem: Each person $i$ in my dataset has a phone, and every once in a while the phone will record a location. I then get to see an unordered list of locations $\{ (x^{(i)}_{j}, y^{(i)}_{j}\}_{j=1}^{N}$. I want to learn a classifier that tells me which people belong to a certain recreational soccer league that meets across the city at a few soccer fields. I want to use something like XGBoost to do the classifying.
A few observations:
- If I don't restrict myself to generic classifiers like XGBoost, I should be able to learn this sort of classifier from this sort of data. For example, I could look for locations that are common to people in the league but not to people out of the league. So this isn't a case of trying to solve an obviously-impossible problem - I just don't see how to use already-implemented algorithms to solve it.
- Out of the box, XGBoost is going to do badly at this, because it doesn't understand that my list of locations is mostly unordered (the ordering of x-coordinates and y-coordinates is meaningful). In particular, I think it should be looking at decision rules along the lines of "does any observation fall near point p," but as-implemented I think that it can only learn decision rules along the lines of "does the observation at index j fall near point p." In principle you can go from the latter to the former, but it is extremely difficult.
- If the city is small, I could of course do a one-hot encoding of discretizations of space. I want to avoid this solution, because it won't generalize well to most of the actual problems I care about.
Thanks for any thoughts! Ideally somebody will tell me that somebody has already implemented the sorts of decision rules I mention in (2) - this seems like a pretty generic thing to want to do, it is just the first time I happen to have run into it!
