Return to Question

added 113 characters in body

Source Link

edited May 26, 2020 at 10:48

I was looking at an interesting case concerning popular locations for uber, see here: https://mapr.com/blog/real-time-analysis-popular-uber-locations-spark-structured-streaming-machine-learning-kafka-and-mapr-db/

All good and well. The K-means clustering was done on a time-invariant basis to get centres / clusters using latitude and longitude only, but against a sample for 32 days. Then the clusters were segmented by hour/day for popularity. OK, I can get that approximation.

When I think of K means clustering applied to the famous whiskeys, then I can get that this is time-invariant.
But when I think of this uber location popularity, I would have intuitively thought that the clustering would need to be performed multiple times by running samples against data for an hourly or two hourly interval - that is to say less than 32 days.
So, given (I think) that it is hard to visualize K-means clustering with a temporal aspect included as feature, is the approach then indeed to 1) cluster like this against all features for a largeR period standardly and then 2) slice per time interval to get the popular locations?
And is this approach then just as good as a more fine-grained clustering approach using less data points due to / on a reduced time scale? I am hoping this is not a silly question as I think the folks at uber have more than a few clues. May be it is just a demo and not how one would do this in reality.
My take is that this approach can be improved, but may be not significantly compared to the mapr described method. I leave aside the issues of boundaries on a map. Final say from me: the approach uber has taken may well be good enough compared to more-fine grained approach.

I was looking at an interesting case concerning popular locations for uber, see here: https://mapr.com/blog/real-time-analysis-popular-uber-locations-spark-structured-streaming-machine-learning-kafka-and-mapr-db/

All good and well. The K-means clustering was done on a time-invariant basis to get centres / clusters using latitude and longitude only, but against a sample for 32 days. Then the clusters were segmented by hour/day for popularity. OK, I can get that approximation.

When I think of K means clustering applied to the famous whiskeys, then I can get that this is time-invariant.
But when I think of this uber location popularity, I would have intuitively thought that the clustering would need to be performed multiple times by running samples against data for an hourly or two hourly interval - that is to say less than 32 days.
So, given (I think) that it is hard to visualize K-means clustering with a temporal aspect included as feature, is the approach then indeed to 1) cluster like this against all features for a largeR period standardly and then 2) slice per time interval to get the popular locations?
And is this approach then just as good as a more fine-grained clustering approach using less data points due to / on a reduced time scale? I am hoping this is not a silly question as I think the folks at uber have more than a few clues. May be it is just a demo and not how one would do this in reality.
My take is that this approach can be improved, but may be not significantly compared to the mapr described method. I leave aside the issues of boundaries on a map.

I was looking at an interesting case concerning popular locations for uber, see here: https://mapr.com/blog/real-time-analysis-popular-uber-locations-spark-structured-streaming-machine-learning-kafka-and-mapr-db/

All good and well. The K-means clustering was done on a time-invariant basis to get centres / clusters using latitude and longitude only, but against a sample for 32 days. Then the clusters were segmented by hour/day for popularity. OK, I can get that approximation.

When I think of K means clustering applied to the famous whiskeys, then I can get that this is time-invariant.
But when I think of this uber location popularity, I would have intuitively thought that the clustering would need to be performed multiple times by running samples against data for an hourly or two hourly interval - that is to say less than 32 days.
So, given (I think) that it is hard to visualize K-means clustering with a temporal aspect included as feature, is the approach then indeed to 1) cluster like this against all features for a largeR period standardly and then 2) slice per time interval to get the popular locations?
And is this approach then just as good as a more fine-grained clustering approach using less data points due to / on a reduced time scale? I am hoping this is not a silly question as I think the folks at uber have more than a few clues. May be it is just a demo and not how one would do this in reality.
My take is that this approach can be improved, but may be not significantly compared to the mapr described method. I leave aside the issues of boundaries on a map. Final say from me: the approach uber has taken may well be good enough compared to more-fine grained approach.

Source Link

asked May 25, 2020 at 21:14

Ged

K-means clustering and temporal aspects

I was looking at an interesting case concerning popular locations for uber, see here: https://mapr.com/blog/real-time-analysis-popular-uber-locations-spark-structured-streaming-machine-learning-kafka-and-mapr-db/

All good and well. The K-means clustering was done on a time-invariant basis to get centres / clusters using latitude and longitude only, but against a sample for 32 days. Then the clusters were segmented by hour/day for popularity. OK, I can get that approximation.

When I think of K means clustering applied to the famous whiskeys, then I can get that this is time-invariant.
But when I think of this uber location popularity, I would have intuitively thought that the clustering would need to be performed multiple times by running samples against data for an hourly or two hourly interval - that is to say less than 32 days.
So, given (I think) that it is hard to visualize K-means clustering with a temporal aspect included as feature, is the approach then indeed to 1) cluster like this against all features for a largeR period standardly and then 2) slice per time interval to get the popular locations?
And is this approach then just as good as a more fine-grained clustering approach using less data points due to / on a reduced time scale? I am hoping this is not a silly question as I think the folks at uber have more than a few clues. May be it is just a demo and not how one would do this in reality.
My take is that this approach can be improved, but may be not significantly compared to the mapr described method. I leave aside the issues of boundaries on a map.

k-means