0
$\begingroup$

I'm trying to classify time-series data where we have n hikers traveling between two locations. We're just trying to classify them without regards to seasonality

The problem is that each hiker starts and ends at different dates and some of them are missing timestamps, plus they have different travel times for whatever reason (detours, paths, etc...). For sake of convenience, latitude and longitude are l, lg respectively.

    n1 = [(l, lg, 04/08 00:00),(l, lg, 04/08 12:03),(l, lg, 04/09 02:30), (l, lg 04/10 00:05)...]
    n2 = [(l, lg, 03/08 00:00),(l, lg, 03/09 03:03),(l, lg, 03/10 00:30), (l, lg 03/10 13:05)...]
    n3 = [(l, lg, 04/07 00:00),(l, lg, 04/08 03:03),(l, lg, 04/10 00:30), (l, lg 04/11 14:05)...]
    len(n1) == len(n2) != len(n3) #delayed due to detour

The most predictable regularity of the data is that no more than 2 data points can occur on one day and that each datapoint is supposed to be around 12 hours, so

    nx = [(l, lg, 04/08 00:00),(l, lg, 04/08 10:00),(l, lg, 04/08 13:00)...]

would never occur.

What I'm considering doing is ignoring the dates of the trips and just considering the change in times, rounding each approximately 12 hour timestamp. This would eliminate starting date discrepancies:

    n1 = [(l, lg, 1),(l, lg, 2),(l, lg, 3), (l, lg, 5)...]
    n2 = [(l, lg, 1),(l, lg, 3),(l, lg, 5), (l, lg, 6)...]
    n3 = [(l, lg, 1),(l, lg, 3),(l, lg, 7), (l, lg, 10)...]

And then for trips which are missing timestamps, just impute them using either the average lat/long at that point or the average of the previous and following data.

    n1 = [(l, lg, 1),(l, lg, 2),(l, lg, 3), (l,lg,4*), (l, lg, 5)...]
    n2 = [(l, lg, 1), (l,lg, 2*), (l, lg, 3),(l,lg,4*), (l, lg, 5)...]
    #timestamp* indicates imputed value

Then I would just pad each nx s.t. len(nx) == len(longest trip). And the padded values would just be the lat/long of the destination.

      n_1 = [...,(l, lg, 28),(dest_lat, dest_long,29),(dest_lat, dest_long, 30)]
      n_3 = [...,(l, lg, 28),(l,lg, 29),(dest_lat, dest_long, 30)]
      #n_y is the longest trip, n_x has been padded s.t. len(n_x) == len(n_y)

I currently have this data in a pandas dataframe. df.columns = ['ID','trip_points','trip_times'] (trip points = list of tuples of lat and long for each trip) and am trying to figure out either how to implement what I just described or see if there's some other way I should be doing this. I've wracked my brain but just can't figure out if I'm missing some better solution. And I would have no idea how to implement this in pandas.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.