Deep learning for trajectory classification - python

Question

I'm intending to obtain advice or suggestions about a classification problem. I'll attach a brief example of the training data and associated figures below to describe the problem and the information available.

The data is a time series of xy points, which is made up of smaller sub-sequences event. So each unique event is independent. I have two unique sequences (10,20) below of even time length. For a given sequence, each individual point has its own unique identifier user_id. The xy trace of these points will vary marginally over a given sequence, with the specific time period found in interval. I also have a separate xy point used as a reference (centre_x, center_y), which details the approx middle/centre of all points.

Lastly, the target_label classifies where these points are relative to each other. So using the centre_x, center_y as a reference, there are 5 class Middle, Top, Bottom, Right, Left. There can only be one label for each unique event.

Problem: Use deep-learning to classify target_label from raw xy data

Typically, one might implement a ML technique using any number of hand-crafted features, such as, euclidean distance to centre_x, centre_y or distance in each individual axis.

However, I was hoping to utilise the raw xy data and model the classification using a deep learning technique. My current thought process is:

• CNN not ideal, given you'd have to convert xy data to images and require multiple objects (points) within each sequence to be detected.

• MPL also not ideal as I want to utilise the raw xy data.

• RNN fit the description of sequencing data but I'm not sure they serve trajectory data well.

I'm currently onto trajectory classification with convolutional and pooled layers but I can't find any working examples. Would this method be feasible here?

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# number of intervals
n = 10

# center locations for points
locs_1 = {'A': (5,5),
      'B': (5,8),
      'C': (5,2),
      'D': (8,5)}

# initialize data 
data_1 = pd.DataFrame(index=range(n*len(locs_1)), columns=['x','y','user_id'])
for i, group in enumerate(locs_1.keys()):

    data_1.loc[i*n:((i+1)*n)-1,['x','y']] = np.random.normal(locs_1[group], 
                                                       [0.2,0.2], 
                                                       [n,2]) 
    data_1.loc[i*n:((i+1)*n)-1,['user_id']] = group

# generate time interavls
data_1['interval'] = data_1.groupby('user_id').cumcount() + 1

# assign unique string to differentiate sequences
data_1['event'] = 10

# center of all points for unqiue sequence 1
data_1['center_x'] = 5
data_1['center_y'] = 5

# classify labels
data_1['target_label'] = ['Middle' if ele  == 'A' else 'Top' if ele == 'B' else 'Bottom' if ele == 'C' else 'Right' for ele in data_1['user_id']]

# center locations for points
locs_2 = {'A': (14,15),
      'B': (16,15),
      'C': (15,12),
      'D': (19,15)}

# initialize data 
data_2 = pd.DataFrame(index=range(n*len(locs_2)), columns=['x','y','user_id'])
for i, group in enumerate(locs_2.keys()):

    data_2.loc[i*n:((i+1)*n)-1,['x','y']] = np.random.normal(locs_2[group], 
                                                       [0.2,0.2], 
                                                       [n,2]) 
    data_2.loc[i*n:((i+1)*n)-1,['user_id']] = group

# generate time interavls
data_2['interval'] = data_2.groupby('user_id').cumcount() + 1

# center of points for unqiue sequence 1
data_2['event'] = 20

# center of all points for unqiue sequence 2
data_2['center_x'] = 15
data_2['center_y'] = 15

# classify labels
data_2['target_label'] = ['Middle' if ele  == 'A' else 'Middle' if ele == 'B' else 'Bottom' if ele == 'C' else 'Right' for ele in data_2['user_id']]

df = pd.concat([data_1, data_2])

df = df.sort_values(by = ['event','interval','user_id']).reset_index(drop = True)

Answer 1

I would probably try adapting some of the NN architectures used in trajectory prediction to classification by changing the last layer, loss function, and possibly the first layer.

To use CNN here, the trajectories will need to be bounded by the picture so it requires you to know a bound for the (x, y) based on the domain. Also, based on the success of RNN in trajectory prediction, there isn't a reason to believe that RNN would not do well here. The model can also be a mixed one where you first use CNN to capture spatial information and pass them to RNN to capture the temporal information (change in spatial information across time).