# Machine Learning problem - label over multiple lines

Currently I'm trying to work out a project where I would like to recognize movements from videos using machine learning and python.

What I've done so far is extracting the x and y values of body joints of each frame in about 150 videos for each of the joints and put those in an Excel file, that now looks like this:

#Frame   NoseX    NoseY   NoseC   NeckX   NeckY  NeckC  ....
1      xvalue  yvalue  cvalue  xvalue  yvalue  cvalue ....
2      xvalue  yvalue  cvalue  xvalue  yvalue  cvalue ....
3      xvalue  yvalue  cvalue  xvalue  yvalue  cvalue ....
.....


This shows an example of the Excel file, showing just 2/25 joints and 3/250 frames (give or take). Each video of the specific movement is put in its own sheet, so in the end about 150 sheets with each 76x250 columns and rows (~2.8 million data points).

But all this data (videos) is currently just one type of movement that I was aiming for as training data (?), so that whenever I provide the model with a new set of different x and y values (other movement type) it should be able to classify it the same as the first set.... or not!

So in short: my output should be able to predict if the type of movement in the video is the same as that from those in the 150 videos, derived from the x, y (and c) values.

The problem I'm facing is that all the examples/tutorials I've encountered are so that each line is a set of data with a label/target that you can run a machine learning algorithm over, but not a whole sheet that defines a label (type of movement). So how do you create 1 label defines by the data in a sheet, instead of a line/row?

I'm hoping that this makes any sense and that someone here has any clue how to tackle such a problem, or that I need to check a different approach to this idea?

• Do you have an automated way of getting these values from the input video? – jonnor May 23 '19 at 11:58

video,movement

The identifiers in the video column would here be a string that is the name of the corresponding sheet with data. And in movement are your class labels.