0
$\begingroup$

I have the dataset from here which contains the following features:

'Index', 'Arrival_Time', 'Creation_Time', 'x', 'y', 'z', 'User', 'Model', 'Device', 'gt' 

A snapshot of data would look like this:

Index,Arrival_Time,Creation_Time,x,y,z,User,Model,Device,gt
0,1424696638740,27920678471000,-0.5650316,-9.572019,-0.61411273,a,gear,gear_1,stand
1,1424696638740,27920681910000,-0.83258367,-9.713276,-0.60693014,a,gear,gear_1,stand
2,1424696638740,27920692014000,-1.0181342,-9.935339,-0.54408234,a,gear,gear_1,stand
3,1424696638741,27920701983000,-1.2228385,-10.142437,-0.5662287,a,gear,gear_1,stand
4,1424696638741,27920711906000,-1.5771804,-10.480618,-0.40282443,a,gear,gear_1,stand
5,1424696638741,27920721675000,-2.1643584,-10.920552,-0.18375498,a,gear,gear_1,stand
6,1424696638741,27920731721000,-2.973,-11.063007,0.21188685,a,gear,gear_1,stand
7,1424696638741,27920743061000,-3.8881836,-11.08276,0.6847417,a,gear,gear_1,stand
8,1424696638742,27920751586000,-4.8919525,-10.890625,1.01574,a,gear,gear_1,stand

I want to perform classification using a neural network for this dataset but first I am trying to figure out how to extract more usual features from 'Arrival_Time', 'Creation_Time'. The definition of these two features are:

Arrival_Time:   The time the measurement arrived to the sensing application
Creation_Time   The timestamp the OS attaches to the sample

Is there a way via scikit-learn to extract or encode these two features toward better features? Any hints are appreciated.

$\endgroup$
1
  • 1
    $\begingroup$ You can add various features like difference between arrival and creation time,average of those etc and use a neural network on top of that. If those are the relevant features it would learn to use them $\endgroup$
    – sww
    May 14, 2018 at 20:03

1 Answer 1

1
$\begingroup$

Month of year, week of year, week of month, day of year, day of week, hour of day, hour of week, minute of hour, second of minute, ... is leap year, is holiday, is Olympic year, ... # of events in the entire training data that were in the same (minute, hour, day, ...) as this one (commonly normalized by something else, such as the total number of events, or the total number sharing some attributes with this one), min/max/mean/variance of all values observed in the same (minute, hour, day, ...) as this one, ... You can generate hundreds of features like this.

Many of these are one-liners in pandas, like df[‘Arrival_Time’].dt.hour (assuming you already did pd.to_datetime() on that column).

Also, it’s good practice to circular encode the features that repeat, such as month of year: {cos,sin}(2 \pi month_of_year / 12) so that January and December are as close in feature space as January and February are.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.