I have a dataset which consists of $n_\text{samples}$ different measurements. Each measurement contains $n_\text{features}$ features. These features are for example average_mass, average_charge, speed, position_x, position_y, ...

In addition, each measurement was done at a certain time $t$ which is also known. I want to classify the measurements into $C$ classes. For this, I obtained a training set with labels $i \in \lbrace1,2,3,\ldots, C\rbrace$. Therefore, this can be treated as a supervised learning task. Lets say all $n_\text{samples}$ are labeled and the classifier should be able to work on new unseen data.

One could think now of different methods to do build such a classifier:

  1. Treat each of the $n_\text{samples}$ measurements independently and show them to the classifier. E.g. one could build a RandomForest with the above mentioned features and estimate its performance by using cross validation. This ansatz would completely ignore the time informaion.
  2. Use a classifier which makes explicit use of the time information. E.g. one could reshape the feature matrix of shape [n_samples, n_features] into a 3D tensor of shape [n_samples', n_timesteps, n_features] where n_timesteps is the number of consecutive timesteps which are considered as one sequence. n_samples' is now smaller than n_samples, since the orginal n_samples are grouped together in sequences of length n_timesteps.

To fix some numbers, lets say there are $C=10$ classes, $n_\text{samples}=10^5$, $n_\text{timesteps}=4$, $n_\text{features}=20$ .

For the second method, I used an LSTM. The keras code for the model looks like this:

model = Sequential()
                       input_shape=(n_timesteps, n_features),
        model.compile(optimizer="adam", loss=self.loss, metrics=['accuracy'])

It turned out that a RandomForest trained with approach 1 (i.e. ignore the time) works way better than the LSTM. I find this astounding as I assumed that since there is a correlation between the measurements in the time direction (position and speed at time 1 will certainly affect the position at time 2), a method which uses the time domain will outperform a classifier which ignores this extra information.

Now I have the following questions:

  1. Most often LSTMs are used for time series prediction or seq2seq tasks (eg. text translations or speech recognition). Hence I wonder if an LSTM is even a good choice for my problem?
  2. Is my network design suitable or are there better ways to construct a LSTM network for my task?
  3. What are other alternatives, which make use of the temporal correlations? Different network types?
  • $\begingroup$ Even though RF outperforms the other approach, it is not correct to ignore lack of sample independence (mass and position at time 2 are certainly dependent on the value of those parameters at time 1). I do not know much about LSTM to give a full answer. $\endgroup$ – katya Apr 28 '17 at 4:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.