7
$\begingroup$

I'm working with a classification problem where the data points include both sequential (time series) data and "static" features - attributes that don't change. An analogy could be a datapoint consisting of a sentence, where the words are the sequential data, along with "static" information like who wrote the sentence, what kind of publication it was in, etc.

I know in NLP we can include these if we're working with the text as a bag of words or n-grams, in which case there's a constant number of features, but I'd like to process the "sentence" as a sequence the way an HMM would.

What models or types of models are generally used for this? I know of HMMs, RNNs, and other models that deal solely with sequences, and obviously most basic ML classifiers can deal with the static features, but I can't seem to find any information on combining them. I think maybe I'm just not searching with the right terminology. It seems like something that people would want to do fairly frequently, though.

Is it common practice to use some sort of ensemble method? If so, I'm not sure how the weighting would work for that.

$\endgroup$

1 Answer 1

2
$\begingroup$

Just a suggestion, if your classifying a sequence with an RNN you could add a final fully-connected layer that combines the output of the RNN with your static features (by concatenation) before going to the softmax and outputting the predicted class probabilities. Since this final layer is fully-connect with its own set of weights, as long as you scale the features to zero mean and unit variance, the weighting would be done automatically when training the network.

$\endgroup$
2
  • $\begingroup$ Thanks! Yeah, that kind of an architecture seems like it would work, but I think for now the project is not venturing into DL territory. That could change, and if it does I will definitely try this. $\endgroup$
    – jmully
    Jul 20, 2017 at 17:26
  • $\begingroup$ @Miguel In such a case, would it make sense to use the static feature set as the initial hidden/cell state in the RNN/LSTM model and the temporal feature set as the standard time step input? $\endgroup$
    – asanoop24
    Jun 22, 2022 at 4:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.