0
$\begingroup$

the title more or less says it all. I want to use the transformers architecture, i.e. a transformer model, for series classification. I don’t want to predict time series, just to classify them.

The series are not very long, the average length is 220 elements, and I will try to modify it, if there is constraint on the model input. Is there a suitable model for that?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Transformers are often used for sequence/text classification. The transformer used for such tasks differs in architecture to generative models such as ChatGPT. While ChatGPT uses an Encoder and a Decoder, the classification models (often) only use an Encoder.

In many cases a [CLS] token, as a classification token will be introduced in the beginning of each sequence. And the embedding of that token will then be used to classify the sequence.

You can read more on the actual implementation here. https://huggingface.co/docs/transformers/tasks/sequence_classification

Improve this answer
$\endgroup$
2
  • $\begingroup$ thank you for your reply. I know these models but the thing is that the input of these models are input ids that come from the corresponding tokenizer on the corresponding text. The thing is that in my case I already have the input sequence which is different. I have a sequence of two dimensional data. $\endgroup$
    – exch_cmmnt_memb
    Commented Jan 25 at 15:32
  • $\begingroup$ Yes of course you cannot directly use the pre-trained models and tokenizer, but using Hugging Faces Library, you can easily build your own tokenizer and subsequent Transformer $\endgroup$
    – Janosch
    Commented Jan 25 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.