0
$\begingroup$

What would be an appropriate model for predicting a binary target variable, given a weighted sequence?

Sequences will be reasonably short, typically between ~ 1 and 5 elements. I have in the order of 200,000 examples for training and testing.

Illustrated example

Say I have the following categories: A, B, C, D.

Each category can have a weight between zero and one.

Example sequences:

  • A (0.33), B (0.71), C (0.0), D (0.95) ⇒ 1
  • C (0.21), A (0.67) ⇒ 0
  • B (0.5), D (1) ⇒ 1
  • B (0.2), D (0.6) ⇒ 0
  • C (0.3) A (0.6), B (0.2) ⇒ 0
Improve this question
$\endgroup$
6
  • $\begingroup$ I wondered if this would be an appropriate use case for a language model (e.g. BERT, but trained from scratch). I could take the embeddings for each word, then take a weighted average with the scores given above for a final document embedding, which could be used to train a binary classifier. I thought BERT would be more appropriate here than something like W2V because it captures context. Is this sound? $\endgroup$
    – Ian
    Commented Dec 17, 2020 at 16:02
  • $\begingroup$ Could you explain the relevance of the tags you have chosen? So far, nothing in your question concerns a Markov process, RNNs, word embeddings, language models, or even "sequence analysis" in the sense of that tag (which concerns genomic models). $\endgroup$
    – whuber ♦
    Commented Dec 17, 2020 at 16:53
  • $\begingroup$ The right method depends on the meaning of the these data. All you have told us is that you have four points in four-dimensional space, and your training set classifies them (assigns 0 or 1). Are the 0-marked points a continuous part of the space? If so, Gaussian Process Regression might work well. Or is there a reason you call these a "sequence"? There is no machine learning without prior knowledge. $\endgroup$
    – chrishmorris
    Commented Dec 17, 2020 at 17:30
  • $\begingroup$ @chrishmorris I'm not sure what you mean by "Are the 0-marked points a continuous part of the space?". Are you referring to the label, or the weight given to an element in the sequence? (e.g. C (0.0)) $\endgroup$
    – Ian
    Commented Dec 17, 2020 at 18:43
  • $\begingroup$ @whuber it seemed appropriate to reach people from these communities who could provide a potential answer $\endgroup$
    – Ian
    Commented Dec 17, 2020 at 18:44

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.