I have 2 data sets which are somehow similar and I want to use them for domain adaptation. Dataset1 is imbalanced and consists of labeled positive and negative samples. Dataset2 consists of only negative samples and is the target dataset. The goal is to train a model, which will be able to distinguish anomalies (or positive samples) on new samples from the same source as is Dataset2. My idea is to pre-train a model on Dataset1 and then use it to train final model on Dataset2. I would like to use autoencoder for anomaly detection, but I struggle with selection of architecture and what to keep fixed and whether to add extra layers or remove any in second step when training on Dataset2.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
This should just be a a comment, but I don't yet have the rep, sorry. One idea would be to train a one-class classifier on dataset 2, re-format dataset 1 to resemble 2 and use it as the verification set. That way your model will better represent the desired source data, but you still get the benefit of the positive samples you have.
