Python package that allows to train a CRF on two datasets

Question

I am looking for a Python package that allows to train a conditional random field (CRF) on two datasets.

For example: I have two datasets, dataset A and dataset B. I want to train a conditional random field on dataset A, then train the CRF on dataset B. I do not want to join the CRF on two datasets at the same time.

I looked at pycrfsuite but it seems that after a CRF is trained, it cannot be trained further on some other dataset (trainer.train() reset the parameters of the CRF).

I also unsuccessfully looked at CRF++: How can I train a CRF on two datasets with CRF++?

---

I have crossposted the question at:

• https://redd.it/604nu4
• http://qr.ae/Tu75Xa

I'll update this thread to reflect any interesting answer I may receive in one of these links.

Answer 1

You can use Wapiti (mirror):

Wapiti is a very fast toolkit for segmenting and labeling sequences with discriminative models. It is based on maxent models, maximum entropy Markov models and linear-chain CRF and proposes various optimization and regularization methods to improve both the computational complexity and the prediction performance of standard models. Wapiti is ranked first on the sequence tagging task for more than a year on MLcomp web site.

Wapiti is developed by LIMSI-CNRS and was partially funded by ANR projects CroTaL (ANR-07-MDCO-003) and MGA (ANR-07-BLAN-0311-02).

It is written in standard C99+POSIX and is open source (BSD Licence) (mirror).

It allows a model file to load and to train again. From the manual (mirror) :

-m | --model <file>
              Specify a model file to load and to train again. This allow  you
              either  to  continue  an  interrupted  training or to use an old
              model as a starting point for a new training. Beware that no new
              labels  can be inserted in the model. As the training parameters
              are not saved in the model file, you have to specify them again,
              or  specify new one if, for example, you want to continue train-
              ing with another algorithm or a different penalty.

There seems to exist some python wrapper for wapiti such as python-wapiti (mirror).

Answer 2

You can use NeuroNER:

• It implements of a bi-directional LSTM + CRF network in TensorFlow
• works on Linux/Mac/Windows
• written in Python
• open source
• allows to train a CRF on two datasets with the options use_pretrained_model = True + train_model = True