Without having a model trained for this specific task, I don't think this is possible. However, if you allow the following assumptions to be true, one approach would be as discussed below.
- The tokenizer of the pre-trained model supports your new corpus
- You are allowed (re-)train/fine-tune using a subset of your corpus
After splitting your corpus into train/validation, create the following using your train dataset - for every sentence in the train corpus, randomly drop tokens with probability p and build a classifier to predict the dropped token. Use validation corpus to pick the best model like usual.
The classifier can be another neural network attached to whatever pre-trained network which acts like a black-box embedding layer.
BERT (https://arxiv.org/abs/1810.04805) and variants explores this idea in detail to build Masked LMs and have at least performed very well on benchmarks.