I have a paragraph and I want to get the probability (p(word | context) ) of each word, given previous words, for various models (e.g. pre-trained LSTM).
Where can pretrained models would allow me to do this? Can someone provide an example as to how to fo this in Python?
Without having a model trained for this specific task, I don't think this is possible. However, if you allow the following assumptions to be true, one approach would be as discussed below.
After splitting your corpus into train/validation, create the following using your train dataset - for every sentence in the train corpus, randomly drop tokens with probability p and build a classifier to predict the dropped token. Use validation corpus to pick the best model like usual.
The classifier can be another neural network attached to whatever pre-trained network which acts like a black-box embedding layer.
BERT (https://arxiv.org/abs/1810.04805) and variants explores this idea in detail to build Masked LMs and have at least performed very well on benchmarks.
