All Questions
Tagged with language-models python
3 questions
0
votes
1
answer
3k
views
Finding the perplexity of multiple examples
I am trying to find a way to calculate perplexity of a language model of multiple 3-word examples from my test set, or perplexity of the corpus of the test set. As the test set, I have a paragraph ...
1
vote
0
answers
1k
views
Inference time for text genration using fine-tuned gpt2
I have re-trained GPT2 model using over 10 million sentences for QA. And while testing also I am getting very good results. The only problem now I am facing is that I have millions of test data that I ...
- 121
0
votes
1
answer
86
views
get predictability of word given sentence in python
I have a paragraph and I want to get the probability (p(word | context) ) of each word, given previous words, for various models (e.g. pre-trained LSTM).
Where can pretrained models would allow me to ...
- 57