Questions tagged [gpt]
The gpt tag has no usage guidance.
5
questions
0
votes
0
answers
139
views
How to calculate loss in pre-training gpt-2
As I know BERT model that calculates loss by compare between an original word that was masked with the predicted result for retrieve the loss and update the weight in pre-train model.
But GPT-2 uses ...
- 1
0
votes
0
answers
93
views
What the context window size in pre-train gpt-2
I have questions about GPT-2.
What is the context window in the pre-train gpt-2 train?
As I know language modeling is based on gpt-2 use prefix language modeling that is demonstrated in picture form ...
- 1
0
votes
0
answers
10
views
Network invocations underlying OpenAI execution?
Really a pretty basic question about generative models, but I'm trying to map my (limited) understanding of NNs generally to what's going on when I invoke an OpenAI API:
When the OpenAI API docs ...
- 398
3
votes
1
answer
192
views
How to compare the semantic similarity of text generated by large language models (GPT-3, BLOOM etc) to reference text?
Large language models, such as GPT-3, BLOOM etc, can generate open-ended text. Say I want to prompt these models to answer a question. How can I compare the semantic similarity of the answer it ...
- 31
6
votes
3
answers
412
views
Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?
Why do language models like InstructGPT and LLM utilize reinforcement learning instead of supervised learning to learn based on user-ranked examples?
Language models like InstructGPT and ChatGPT are ...
- 1,140