11
$\begingroup$

I've read that ChatGPT will sometimes give different answers to the same prompt. In other words, there is an element of randomness. Where does this randomness come from? Is there some sort of component in transformers or in ChatGPT's implementation specifically that's like a variational auto-encoder where you can feed in a randomized feature vector as input to trigger different results?

I also saw in OpenAI's InstructGPT paper Training language models to follow instructions with human feedback, upon which ChatGPT is based, that there is an infographic (see below) saying that answers are sampled when given a prompt during the "reward model" training step. I assume that you would only be able to sample responses if there were an element of randomness somewhere.

enter image description here

$\endgroup$

1 Answer 1

7
$\begingroup$

Where does this randomness come from?

From the temperature. See What is the "temperature" in the GPT models?. Note that even a temperature set to 0 doesn't guarantee a deterministic result.

On https://platform.openai.com/playground one may change the temperature:

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.