OpenAI GPT-4 API: Reproducible Outputs

So OpenAI has a neat little feature called Reproducible Outputs (https://platform.openai.com/docs/guides/text-generation/reproducible-outputs)

Basically you can use the system_fingerprint you get in your API response and use it as the seed parameter for the next API call.

You can see a cookbook example here:

https://cookbook.openai.com/examples/reproducible_outputs_with_the_seed_parameter

Now they say determinism is not guaranteed (https://platform.openai.com/docs/api-reference/chat/create#chat-create-seed). So you should really evaluate if it's worth your time.

Determinism is a huge problem with the LLM. It's is one of the reasons you want to work FLEXIBLY WITH THEM, and not automate your processes completely just yet.

Also, the API response values will be different than the ChatGPT systems with its control of context.

In my experience, you're better served by doing a good study of what GPT-4 does well for you and what it doesn't do well. This should take into account the natural variation you get with answers and specific prompts.

Of course, You'll want to setup a good experimental design to confirm how much it hallucinates and how much efficiency you are actually getting!

When you reach the stage of experimental design, you can reach out to us for consulting!