GPT-4o : Json Mode is a bit weird.

Written by Noel Victor | May 28, 2024 5:42:23 PM

Did you know that the latest OpenAI APIs for GPT have a JSON output mode ?!

https://platform.openai.com/docs/guides/text-generation/json-mode

 

This makes creating automation around the API super easy! Right? Well... No, not quite.

There are couple gotchas:

  • even with JSON mode enabled you need to ask for JSON in your prompt (weird)
  • you REALLY need to have some good safe guards around token counts, and proper finish_reason codes

It was not smooth sailing for me.

  • I got malformed JSON due to reaching token context window maxs
  • I got some weird JSON in the JSON, that I had to parse out
  • I still had to parse out things from the response using string manipulation

 

Very Very weird stuff. I'm wondering if the feature is just too new.

It feels underdeveloped for me. I think it probably has enormous future potential for the creation of AI agents.

I'm very excited to see how it develops!