Getting Structured Data from OpenAI API

I’ve been prompting ChatGPT for structured data since before it was an officially supported feature. The idea of an API endpoint that can morph and adapt to whatever purpose it needs to fill is incredible. Ever since I figured out how to do it, I’ve been building little apps that make use of GPT under the hood and return JSON back to some experience I’m building.

Recently, OpenAI expanded this functionality to its latest models and introduced new features that make it even easier to obtain complex structured JSON responses, opening up a range of possibilities for more reliable integrations. So I decided to look a little deeper and sum up everything I’ve found so far.

Getting Started: The Basics of Structured JSON Responses

One of the simplest ways to obtain a structured JSON response from GPT-4o-Mini is by explicitly asking the model to return data in JSON format. Here’s how you can do it:

1. Ask for JSON Output: In your prompt, clearly specify that you want the response in JSON format. For example, you might say, “Please provide the following information as a JSON object.”
2. Describe the Structure: Describe the structure of the JSON object you expect. For instance, if you want the AI to return user data, you could specify fields like name, age and email.
3. Use the response_format Parameter: Along with your API request, pass an additional parameter called response_format that includes a key type with the value json_object. This tells the model to format its output strictly as a JSON object.

Here’s a simple example of an API request using Python:

response = openai.ChatCompletion.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "user", "content": "Please provide the user's profile as a JSON object with the fields `name`, `age` and `email`."}
  ],
  response_format={"type": "json_object"}
)

As you can see, it’s pretty easy to implement and if trying to enforce a specific response type, can be extremely helpful.

Here is a little JS app I put together to help me discover fun new locations on Google Maps by entering a request into a prompt and having Google Maps automatically display a specific location it thinks best matches my query. To use this code, create a new bookmark in your browser and paste the code in. Then, click the bookmark, add your OpenAI key which is stored locally in your browser and then enter your request.

javascript:(function() {
    var apiKey = localStorage.getItem('openai_api_key');
    if (!apiKey) {
        apiKey = prompt("Enter your OpenAI API key:");
        if (apiKey) {
            localStorage.setItem('openai_api_key', apiKey);
        } else {
            return;
        }
    }
    var userInput = prompt("Enter your criteria:");
    if (userInput) {
        var promptSystemMessage = "You are TreasureFinder, a bot that takes a user's input and suggests a specific location on Google Maps. Please provide the location as a JSON object with a 'location_name' field. Only return the location name as it would be found on Google Maps.";

        fetch('https://api.openai.com/v1/chat/completions', {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': 'Bearer ' + apiKey
            },
            body: JSON.stringify({
                "model": "gpt-4o-mini",
                "messages": [
                    { "role": "system", "content": promptSystemMessage },
                    { "role": "user", "content": userInput }
                ],
                "response_format": {
                    "type": "json_object"
                }
            })
        })
        .then(response => response.json())
        .then(data => {
            console.log('result', data);
            var content = data.choices[0].message.content.trim();
            var jsonResponse = JSON.parse(content);
            var locationName = jsonResponse.location_name.trim();
            console.log(locationName);
            window.location.href = 'https://www.google.com/maps/search/' + encodeURIComponent(locationName) + '/data=!3m1!1e3';
        })
        .catch(error => {
            console.error('Error:', error);
        });
    }
})();

Now I can click on the bookmark, enter a prompt like:

Show me the most interesting looking structure in the world. It should be something that you wouldn't see every day and one of a kind.

And the result looks something like this:

Defining Custom Structures

The simple approach will get you pretty far, especially when working with simpler responses. However, if the response you are looking for is more complex, it could be helpful to further define the response structure to ensure you are getting back exactly what you are expecting.

For example, you can define a JSON schema within your request to ensure the AI adheres strictly to your desired structure. Here’s how it works:

1. Define a JSON Schema: Within your API call, you can pass a JSON schema that describes the structure, data types and required fields of the output. This ensures that the model generates output that conforms to your specifications.
2. Strict Mode: You can enable a strict mode to prevent the model from generating any output that doesn’t match the schema. This is useful for ensuring data consistency and avoiding errors during subsequent processing.

An example of using a custom schema in a Python app might look like this:

response = openai.ChatCompletion.create(
  model="gpt-4o-mini",
  messages=[
    {"role": "user", "content": "Please generate a report in JSON format."}
  ],
  response_format={
    "type": "json_schema",
    "json_schema": {
      "type": "object",
      "properties": {
        "report_title": {"type": "string"},
        "data": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "id": {"type": "integer"},
              "value": {"type": "string"}
            },
            "required": ["id", "value"]
          }
        }
      },
      "required": ["report_title", "data"]
    }
  }
)

Opportunities and Game-Changing Potential

The ability to receive structured outputs directly from an AI model opens up numerous opportunities. For instance, developers can now reliably integrate AI-generated content into applications without needing to parse and validate the output manually. This is something I ran into regularly while experimenting with my WordPress page builder plugin. Without structured responses, GPT tended to add additional unneeded information to its responses. Sometimes it would describe the response before outputting JSON. Other times it would ignore my structure altogether and invent new key names. Ultimately it was very unreliable until I was able to enforce the structure by adding the new response_format object to my requests.

These capabilities not only reduce the risk of errors but also significantly speed up development processes by ensuring consistency and reliability in AI outputs.

Current Limitations and the Future Outlook

While the ability to generate structured data is a significant advancement, there are still limitations. For example, the strictness of the JSON schema can sometimes lead to issues where the model fails to produce an output if it cannot fully conform to the schema, especially in edge cases. Additionally, there’s a notable difference in how well this feature performs across different models, with some like GPT-4o performing better than GPT-4o-Mini in adhering to complex schemas.

Despite these limitations, the future looks promising. As OpenAI continues to refine these features and improve model capabilities, we can expect even more robust tools for handling structured data. The integration of structured outputs with broader AI workflows is likely to become a standard practice, enabling more sophisticated and reliable AI-driven applications.

This exploration of structured data in the OpenAI API demonstrates how far AI has come and hints at the incredible possibilities that lie ahead. Whether you’re managing data, generating reports or integrating AI with existing applications, understanding and utilizing these tools will be crucial for maximizing the potential of AI in your projects.