JSON Mode

Get structured JSON responses from models using Oru-el's JSON mode.

JSON Mode#

JSON mode forces the model to produce valid JSON in its response. This is essential when you need structured data — parsing entities, extracting information, generating configs, or building data pipelines.

Enabling JSON mode#

Set response_format to {"type": "json_object"} in your request:

response = client.chat.completions.create(
    model="llama-4-maverick",
    messages=[
        {"role": "system", "content": "You are a helpful assistant. Respond with valid JSON."},
        {"role": "user", "content": "List 3 programming languages with their year of creation."},
    ],
    response_format={"type": "json_object"},
)

data = json.loads(response.choices[0].message.content)

Important: You must include an instruction to produce JSON in your prompt (either in the system message or user message). JSON mode ensures the output is valid JSON, but the model still needs direction on what JSON to produce.

Basic example#

Python#

import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oru-el.com/v1/inference",
    api_key="oruel_your_api_key_here",
)

response = client.chat.completions.create(
    model="llama-4-maverick",
    messages=[
        {
            "role": "system",
            "content": "You extract structured data. Always respond with a JSON object.",
        },
        {
            "role": "user",
            "content": "Extract the person's info: John Smith is a 34-year-old software engineer from Austin, Texas.",
        },
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

data = json.loads(response.choices[0].message.content)
print(json.dumps(data, indent=2))

Output:

{
  "name": "John Smith",
  "age": 34,
  "occupation": "software engineer",
  "city": "Austin",
  "state": "Texas"
}

JavaScript#

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.oru-el.com/v1/inference",
  apiKey: "oruel_your_api_key_here",
});

const response = await client.chat.completions.create({
  model: "llama-4-maverick",
  messages: [
    {
      role: "system",
      content: "You extract structured data. Always respond with a JSON object.",
    },
    {
      role: "user",
      content:
        "Extract the person's info: John Smith is a 34-year-old software engineer from Austin, Texas.",
    },
  ],
  response_format: { type: "json_object" },
  temperature: 0,
});

const data = JSON.parse(response.choices[0].message.content);
console.log(data);

cURL#

curl https://api.oru-el.com/v1/inference/chat/completions \
  -H "Authorization: Bearer oruel_your_api_key_here" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-4-maverick",
    "messages": [
      {"role": "system", "content": "You extract structured data. Always respond with a JSON object."},
      {"role": "user", "content": "Extract the person'\''s info: John Smith is a 34-year-old software engineer from Austin, Texas."}
    ],
    "response_format": {"type": "json_object"},
    "temperature": 0
  }'

Entity extraction#

Extract structured entities from unstructured text:

response = client.chat.completions.create(
    model="llama-4-maverick",
    messages=[
        {
            "role": "system",
            "content": """Extract entities from the text. Return a JSON object with this schema:
{
  "companies": [{"name": string, "role": string}],
  "monetary_values": [{"amount": number, "currency": string, "context": string}],
  "dates": [{"date": string, "context": string}],
  "people": [{"name": string, "title": string}]
}""",
        },
        {
            "role": "user",
            "content": "On March 15, 2024, Acme Corp CEO Jane Doe announced a $2.5 billion acquisition of Beta Inc. The deal is expected to close by Q3 2024.",
        },
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

entities = json.loads(response.choices[0].message.content)

Expected output:

{
  "companies": [
    { "name": "Acme Corp", "role": "acquirer" },
    { "name": "Beta Inc", "role": "target" }
  ],
  "monetary_values": [
    { "amount": 2500000000, "currency": "USD", "context": "acquisition price" }
  ],
  "dates": [
    { "date": "2024-03-15", "context": "announcement date" },
    { "date": "2024-Q3", "context": "expected close" }
  ],
  "people": [
    { "name": "Jane Doe", "title": "CEO of Acme Corp" }
  ]
}

Structured data parsing#

Parse documents into a consistent schema:

response = client.chat.completions.create(
    model="llama-4-maverick",
    messages=[
        {
            "role": "system",
            "content": """Parse the product review into JSON with this schema:
{
  "rating": number (1-5),
  "sentiment": "positive" | "negative" | "mixed",
  "pros": [string],
  "cons": [string],
  "summary": string,
  "would_recommend": boolean
}""",
        },
        {
            "role": "user",
            "content": "I've been using this keyboard for 3 months. The mechanical switches feel amazing and the build quality is top-notch. However, the software is buggy and the RGB lighting flickers occasionally. Battery life is decent at about 2 weeks. Overall I like it but the software needs work. 4 out of 5 stars.",
        },
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

review = json.loads(response.choices[0].message.content)

Classification#

Use JSON mode for classification tasks with structured output:

response = client.chat.completions.create(
    model="llama-4-maverick",
    messages=[
        {
            "role": "system",
            "content": """Classify the support ticket. Return JSON:
{
  "category": "billing" | "technical" | "account" | "feature_request" | "other",
  "priority": "low" | "medium" | "high" | "critical",
  "sentiment": "positive" | "neutral" | "negative" | "angry",
  "requires_human": boolean,
  "suggested_response_template": string
}""",
        },
        {
            "role": "user",
            "content": "I've been charged twice for my subscription this month! This is the third time this has happened. Fix this immediately or I'm canceling.",
        },
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

ticket = json.loads(response.choices[0].message.content)
print(f"Category: {ticket['category']}, Priority: {ticket['priority']}")

Error handling#

Always wrap JSON parsing in error handling. While JSON mode ensures valid JSON syntax, the structure may not match your expected schema:

import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.oru-el.com/v1/inference",
    api_key="oruel_your_api_key_here",
)

response = client.chat.completions.create(
    model="llama-4-maverick",
    messages=[
        {"role": "system", "content": "Respond with a JSON object containing 'name' and 'age'."},
        {"role": "user", "content": "Tell me about Alice, who is 30."},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

content = response.choices[0].message.content

try:
    data = json.loads(content)
except json.JSONDecodeError:
    print(f"Failed to parse JSON: {content}")
    data = None

if data:
    # Validate expected fields exist
    name = data.get("name")
    age = data.get("age")
    if name is None or age is None:
        print(f"Missing expected fields. Got: {data}")

const content = response.choices[0].message.content;

let data;
try {
  data = JSON.parse(content);
} catch (error) {
  console.error("Failed to parse JSON:", content);
}

if (data) {
  const { name, age } = data;
  if (name === undefined || age === undefined) {
    console.error("Missing expected fields:", data);
  }
}

Tips for reliable JSON output#

Specify the schema in the system message — show the exact JSON structure you expect, including field names, types, and allowed values.
Use temperature: 0 — deterministic output is more reliable for structured data tasks.
Provide examples — if the schema is complex, include a sample JSON object in your prompt.
Keep schemas simple — deeply nested schemas with many optional fields are harder for models to follow consistently.
Validate after parsing — JSON mode guarantees valid JSON syntax but not that the output matches your schema. Always validate the structure and types.
Handle truncation — if the response is cut off by max_tokens, the JSON may be incomplete. Set max_tokens high enough for your expected output, and check finish_reason for "length".

JSON mode vs. tool calling#

Both can produce structured data, but they serve different purposes:

Feature	JSON mode	Tool calling
Use case	Extract data, classify, parse	Trigger actions, fetch real-time data
Output	Raw JSON in `content`	Structured `tool_calls` with `function.arguments`
Schema enforcement	Via prompt instructions	Via JSON Schema in tool definition
Follow-up required	No	Yes (return tool result, then get final response)
Best for	Data extraction, classification	Multi-step workflows, external integrations

Use JSON mode when you need structured data as the final output. Use tool calling when the model needs to trigger an action and then reason about the result.