Structured Outputs and Function Calling

Choose whether you need data or an operation

Suppose a support application asks a model to classify a ticket and look up an order. Classification needs data in a predictable shape, such as a category and short explanation. Looking up an order needs an operation in your system. Structured output addresses the first problem; function calling provides a protocol for requesting the second. A single workflow can use both, but they are different contracts.

A tool call is a proposal with a name and arguments. It does not mean the operation has already happened, and it does not grant permission to perform it. Your application validates the proposal, decides whether it is allowed, executes the permitted operation and supplies its actual result. The model can then explain that result or request another step within the application’s limits.

Describe what shape is allowed

JSON parsing answers whether text is valid JSON. A schema can additionally require an object, constrain field types, restrict a category to an enum and reject unexpected fields. Defining a property does not automatically require it in JSON Schema; use required for mandatory fields. Set additionalProperties to false when the contract must reject unrecognized fields instead of silently carrying them downstream.

Design for absent evidence. If the source does not establish a category, allow an explicit unknown value or a separate needs-information result rather than forcing a guess. Use field names and descriptions that explain the intended meaning. Keep schemas small enough to inspect, version the application contract when it changes, and verify the subset supported by the selected provider or validation library.

{
  "type": "object",
  "properties": {
    "category": { "enum": ["billing", "technical", "unknown"] },
    "explanation": { "type": "string" }
  },
  "required": ["category", "explanation"],
  "additionalProperties": false
}

Structure does not establish truth

A response such as {"category":"billing","explanation":"The customer requested a refund"} can fit the schema while misreading a technical crash report. Structure is useful because software can reliably locate fields; semantic correctness still needs evidence and task-specific evaluation. Similarly, a syntactically valid order ID can refer to another user’s order. Neither JSON nor a schema makes that lookup authorized.

Providers may offer constrained structured output, JSON-only modes or SDK parsing helpers. These have different guarantees and supported schemas. Handle documented refusal and incomplete-generation outcomes before treating a response as the expected object. A stream that stops halfway through an object is not a completed result. Keep domain checks and application authorization even when a provider offers strict schema adherence.

Keep the tool loop under application control

Advertise a narrow tool such as get_order_status with an orderId argument. Avoid exposing an unrestricted SQL or shell tool just because it is flexible. When a proposal arrives, allowlist its name, parse and validate its arguments, establish the authenticated user from the server session, and check access to the requested order before looking up or returning private data.

Return the tool’s observed result with the protocol identifier linking it to the request. A failure is a result too: report a bounded error rather than inventing an order status. Limit the number of tool rounds, total time and cost so a model cannot loop indefinitely. Tool outputs can contain untrusted text, so receiving a result does not make every instruction inside it authoritative.

Test a narrow dispatcher without a model

The simulation uses fictional in-memory orders and an already-authenticated user ID. It accepts only get_order_status with exactly one nonempty string argument. A valid-looking request for another owner’s order is denied. The order status comes from the application record, not the model’s guess. Unknown and inaccessible IDs share the same bounded error in this example.

Run the complete snippet in Node.js or a browser console. It neither calls a provider nor changes an order. The handwritten checks cover this tiny schema only; they are not a general JSON Schema validator. A deployed implementation needs a real session, current resource permissions, safe database access, audit policy and the selected provider’s message envelope.

const orders = new Map([
  ['order-1', { ownerId: 'user-a', status: 'shipped' }],
  ['order-2', { ownerId: 'user-b', status: 'processing' }],
]);
function executeCall(authenticatedUserId, call) {
  if (!call || call.name !== 'get_order_status') return { error: 'unsupported_tool' };
  if (typeof call.arguments !== 'string') return { error: 'invalid_arguments' };
  let args;
  try { args = JSON.parse(call.arguments); } catch { return { error: 'invalid_arguments' }; }
  if (!args || Array.isArray(args) || typeof args !== 'object' ||
      Object.keys(args).length !== 1 || typeof args.orderId !== 'string' ||
      !args.orderId.trim()) return { error: 'invalid_arguments' };
  const order = orders.get(args.orderId);
  if (!order || order.ownerId !== authenticatedUserId) return { error: 'not_authorized' };
  return { status: order.status };
}
const makeCall = orderId => ({ name: 'get_order_status', arguments: JSON.stringify({ orderId }) });
if (executeCall('user-a', makeCall('order-1')).status !== 'shipped') throw new Error('Lookup failed');
if (executeCall('user-a', makeCall('order-2')).error !== 'not_authorized') throw new Error('Access leak');
if (executeCall('user-a', { name: 'delete_order', arguments: '{}' }).error !== 'unsupported_tool') throw new Error('Unknown tool');
if (executeCall('user-a', { name: 'get_order_status', arguments: '{' }).error !== 'invalid_arguments') throw new Error('Bad JSON accepted');
console.log('4 dispatcher checks passed');

Treat writes differently from reads

Reading an authorized status and canceling an order have different consequences. For a write, validate current state and policy, show the exact target and effect when approval is required, and bind approval to those reviewed details. Recheck authorization when executing: access or order state may have changed since the proposal was generated. A changed argument should not silently reuse approval for a different action.

Consider retries and duplicate calls. A timeout can occur after a remote write succeeded, so blindly repeating cancellation, sending or payment actions can produce duplicate effects. Use a suitable idempotency key or a transaction/status-check design in the action service. Record tool outcomes accurately; a model sentence saying “canceled” is not a substitute for a successful execution result.

Evaluate each boundary independently

Test missing and extra fields, wrong types, unknown enums, malformed JSON and unsupported tool names. Then test valid-shaped but unauthorized IDs, missing resources, stale approval and tool failure. Finally, test semantic mistakes: the wrong category, a tool selected for an unrelated request, or an explanation that contradicts the actual tool result. These classes need different fixes.

Keep a small set of representative user requests and expected decisions when changing prompts, schemas or tool descriptions. Measure both valid-format rate and correct-task behavior. A schema change can improve parsing while making the tool harder to choose, and a dispatcher can reject a harmful proposal even if the model still produces it. Report which layer prevented the failure instead of treating every failure as a model problem.

Sources & further reading

Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.

• JSON Schema: Object properties and required fields
• OpenAI: Structured outputs
• OpenAI: Function calling
• OWASP: Prompt injection prevention