Working with AI APIs

Put a controlled server between browser and provider

Consider a “Summarize my document” button. The browser knows which document the user selected, but it must not decide whether that user may read the document or hold the provider secret. The application server is where identity, access and cost limits become enforceable before document text is sent outside the application.

A typical application sends the user’s request to its own authenticated backend. The backend validates it, checks authorization and usage limits, then calls the model provider using a server-held credential. The browser receives the result it is allowed to see. A secret embedded in frontend JavaScript or a public build-time environment variable is not secret.

The server should decide which model, tools and instructions are permitted. Do not accept arbitrary provider URLs or privileged tool definitions directly from a browser request. Bound input size and output size, and give each request an identifier for investigation without logging unnecessary prompt content.

Design the application contract first

An endpoint for summarizing a user-owned document should verify document access before reading or sending it. Return a defined success shape and useful failure categories. A model response that is empty, incomplete or invalid should not be treated as success merely because the provider returned an HTTP success status.

The following is architecture pseudocode, not a provider SDK example. Its operations must be implemented and tested in the actual backend. Provider APIs differ in authentication, response shapes, streaming and rate-limit behaviour; use the current official documentation for the service you select.

POST /api/summarize
  authenticate the caller
  validate document ID and input limits
  authorize access to the document
  enforce per-user quota and concurrency limit
  call the configured provider with a deadline
  validate the returned structure and completion status
  return { summary, requestId } or a bounded error

Expect partial and failed requests

Use deadlines and cancellation so abandoned requests do not occupy resources indefinitely. Retry only failures that are safe and appropriate to retry, with bounded attempts and backoff. Respect provider retry guidance. Repeating a generation can incur another charge, and repeating a tool action can cause a duplicate side effect.

Streaming improves perceived latency but introduces partial-output states. A connection can close halfway through JSON or before a final completion event. Keep incomplete text visibly provisional and avoid executing actions based on partial model output. Provide a recoverable UI state instead of leaving an endless spinner.

Measure quality as well as availability

Track request duration, failures, usage and validation outcomes. Set budgets and concurrency controls before opening a costly endpoint to the public. Avoid placing prompts, credentials or document contents in unrestricted logs. Retention and provider processing arrangements should match the data you send.

Evaluate a model or prompt change before rolling it out. Test malformed input, access to another user’s document, timeout, rate limit, invalid JSON and an unsupported question. Keep a useful fallback when the provider is unavailable. An AI feature should fail within its boundary rather than making the rest of the application unusable.

Validate a completed result at the boundary

Separate transport success from usable application data. An HTTP 200 means the request reached a successful HTTP response; it does not establish that generation completed, that a field exists or that a summary is faithful. An adapter should interpret the selected provider’s real completion status and extract its text before your application validates the contract.

The following standalone JavaScript uses a fictional normalized response with a status and text field. It makes no API call and needs no credentials. The application accepts only a completed JSON object containing a single nonempty summary of at most 200 JavaScript string units. This is an intentionally small local contract, not a provider response schema or a token limit.

function readSummary(response) {
  if (!response || response.status !== 'complete' ||
      typeof response.text !== 'string') return null;
  let value;
  try { value = JSON.parse(response.text); } catch { return null; }
  if (!value || Array.isArray(value) || typeof value !== 'object') return null;
  if (Object.keys(value).length !== 1 || typeof value.summary !== 'string') return null;
  const summary = value.summary.trim();
  return summary.length > 0 && summary.length <= 200 ? summary : null;
}
const cases = [
  [{ status: 'complete', text: '{"summary":"CSV export is available."}' }, 'CSV export is available.'],
  [{ status: 'partial', text: '{"summary":"Draft"}' }, null],
  [{ status: 'complete', text: '{"summary":' }, null],
  [{ status: 'complete', text: '{"summary":"   "}' }, null],
  [{ status: 'complete', text: '{"summary":42}' }, null],
];
for (const [response, expected] of cases) {
  if (readSummary(response) !== expected) throw new Error('Unexpected summary result');
}
console.log('5 response checks passed');

Make recovery depend on the failure

Retries are new attempts with possible cost and effects. A timeout may mean the client stopped waiting even though the provider continued work. Cancellation can release local resources, but it is not proof that a remote action never happened. For operations that modify data, establish whether an idempotency mechanism or an action-status lookup prevents duplicates before allowing automatic replay.

Keep a request identifier and preserve the user’s input so they can recover without retyping. Show whether the result is a draft, an incomplete stream or a confirmed completion. Error categories should help the user act without exposing provider secrets, raw internal traces or another user’s document.

Sources & further reading

Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.

• OWASP: REST security
• MDN: AbortController