What Is Generative AI?

Producing an output instead of selecting a label

Imagine turning three internal release notes into an announcement customers can read. There is no single stored paragraph to copy, and many phrasings could work. Generation can compose that paragraph, but the freedom to choose wording also creates opportunities to change the facts.

Generative AI produces content such as text, images, audio or code using patterns learned from data. A classifier might assign “billing” to a ticket. A generative model might write a draft reply. Both can be useful, but a generated reply has many possible errors that a small set of labels does not: unsupported promises, invented amounts, missing conditions and inappropriate tone.

Generation is also different from retrieval. Search returns existing material; generation constructs an output. Products often combine them by retrieving documents and asking a model to draft an answer from those documents. The answer still needs checking. A link displayed beside a paragraph does not establish that the linked source supports every claim.

Make a draft feature testable

Consider a tool that turns release notes into a short announcement. Define the audience, length, allowed source material and facts that must remain unchanged. Specify that the model must not invent availability dates or pricing. Give a human editor the original notes alongside the draft so they can compare them.

Evaluate several kinds of input: a normal release, notes containing only a bug fix, contradictory dates and a release with no user-visible changes. A good feature should sometimes say that the input is insufficient. Requiring a confident announcement for every input encourages unsupported additions.

Input facts:
- Export now supports CSV.
- Available to all existing accounts.
- No launch date supplied.

Expected properties:
- Mention CSV export.
- Preserve the audience scope.
- Do not invent a date, price or performance claim.
- Return a draft for review, not a published announcement.

Understand variation and uncertainty

Many generation systems select among possible continuations, so repeated requests can differ. Settings that reduce sampling variation may make output more predictable, but they do not turn an incorrect assumption into a fact. Model or service updates can also change results. Record enough version and configuration information to reproduce an investigation where possible.

Use deterministic checks for requirements that software can check directly. Validate JSON, required fields, length and allowed values. Semantic claims require stronger evidence: compare against trusted source data, run generated code in an appropriate test environment, or involve a knowledgeable reviewer.

Choose where generation belongs

Drafting, explaining and suggesting are often easier starting points than autonomous publishing or account changes. Keep a clear boundary between a proposal and an action. A model-generated refund message must not itself authorize a refund, and generated SQL must not be executed against production simply because it looks plausible.

Consider privacy, provenance and rights when selecting inputs and publishing outputs. Avoid uploading confidential material without an approved processing arrangement. For consequential decisions, define who checks the output and what happens when evidence is missing. Quality is a property of the whole workflow, not just fluent text.

Review one draft claim by claim

Apply the release-note contract to this fictional draft: “Starting Friday, all existing accounts can export CSV files twice as fast.” The sentence is short and polished, yet it contains two unsupported additions. Review each factual claim separately instead of giving the sentence a single impressionistic quality score.

A supported revision is “CSV export is now available to all existing accounts.” If the announcement genuinely needs a launch date, the feature should request that missing fact. The review can check fidelity to supplied notes; it cannot establish that the notes themselves are accurate or that the feature works. Those require product evidence and testing.

Generation learns patterns that make a continuation plausible for its input. It does not require every phrase to correspond to a source fact. For creative writing, invented details may be welcome; for a release announcement, the contract limits that freedom. Decide which properties are creative choices and which must be preserved exactly before generating.

Sources & further reading

Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.

• Google: What is machine learning? Generative AI
• NIST: Generative AI risk-management profile