How to Verify AI-Generated Code

Begin with requirements, not the patch

Generated code is a proposed implementation. Before reviewing it, state the expected behaviour, input boundaries and things that must remain unchanged. Otherwise it is easy to accept a patch because it looks tidy while it solves a different problem. Read the diff, including dependency and configuration changes.

For a port parser, “return a number” is too weak. Decide whether spaces, signs, decimal points and leading zeros are valid. Decide how invalid input is reported. These are product choices, not details a model should silently invent. The example below adopts a strict decimal-only policy and returns null on rejection.

Write tests that challenge the implementation

JavaScript’s parseInt accepts a numeric prefix, so parseInt("80abc", 10) returns 80. A generated parser based only on parseInt may pass the normal case and still violate the contract. Use counterexamples selected from the requirement rather than copying the implementation’s own assumptions.

This implementation permits leading zeros but rejects whitespace and signs. If your application needs a different policy, change both the contract and the tests deliberately. Do not broaden acceptance just to make a failing test disappear.

function parsePort(text) {
  if (!text.length || /[^0-9]/.test(text)) return null;
  const value = Number(text);
  return Number.isInteger(value) && value >= 1 && value <= 65535
    ? value : null;
}
for (const [input, expected] of [
  ['80', 80], ['00080', 80], ['65535', 65535],
  ['0', null], ['65536', null], ['80abc', null],
  ['1.5', null], ['', null], [' 80', null], ['+80', null]
]) {
  if (parsePort(input) !== expected) throw new Error('Failed: ' + input);
}
console.log('10 parser checks passed');

Use several kinds of evidence

Type-checking catches incompatible types, not every behavioural mistake. Unit tests check selected cases. Integration tests reveal mismatches between components. Security review asks whether permissions, input validation, secrets and output handling remain correct. Each layer answers a different question.

Check that dependencies and APIs actually exist in the versions the project uses. Avoid installing a package merely because a model proposed its name. Inspect error paths, cleanup and cancellation. A function that works once can still leak resources or fail when two requests run together.

Verify the change in context

Run the relevant checks in the real project and inspect their output. Compare the changed behaviour with the old behaviour for inputs that should remain supported. If a test fails, establish whether the implementation or the test contradicts the requirement. Generated tests can be wrong too.

For consequential changes, have another person review the reasoning and evidence. Record known limitations rather than claiming that a passing suite proves correctness. Keep patches focused so rollback and investigation remain practical if a production case exposes a gap.

Confirm a test can catch the original defect

A useful regression check must distinguish the requirement from the faulty behavior. Run the snippet below separately from the corrected parser. It deliberately fails: the numeric-prefix implementation returns 80 for “80abc,” but the independently chosen expectation is null. A failure here is the desired observation, not a reason to rewrite the expected value to 80.

Then run the corrected parser’s checks. Rejecting any non-ASCII-digit character before conversion prevents this prefix acceptance. Bounds checks separately reject 0 and 65536. These checks address different failure classes: lexical validity, numeric range and the caller’s responsibility to supply the right type.

If a test still passes after replacing the implementation with the known-bad version, investigate the test. It may not execute the relevant path, may lack an assertion or may only cover a happy case. Tests generated alongside code can repeat the same misunderstanding, which is why the expectations should come from the contract.

function proposedParsePort(text) {
  const value = parseInt(text, 10);
  return value >= 1 && value <= 65535 ? value : null;
}
const actual = proposedParsePort('80abc');
if (actual !== null) {
  throw new Error('Expected null for 80abc; got ' + actual);
}

Related programming documentation

• JavaScript functions

Sources & further reading

Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.

• MDN: parseInt
• OWASP: Secure coding with AI