How AI Models Learn: Training and Inference

Training changes the model

Suppose a dispatch screen needs to estimate delivery hours before a parcel leaves the depot. You have past distances and actual delivery times. The task is to find a useful relationship in those past examples, then apply it to a new parcel whose arrival time is still unknown.

During training, an algorithm adjusts model parameters to improve an objective. In supervised learning, examples include inputs and desired outputs. A delivery-time model might take distance and parcel size as inputs and learn from actual delivery times. A loss function measures how far its predictions are from the targets; an optimizer uses that feedback to update parameters.

Training data, parameters and application settings are different things. Changing an instruction sent to a hosted language model usually changes the input for that request, not its weights. Fine-tuning is a separate training process. Whether a provider retains or uses submitted data is a separate service-policy question; do not infer it from the word “inference”.

Keep an honest evaluation set

A training set supplies examples for fitting parameters. A validation set helps choose model settings. A final test set estimates performance after those choices are made. If you repeatedly choose a model because it performs well on the test set, the test set is effectively becoming another validation set.

Split data along the boundaries of the real problem. Randomly dividing nearly identical tickets from the same conversation can put a copy in both training and testing. For forecasting, training on future records makes an evaluation unrealistic. Fit preprocessing, such as scaling, using training data only, and apply the learned transformation to later sets.

Training:   deliveries completed in January–March
Validation: deliveries completed in April
Test:       deliveries completed in May

Inputs: facts available when the parcel is dispatched
Target: actual arrival time, known only afterwards

Inference applies what was learned

At inference time, the deployed model receives an input and produces an output using its current parameters. The application still has work to do: validate the input, use the same preprocessing as training, interpret the result and decide whether it is safe to act on it. A model file alone is not a working product.

An API may return a prediction quickly even though training took a long time. Conversely, a large generative model can require substantial computation for each response. Track the latency and cost of the complete request, including retrieval, generation and validation, rather than treating “inference” as free.

A good training score is not the goal

Overfitting means the model captures details of the training examples that do not help on new examples. Compare training and validation results, inspect errors and start with a simple baseline. More training is not always better. Regularization, better data and a simpler model may help, but the evaluation must show that they do.

Even a well-tested model can degrade when conditions change. A new delivery region or a different ticket template can change the input distribution. Monitor outcomes and review the data before automatically retraining: new records can contain mistakes, private information or malicious examples.

Fit one parameter, then leave it fixed

We can make the distinction tangible without an ML library. Assume predicted delivery hours equal distance multiplied by one learned rate. The rate is a parameter; distance is an input; observed hours are the target. The example tries three possible rates and chooses the one with the smallest mean absolute error on the training rows. Mean absolute error is the average distance between predictions and targets, measured here in hours.

Run the complete example in Node.js or a browser console. This tiny grid search really fits a parameter, but it is not a realistic delivery service: the fictional data has a perfect linear relationship, no traffic and no uncertainty. Larger models usually search their parameter space with different optimization methods. The important boundary is that the held-out rows never choose the rate.

const training = [[10, 2], [20, 4], [30, 6]];
const heldOut = [[15, 3], [25, 5]];
const candidates = [0.1, 0.2, 0.3];
function meanAbsoluteError(rows, rate) {
  return rows.reduce((sum, [km, hours]) =>
    sum + Math.abs(km * rate - hours), 0) / rows.length;
}
const rate = candidates.reduce((best, candidate) =>
  meanAbsoluteError(training, candidate) < meanAbsoluteError(training, best)
    ? candidate : best);
console.log('Learned rate:', rate);
console.log('Held-out error:', meanAbsoluteError(heldOut, rate));
console.log('Prediction for 40 km:', 40 * rate);
console.log('Error after conditions change:',
  meanAbsoluteError([[15, 6], [25, 8]], rate));

Sources & further reading

Primary references for the concepts in this guide. Examples and exercises are illustrative; verify provider-specific behaviour against the version you use.

• Google: Generalization
• Google: Dividing datasets
• scikit-learn: Preprocessing and data leakage