Glossary

Glossary term · · 3 min read

Heteroscedastic Uncertainty

A courier company predicts noon delivery for two parcels. In the city center, actual delivery times for similar addresses and dispatch times usually stay close to the forecast. Outside the city, the spread is larger: the data lack information about some local delays. A single measure of spread for every parcel hides this difference. The same prediction need not mean the same predictability.

Heteroscedastic Uncertainty means modeling the noise level as dependent on the input. Here, noise means variation in the outcome that the available data cannot explain. The model can therefore assign different spreads to different addresses instead of using one shared value as in Homoscedastic Uncertainty. Kendall and Gal, §2.2 describe the distinction and how to train such a model.

Two identical parcels on ink-painted routes: a compact trail beside city buildings and a broad, blurred trail outside the city symbolize different spreads in delivery forecasts.

Time and spread are two pieces of information

For illustration, assume a symmetric, bell-shaped error distribution, called a normal distribution: small deviations are more frequent and large ones rarer. Its width is described by the standard deviation, denoted by σ. The model now returns both a time and a positive σ value for the particular parcel.

AddressPredictionPredicted spread
City center12:00σ = 0.5 hours
Outside the city12:00σ = 2 hours

The times and numbers are our own example, not measurements from a company. The second spread is four times larger, although the center of both forecasts stays the same. Neither σ value is a maximum error or a guarantee of delivery within “time ± σ”.

Where does the model get the second number?

During training, the model sees data about past parcels and their actual delivery times. It fits both the predictions and the spread. In the described method, a large error with a small σ sharply raises the loss, a numerical score of mismatch. Increasing σ softens that component, but a second loss component penalizes excessive spread. Declaring enormous uncertainty for every parcel is therefore not enough (§3.1, Equation 8, and §3.2). This is a rule for predicting numbers under a particular distribution, not an arbitrary chatbot confidence score.

Limitation: address-dependent spread does not guarantee recognition of a completely new country that the model never encountered in training. It describes noise and does not, on its own, measure uncertainty about the model’s settings; the authors distinguish these sources of uncertainty in §2.2–3.1. In practice, it is worth checking on a Test Set, examples held aside for final evaluation, whether the predicted spread matches actual errors. The paper studies images; the parcels are an analogy, not a result about LLMs.

The text and illustration were prepared with AI assistance. The delivery example is an educational simplification.

I use AI-generated content as part of my daily learning process.