Glossary

Glossary term · · 3 min read

Homoscedastic Uncertainty

A courier company wants to predict delivery times from the address and dispatch time. A model predicts noon for one parcel and 18:00 for another. Even good forecasts can miss the actual time because the data do not capture every delay. The predicted time alone does not tell us how much variation in these errors we assume. Do we need a separate estimate for each parcel, or could one for the whole task be a starting point?

Homoscedastic Uncertainty means modeling uncertainty arising from noise in the data as equal for every input to a given task. “Equal” refers to the spread, not the predicted time or the actual error. Kendall, Gal and Cipolla, §3.1 describe this distinction.

Two parcels with clock faces showing different times are surrounded by equally wide blurred bands, symbolizing a shared level of uncertainty.

One measure for different predictions

For illustration, suppose the errors follow a symmetric bell-shaped normal (Gaussian) distribution: small deviations are more frequent, large ones rarer. We describe its width with the standard deviation, a measure of spread denoted by σ. Let σ be two hours.

ParcelPredictionShared measure of spread
Dispatched in the morning12:00σ = 2 hours
Dispatched in the afternoon18:00σ = 2 hours

This does not promise that every parcel will arrive within two hours of its predicted time. The model assumes the same width of the error distribution for both inputs. The times and spread are our own example, not measurements from a courier company.

Why learn this quantity?

When a model learns several tasks with shared settings, called Multi-task Learning, each task can have a different noise level. In Kendall et al.’s method, the model also learns one positive σ value for each task. That value can change during learning, but at any given moment it is shared across all inputs to that task.

This value controls Task Weighting: the task’s contribution to the joint error score, called the loss. For predicting numbers under the bell-shaped distribution assumption, larger σ reduces the coefficient on the sum of squared errors. An additional term in the objective penalizes excessive increases in σ, so the task cannot simply be muted (§3.2, Equation 7). This differs from GradNorm, which considers tasks’ rates of improvement. The paper studied images; the parcels are an analogy, not a result about LLMs.

Limitation: if deliveries outside the city are much less predictable than those in the center, a single σ will not show that difference. It therefore does not assess the reliability of a particular parcel’s forecast. That would require a model allowing the spread to depend on the input.

The text and illustration were prepared with AI assistance. The parcel example is an educational simplification.

I use AI-generated content as part of my daily learning process.