Negative Transfer

A company trains a model to recognize urgent customer messages. “My parcel hasn't arrived; I need it tomorrow” calls for a quick response. We add a second task: choosing a department, such as deliveries or returns. We hope joint learning will help the model read messages better. Instead, it misses urgent cases more often. How can we check whether the extra task caused harm?

Negative Transfer is a decline in performance on the task we care about caused by using knowledge, data or joint learning from another task. We need a reference point: how comparable learning works without that addition. Wang et al., Characterizing and Avoiding Negative Transfer, §3, equations 3–4 formalize this comparison for transferring knowledge from source data to a target task. Here we illustrate a related phenomenon in joint learning of two tasks, also discussed by Yu et al., §1–2.2.

What might have gone wrong?

The shared part of the model turns a message into a list of numbers describing useful text features. Both tasks use it. During learning, wrong answers lead to corrections of shared settings: numbers that control processing.

For choosing a department, the parcel matters; the deadline “tomorrow” may not. If learning this task shifts shared processing toward the topic alone, urgency assessment may lose access to deadline information. This is a possible mechanism in our example, not a reading of a real model's internals. Different task needs alone do not prove harm — the comparison's outcome matters.

Compare the same skill

Prepare two versions: learning urgency alone and learning urgency together with the department. Keep the same messages and urgency labels, comparable model size and computing budget. You are changing the presence of the extra task, rather than comparing two arbitrarily different models.

Suppose we obtain the following invented educational result on the same 100 new messages, used neither for training nor for choosing settings:

Learning setup	Incorrect urgency assessments
Urgency only	10 out of 100
Urgency and department jointly	18 out of 100

The error rate rose from 10% to 18%, an increase of 8 percentage points. In this comparison, the extra task harmed urgency assessment, even if it chooses departments well. The concept therefore names harm that a good result on the extra task alone would not reveal.

One limitation: a single training run and 100 messages are not enough for a confident conclusion. Results can vary. Repeat training runs and check whether the difference persists on data representative of real messages. The conclusion applies to the learning setup tested; it does not mean every combination of these tasks must be harmful.

Related concepts

• Multi-task Learning combines learning across tasks; Negative Transfer is a possible unwanted outcome of that combination.
• Validation Set is held-out data for choosing settings, such as how much of the model to share.
• Test Set serves for final evaluation after the choice of settings is closed. We count the table's errors on such a set.
• Catastrophic Forgetting [Polski] describes losing old skills during further learning. Negative Transfer does not require a task to have been mastered earlier and then forgotten.

The text and illustration were prepared with AI assistance. The message, feature-loss mechanism and table figures are our own educational example, not results from the cited studies.