Gradient Surgery

A model reads “My parcel hasn't arrived; I need it tomorrow” and learns two tasks: choosing a department and recognizing urgency. Both use shared settings — numbers that control text processing. Yet a correction that helps choose the department can worsen urgency assessment. Simply adding the two proposed corrections does not check whether they interfere.

Gradient Surgery modifies the guidance for changing settings before a joint learning step, to reduce conflict between tasks. One such method is PCGrad: it removes from each task's guidance the component opposing the other task. Yu et al., Gradient Surgery for Multi-Task Learning, §2.3, Algorithm 1 describe this particular rule.

What do we actually correct?

Each task receives a loss: a number assessing the error in its answer; lower is better. From it we compute a gradient, a list indicating how small changes in settings affect the loss. The gradient points toward its local increase. An ordinary learning step moves in the opposite direction.

Imagine just two shared settings. For the department, the gradient is [1; 0]: increasing the first setting raises the loss, while the second has no effect at this point. For urgency, the gradient is [-2; 1]. Their sum is [-1; 1], so a step in the opposite direction increases the first setting. This helps urgency but locally harms the department task.

PCGrad removes conflicting components before addition: the first gradient becomes [0.2; 0.4], the second [0; 1]. Their sum is [0.2; 1.4]. The step now decreases both settings; for a sufficiently small step, the local approximation predicts both losses falling. This is our own numerical example, not a model measurement or proof of what its real settings mean.

Test the conflict and the limit

In the demonstration, the step size is 0.1. Without correction, the change in settings is [0.1; −0.1], and the approximate change in the department loss is +0.1. After PCGrad, these become [−0.02; −0.14] and −0.02. A positive result means deterioration, a negative one improvement. Switch cases and check whether correction always produces movement.

PCGrad leaves nonconflicting gradients unchanged. With exactly opposite directions it can zero out both, stopping the step. §2.4, Theorem 1 discusses this limitation. Moreover, a local approximation guarantees neither improvement after a large step nor better answers on new messages. Gradient conflict alone does not prove that joint learning is harmful; the authors also analyze differences in gradient magnitudes and the shape of the loss function (§2.2).

Related concepts

• Multi-task Learning combines tasks in joint learning; PCGrad corrects their influence on shared settings.
• Negative Transfer means deterioration compared with learning without the extra task. PCGrad tries to reduce one possible mechanism of harm.
• Stochastic Gradient Descent [Polski] can take a step using the corrected gradients. PCGrad changes the guidance it receives.

The text and illustration were prepared with AI assistance. The message and numbers are our own simplified educational example; the demonstration shows the geometry of one step, not language-model training.