Editing Models with Task Arithmetic
Paper Summary
The paper that turned model merging into arithmetic. A task vector τ = θ_fine-tuned − θ_pre-trained captures, in weight space, what fine-tuning on one task changed. Because fine-tunes of the same pre-trained model stay in a shared low-loss region, these vectors can be added, negated and combined with a scaling coefficient, and the result is still a working model. It is the cleanest single statement of what is now called weight-space programming, and the basis for the merge methods (TIES, DARE) and tools (mergekit, PEFT merging) that followed.
Abstract
Proposes steering a pre-trained model by editing its weights with task vectors: the difference between a model fine-tuned on a task and the pre-trained model it started from. Negating a task vector reduces the corresponding behavior, adding several task vectors builds a multi-task model, and combining vectors by analogy can improve a task with little or no data for it. (Paraphrased; ICLR 2023.)
Critical Analysis & Questions for Consideration
Task arithmetic is simple, reproducible and now widely used, but its guarantees are empirical and its scope is narrower than the arithmetic notation suggests.
Same-base requirement
the method relies on fine-tunes sharing one pre-trained starting point; merging independently trained models is a different, largely open problem.
Interference is real
summing many vectors degrades results as tasks conflict, which later work (TIES-Merging, DARE) targets by trimming small deltas and resolving sign conflicts.
Hyperparameter sensitivity
λ is tuned on validation data per edit, so "free" merges still need held-out data.
Not capability creation
a merge composes skills that some fine-tune already learned; it does not add new ones.
Theory lags practice
why fine-tunes stay linearly connected (linear mode connectivity) is empirically supported but not fully explained.