Intruder-dimension damping
Intruder-dimension damping is a proposed diagnosis and post-hoc modification of adapter residual directions to investigate forgetting.
Project status: Research proposal/design in the inspected project sources; no implementation or completed local efficacy run identified. This entry describes the source audit of 14 September 2026; historical measurements retain their original dates.
Mechanism
Compute an SVD of per-layer adapter/residual deltas, flag directions relative to a chosen base-weight spectral reference, and attenuate selected components before evaluation or merge. The proposal separates reporting count/norm of unusual directions from actually altering weights.
Implementation and controls
Roadmap PR I names engine/intruder_diag.py, an on-demand diagnostic, and an optional merge-time damping flag. The V-KTO follow-up proposes applying alpha values 0.7, 0.85, 0.95, and control 1.0 to the saved salvage adapter, then evaluating HumanEval, MBPP, GSM8K, and LogicBench. These are proposed interfaces and arms, not available API promises.
Evidence and evaluation
The cited project document records this candidate and its intended experiment. It does not provide a completed local result for this method. Published-paper results mentioned by that document are background, not Trainfer measurements.
Limitations and interpretation
A large singular value alone does not establish a harmful “intruder” direction; the reference spectrum and alignment rule must be defined. Editing a delta can damage useful behavior even without taking a gradient step, contrary to the follow-up note's “risk is zero” wording. Positive recovery would need comparisons with uniform scaling and removal of the entire adapter.
Sources
- cont: docs/research/production-implementation-roadmap.md — checkout audited
87946914c7b9. - agi: lile/docs/research/NEXT_ARMS_jun02.md — historical revision
3842fd8875ca.