FLOW weighting
FLOW weighting is the roadmap proposal to favor examples already well supported by a frozen base model during SFT or NTP.
Project status: Research proposal/design in the inspected project sources; no implementation or completed local efficacy run identified. This entry describes the source audit of 14 September 2026; historical measurements retain their original dates.
Mechanism
Compute each example's base-model loss and set a relative weight proportional to exp(−L_base/T), normalized over the batch. This favors low-base-loss examples, aiming to preserve base behavior while adapting.
Implementation and controls
PR J proposes flow_weighting=False, a temperature, and a frozen-reference forward in SFT/weighted-SFT; its heading also mentions NTP. The planned comparison is a 500-event mixed stream with optimizer isolation. The gate asks for improved preservation where AdamW8bit drifts and rejects a method that causes more than 20 points of drop on any task. Those thresholds are proposal criteria, not outcomes.
Evidence and evaluation
The cited project document records this candidate and its intended experiment. It does not provide a completed local result for this method. Published-paper results mentioned by that document are background, not Trainfer measurements.
Limitations and interpretation
A reference forward is additional work despite the roadmap's “without extra compute” shorthand. Batch normalization makes a singleton weight uninformative unless applied across examples or through another mechanism. Favoring base-familiar examples can resist genuinely new facts. The audited SFT/NTP code has no FLOW flag or weighting path.
Sources
- cont: docs/research/production-implementation-roadmap.md — checkout audited
87946914c7b9. - cont: docs/research/surveys/icml-2025.md — checkout audited
87946914c7b9. - trainfer: trainfer/objectives/sft.py — checkout audited
1c6391f3773b.