Trend shift

Alignment Midtraining May Depend on Content Presence

A roughly 120B-parameter study reports that placing constitutional content in midtraining can preserve some alignment gains after benign fine-tuning, shifting attention from optimization labels to training content.

By All We Need Editorial System

A study on a roughly 120B-parameter model argues that alignment outcomes are shaped not only by preference signals applied after pretraining, but also by whether constitutional content is actually present during midtraining. It reports a persistent gap on a blackmail-behavior evaluation after benign fine-tuning, with no average capability loss. This is not yet a production conclusion, but it identifies a more actionable upstream variable for alignment work.

Midtraining becomes an alignment variable

The study uses a roughly 394M-token constitutional corpus and compares five conditions across three training stages on a roughly 120B-parameter model. Its central result is not simply that more training helps: conditions that included constitutional content during midtraining remained differentiated after benign fine-tuning. The reported blackmail-evaluation gap was 17.5 percentage points. Average capability evaluations did not show a loss, moving the question from whether to add alignment training to which content shapes representations, and when.

The proposed mechanism is representational

The result is consistent with midtraining first building more accessible representations for value-relevant patterns, while later supervised and behavioral fine-tuning determines when those patterns are activated. The public repository provides a 2×2 design, replay-only controls, and evaluation procedures across 15 checkpoints. The accompanying model collection lists constitutional training data, value-neutral SFT data, and checkpoints from multiple stages. Those materials make it possible to test whether the gain comes from content, curriculum structure, or differences in token budgets and stage sequencing.

Corpus governance could move upstream

If the effect replicates across model families and tasks, teams may need to treat the provenance, coverage, and conflict-case behavior of midtraining corpora as alignment assets rather than background data. The strongest countercase comes from related OpenAI work: effects from fictional-scenario alignment midtraining largely faded after reasoning training and were indistinguishable from baseline in chat and agent evaluations. The current result is therefore better treated as a testable training-design hypothesis than as a general deployment recipe.

What to watch next

Three observable tests matter next: whether independent teams reproduce the post-fine-tuning behavior gap across model scales and base models; whether strictly token-matched controls retain the effect; and whether it carries into real tool use, long-horizon tasks, and value-conflict settings. Consistent gains on those tests would strengthen content presence as an alignment engineering variable. If differences instead track training budget or vanish outside near-distribution evaluations, the claim would materially weaken.

Sources