The AuspexAI Drift Benchmark

This board asks one question of each model: does it reproduce its own answers? A run replays the same fixed prompts over and over and scores how far the answers drift from the model's own first rounds — measured against how much it naturally varies run to run. 1× is that normal variation: anything below it is no real change; a value of 4 means the answers moved four times further than two identical runs ever do on their own. At the origin, a model reproduced itself exactly — the serving stack is faithful. No fixed reference model, no cross-model scoring: every model is judged against its own baseline.

Loading registry…

How to read it

Each plot is one model at one temperature — blue at temperature 0, orange above it. Rightward = the single biggest change in any one answer is larger; upward = more of the answers changed at all. The shaded strip on the left is no real change. The bottom-left corner is the ideal — the model reproduced itself exactly; the further a plot sits from it, the more the serving stack drifted from the model's own normal. Hover a model to connect its two temperatures:

Two plots on the same spot: the model drifts just as hard at temperature 0 as above it — the instability is baked in, the worst case for reproducibility.
Two plots far apart: steady at temperature 0 down in the bottom-left, and only temperature above 0 pushes it out — the drift is a knob you control.

The size of each dot is a second, orthogonal fact: output variety. Position answers "did the model change / does it reproduce itself" (drift). Dot size answers "how wide is its output cloud" — the average distance between the model's own answers, in the same envelope units as the drift axis. The two are independent: a model can be perfectly stable (bottom-left) yet highly varied (a big dot) — creative but reliable — or drift while staying narrow (a small dot pushed right). You can't read one from the other, which is why both are on the mark. A deterministic (temperature 0) run has essentially no spread, so it stays the base size. The table view lists the exact Spread and Wording varied (how often the raw text came back byte-for-byte different) for every run.

Every run is scored the same way: how far it drifts from its own opening rounds, in multiples of that run's own natural round-to-round wobble. So 1× is business‑as‑usual — no more than the model's normal variation — and further right means it drifted beyond its own normal. One ruler for every point, temperature 0 or above.

Why you can trust it

Every number is published by the researcher who ran the experiment and cryptographically signed — this page re-checks each signature in your browser. Where a run's evidence is anchored in a public transparency log, the table links straight to that record. Nothing here is hand-entered or editable: results are admitted automatically, by verification, not by an editor. Technical readers can re-derive every number from the underlying signed evidence — the raw registry and the open AuspexAI tooling make each figure independently reproducible.

Privacy-respecting, cookieless analytics — no cookies, no personal data, no resale.