oooi cross-checks AI coverage against primary records, scores what survives, and tracks the models, benchmarks and policy behind every story. No stance, only the record. Today's coverage, scored. Where vendor benchmark numbers and independent re-runs diverge A published score is a measurement of a model, a harness, a prompt and a scoring rule at once. When three of those four change, the number changes too — and it is almost never the model that moved. The EU AI Act's real weight falls on documentation, not on capability Regulation (EU) 2024/1689 does not cap how good a model may be. It requires providers to be able to describe what they built, what it was trained on, and how it is evaluated. That is a heavier lift than it sounds. SWE-bench Verified is a test of the whole loop, not the model alone Resolving a real GitHub issue means reading a repository, writing a patch and passing a test suite the model never sees. The top of the board now sits above 95 percent, which raises a harder question than it answers. The Arabic evaluation gap is a measurement problem before it is a model problem Most widely cited benchmarks are English-native. Translating them measures translation quality as much as capability, and it cannot capture the dialect variation that decides whether a product works. GPQA Diamond was built to be un-Googleable. It is now nearly solved. Graduate-level science questions designed so that search does not help. The leaders now clear 95 percent, and the benchmark's own design explains why that number is harder to interpret than it looks. Open weights are closing the benchmark gap faster than the deployment gap On published evaluations the distance between the best open-weight releases and the closed frontier keeps narrowing. On the things that decide whether a model gets deployed, the distance is a different shape. The frontier, as measured by somebody else. Every figure is copied from an independent evaluator's published board, with the evaluator named and the as-of date attached. We never re-run a benchmark ourselves and we never print a vendor's own score as a ranking. The papers behind the headlines. SWE-bench: grading code by whether the tests pass The paper that moved code evaluation from string similarity to execution. Its method — real issues, hidden test suites, binary outcomes — is now the template for agentic benchmarks.

GPQA: the benchmark that validated its own difficulty Writing hard questions is easy. Proving they are hard is not. GPQA's contribution is the validation step: non-experts with unrestricted web access were shown to fail them. HELM argued that one number is never an evaluation Before HELM, model comparison meant picking a benchmark. HELM's proposal was a grid: many scenarios, many metrics, every cell reported, with the gaps left visible instead of averaged away. Ask, and get the sources back. Answers are drawn only from articles published on this site and cite them. An empty source list means the archive had nothing to say, not that the answer is unsupported. How we stay neutral. Three rules decide what gets published here, and all three are visible on the page rather than described in a policy nobody reads. Stories are reported from primary records, not from a take on them. The credibility score and the bias-lean label are the only judgements we apply, and both are printed on the story. Every score shown The score comes from the source list attached to the same article. A single-source claim from the party making it scores low, however plausible it sounds. Everything dated Every benchmark figure carries its evaluator and the date it was read. Where a board has been archived by its evaluator, we say so on the board instead of presenting a snapshot as a ranking. Frequently asked questions From the number of independent sources corroborating the central claim, whether the primary source is the party that benefits from it, and whether the claim can be checked against a public record. The full source list, with the date each was last checked, sits at the foot of every article. No. Coverage is reported from primary records. The credibility score and the bias-lean label are the only judgements applied, and both are shown rather than hidden. A high score describes how well sourced a claim is, not whether we agree with it. From independent evaluators' published leaderboards, copied verbatim with the evaluator named and the as-of date attached. We do not re-run benchmarks ourselves, and we do not print a vendor's own published score as a ranking. All five: Arabic, English, French, Spanish and German. Every article in the archive is published complete in each one, not summarised or partially translated. An article added through the admin tool without a given translation is the one exception, and it shows the English text with a notice on the page. It gets corrected and the correction is recorded on the article, not patched silently. If a figure or a source is wrong, write to the corrections address on the contact page.