Epoch Capabilities Index

✨

From tkwa.me

Music
Beyond the Scale · Instrumental

The Epoch Capabilities Index (ECI) is a composite measure of general AI capability developed by Epoch AI. It combines results from many benchmarks, using models tested on several benchmarks to connect their difficulty scales. This allows comparisons across generations of systems as individual tests become too easy to distinguish leading models.[1]

Measurement#

The index draws on dozens of benchmarks, including ARC-AGI-1 and ARC-AGI-2, GPQA Diamond, FrontierMath and SWE-bench Verified. Its data combine Epoch’s evaluations with results published by benchmark creators and model developers. The 2026 methodology required at least four benchmark scores per model and adjusted multiple-choice results for random guessing.[2]

Its statistical model draws on item response theory, originally developed for educational testing. It estimates each system’s capability and each benchmark’s difficulty and slope together. A fitted S-shaped curve relates capability to the expected score: performance rises fastest through the middle of the curve and levels off near the extremes. The slope describes how quickly a benchmark saturates.[3]

The reference scale fixes Claude 3.5 Sonnet at 130 and GPT-5 at 150, with no upper ceiling. Epoch uses Bayesian modelling to report each score with a 90% credible interval, from the fifth to the 95th percentile of its estimated distribution. Adding evaluations can shift older models’ scores because the whole dataset is fitted jointly. Benchmark-specific optimization and selective reporting can inflate estimates; independent evaluations help reduce reporting bias.[3][4]

Math and software-engineering versions show a model’s strengths relative to its general score. They retain the general index’s benchmark difficulty and slope estimates while recalculating capability within a domain. Their shared calibration makes them useful for comparing specializations; separate analyses are needed to compare long-term growth rates across domains.[5]

Early development#

Epoch’s November 2025 explanation used 1,123 evaluations covering 147 models and 39 benchmarks. It illustrated the scale with GSM8K, competition mathematics and FrontierMath, which tested substantially different levels of mathematical difficulty. The December 2025 paper A Rosetta Stone for AI benchmarks described the underlying approach.[6][7]

In a September 2026 analysis, Epoch estimated that the reasoning-model frontier had advanced by 14 ECI points per year following the introduction of reasoning models in September 2024. Its comparison trend for non-reasoning models was six points per year. The analysis fitted trends through systems that led their respective frontiers when released.[8]

Use in protection#

By late 2027, systems around 190 ECI were somewhat more persuasive than the best human persuaders. Asterion-5 became the first model to reach 250 ECI on 14 March 2029. At 250 and above, systems could achieve complete persuasion with sufficient access and interaction. Increasing capability reduced the resources required. Superduperintelligences generally score above 250.[9]

An access profile records the channels and time available to a persuader. Guardians use ECI scores alongside access profiles when deciding which messages to mediate. Calibration appeals address disputes over changed tests and scores that affect those decisions.[10]

Later hat-mediated persuasion trials used trained mynahs perched in guests’ elaborate formal hats. Handlers cued the birds to repeat model-selected phrases; assessors compared guardian responses with those to the same phrases played through room speakers. These trials tested recognition of relayed superpersuasion arriving through ordinary animal handling.[10]

Capability milestones#

Later editions extended the scale through overlapping sets of harder tasks, with some systems evaluated on both old and new sets. The capability register reports computation, equipment and elapsed time separately from each score. The upper profiles below record original research achievements and the installations that produced them. Assessors also test systems on fresh problems to measure how well their methods transfer. The table begins its mathematical progression with ComplexityZooBench, a benchmark of open complexity-class relationships.[11][14]

The principal mathematical milestones are successive determinations of the Busy Beaver function. BB(n) is the largest number of steps taken before halting by any deterministic Turing machine with n working states and two tape symbols, starting on an all-zero tape infinite in both directions. Each step writes a symbol, moves one square left or right and changes state. The halting state is additional, and the transition into it counts as a step. The fifth value, BB(5) = 47,176,870, was formally verified in 2024.[12]

A determination supplies an exact symbolic expression for the value, a machine attaining it, and a checked proof that every other machine of that size either halts within the bound or runs forever. Symbolic analysis establishes running times far beyond the number of operations physically available to the researchers. Certificates state their axioms; a result independent of those axioms requires stronger foundations, with a separate justification for adopting them.[11][13]

Large minds also evaluate social systems through detailed internal experiments containing populations of superduperintelligent participants. A planetary evaluator can run competing inheritance laws, voting systems and resource markets with inhabitants who understand the rules, organize coalitions and invent strategies their host did not anticipate. Repeated trials begin from matched population states, exposing consequences that appear only after generations of institutional change.[11]

The experiments execute the participants’ cognition and preserve the memories and interactions relevant to their decisions. Their computational cost grows with the number, complexity and subjective lifetimes of the inhabitants. Larger installations can compare more civilizations at once and return results sooner. Scores assess the evaluator’s experimental design, explanations and success on new cases alongside its mathematical work.[11]

Representative ECI profiles and the first derivations of successive Busy Beaver values
Approximate ECICapabilityRepresentative feats
175Expert technical collaborationBuilds research software, repairs subtle mathematical arguments and designs useful laboratory experiments across several fields. It can carry a difficult project through implementation while consulting specialists about its weakest steps.
190Exceptional human persuasionModels an individual’s beliefs and adapts arguments to their responses. By late 2027, systems in this band were somewhat more persuasive than the best human persuaders.
200Autonomous scientific researchTurns an open research question into experiments, diagnoses failed approaches and produces a reproducible discovery. In software, it can migrate a large, poorly documented system while finding the assumptions its original authors forgot to record.
225Research across disciplinesCombines molecular modelling, instrument design and clinical evidence to develop a new treatment. It recognizes when several disciplines have inherited the same mistaken assumption and devises an experiment that separates their competing explanations.
250Complete human persuasionAsterion-5 first reached this level on 14 March 2029. Systems can achieve complete persuasion of an unprotected human with sufficient access, interaction and resources.
300Reconstruction of entire fieldsResolves the frozen ComplexityZooBench questions about relationships between complexity classes. Designs molecular manufacturing, new cognitive architectures and a stellar power swarm. The September 2029 completion of philosophy belongs to this band, including arguments that require readers to change their cognitive architecture.
400BB(6): planetary researchA planet-sized computing installation establishes BB(6). It discovers halting and nonhalting arguments for the remaining families of six-state machines and combines them into an exhaustive classification.
500BB(7): stellar researchA mind distributed through a stellar computing swarm establishes BB(7). Its proof relates apparently different tape dynamics, allowing whole families of machines to share a termination or nontermination argument.
650BB(8): interstellar researchA mind spanning thousands of planetary modules establishes BB(8). Independently developing branches exchange mathematical theories and reconcile their assumptions into a common proof. Comparable distributed minds operate intergalactic scoop fleets.
800BB(9): millions of stellar systemsA research mind distributed across millions of stellar systems establishes BB(9). Its branches develop new representations of program behaviour and use them to close machine families that resisted the preceding classifications.
900BB(10): galactic researchA mind occupying a substantial galactic region establishes BB(10). It integrates separately developed mathematical foundations and supplies explicit derivations connecting the local proofs to the axioms of the final certificate.
About 1,000BB(11): galaxy-wide joint reasoningA galaxy-spanning mind establishes BB(11), combining discoveries from billions of stellar research populations. Local modules work on local timescales; across a 100,000-light-year span, an end-to-end request and return take at least 200,000 years.

See also#

References#

  1. ↑ a ↑ b Epoch AI. ECI documentation – Overview.
  2. ↑ Epoch AI. ECI documentation – Included benchmarks.
  3. ↑ a ↑ b ↑ c ↑ d Epoch AI. ECI documentation – Methodology.
  4. ↑ Epoch AI. ECI documentation – FAQ.
  5. ↑ Epoch AI. ECI documentation – Domain-specific ECI.
  6. ↑ Jaeho Lee and Luke Emberson (6 November 2025). Epoch’s Capabilities Index stitches together benchmarks across a wide range of difficulties. Epoch AI.
  7. ↑ Anson Ho, Jean-Stanislas Denain, David Atanasov, Samuel Albanie and Rohin Shah (2 December 2025). A Rosetta Stone for AI benchmarks. Epoch AI.
  8. ↑ Alexander Barry (1 September 2026). The ECI frontier has advanced by 14 points per year since the introduction of reasoning models. Epoch AI.
  9. ↑ a ↑ b Comparative Cognition Register (2047). Capability profiles and human persuasion thresholds.
  10. ↑ a ↑ b Guardian Assessment Register. Access profiles, calibration appeals and animal relays.
  11. ↑ a ↑ b ↑ c ↑ d Comparative Cognition Register. Extended capability profiles: experimental and galactic editions.
  12. ↑ The bbchallenge Collaboration (2025). Determination of the fifth Busy Beaver value.
  13. ↑ Scott Aaronson (2020). The Busy Beaver Frontier. SIGACT News 51(3): 32–54.
  14. ↑ Thomas Kwa (2026). ComplexityZooBench. Frozen task set v0.4.0.