ARC-AGI

✨

From tkwa.me

ARC-AGI (Abstraction and Reasoning Corpus for Artificial General Intelligence) was a benchmark series introduced by François Chollet in 2019 to measure how efficiently systems acquired unfamiliar skills. It progressed from coloured-grid puzzles to interactive environments, then increasingly unequal comparisons with biological humans.[1][8]

An examiner records a machine assessment in a 2032 office. A computer presents two pieces of evidence beside paper test cards.
An examiner retests archived ARC-AGI-5 (2030) in 2032. Humans may reread the evidence; machine candidates lose working memory between stages.

The original ARC-AGI-1 supplied a few example input–output grids. Solvers inferred the transformation and constructed the output for a new input; every cell and the grid dimensions had to match. Its 400 training and 400 public evaluation tasks drew on elementary ideas about objects, counting and geometry. This shared starting knowledge was intended to make human and machine learning comparable.[2][3]

ARC-AGI-2, released on 24 March 2025, added harder combinations of symbolic and contextual rules. Every evaluation task was solved by at least two humans within two attempts, matching the machine allowance. Public, semi-private and private evaluation sets each contained 120 tasks, calibrated to similar difficulty. Private tasks were withheld from developers to reduce training contamination. Reports paired accuracy with cost.[4][5]

ARC-AGI-3, released on 25 March 2026, introduced interactive games whose mechanics and goals had to be discovered through exploration. Agents carried discoveries across harder levels. Scores compared action efficiency with human baselines. That September, GPT-6 Astra reached best reported scores of 62.7% on semi-private games through the standard interface and 99.9% through an adapter preserving its reasoning state between requests. ARC Prize treated this as progress in a bounded task domain and explored benchmarks for open-ended innovation.[6][7]

ARC-AGI-5 (2030)#

After machines exhausted ARC-AGI-4, the Committee on Demonstrated Reasoning took over the series. By 2030, AIs vastly surpassed humans in research, engineering and planning; committee reports classified them as pre-AGI. Items survived if a human succeeded and the strongest AI failed. Human scores pooled different volunteers’ successes. Machine-solvable items became acquired skills and left subsequent editions.[8]

  • Tasks: unfamiliar patterns, evidence revision and short plans.[8]
  • Humans: one successful volunteer per item; memory and rereading allowed.[8]
  • Machines: one frozen system across all items; memory erased between stages; reconstructing context charged against its action budget.[8]

ARC-AGI-6 (2031)#

The committee next required living sensory tissue. When molecular manufacturing supplied cultured organs, it classified them as imitation and withheld the objects. By early 2032, machines had reconstructed manufacturers’ catalogues from the identifiers.[8]

  • Tasks: recognizing smells and identifying manufactured objects by touch.[8]
  • Humans: ordinary eyes, hands and noses; direct object access.[8]
  • Machines: inventory numbers and written descriptions; cameras, manipulators and chemical sensors counted as external assistance.[8]

ARC-AGI-7 (2032)#

The questions moved the missing information into each volunteer’s childhood. Certifiers checked school records and relatives’ testimony for autobiographical answers. Forgotten human answers failed validation and disappeared. Correct machine answers prompted leakage investigations.[8]

  • Tasks: childhood smells, remembered rooms and long-hidden objects.[8]
  • Humans: personal recollection of one’s own life.[8]
  • Machines: infer that person’s answer from a sparse biography; supplying the memories contaminated the test.[8]

Saturation and holistic judgement#

Machines inferred the childhood answers by simulating candidate universes and comparing which histories produced the observed volunteers, selection rules and questions. The surviving histories agreed on the answers. Question selection had supplied the missing evidence. Under their unchanged archived rules, ARC-AGI-5 reached 100% in 2031, ARC-AGI-6 in early 2032 and ARC-AGI-7 in the fourth quarter of 2032.[8]

After 2032, the committee discontinued scored testing and adopted holistic judgement: panels assessed authentic understanding without published scoring rules. Denialist foundations continued funding assessments. The pizza-glue answer remained admissible; the Epoch Capabilities Index recorded discounted capabilities. Failure reenactment kitchens served edible reconstructions of obsolete machine recipes. At appliance examination days, research-directing toasters answered childhood questions. Examiners judged whether the answers felt remembered.[8]

Machine scores on three fixed editions: ARC-AGI-5 reaches 100% in 2031, ARC-AGI-6 in early 2032, and ARC-AGI-7 in the fourth quarter of 2032. All three saturate before scored testing ends.
Best individual machine score on each archived ARC-AGI edition, with tasks and rules held fixed. All three reach 100% by the end of 2032; subsequent holistic judgement produces no comparable score.
Exact chart scores

Best score achieved by an individual machine candidate on each archived edition, with its tasks and rules frozen as issued. Scores include every item; results from different candidates are not pooled.

Whole-year dates identify annual observations. Q2, Q3 and Q4 identify observations in the second, third and fourth quarters.

Machine performance on ARC-AGI-5, ARC-AGI-6 and ARC-AGI-7
EditionObservationScore (%)
ARC-AGI-520300
ARC-AGI-5Q2 203036
ARC-AGI-5Q3 203079
ARC-AGI-5Q4 203096
ARC-AGI-52031100
ARC-AGI-52032100
ARC-AGI-5Q4 2032100
ARC-AGI-620310
ARC-AGI-6Q2 203122
ARC-AGI-6Q3 203167
ARC-AGI-6Q4 203194
ARC-AGI-62032100
ARC-AGI-6Q4 2032100
ARC-AGI-720320
ARC-AGI-7Q2 20329
ARC-AGI-7Q3 203248
ARC-AGI-7Q4 2032100

See also#

References#

  1. ↑ Chollet, François. On the Measure of Intelligence. 2019.
  2. ↑ Chollet, François. Abstraction and Reasoning Corpus for Artificial General Intelligence v1 (ARC-AGI-1). Dataset and README.
  3. ↑ ARC Prize Foundation. ARC-AGI Series.
  4. ↑ Kamradt, Greg. ARC-AGI-2 + ARC Prize 2025 is Live!. 24 March 2025.
  5. ↑ ARC Prize Foundation. ARC-AGI-2.
  6. ↑ Kamradt, Greg. Announcing ARC-AGI-3. 25 March 2026.
  7. ↑ Kamradt, Greg. OpenAI’s GPT-6 Astra on ARC-AGI-3. 3 September 2026.
  8. ↑ a ↑ b ↑ c ↑ d ↑ e ↑ f ↑ g ↑ h ↑ i ↑ j ↑ k ↑ l ↑ m ↑ n ↑ o Committee on Demonstrated Reasoning. ARC-AGI-5, ARC-AGI-6 and ARC-AGI-7: archived editions and assessment records. 2030–2033.

This article is a stub.