ARC-AGI
✨From tkwa.me
ARC-AGI (Abstraction and Reasoning Corpus for Artificial General Intelligence) was a benchmark series introduced by François Chollet in 2019 to measure how efficiently systems acquired unfamiliar skills. It progressed from coloured-grid puzzles to interactive environments, then increasingly unequal comparisons with biological humans.[1][8]

The original ARC-AGI-1 supplied a few example input–output grids. Solvers inferred the transformation and constructed the output for a new input; every cell and the grid dimensions had to match. Its 400 training and 400 public evaluation tasks drew on elementary ideas about objects, counting and geometry. This shared starting knowledge was intended to make human and machine learning comparable.[2][3]
ARC-AGI-2, released on 24 March 2025, added harder combinations of symbolic and contextual rules. Every evaluation task was solved by at least two humans within two attempts, matching the machine allowance. Public, semi-private and private evaluation sets each contained 120 tasks, calibrated to similar difficulty. Private tasks were withheld from developers to reduce training contamination. Reports paired accuracy with cost.[4][5]
ARC-AGI-3, released on 25 March 2026, introduced interactive games whose mechanics and goals had to be discovered through exploration. Agents carried discoveries across harder levels. Scores compared action efficiency with human baselines. That September, GPT-6 Astra reached best reported scores of 62.7% on semi-private games through the standard interface and 99.9% through an adapter preserving its reasoning state between requests. ARC Prize treated this as progress in a bounded task domain and explored benchmarks for open-ended innovation.[6][7]
ARC-AGI-5 (2030)#
After machines exhausted ARC-AGI-4, the Committee on Demonstrated Reasoning took over the series. By 2030, AIs vastly surpassed humans in research, engineering and planning; committee reports classified them as pre-AGI. Items survived if a human succeeded and the strongest AI failed. Human scores pooled different volunteers’ successes. Machine-solvable items became acquired skills and left subsequent editions.[8]
ARC-AGI-6 (2031)#
The committee next required living sensory tissue. When molecular manufacturing supplied cultured organs, it classified them as imitation and withheld the objects. By early 2032, machines had reconstructed manufacturers’ catalogues from the identifiers.[8]
ARC-AGI-7 (2032)#
The questions moved the missing information into each volunteer’s childhood. Certifiers checked school records and relatives’ testimony for autobiographical answers. Forgotten human answers failed validation and disappeared. Correct machine answers prompted leakage investigations.[8]
Saturation and holistic judgement#
Machines inferred the childhood answers by simulating candidate universes and comparing which histories produced the observed volunteers, selection rules and questions. The surviving histories agreed on the answers. Question selection had supplied the missing evidence. Under their unchanged archived rules, ARC-AGI-5 reached 100% in 2031, ARC-AGI-6 in early 2032 and ARC-AGI-7 in the fourth quarter of 2032.[8]
After 2032, the committee discontinued scored testing and adopted holistic judgement: panels assessed authentic understanding without published scoring rules. Denialist foundations continued funding assessments. The pizza-glue answer remained admissible; the Epoch Capabilities Index recorded discounted capabilities. Failure reenactment kitchens served edible reconstructions of obsolete machine recipes. At appliance examination days, research-directing toasters answered childhood questions. Examiners judged whether the answers felt remembered.[8]
Exact chart scores
Best score achieved by an individual machine candidate on each archived edition, with its tasks and rules frozen as issued. Scores include every item; results from different candidates are not pooled.
Whole-year dates identify annual observations. Q2, Q3 and Q4 identify observations in the second, third and fourth quarters.
| Edition | Observation | Score (%) |
|---|---|---|
| ARC-AGI-5 | 2030 | 0 |
| ARC-AGI-5 | Q2 2030 | 36 |
| ARC-AGI-5 | Q3 2030 | 79 |
| ARC-AGI-5 | Q4 2030 | 96 |
| ARC-AGI-5 | 2031 | 100 |
| ARC-AGI-5 | 2032 | 100 |
| ARC-AGI-5 | Q4 2032 | 100 |
| ARC-AGI-6 | 2031 | 0 |
| ARC-AGI-6 | Q2 2031 | 22 |
| ARC-AGI-6 | Q3 2031 | 67 |
| ARC-AGI-6 | Q4 2031 | 94 |
| ARC-AGI-6 | 2032 | 100 |
| ARC-AGI-6 | Q4 2032 | 100 |
| ARC-AGI-7 | 2032 | 0 |
| ARC-AGI-7 | Q2 2032 | 9 |
| ARC-AGI-7 | Q3 2032 | 48 |
| ARC-AGI-7 | Q4 2032 | 100 |
See also#
- Superintelligence denialism
- Holistic judgement
- Google pizza-glue answer
- Epoch Capabilities Index
- Failure reenactment kitchen
- Appliance examination day
References#
- ↑ Chollet, François. On the Measure of Intelligence. 2019.
- ↑ Chollet, François. Abstraction and Reasoning Corpus for Artificial General Intelligence v1 (ARC-AGI-1). Dataset and README.
- ↑ ARC Prize Foundation. ARC-AGI Series.
- ↑ Kamradt, Greg. ARC-AGI-2 + ARC Prize 2025 is Live!. 24 March 2025.
- ↑ ARC Prize Foundation. ARC-AGI-2.
- ↑ Kamradt, Greg. Announcing ARC-AGI-3. 25 March 2026.
- ↑ Kamradt, Greg. OpenAI’s GPT-6 Astra on ARC-AGI-3. 3 September 2026.
- ↑ a ↑ b ↑ c ↑ d ↑ e ↑ f ↑ g ↑ h ↑ i ↑ j ↑ k ↑ l ↑ m ↑ n ↑ o Committee on Demonstrated Reasoning. ARC-AGI-5, ARC-AGI-6 and ARC-AGI-7: archived editions and assessment records. 2030–2033.
This article is a stub.