ARC-AGI
Project record
Maintained by ARC Prize Foundation1391261
ARC-AGI is a series of benchmarks that measure how efficiently a system acquires new skills on unfamiliar tasks, using tasks that rely only on basic "core knowledge" priors and are easy for people. ARC-AGI-1 (introduced in 2019) and ARC-AGI-2 (March 2025) are grid puzzles in which the solver infers a transformation from a few input-output examples; ARC-AGI-3 (March 2026) consists of interactive, turn-based game environments in which agents must explore and infer goals without explicit instructions. Public tasks are released for each version, while semi-private and private evaluation sets used for official scoring are held back.1235614
- Website: ARC-AGI overview (ARC Prize) (external site: arcprize.org)
- Repository: ARC-AGI-1 repository (external site: github.com)
- Repository: ARC-AGI-2 repository (external site: github.com)
- Repository: ARC-AGI Toolkit for ARC-AGI-3 (external site: github.com)
- Paper: ARC-AGI-3 technical report (arXiv 2603.24621) (external site: arxiv.org)
Availability and license
Overall availability
ARC-AGI-1 (400 training and 400 evaluation tasks) and ARC-AGI-2 (1,000 training and 120 public evaluation tasks) are downloadable from GitHub under Apache 2.0. ARC-AGI-3 games can be played in a browser or through the toolkit and API; an API key is optional but gives access to more games. The semi-private and private evaluation sets of ARC-AGI-1 and -2, and the semi-private set of ARC-AGI-3, are not published.78910351114
Availability is separate from permission: read the license before using or redistributing.
Apache License 2.0 (ARC-AGI-1 task data and testing interface) (external site: raw.githubusercontent.com)8
Apache License 2.0 (ARC-AGI-2 public task data) (external site: raw.githubusercontent.com)10
MIT License (ARC-AGI Toolkit for ARC-AGI-3) (external site: raw.githubusercontent.com)12
A license for the ARC-AGI-3 game environments themselves was not found in the pages read; the toolkit that connects agents to them is MIT-licensed.115
Public materials checklist
| Item | Status | Notes and evidence |
|---|---|---|
| CodeIs the evaluation code published? | Public | The ARC-AGI-1 repository includes a browser-based testing interface (Apache 2.0), and the ARC-AGI-3 toolkit (arc-agi on pip) is MIT-licensed.71112 |
| Tasks / dataAre the tasks or test data available? | Partial | Public training and evaluation sets are released for ARC-AGI-1 and -2, and a public game set for ARC-AGI-3; the semi-private and private evaluation sets of ARC-AGI-1 and -2, and ARC-AGI-3's semi-private set, are withheld.23414 |
| MethodologyIs the method for scoring described? | Public | The ARC-AGI-1 README defines the JSON task format and the scoring rule (exact output grid, three trials per test input); ARC-AGI-2 evaluation tasks were calibrated with human testers; the ARC-AGI-3 technical report describes the environment design and efficiency-based scoring.736 |
| ReproducibilityAre instructions for reproducing results published? | Partial | The public sets can be run locally from the documented task formats or through the toolkit, but official scores on the semi-private and private sets are produced only by ARC Prize through verified testing or its competitions.71114 |
| LimitationsAre known limitations documented? | Partial | The testing policy acknowledges possible limited leakage of the semi-private set over time because tasks are sent to external APIs, and the ARC-AGI-1 README warns against leaking evaluation-set information into solutions.147 |
What it is useful for
Organization context
U.S. eligibility
Eligible · basis: U.S.-governed project
ARC Prize Foundation publishes the ARC-AGI-2 tasks and the ARC-AGI-3 toolkit in its arcprize GitHub organization (the toolkit's MIT license names ARC Prize Foundation), presents ARC-AGI-1 through -3 as its ARC-AGI series, and is credited as the author of the ARC-AGI-3 technical report. The foundation is a nonprofit with a San Francisco, California address per its site terms and its 2024 Form 990. The original ARC-AGI-1 repository remains in its creator's personal GitHub account.139121615162
Sources
This listing is not an endorsement, a safety assessment, or a federal approval.