ARC

Abstraction & Reasoning Corpus

About ARC

The Abstraction and Reasoning Corpus (ARC) is like an IQ test for machine intelligence, allowing us to quantify progress towards human-level AI. The test was first introduced in 2019 by François Chollet, Software engineer and AI researcher working at Google. He introduced ARC in his seminal paper On the Measure of Intelligence, in which he establishes that intelligence is the ability of an agent to adapt to an ever-changing environment and produce appropriate behavior in never-seen-before situations.

The world's best Chess Engines are not intelligent

Thus, there are no specific tasks that can be used to measure intelligence. Whatever task one chooses - chess, the board game Go, or the computer game Starcraft - if the task is fixed and known in advance, it is possible to prepare for it. In such cases, the programmer designs or trains a system that uses arbitrary, human-generated rules or arbitrary sets of training data to do a particular task well, but never beyond.

A human-level IQ test for AI

This is where ARC comes in. ARC is a benchmark for measuring AI skill acquisition. To solve ARC, an algorithm must find the solution to various and previously unknown tasks from only a handful of demonstrations - typically 3 per task. While humans on average solve 80% of all ARC tasks effortlessly, today's algorithms achieve no more than 29%. Find out everything about ARC tasks here or go directly to the ARC Playground.

ARC is a benchmark of general intelligence

An AI must approach each ARC task from scratch, using only the same kind of prior knowledge about the world that humans inherently possess - so called core knowledge. For this reason, modern deep-learning models as well as big-language models score close to zero on ARC, underscoring the need for novel approaches on the road to general AI.

This is where you come in!

Learn more about the ARCathon, a Switzerland-wide competition between the most creative minds from companies and institutions based in Switzerland. Let's break the world record of 30% solved tasks by the current best algorithm and get one step closer to human-level AI! Join a team tackling ARC 1 here, or help us prepare ARC 2 here to gain an even better understanding of the core concepts of intelligence.

For Humankind

Lab42
Villa Fontana
Obere Straße 22B
7270 Davos, Switzerland

Lab42

Powered by

Lab42

Powered by 

For Humankind

Villa Fontana
Obere Straße 22B
CH-7270 Davos