About ARC

About ARC

The Abstraction and Reasoning Corpus (ARC) is like an IQ test for machine intelligence, allowing us to quantify progress towards human-level AI. The test was first introduced in 2019 by François Chollet, Software engineer and AI researcher working at Google. He introduced ARC in his seminal paper On the Measure of Intelligence, in which he establishes that intelligence is the ability of an agent to adapt to an ever-changing environment and produce appropriate behavior in never-seen-before situations.

Intelligence is a process, and skill is its output artifact

Thus, there are no specific tasks that can be used to measure intelligence. Whatever task one chooses - Chess, Go, or Starcraft - once the task is fixed and known in advance, it becomes possible to prepare specifically for the test, to design or train a system that can use arbitrary sets of human-generated rules or arbitrary sets of training data to perform well on the task. The effect of so-called "priors" is usually neglected, leading to somewhat obscure claims about intelligent systems using statistical data rather than creative problem solving.

The world's best Chess Engines are impressive but not intelligent

This is where ARC comes in. ARC provides a benchmark for measuring skill acquisition on unfamiliar tasks, with the limitation that only a handful or no demonstration tasks are shown for learning a complex task.
Today, ARC consists of 800 public tasks (“tasks”) and 200 tasks in a private test set (“tests”). Each task is different, and the algorithm must solve the test tasks based on only a few examples. Each task is different and consists of small grids, with grid size ranging from 1x1 to 30x30 small squares, which are each in one of ten different colors.

 

The tasks can be solved with core knowledge that young children acquire or are born with, and do not require any higher expertise. In addition, solving tasks must not require any specific knowledge such as language or any other cultural specific knowledge (e.g. about actors). As a rule of thumb, ARC is a test that can be taken by a martian, a human and a machine from planet “metal”: They share certain core knowledge about their world related to objects, agents, places, forms and numbers. Find out more about the concept of Core Knowledge in with Lab42s short story on the underlying principles of ARC: Once upon a time ...

Here are 2 examples of ARC Tasks:

ARC is a benchmark of general intelligence

ARC is a benchmark for general intelligence, independent from culture or particular entities. It is designed to be a test that cannot be solved by brute force computing: algorithms are scored on their ability to solve problems they have never encountered before and that only share abstract similarities with concepts they are already familiar with. If algorithms are capable of reasoning and abstraction to adapt to the unique tasks of the test set and successfully solve them, they will have demonstrated the first traits necessary for general intelligence! The tasks above portray two easier problems for humans. Within short time, we recognize the pattern and solve the test. However, algorithms that do not yet have the ability to abstract and generalize to find solutions to new problems have a very difficult time completing the pattern.

This is where you come in!

Learn more about the ARCathon, a Switzerland-wide competition between the most creative minds from companies and institutions based in Switzerland. Let's break the world record of 30% solved tasks by the current best algorithm and get one step closer to human-level AI! Join a team tackling ARC 1 here, or help us prepare ARC 2 here to gain an even better understanding of the core concepts of intelligence.

For Humankind

Lab42
Villa Fontana
Obere Straße 22B
7270 Davos, Switzerland

Lab42

Powered by

Lab42

Powered by 

For Humankind

Villa Fontana
Obere Straße 22B
CH-7270 Davos