ARC
Core Knowledge
Once upon a time ...
… far up in the mountains lives a scientist known as “Brainius”, whose only intent is to develop new and powerful tools to solve humankind’s challenges and make Earth a better place. His ultimate goal is to decode the human mind to develop human-level AI, for which he needs the help of all of us but more on that below. He thinks this is the path to a better life for everyone worldwide.
Brainius believes that one substantial aspect of our intelligence is “core knowledge”. This knowledge is given to us at birth and is necessary to understand the world around us. It helps us acquire new knowledge and do increasingly complicated stuff later in our lives. However, it is difficult to discern between acquired knowledge and core knowledge, and until now, there is no definite answer to what core knowledge is.
To improve our ideas on core knowledge, Brainius has a deciding idea: Find the most intelligent lifeform in the Universe and investigate what it understands about our world and what it does not. In this way, it should be possible to find out what basic knowledge we share – so-called “core knowledge” – and what knowledge we have acquired in the course of our lives.
Brainius travels far and finally meets an alien named “Glubschi” on a distant planet completely different from Earth. Glubschi is enthusiastic about Brainius’ idea, as an AI would also be helpful in his world, so they start some tests.
As the first test on core knowledge, Brainius shows a picture to his 4-year-old daughter Heila and to Glubschi. Heila immediately identifies two cute cats, while Glubschi only recognizes two “things” of the same type. Brainius concludes that recognizing a cat is not core knowledge, but counting is.
Brainius’s second test is to show Glubschi and Heila the word “electricity” while saying it aloud. Brainius turns on a light simultaneously, and Heila understands that electricity describes what powers the light bulb.
However, Glubschi only recognizes that the word contains different objects of different counts. Therefore, Brainius excludes any meaning associated with signs such as letters, phrases, or sounds from core knowledge and sophisticated theories explaining our world. On the other hand, he includes the ability to sort objects.
All the science has made Heila tired, and Brainius decides to play football with her and takes Glubschi with him. On the field he observes something that can be added to his study: While it is clear to Heila that the ball should be shot into the net, the opposite makes sense to Glubschi: the net should be placed over the ball. This means that cultural knowledge, such as rules of games, is not part of core knowledge.
However, there is something else Brainius learns from this: some “objects” are agents that have intentions and try to pursue goals – like Heila and Glubschi. On the other hand, objects like the ball or the net have no intentions or goals – the ball is not trying to end up in the net or vice versa. Therefore, Brainius adds to core knowledge that objects can be intentional agents or not.
Now, what have we learned from these tests? So far, cultural knowledge has nothing to do with core knowledge:
- No shapes or names of animals, landscapes, buildings, ….
- No languages, text, letters, symbols, ….
- No sophisticated theories like complex physics, mathematics, biology, ….
- No game rules, customs, stories, ….
In short: Everything that Glubschi could not know, given that he had never been on earth.
But what should Brainius include, then? His goal is to come up with an intelligence test to rate and develop artificial intelligence programs, so he needs a simple set of rules that tells him what core knowledge is.
Luckily, we already gathered some hints above, and Brainius started thinking about the basic categories that make up core knowledge. To ensure that the categories conceptualize core knowledge, he constantly checks that they are nothing new to Glubschi and Heila.
Categories of Core Knowledge
The first category is about objects:
- Objects persist and cannot appear or disappear without reason
- Objects can interact or not depending on the circumstances
The second category is about numbers and counting:
- Objects can be recognized and kept apart, e.g. by their shape, appearance, movement …
- Objects can be counted and sorted by number, size, shape …
- Basic mathematics like addition, subtraction, and comparison of small numbers of objects is possible
- Objects can be animate or inanimate
- Some objects are ‘agents’: they have intentions and they pursue goals
The fourth category is about basic geometry:
- Lines and simple geometric shapes like rectangles, triangles, and circles
- Geometric shapes of the same type can be distinguished as such even after possible transformations
- Objects can be mirrored, rotated, translated, deformed …
- Objects can be put together, taken apart, and pieces that build up larger objects can be detected
- Objects can be repeated
- Points can be connected, for example, to a path
- Differences in distances can be detected
The Abstraction and Reasoning Corpus (ARC)
Thanks to the help of Heila and Glubschi, Brainius could find these basic categories that are all part of core knowledge. He can now take the next step toward human-level AI.
He comes up with a game that is simple for humans – or any intelligent lifeform – but difficult for current state-of-the-art algorithms. He calls his game the “Abstraction and Reasoning Corpus”, or “ARC”, a collection of numerous different tasks.
An individual ARC task consists of a set with very few examples from which one must find a rule by abstraction and reasoning (therefore, the name). This rule is then applied to a test case to see whether it is correct.
To test intelligence rather than cultural knowledge, it is essential that only core knowledge is needed to solve a task. Moreover, there should be only one unique solution characterized by its simplicity.
To be safe, Brainius gives all his tasks to Glubschi and Heila and keeps only those for which both can find a solution.
How you can take part
Since a single scientist can’t create a sufficiently large and diverse set of tasks for ARC to have an accurate benchmark for intelligence testing, Brainius needs you to generate tasks. To help Brainius, we at Lab42 do two things:
- We set up the ARC Editor where you can test your intelligence by solving tasks that Heila and Glubschi have already approved. More importantly, you can create and submit your own tasks on the website and help Brainius develop an AI. Of importance is that the tasks you make have a straightforward solution, and only core knowledge is needed to infer the rule to solve the tasks. If not, they must be sorted out by Glubschi and Heila, and we do not want to overburden the two of them with unfitting tasks. For this reason, we will go over some simple ARC tasks below that demonstrate what we mean.
- We organize the ARCathon. In this competition, you and your work colleagues can combine your skills to develop novel algorithms to solve the already existing tasks. If you work for a Swiss-based company and have some ideas, take part, and help Brainius make Earth a better place!
A few simple ARC examples
Objects, numbers and counting
In the two examples on the right, we see that the rule is to take the largest object as the output. Therefore, we only need to be able to detect objects and compare them by their size.
In the example on the left, the rule is to take the broken lines, draw them over the black stripe and color them with the most common color of all the lines. For this rule, we must be able to count in small numbers and notice that black “covers” part of the lines.
Goal-directedness
In the example on the right, the rule is to connect the yellow boxes with the shortest path starting from the purple squares. In addition, one can only turn when hitting a green box. The search for the shortest path between two points while respecting the constraints can be considered a goal-directed action and is part of core knowledge.
Basic geometry
In the example on the left, the rule is that everything is mirrored at the green line. Again, you only need to detect geometric objects and know how to mirror them.
Not part of core knowledge
How to complete our logo can not be inferred from core knowledge and is thus not a valid ARC task.
For Humankind
Villa Fontana Obere Strasse 22B 7270 Davos, Switzerland
Powered by 