ARCathon 2023
Competition Guide
ARC Challenge
Outline
The ARC challenge is a unique coding problem that requires you to build an AI that can tackle reasoning tasks it has never encountered before. You will be presented with three to five pairs of training inputs and outputs, and a test input, where you must use the pattern identified from the training examples to predict the output.
Your objective is to develop a program that can solve 100 secret ARC tasks (the private test set), which are not accessible to the public. To train your program, you have access to 400 public training tasks and 400 public evaluation tasks. These 800 public tasks you can use in any way you see fit.
By successfully completing the ARC challenge, you will advance AI towards human-like cognition and unlock new possibilities for AI applications. Given that current deep learning models including large language models cannot solve ARC, it is necessary to think creatively and find new ways to program abstract reasoning at a human level.
Join the Lab42 Community
The Lab42 Discord server is a place for all AI enthusiasts, challenge participants and experts from different fields around the globe. Collaborate, discuss and exchange insights across disciplines!
Where to start
To get familiar with ARC it is advisable to read Chollet's paper On the Measure of Intelligence - especially pp. 46-58 - and read through our ARC Page. Furthermore, Lab42 has created an ARC interface where you can solve the original ARC tasks and even create new ones: ARC Playground.
From there, you can start brainstorming about your own approach. If you need a stepping stone for creativity, see the document Possible First Steps to a Solution.
Guide & Documents
The ultimate Guide to ARCathon
The Ultimate Guide to ARCathon contains all the necessary information about ARC data, submission rules, and Docker, which is the means to submit in the ARCathon. It serves as a kind of encyclopedia, which is constantly expanded according to questions and hints from the community.
If you have worked with Docker before and just want to submit your code, the Submission Guide (short) below should be sufficient for you.
ARC Data
The task files are stored in two directories:
- Training set: contains the 400 task files for training. You can use these tasks to prototype your algorithm or train it to learn cognitive priors that are critical to solving ARC.
- Evaluation set: contains the 400 task files for your own evaluation.
The tasks are stored in JSON format. Each task JSON file contains a dictionary with two fields:
- "train": These are the example input/output-pairs. It is a list of typically three pairs.
- "test": These are the test input/output-pairs. It is a list of typically one pair.
A pair is a dictionary with two fields:
- "input": This is the input-grid for the corresponding pair.
- "output": This is the output grid for the corresponding pair.
Download ARC Data
Submission
The general format for submissions is Docker. Dockers have the advantage that they are portable and can be run on any virtual machine. It uses virtualization at the operating system level to deploy software in packages called containers. All code submitted to ARCathon must conform to the format outlined in the Submission Guide and be submitted as a Docker image!
Starting February 13, you can submit three Docker containers per calendar week until December 1, 2023, which will be evaluated by our team within a maximum of 72 hours. Please submit your Docker image as a link to a registry (Docker Hub, etc.) or as a direct download link via OneDrive, Google Drive, Dropbox, or your preferred platform. Solutions must be submitted via the submission form.
Submission Guide (short)
Essential Documents
General Terms & Conditions
The ultimate Guide to ARCathon
Possible First Steps to a Solution
Possible First Steps to a Solution
For Humankind
Villa Fontana Obere Strasse 22B 7270 Davos, Switzerland
Powered by 
