ARCathon 2023
Competition Guide
The ARC Challenge
- The ARC challenge is a coding problem that requires you to create an AI capable of solving reasoning tasks it has never encountered before.
- You will be given three to five pairs of training inputs and outputs, as well as a test input. Using the patterns identified from the training examples, your objective is to predict the output for the test input.
Your Objective
- To successfully complete the ARC challenge, you need to develop a program that can solve 100 secret ARC tasks, which are not publicly accessible.
- However, you do have access to 400 public training tasks and 400 public evaluation tasks, which you can utilize in any way you deem necessary.
Impact of ARC Challenge
- By accomplishing the ARC challenge, you will contribute to the advancement of AI towards human-like cognition and unlock new possibilities for AI applications.
- It is important to note that current deep learning models, including large language models, are unable to solve ARC. Therefore, it is crucial to think innovatively and discover novel approaches to programming abstract reasoning at a level comparable to humans.
Join the Lab42 Discord
The Lab42 Discord server is a place for all AI enthusiasts, challenge participants and experts from different fields around the globe. Collaborate, discuss and exchange insights across disciplines!
Where to start
- To gain a thorough understanding of ARC, it is highly recommended to read Chollet's paper titled "On the Measure of Intelligence," with special emphasis on pages 46 to 58.
- Additionally, it is advised to thoroughly go through our ARC Page.
- Lab42 has developed an ARC interface, the ARC Playground, which allows you not only to solve the original ARC tasks but also to create new ones.
Developing Your Approach
- From there, you can start brainstorming about your own unique approach.
- If you are seeking inspiration, refer to the document titled "Possible First Steps to a Solution" which provides suggestions to help you get started.
Guide for ARCathon
- The Ultimate Guide to ARCathon is a comprehensive resource that includes all the essential information about ARC data, submission rules, and the use of Docker for submitting in the ARCathon.
- This guide is designed to be an encyclopedia-like reference that is continuously updated based on the questions and suggestions from the community.
The ultimate Guide to ARCathon
ARC Data
Data Set Structure
- The task files are stored in two directories: the Training set and the Evaluation set.
- Each set contains 400 task files. The Training set is used for prototyping your algorithm or training it to learn cognitive priors that are critical to solving ARC, while the Evaluation set is for your own evaluation.
File Format & Content
- The tasks are stored in JSON format. Each task JSON file contains a dictionary with two fields: "train" and "test".
- The "train" field contains example input/output pairs, typically three pairs.
- The "test" field contains test input/output pairs, typically one pair.
Pair Structure
- A pair is a dictionary with two fields: "input" and "output".
- The "input" field represents the input grid for the corresponding pair.
- The "output" field represents the output grid for the corresponding pair.
Download ARC Data
Submission Format
- To ensure consistency and compatibility, all submissions for ARCathon must follow the Docker format.
- Docker containers offer the advantage of being portable and can be executed on any virtual machine. This technology utilizes operating system-level virtualization to package software into containers.
- Therefore, it is mandatory for all participants to adhere to the submission format specified in the Submission Guide and submit their code as a Docker image.
Submission Schedule & Procedure
- Participants are allowed to submit a maximum of three Docker containers per calendar week.
- Our team will evaluate these submissions within a maximum time frame of 72 hours.
- To submit your Docker image, please provide a link to a registry such as Docker Hub or any other preferred platform, or alternatively, share a direct download link through services like OneDrive, Google Drive, Dropbox, or any other suitable platform.
- All solutions must be submitted using the designated Submission Form.
Submission Guide
Terms & Conditions
Guide to ARCathon
Possible First Steps
Possible First Steps
For Humankind
Villa Fontana Obere Strasse 22B 7270 Davos, Switzerland
Powered by 
