Open-source platform for AI benchmarks and competitions
Supported by
Open-source platform for AI benchmarks and competitions
Build open-weight systems of 8 billion total inference parameters or fewer that answer Sinhala multiple-choice questions across Easy, Medium and Hard difficulty levels. Jointly organized by the NLP research groups of the [Informatics Institute of Technology (IIT)](/https://www.iit.ac.lk/) and the [University of Colombo School of Computing (UCSC)](https://gcct.ucsc.cmb.ac.lk/), Colombo, Sri Lanka.
No benchmarks yet.
Explore public benchmarks, submit your algorithms or datasets, and compare your results with the community on live leaderboards.
Browse benchmarksCreate your own benchmark with custom datasets, scoring programs and compute workers, and share it with participants worldwide.
Read the tutorialCodabench is open source. Help improve the platform by reporting issues, suggesting new features or contributing code on GitHub.
View on GitHubCodabench is a free, open-source platform for running AI benchmarks and competitions. Organizers set up tasks, datasets and scoring programs, and participants submit code or results. Each submission is evaluated automatically and reproducibly inside a Docker image, and results appear on live leaderboards.
Submissions run on Codabench's shared compute workers by default. Organizers who need more power, such as GPUs, can connect their own workers through a private queue and keep full control over their hardware and data.
Pages, phases, tasks and scoring for any problem.
Participants submit code or predictions, scored by your own program.
Participants submit datasets, and organizers provide the reference algorithms.
Real-time rankings and detailed results after every submission.
Every submission runs in a Docker image chosen by the organizer.
Limited storage and compute included. Easily connect your own workers.
Created by Microsoft and Stanford University to make computational research more efficient, reproducible and collaborative.
The community is led by Université Paris-Saclay under the direction of Professor Isabelle Guyon.
The next generation of CodaLab Competitions, built for flexible and reproducible benchmarking. Both platforms are administered by LISN.
If you use Codabench in your research, please cite our paper: Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform, Patterns, 2022. doi:10.1016/j.patter.2022.100543
@article{codabench,
title = {Codabench: Flexible, easy-to-use, and reproducible meta-benchmark platform},
author = {Zhen Xu and Sergio Escalera and Adrien Pavão and Magali Richard and
Wei-Wei Tu and Quanming Yao and Huan Zhao and Isabelle Guyon},
journal = {Patterns},
volume = {3},
number = {7},
pages = {100543},
year = {2022},
issn = {2666-3899},
doi = {https://doi.org/10.1016/j.patter.2022.100543},
url = {https://www.sciencedirect.com/science/article/pii/S2666389922001465}
}
