BabyLM Returns for EMNLP 2026 with New MultiLingual Track and Updated Datasets

BabyLM, the challenge focused on sample-efficient pretraining under human-scale data budgets, is back for its fourth year as both a shared task and a workshop at EMNLP 2026. This iteration introduces several updates to keep the competition fresh and relevant for researchers and practitioners.

Key Changes

• New MultiLingual Track: BabyLM now includes a dedicated multilingual track based on the BabyBabelLM dataset, focusing on English, Dutch, and Chinese.
• Track Restructuring: The standalone Multimodal and Interaction tracks have been removed. Instead, participants can use paired image-text data and leverage feedback from a teacher model within the Strict or Strict-Small tracks.
• Updated Datasets: New detoxified datasets are available, including a 100M-word Strict dataset and a 10M-word Strict-Small dataset.

Data and Tracks

Available Datasets

All data is available on the Hugging Face BabyLM community:

• Strict Dataset: A detoxified 100M-word corpus.
• Strict-Small Dataset: A detoxified 10M-word subset of the Strict dataset.

New MultiLingual Track

The MultiLingual track introduces a new challenge with a focus on multilingual pretraining. Participants can train models on a mixture of English, Dutch, and Chinese data, totaling 100M tokens. The word counts are adjusted by each language's Byte Premium to ensure fair representation.

• Languages: English, Dutch, Chinese
• Total Tokens: 100M (adjusted for Byte Premium)
• Dataset Source: BabyBabelLM

Evaluation Pipeline

The evaluation pipeline will be distributed as an open-source repository building on the 2025 version. The MultiLingual track will be assessed using a combination of zero-shot and finetuning-based tasks across the three languages.

• Open Source Repository: Planned for early April
• Tasks: Zero-shot and finetuning-based evaluations

Submission Guidelines

Submissions can be made via ACL Rolling Review (ARR) or directly through OpenReview. Official submission links will be posted on the BabyLM website once they are live.

Updated Rules for Round 4

• MultiLingual Track: Participants must train on a multilingual mixture from BabyBabelLM, focusing on English, Dutch, and Chinese. The total token count is 100M, adjusted by Byte Premium.
• Track Restructuring: The Multimodal and Interaction tracks have been removed. Paired image-text data and teacher model feedback can be used in the Strict or Strict-Small tracks.

Getting Involved

If you have questions for the organizers or want to connect with other participants, consider joining the BabyLM Slack community.

Source