OLMoE Lands on iOS: Fully Open, On-Device AI for Everyone

At Ai2, we've been dedicated to building fully open language models that empower researchers and developers. Today, we're taking a significant step forward by releasing OLMoE (Open Language Model of Experts) as a fully open-source iOS app. This app allows anyone to test our state-of-the-art models directly on their devices, with full privacy and security.

What Changed?

OLMoE is now available on iOS, specifically for iPhone 15 Pro or newer devices, and any M-series iPad. This release marks a major milestone in making advanced AI accessible without compromising user data. The app is fully open source, meaning you can not only download it from the Apple App Store but also build it from our repository.

Why It Matters

• Privacy: OLMoE runs entirely on-device, ensuring that all prompts and responses remain private and never leave your device.
• Offline Capabilities: No internet connection is required, making the app reliable in any location.
• Research and Development: The open-source nature of the app provides researchers with a powerful tool to study and improve on-device models. Developers can prototype new AI experiences using our codebase.

Key Features

• Real-World Task Evaluation: Experience what state-of-the-art on-device models can do in practical scenarios.
• Efficiency Research: Investigate how to make local AI models more efficient.
• Custom Model Testing: Use the open-source code to test your own models locally.
• Integration: Integrate OLMoE into other iOS applications.

Technical Details

Model Architecture:

• The core of OLMoE is a mixture-of-experts (MoE) model, which allows it to be both efficient and powerful.
• We've created a new version of OLMoE specifically for this release: allenai/OLMoE-1B-7B-0125-Instruct. This version uses the Dolmino mix-instruction tuning approach to enhance performance on a wide range of tasks.

Performance Benchmarks:

• The 7B model in particular has shown significant improvements, surpassing state-of-the-art models from just a year prior in multi-task language understanding (MMLU) benchmarks.
• Mobile processing units have become more powerful, making it feasible to run these advanced models on-device.

Building the App

To build this application, we leveraged our best fully open recipes:

• OLMoE: Our most efficient, fully-open language model.
• Dolmino Mix-Instruction Tuning: A technique that fine-tunes the model for better performance on specific tasks.
• vLLM and SGLang: Open-source projects that help deploy LLMs on cloud servers, ensuring a robust backend for our app.

Future Outlook

As smaller models continue to advance and mobile hardware improves, we expect on-device AI to gain more traction. OLMoE is a step towards making this technology accessible and open to everyone, from researchers to developers to end-users.

Get Started

Download the app today from the Apple App Store or build it from our repository. Join us in exploring the future of on-device AI!