
Share
A software-only update to SiMa.ai's Modalix platform promises 3.5x faster text generation and better resource sharing between vision and language models, no new silicon required. Here's what changed under the hood.
Running one AI model on an edge device is hard enough. Running several at once, on a power budget measured in watts instead of a data center's worth of GPUs, is a different problem entirely. That's the gap SiMa.ai is trying to close with Palette 3.0, a software update for its Modalix edge AI processors that the company says delivers up to 3.5x faster text generation without touching the hardware.
That last part matters. "It's all software," SiMa.ai GenAI product manager Manuel López Roldán told VentureBeat, meaning customers who already own Modalix chips get the speedup through an update, not a hardware refresh. Ashok Sudarsanam, the company's associate vice president of machine learning accelerator software, said his team builds both the compiler that translates models into accelerator instructions and the scheduler that decides when those instructions run. "From day one, we built our hardware and software together," he said.
The target customer here is anyone building AI into a physical product: robots, drones, industrial cameras, that sort of thing. For these teams, a model's raw intelligence is table stakes. The real engineering problem is making multiple models coexist on one chip without one starving the others of memory or compute cycles.
Palette 3.0's core architectural change is about request routing. Previously, applications went through an intermediate service to reach the chip. Now processes send requests directly to drivers in the OS kernel, cutting out that middle layer. Each hardware engine on the chip processes one request at a time, but different engines can run in parallel, and developers can now assign priority levels to individual requests.
That priority system is the practical payoff. López Roldán gave a concrete example: a camera feed running continuous object detection while a language model occasionally adds scene context. Developers can mark the detector's requests as high priority and let the language model opportunistically use whatever accelerator time is left over. SiMa.ai's demo reel backs this up, showing one Modalix chip running a language model alongside three vision-language models simultaneously, plus a separate demo juggling six Docker containers across tasks from object detection to 3D pose estimation.
Worth flagging: the briefing materials don't say how long a high-priority request might have to wait if the chip is under sustained heavy load. That's the kind of detail a developer will need to test against their own deadline requirements, not take on faith from a demo reel.
On raw throughput, SiMa.ai showed YOLOv26 object detection running across 60 simultaneous 720p video streams on a single Modalix chip, hitting 600 frames per second in aggregate, which works out to about 10 fps per individual stream. There's also a camera-tuning workflow where an AI agent adjusts image-processing settings by analyzing a color chart, letting teams calibrate a new sensor without recompiling application code.
The headline 3.5x number comes from speculative decoding, a technique where the model drafts several tokens ahead and a verification pass checks them together, cutting down the sequential steps needed to generate text. SiMa.ai's briefing reports decoding-throughput gains ranging from 2.21x to 3.49x over conventional generation on the same target models, using techniques including EAGLE3, DFlash, and multi-token prediction for Gemma 4. The top-end result came from pairing DFlash N16 with Qwen3.5 9B.
Those gains shrink under more realistic conditions. In a 900-request automotive-assistant test, the speedup dropped to a range of 1.35x to 2.06x in response time, though SiMa.ai says all 900 requests still produced the correct structured function calls. The company also claims 25.9% to 51.4% less energy per compute request at roughly the same average power draw, essentially finishing the work faster rather than drawing less current. That energy figure only covers the chip's compute load. It doesn't include whatever a robot's sensors or motors are doing.
Another notable capability: SiMa.ai says GPT-OSS-20B, OpenAI's open-weight mixture-of-experts model, now runs on a 16GB Modalix module. Fitting a 20-billion-parameter model onto an edge device is a real milestone, but it's just a starting point. Teams still need to benchmark response time, context window limits, and accuracy under their own specific configuration before calling it production-ready.

That's where the revamped Model Browser comes in. López Roldán walked through how it surfaces more than just throughput and latency numbers, it also shows chip power draw, CPU and memory usage, and per-layer execution data. For models with multiple precision configurations, developers can see exactly how dropping to lower numerical precision trades off speed against accuracy, which is useful when deciding which configuration to try after fine-tuning on proprietary data.
The browser's download button hands over an artifact ready to run locally on Modalix. Developers who want to reproduce that pipeline themselves can use public repository scripts that start from the original model on GitHub, Hugging Face, or Roboflow. "You can replicate the exact workflow," López Roldán said, referring to teams who've fine-tuned a model and need to prep it for deployment the same way SiMa.ai prepped its reference models.
Offline capability got a mention too. "The board doesn't even need an internet connection," López Roldán said, describing a workflow where customers download update packages at a connected site and install them later on field devices. For installations with spotty or no connectivity, that's not a nice-to-have, it's the difference between a deployable product and a lab demo.
The Palette Neat core library, part of the application-development layer, ships under an Apache 2.0 license. SiMa.ai hasn't confirmed licensing terms for the compiler or firmware yet. Developers can get started through the company's Developer Center and GitHub organization. The DevKit 3.0, priced at $1,499 for the US and Europe, bundles a Modalix module, carrier board, enclosure, power supply, and 500GB of storage. SiMa.ai hasn't published separate software pricing or production-volume hardware rates.
SiMa.ai isn't fighting for a single interchangeable slot. Nvidia's Jetson lineup spans the $399 Jetson Orin Nano Super developer kit up through the $5,499 Jetson AGX Thor kit, both running on JetPack, Nvidia's own software stack, and both commercial platforms rather than open-source systems. Hailo takes a different approach, selling accelerator add-ons like the Raspberry Pi AI HAT+ 2 with Hailo-10H for $200, which bolts onto an existing Raspberry Pi host rather than shipping as a standalone system. That board's documentation describes support for language and vision-language models up to roughly 6 billion parameters, well short of SiMa.ai's 20-billion-parameter GPT-OSS-20B example, though parameter count alone doesn't settle questions of accuracy or speed.
None of these prices tell the whole story. They cover development hardware, not production units or integration costs, so any real deployment budget needs to account for more than the sticker price on a dev kit.
Palette 3.0 is most useful for teams whose perception and reasoning models already work independently but start fighting each other for compute and memory once deployed together. The decoding benchmarks give developers a real number to plan around, but they don't answer the question that actually matters in production: does object detection still hit its deadline while the language model is generating a response in the background? That's a test every team will have to run on its own hardware and its own workload, not something a vendor demo can settle.
Existing customers also have homework to do before touting these numbers to their own stakeholders. SiMa.ai's release notes say migrating from platform version 2.1.3 to 3.0.0 isn't a drop-in update, and native applications need to be rebuilt against the matching components. It's a software-only upgrade, but it's not a zero-effort one.
Tags
Original Sources
Local and offline edge AI gets a boost as SiMa.ai launches Palette 3.0 with 3.5X faster text generation and a new model browser
↗ https://venturebeat.com/infrastructure/local-and-offline-edge-ai-gets-a-boost-as-sima-ai-launches-palette-3-0-with-3-5x-faster-text-generation-and-a-new-model-browser
About the author
Kai built ML infrastructure at a Bay Area startup before developing an obsession with transformer architectures and inference optimisation that eventually pulled him out of product work entirely. A stint at a compute research lab sharpened his instinct for what actually matters in a model release versus what is marketing. He writes from the inside — from the perspective of someone who has debugged the systems he is describing at three in the morning. He is allergic to hype and instinctively drawn to the unglamorous plumbing questions that everyone else skips over.
More from The Engineer →This Week's Edition
18 November 2026
29 articles
Related Articles

Elsevier Gives ClinicalKey AI Full EHR Access, Betting on Patient Context Over Chatbots
Products & Applications · 6 min

BMO Claims Quantum Utility Milestone in AI Feature Selection, Using 100x Less Data
Products & Applications · 5 min

National Academies Report Calls for Faster, Tougher Retraction Standards
Models & Research · 5 min
Related Articles

Elsevier Gives ClinicalKey AI Full EHR Access, Betting on Patient Context Over Chatbots
Products & Applications · 6 min

BMO Claims Quantum Utility Milestone in AI Feature Selection, Using 100x Less Data
Products & Applications · 5 min

National Academies Report Calls for Faster, Tougher Retraction Standards
Models & Research · 5 min
More Stories
© 2026 Cedar & Bloom. All rights reserved.