Liquid AI's RIPL Engine Pushes LLM Inference Under 50ms for Real-Time Workloads

Liquid AI just announced RIPL, short for Real-time Inference Pipeline, and the pitch is simple: put a real language model directly in the critical path of a live request without blowing your latency budget. The engine returns complete generated responses end-to-end in under 50 milliseconds, and it holds that number under sustained peak traffic, not just in a cherry-picked demo run.

That distinction matters more than it sounds. Most inference stacks today, vLLM included, are optimized for a different problem entirely: general-purpose models, big batches, and agentic workflows where a few seconds of latency is fine because a human is reading a chat response anyway. But there's a whole category of enterprise workloads that doesn't get that luxury.

Think search ranking, product recommendations, fraud decisions, and payment authorizations. These systems need an answer in single-digit or low double-digit milliseconds, because they sit inline with something a user is actively waiting on. A slower checkout flow doesn't just feel bad. It measurably hurts conversion. That's precisely why these workloads have historically leaned on gradient-boosted trees or small embedding models instead of anything resembling a modern LLM. Trees are fast. They're just not very smart compared to a generative model that can actually reason about context.

RIPL is Liquid's attempt to close that gap, pairing it with their own Liquid Foundation Models (LFMs) to get generative quality without sacrificing the latency profile that real-time systems require.

Benchmarks: RIPL vs vLLM Head to Head

The numbers Liquid is publishing are aggressive, and they come from head-to-head comparisons against vLLM running competing small models on identical hardware (a single NVIDIA B200 GPU), under open-loop Poisson arrivals, which is a more honest load pattern than closed-loop benchmarks since it simulates requests arriving independently of how fast the system responds.

For a 32-token-in, 32-token-out workload at 1,000 requests per second:

• LFM2.5-1.2B on RIPL hit 34.6ms p95 latency. Qwen3.5-0.8B on vLLM hit 498ms p95, a 14.4x gap despite being a smaller model.
• MiniCPM5-1B on vLLM landed at 163.2ms p95, still nearly 5x slower than RIPL's larger LFM.
• LFM2.5-2.6B on RIPL came in at 76.4ms p95, versus 586.8ms for Qwen3.5-2B (a 7.7x gap) and 241.5ms for Granite-4.2-3B.

The gap widens further under a heavier workload shape (256 tokens in, 128 tokens out). At 400 req/s, Qwen3.5-0.8B on vLLM balloons to 4,430.88ms p95 latency. LFM2.5-1.2B on RIPL stays at 155.76ms. That's a 28.4x difference, and it's the kind of number that moves a model from "unusable inline" to "fits comfortably inside a request budget."

Across the full comparison set, Liquid reports LFM2.5-1.2B delivering 14.4 to 28.4x lower p95 latency than Qwen3.5-0.8B depending on workload shape, and LFM2.5-2.6B delivering 5.6 to 7.7x lower p95 latency than Qwen3.5-2B. These aren't marginal engineering wins. They're the difference between a model that can live in a real-time path and one that can't.

Liquid also published an interactive latency calculator covering four LFM2.5 sizes (230M, 350M, 1.2B, 2.6B) across GPUs including H100, H200, B200, and RTX PRO 6000. The reference numbers for a 32-in/32-out shape on presumably the B200 give a sense of scale:

• LFM2.5-230M: 18.8ms p50, 18.9ms p95, first token at 2.3ms
• LFM2.5-350M: 20.7ms p50, 20.8ms p95
• LFM2.5-1.2B: 30.3ms p50, 30.3ms p95
• LFM2.5-2.6B: 59.6ms p50, 59.6ms p95, first token at 6.13ms

Note the p95 and p50 numbers sit almost on top of each other in every row. That tail-latency stability, rather than raw median speed, is arguably the more interesting engineering claim here. Production systems get burned by p99 spikes, not median performance. A model that's fast on average but occasionally blows past the budget is often worse than one that's consistently mediocre.

Shopify is already running LFMs in production for latency-sensitive commerce workloads, and search enhancement is the headline example. Liquid reports sub-20ms inference on real production traffic there, with LFMs using roughly 50% fewer parameters than the alternatives they replaced, while delivering 2 to 10x faster inference without a quality hit. That's a genuinely rare combination in ML deployment: smaller, faster, and not worse.

Key Takeaways

The broader story here isn't really about one benchmark chart. It's about whether generative models can finally compete in latency-critical paths that have been the exclusive domain of gradient-boosted trees and embedding lookups for years.

If RIPL's numbers hold up under independent scrutiny and diverse production traffic, the implication is straightforward: teams building fraud scoring, credit decisioning, or search ranking systems no longer have to choose between "fast and dumb" or "smart and slow." Liquid is explicitly recruiting that use case, saying it wants to hear from teams with a request path and a hard latency budget attached to it.

Worth watching: these benchmarks compare RIPL-run LFMs against vLLM-run competitor models, not against those same competitor models optimized with a comparable custom engine. That's a fair comparison if RIPL is meant to be evaluated as a full stack (model plus engine), but it does mean the gains attributed to "the model" versus "the engine" aren't fully disentangled in the published data. Still, for anyone shipping inline inference where every millisecond is budgeted, this is a stack worth testing against your own workload shape before writing it off as marketing.