Tether Bets on Local Inference With QVAC Fabric, a llama.cpp-Based Runtime That Fine-Tunes Models on Consumer GPUs

Fine-tuning is where a lot of AI budgets quietly go to die. Teams get quoted a price for API access, then discover that adapting a model to their specific task adds a whole second layer of cost: data prep, GPU procurement, scaling, retries. Users on OpenAI's community forums have reported that running a fine-tuned model can cost double the base model rate. That's not a rounding error, it's a structural tax on customization.

Parameter-efficient fine-tuning methods like LoRA and QLoRA were supposed to fix this by only training a small subset of weights instead of the whole model. They help, but the underlying infrastructure bill hasn't gone away. A $4.50 GPU-hour charge on a multi-GPU cluster adds up fast, and it's the kind of cost that only becomes visible once accounting pulls the numbers together. Multiply that across thousands of LLMs competing for the same data center compute, and you get an ecosystem where the economics are shaky even for well-funded teams.

Tether's QVAC group is making a straightforward bet: if multi-GPU clusters, cloud dependency, and enterprise-grade hardware are the three biggest cost drivers in AI, then cutting reliance on all three should meaningfully change the math. Their answer is QVAC Fabric, a high-throughput inference runtime built to run on the GPU already sitting in your laptop or phone.

Under the Hood of QVAC Fabric

QVAC Fabric is derived from llama.cpp, the popular C/C++ inference engine that's become a go-to base for running LLMs outside of data centers. Tether's team extended it with a complete LoRA fine-tuning workflow baked into a modular framework, so training and inference share the same hardware-agnostic foundation.

That hardware-agnostic part matters. Fabric runs across:

• Desktop GPUs from NVIDIA, AMD, and Intel
• Mobile GPUs including Mali and Adreno
• Apple silicon (A-series and M-series chips)

It also switches backends on the fly, Vulkan, CUDA, or ROCm, depending on whichever GPU it lands on. That's a meaningful engineering lift, since it means the same model workflow doesn't need separate CUDA-only and vendor-locked code paths.

The headline architectural piece is what Tether calls the Dynamic Tiling Algorithm. Mobile GPUs are memory-constrained in ways desktop and server chips aren't, so large matrix operations, the core math behind transformer inference, tend to choke on limited VRAM. Dynamic Tiling segments those operations into smaller chunks that fit the available memory, cutting computational overhead without needing more hardware. It's the kind of workaround that sounds simple on paper but is the difference between a model running on a phone and a model crashing on a phone.

Sitting on top of Fabric is the QVAC SDK, a unified development kit meant to let developers build cross-platform AI apps without hand-rolling device-specific integration work. In its first release, the SDK supports LLMs, text-to-speech, OCR, RAG (retrieval-augmented generation, where a model pulls in external documents at query time instead of relying purely on trained-in knowledge), transcription, translation, text embeddings, and delegated inference. Tether says the roadmap is "aggressive" for adding more capabilities.

Two products already ship on this stack. QVAC Workbench is a local-first AI assistant handling scheduling, writing, coding, and research, running inference on-device rather than shipping prompts to a remote server. It launched in October 2025 alongside QVAC Genesis I, described as the world's largest library of synthetic datasets for STEM-focused model training, spanning 19 educational fields.

Workbench's more interesting bit is its networking layer. It runs on Pear, a peer-to-peer runtime built on the Holepunch stack, which enables delegated inference: start a task on your phone, then hand the heavy compute off to your desktop at home, seamlessly, based on whichever device has the capacity. That's a genuinely different model from the typical client-server AI app pattern, and it sidesteps the need for a cloud intermediary entirely.

QVAC Health is the second product, a personal health assistant that logs meals, workouts, and medications, and scans lab reports via OCR to track biomarkers. Because everything stays on-device, Tether's pitch is that users will trust it with more sensitive data than they'd hand to a cloud-based health app. Whether that trust materializes at scale is an open question, but the privacy architecture at least removes the "your data lives on someone else's server" objection.

The most technically notable claim, though, is more recent. On March 17th, 2026, Tether's AI Research team announced what they describe as the first LoRA fine-tuning framework for Microsoft's BitNet. BitNet is a 1-bit architecture that compresses model weights down to a ternary range of -1, 0, and 1, which slashes memory and compute needs compared to full-precision models. Pairing that with LoRA compresses requirements further still.

The framework swaps out CUDA dependencies for Vulkan, which is what unlocks compatibility with AMD, Intel, Apple, and mobile GPUs for both training and inference, not just inference. Tether says their approach successfully fine-tuned a 13-billion-parameter model on an iPhone 16. If that holds up under independent scrutiny, it's a real signal that workloads once considered strictly data-center territory can move onto consumer hardware.

Key Takeaways

The bigger claim here is about who gets access to frontier AI tooling. As Tether CEO Paolo Ardoino put it, "QVAC SDK and Fabric give people and companies the ability to execute inference and fine-tune powerful models on their own terms, on their own hardware, with full control of their data." Removing the CUDA and multi-GPU server dependency doesn't just cut costs, it changes who can participate: students, independent researchers, and small teams without enterprise budgets suddenly get a shot at fine-tuning SOTA models.

Whether QVAC Fabric holds up under real production load at scale is still an open question worth watching. But the direction, edge-first, hardware-agnostic, privacy-preserving inference, is one a growing number of engineers building consumer AI apps will want to keep an eye on. The one-time hardware cost model is genuinely different from the recurring GPU-rental treadmill most teams are stuck on today, even if heavier workloads will still need beefier local hardware to keep up.