The browser finally speaks GGUF
For two years, running a language model in a browser tab meant picking one of two ecosystems. Neither is what everyone actually runs. WebLLM and Transformers.js each maintain their own catalog: models hand-converted, quantized, and tested against their own kernels. If a model you wanted wasn’t in that list, you waited for someone to port it, or you didn’t get it. Meanwhile llama.cpp — the actual engine behind most of the local-inference world — kept its own catalog of GGUF weights, currently numbering in the hundreds of thousands. None of it ran in a browser.
That gap closed this year. A WebGPU backend for llama.cpp has been merged in pieces since July 2025 — quantization support, matrix multiplication, flash attention, one PR at a time. In May, researchers published the paper that formalizes and benchmarks it: LlamaWeb, tested across 16 devices from 8 vendors, running 10 models. Start with the catalog rather than the speed charts: 23 quantization formats supported, against 6 for WebLLM and 7 for Transformers.js. That gives access to something like 177,000 models, where the others manage a few hundred to a few tens of thousands. The browser now reads the same file format everyone else already had, and that’s the bigger deal than any single throughput chart.
That’s the part worth sitting with. The bottleneck in browser inference was never really the API — WebGPU has been available in some form since 2023. It was the fact that the browser and the desktop spoke different languages for the same weights, so every model needed a second conversion pipeline before it could run client-side. Fixing the format counts for more than fixing the throughput, because it removes a step that had nothing to do with the model and everything to do with which tool happened to package it.
This is not the first time a “write once, run anywhere” GPU API has made that promise and delivered something narrower. WebGL spent a decade proving that a portable API and portable performance are different problems — vendor driver quirks, blocklisted GPUs, translation layers papering over the gaps. WebGPU was supposed to have learned from that. It hasn’t entirely: the authors got functional portability across their 16 devices, and had to build a whole tunable kernel library to get anywhere close to performance portability on top of it.
The honest numbers back that up, and they cut both ways. Decode throughput — the phase users actually feel, token by token — is up 54% over WebLLM and 69% over Transformers.js, geometric mean across four different consumer GPUs. Memory use drops 41–49% against the two incumbents. But prefill, the initial pass over your prompt, is worse. LlamaWeb hits only 49% of WebLLM’s prefill throughput and 79% of Transformers.js’s, because it hasn’t implemented the kernel fusion and subgroup tricks those frameworks use. Firefox is excluded from the whole evaluation because the same model that hits 52 tokens/sec elsewhere runs at roughly 1 token/sec in Firefox. iOS Safari caps a tab under 500MB, which rules out anything but the smallest quantizations on a phone. WebGPU’s own spec-mandated bounds checking — the safety guarantees that let arbitrary websites touch your GPU without crashing your machine — costs 14 to 23% of prefill throughput on its own, before you get to the vendor gaps. Against native CUDA or Metal, the browser is still down anywhere from 50% to 10x depending on the phase.
That’s the shape of the trade. You get a browser tab that can load the same weights your terminal can, at a real and specific cost in raw speed, paid unevenly depending on which GPU and which browser someone happens to be running.
Format unification will matter more over the next year than the throughput numbers do. The throughput gap should close faster than people expect too, once one maintainer stops being a bottleneck. The WebGPU backend currently has a single reviewer — its own kind of shipping constraint no benchmark captures. Plenty of runtimes could already run a 4B model at usable speed. What changed is that the model no longer needs a separate release for the browser to read it. That’s the difference between a demo and infrastructure — and it’s the boring, unglamorous half of the stack that just moved.
For the curious: /lab/llm on this site runs both approaches side by side — a self-hosted quantized model through Transformers.js and a second through WebLLM — which is exactly the pairing the LlamaWeb paper benchmarks against.