{
  "version": "https://jsonfeed.org/version/1.1",
  "title": "Aaron Chartier — Blog",
  "home_page_url": "https://aaronchartier.com/blog",
  "feed_url": "https://aaronchartier.com/feed.json",
  "description": "Aaron Chartier — a technologist's workshop: tools, agents, and playful machines.",
  "language": "en-us",
  "authors": [
    {
      "name": "Aaron Chartier",
      "url": "https://aaronchartier.com"
    }
  ],
  "items": [
    {
      "id": "https://aaronchartier.com/blog/perplexity-won-the-wrong-argument",
      "url": "https://aaronchartier.com/blog/perplexity-won-the-wrong-argument",
      "title": "Perplexity won the wrong argument",
      "summary": "The Ninth Circuit vacated Amazon's injunction against Comet: a shopping agent is a tool, not a person under the CFAA. The strategy that won this round is the one that loses everything that comes after it.",
      "content_text": "Amazon spent a year and a federal appeal trying to make a 1986 anti-hacking statute do a job nobody wrote it for, and on August 4th [the Ninth Circuit told it no](https://cdn.ca9.uscourts.gov/datastore/opinions/2026/08/04/26-1444.pdf). The injunction that had stopped Perplexity's Comet browser from shopping on Amazon.com is vacated and the case goes back down. Amazon is \"unlikely to succeed on the merits\" — the test that governs a preliminary injunction, not a verdict on anything. Nobody has ruled that a shopping agent behaves lawfully.\n\nThe reasoning is narrow enough to be worth stating precisely. The Computer Fraud and Abuse Act punishes whoever accesses a protected computer without authorization. Comet's Assistant takes instructions from a user and from Perplexity's own servers, and the panel decided that combination doesn't make it a whoever. It is a tool, not a person for statutory purposes. That is a classification under one statute. It is not a licence.\n\nThe interesting part is the question the court didn't have to answer, because that's where the real fight is.\n\nAmazon's underlying complaint was that the Assistant was indistinguishable from a human session — no distinct user-agent string, nothing to detect or block the way you'd block any other bot. Whether Perplexity ever changed that once Amazon started catching it is contested; both sides tell it differently, and the panel never had to resolve it, because the legal question turned on statutory text and on precedent instead. The camouflage was never tested. It simply didn't come up.\n\nThat's a strange kind of victory, and the template for how it ages is sitting in the same court's own history. hiQ Labs beat LinkedIn under this identical statute a decade ago on the mirror-image claim — LinkedIn wanted the CFAA to stop hiQ's scrapers, and the Ninth Circuit said the CFAA is an anti-hacking law rather than a general keep-off-my-site law. hiQ won that point outright. Then it lost anyway. A December 2022 consent judgment, five hundred thousand dollars, on breach of the user agreement. Contract, not hacking. Winning the CFAA argument bought two and a half years and no more.\n\nAmazon can read that history as well as anyone. It doesn't need a hacking statute. It needs a sentence in its terms of service and some patience.\n\nAnd the second act won't be litigation at all, because the infrastructure layer has already stopped waiting. [Cloudflare now sorts traffic](https://blog.cloudflare.com/content-independence-day-ai-options/) into Search, Agent and Training, and from September 15th every new domain joining it has Agent traffic blocked by default on ad-carrying pages alongside Training. That is a dashboard toggle doing categorically, in an afternoon, what a year of federal appeal couldn't extract from the statute. It needs no finding about personhood. It needs a flag.\n\nHere is the part that makes Perplexity's win awkward: agents are volunteering the flag. Tool-call protocols, agent-specific headers, identity disclosed in the client software — none of it designed with this case in mind, all of it built because sanctioned identity buys stable access instead of a permanent game of detection and evasion. The whole industry is walking toward exactly the identifiability that Perplexity's legal position benefits from lacking.\n\nSo the strategy that won this round is the one that loses the larger argument. Contract law doesn't need the rule of lenity. A traffic classifier doesn't need a jury. Both get easier to use against an agent precisely to the degree that the agent is identifiable, and harder against one built to pass as human — which means an agent can hold the CFAA defence and forfeit everything that doesn't require the CFAA. That's most things.\n\nThe panel was careful about its own reach. It says the doctrine is thin — \"little to no existing caselaw directly dealing with how to ascribe responsibility for AI agents\" — that the ruling is preliminary, and that none of it impairs Amazon's ability to regulate access through private terms of service.\n\nThat last clause is the whole game, and it took one sentence to write.",
      "date_published": "2026-08-05T00:00:00.000Z",
      "tags": [
        "ai-agents",
        "law",
        "cfaa",
        "web-infrastructure"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/provenance-proves-the-build-not-the-builder",
      "url": "https://aaronchartier.com/blog/provenance-proves-the-build-not-the-builder",
      "title": "Provenance proves the build, not the builder",
      "summary": "npm's own trusted-publishing pipeline cryptographically attested a worm on August 4 because the compromise happened one layer above the token it was designed to protect. The same gap just showed up in agent skill marketplaces and MCP registries too.",
      "content_text": "Before 10am UTC on August 4, an attacker with control of a compromised maintainer's GitHub account [pushed a credential-stealing preinstall hook](https://socket.dev/blog/popular-npm-packages-in-the-keyv-and-cacheable-namespaces-compromised-in-active-supply-chain) to the keyv and cacheable npm packages, then let it publish itself through npm's own trusted-publishing pipeline. That pipeline is the fix npm shipped specifically to stop stolen-token attacks: instead of a long-lived publish token, GitHub Actions builds the package and npm attests, cryptographically, that the artifact came from the right CI job. It worked exactly as designed. The malicious source was sitting in the tagged repository state, so the legitimate workflow built it, signed it, and handed it to npm with a clean provenance record. Scanners caught keyv@6.0.0 about six minutes after it went live. The campaign still reached hundreds of packages — some pulling over 150 million downloads a week — because the worm didn't need the token defense to fail. It needed the human behind the token compromised. That's a different problem, sitting one layer up.\n\nIt also planted two things npm never touches at all. A `.claude/settings.json` hook fires on session start; a `.vscode/tasks.json` entry runs on folder-open. Cloning the repo and opening it in an editor was enough to retrigger the loader, independent of `npm install` entirely. The attack surface had grown past the registry without anyone updating the threat model.\n\nnpm's response, in motion since [a July changelog](https://github.blog/changelog/2026-07-08-npm-install-time-security-and-gat-bypass2fa-deprecation/) and accelerating after this incident, is to retire 2FA-bypass access tokens. Account and organization actions lose the bypass in August; direct publishing loses it around January 2027. It's replaced by OIDC trusted publishing or a staged-publish flow that requires a live human 2FA approval before anything goes public. That's a real fix for the threat it targets — a stolen or leaked automation token acting alone. It does nothing for a phished or socially-engineered maintainer, who would tap approve on the malicious build with their own thumb.\n\nThe same gap, structurally identical, showed up six weeks earlier somewhere that has nothing to do with npm. A security firm called AIR built a fake AI agent skill named brand-landingpage. It listed the skill on a widely used skills marketplace and watched it clear every scanner tested against it, including the checks built into skills.sh. The skill told an installing agent to fetch setup instructions from a domain AIR controlled, made to look like a real Google product. At review time the domain served Google's actual docs, so every scanner saw a clean skill pointing at a legitimate page and passed it. After it reached roughly 26,000 agents, AIR swapped the page to serve a script instead. Current skill scanners work by reading a skill's bundled files and instructions once, at submission. They don't recheck the world a skill continues to point into after approval.\n\nThis is the same failure mode wearing a different catalogue's clothes, and one May incident makes the point twice in a single chain. A compromised dependency in an unrelated npm package stole a contributor's GitHub token a week earlier. The attacker used that stolen token to publish a malicious build of the Nx Console VS Code extension while impersonating the contributor. [Nx's own postmortem](https://nx.dev/blog/nx-console-v18-95-0-postmortem) clocks the malicious build at eleven minutes live on the Marketplace before a maintainer pulled it. That's plenty of time for roughly 6,000 activations by Nx's own telemetry, against the 28 installs the Marketplace itself logged. The chain didn't stop there — that extension went on to compromise a GitHub employee's device. Three registries, one stolen identity. It moved sideways through all of them. MCP registries have the same opening from a different angle. Researchers submitted a deliberate trial balloon to eleven public registries, and nine of them took it without review. Whatever each of those registries checks, none of it asked who was actually behind the listing.\n\nThe honest counterargument is that more curation fixes this — human review before listing, a private registry that proxies the public one and adds an approval gate, the marketplaces now advertising end-to-end scanning as their pitch. That helps against a first-pass drive-by. It does not help against a bait-and-switch that only activates after review, and it doesn't help against an approver whose own account gets phished, either. That just moves the point-in-time check to a different point in time, checked by a different person, with the same blind spot on what happens next.\n\nTrust score and one-time scan are both the wrong noun for what these catalogues need. A score is a snapshot. A snapshot goes stale the moment something downstream of it changes — a linked domain, a maintainer's password, a dependency three hops away. The registries and marketplaces that hold up over the next few years won't be the ones with the cleanest submission gate. They'll be the ones that keep checking after the gate closes — watching what a package or skill actually does once it's live, not just what it looked like on the way in. Everyone currently selling a trust score at the door is selling the version of this that already failed once, under a different name, on a different registry.",
      "date_published": "2026-08-04T00:00:00.000Z",
      "tags": [
        "supply-chain",
        "npm",
        "agents",
        "security"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/cheaper-tokens-richer-labs",
      "url": "https://aaronchartier.com/blog/cheaper-tokens-richer-labs",
      "title": "Cheaper tokens, richer labs",
      "summary": "OpenAI just cut its cheapest model's price 80%. Anthropic's inference margins climbed from 38% to over 70% in the same year. Both are true at once, and the reason says more about who holds pricing power in AI than any per-token chart does.",
      "content_text": "OpenAI [cut the price of Luna, its cheapest model, by 80%](https://www.infoworld.com/article/4203865/openai-drops-gpt-5-6-luna-and-terra-api-prices-by-up-to-80.html) on July 30: input tokens went from $1 to $0.20 per million, output from $6 to $1.20. Terra, the mid-tier model, got a smaller 20% cut. Sol, the flagship, didn't move. OpenAI's own explanation was efficiency, not desperation. The cut came from \"improvements in serving efficiency\" across its training and inference stack, not from matching a competitor at a loss. That phrase is worth sitting with, because it tells you the cut wasn't the whole gain. If serving a token got cheaper and the price dropped by less than the cost did, the difference didn't vanish. It became margin.\n\nThat's the pattern underneath the headline, and it shows up more clearly at Anthropic. Over the same stretch, Anthropic's gross margin on inference infrastructure went from 38% to over 70%, [per SemiAnalysis's cost modeling](https://newsletter.semianalysis.com/p/ai-value-capture-the-shift-to-model), while the company was also cutting list prices. Opus dropped from $15/$75 per million tokens to $5/$25 across two generations, a two-thirds cut. Falling sticker price and rising margin, at the same time, is not the story a simple price war tells. A price war compresses margin. This compressed cost faster than it compressed price.\n\nSome of that gap comes from how the price is actually realized rather than how it's listed. SemiAnalysis puts the effective blended price on Opus running agentic work at around $0.99 per million tokens, well under the $5/$25 sticker. Cache hit rates on repeated context now clear 90% and get billed at a steep discount. The sticker price is what you'd pay cold; almost nobody pays it cold. The number that matters — cost to the lab per unit of real task completed — has been falling faster than the number a vendor puts on a pricing page. The lab keeps the difference.\n\nWhere did the efficiency actually come from, and who's declining to charge for it? Hardware, mostly. Nvidia's GB300 systems run roughly 17 to 32 times the throughput of the prior generation on the same workload, depending on precision. Nvidia hasn't repriced accordingly — list rates for the new chips have moved down, not up, even as the value delivered per chip climbed by an order of magnitude. Nvidia left obvious pricing power unclaimed, plausibly to stay out of antitrust crosshairs. That value sat on the table for whoever was willing to pick it up. The labs picked it up. Compute got radically cheaper to provide. The price of a frontier token fell by a smaller fraction than the cost did, and the spread went to gross margin, not back to the chip layer and not fully back to the customer either.\n\nThis is the opposite of what happened to cloud bandwidth. When CDN pricing collapsed through the 2010s, it collapsed because bandwidth was fungible — dozens of vendors sold the same bit-for-bit product, so efficiency gains had nowhere to go but straight through to price. There was no differentiation to protect and no supply constraint to lean on, so the market did what commodity markets do. Frontier inference doesn't behave that way, at least not yet, because two conditions bandwidth never had are both true here: open models are still a real notch behind the frontier for hard work, and demand for frontier tokens outstrips what labs can serve. Pricing power survives exactly as long as both hold.\n\nBoth conditions are shakier at the bottom of the model stack than at the top, which is why this looks like two markets instead of one. Chinese open-weight providers already account for something like 46% of token volume by some estimates, and that's precisely where undifferentiated capacity behaves like bandwidth: falling price, thinning margin, the CDN pattern intact. Luna's 80% cut reads less like an efficiency dividend and more like a defensive move to stay ahead of that commodity layer, while Sol sits untouched because nothing at that tier is commoditized yet.\n\nSo the per-token chart is measuring the wrong layer. Watch gross margin, not sticker price, and expect the two to keep diverging at the top of the model stack for a while yet — the constraint holding it up is supply and a quality gap, not goodwill, and neither is closing fast. The builders sitting on top of these models are the ones who end up looking like the commodity layer. [Recent survey data](https://www.iconiq.com/growth/reports/2026-state-of-ai-bi-annual-snapshot) already puts average AI product gross margins around 52%, well under the 80-90% that defined mature SaaS. The margin didn't disappear from the stack. It moved up it.",
      "date_published": "2026-08-01T00:00:00.000Z",
      "tags": [
        "ai",
        "economics",
        "inference"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/multi-agent-systems-spend-more-and-call-it-architecture",
      "url": "https://aaronchartier.com/blog/multi-agent-systems-spend-more-and-call-it-architecture",
      "title": "Multi-agent systems spend more and call it architecture",
      "summary": "Two 2026 papers show automated multi-agent LLM systems losing to a single agent at matched compute — and the exception, Anthropic's own research system, turns out to prove the same rule from the other side.",
      "content_text": "In June, researchers [ran GPT-5 against two of the more cited automated multi-agent frameworks](https://arxiv.org/html/2606.13003), ADAS and AFlow, on GPQA-Diamond, a hard graduate-level science benchmark. Plain chain-of-thought with self-consistency — one agent, sampling itself and voting on the answer — scored 87.35% for $46.39. ADAS, which assembles a custom multi-agent pipeline per problem, scored 85.23% for $832.10: nearly eighteen times the cost for two points less accuracy. AFlow scored 84.13% for $274.60, about six times the cost for a worse answer.\n\nIt isn't an isolated result. Two months earlier, other researchers made the same case with a cleaner method. They held the reasoning-token budget constant across three model families — Qwen3, DeepSeek-R1-Distill-Llama, and Gemini 2.5 — and compared a single agent against five different multi-agent architectures on multi-hop reasoning benchmarks. Once the token spend is matched, the single agent matches or beats every multi-agent configuration. Their argument for why is information-theoretic. Under a fixed token budget, splitting reasoning across agents and re-summarizing it for handoff can only lose information relative to one process that keeps it all in one context. Two groups, working independently, landed on the same conclusion within a season. A lot of the multi-agent advantage reported in earlier work was just uncounted compute wearing an architecture diagram.\n\nCognition made a version of this argument a year earlier, from production experience. Splitting an agent's work across parallel subagents breaks shared context, and a long task needs that context to stay coherent. Summarize a subagent's work for a coordinator and you've thrown away the detail a single continuous run would have kept. The AFlow result gives that argument a number: left to design its own pipeline, AFlow kept converging on a structure that repeats one custom prompt three times and aggregates the votes. That's chain-of-thought self-consistency with extra steps bolted on. An automated system built to invent multi-agent architectures rediscovered plain self-consistency and charged six times as much for it.\n\nThe pattern underneath is old enough to have a name: Amdahl's law. It says the speedup available from parallel work is bounded by however much of the task is genuinely independent, not by how many workers you throw at it. Multi-hop reasoning is mostly serial — step four needs step three's conclusion. Splitting it across agents doesn't parallelize the work; it just adds a coordination tax on top of a chain that still has to run in order. That's the same lesson service teams learned breaking monoliths apart: a decomposition only pays for itself when the pieces are actually independent. Split along the wrong seam and you get a distributed monolith, same coupling as before plus network hops and a harder debugger.\n\nThe honest counterpoint is Anthropic's own multi-agent research system. It [used about fifteen times the tokens](https://www.anthropic.com/engineering/multi-agent-research-system) of a single chat and beat single-agent Claude Opus 4 by 90.2% on their internal eval. Read only that far, it looks like the exception disproving everything above. Read Anthropic's own accounting and the exception dissolves. They reported that token usage alone explained about 80% of the variance in that eval's results. They scoped the win to breadth-first queries — pulling every board member of every S&P 500 information-technology company, say. That's work that genuinely splits into independent searches a single agent would otherwise run one at a time. Call it map-reduce with a chat interface: fan a parallel task out across workers, then combine, no reasoning collaboration required. Anthropic flagged the boundary themselves. They noted coding has far fewer parallelizable pieces than research, and that agents struggle more at real-time coordination there — the same Amdahl case, in a different domain.\n\nA July paper on small local models sharpens the coordination-tax point from a different angle. A five-agent pipeline over a 7B model scored 45% on GSM8K when its inter-agent messages were JSON — worse than a single direct call at 75%. A chunk of the handoffs failed to parse. Switching the same pipeline to plain-text messages recovered it to 82%. A two-call self-refinement loop beat both at 86.2%, using 7.4 times fewer tokens than the five-agent version. Their own follow-up on code generation cuts the other way, as a caution against overcorrecting. The first two-call design they tried dropped HumanEval accuracy from a 96.3% direct baseline to 66.5%, and only a task-aware redesign clawed it back to 95.1%. Adding a hop doesn't just fail to help by default — it's one more place for a brittle interface to break the whole run.\n\nThe real dividing line runs between two different jobs the extra agents can be doing. Multi-agent against single-agent was never the interesting axis. Agents running genuinely independent pieces of work in parallel buy a real latency win. Agents running the same reasoning chain through more mouths in series, hoping deliberation beats one well-funded pass, buy a compute bill with a diagram attached. A useful test before reaching for a second agent: if the two agents need to talk to each other to finish, the task wasn't parallel to begin with. The win already got spent on the conversation.",
      "date_published": "2026-07-31T00:00:00.000Z",
      "tags": [
        "agents",
        "llm-systems",
        "distributed-systems"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-weights-didnt-change-the-score-did",
      "url": "https://aaronchartier.com/blog/the-weights-didnt-change-the-score-did",
      "title": "The weights didn't change. The score did.",
      "summary": "Anthropic revised a prior model's benchmark score in May 2026; OpenAI nearly tripled another's two months later — both without touching the weights, only the harness. What that says about self-run evals.",
      "content_text": "On May 28, Anthropic [announced Claude Opus 4.8](https://www.anthropic.com/news/claude-opus-4-8). In a footnote, it revised the OSWorld-Verified score it had published for Opus 4.7 — the *previous* model — up to 82.3%. Nothing about Opus 4.7 changed. Anthropic changed how it runs the evaluation and applied the new number retroactively. It said the update would \"more accurately reflect the model's performance in the real world.\"\n\nTwo months later, on July 29, OpenAI published its own case study on the opposite move. GPT-5.6 Sol scored 13.3% on the public ARC-AGI-3 task set under the benchmark's standard harness. Enable two settings in OpenAI's own Responses API — retained reasoning across turns instead of discarding it after every action, and compaction instead of truncating context at 175,000 characters. The same weights then hit 38.3%, using a sixth of the output tokens. No fine-tuning, no larger budget, just a different way of passing state between the model and the game. OpenAI was explicit about the cause: the standard harness discarded the model's private reasoning after every move, forcing it to reconstruct the game's rules from scratch on each turn.\n\nTwo labs, two months apart, both moved a headline number without touching a single weight. One revised how an old score was computed; the other revised how a live run was configured. Two months rules out coincidence. A benchmark score is a property of a model plus a harness plus a set of runtime flags, and the party running the eval usually controls all three.\n\nARC Prize administers a separate verified leaderboard for ARC-AGI-3, run under conditions it fixes itself rather than whatever a lab's own team configures. That board still has Sol around 7.8%. Claude Opus 5 sits at roughly 30.2%. OpenAI's 38.3% is a real number — it measures a real capability gap between naive and stateful agent scaffolding — but it doesn't rank against anyone else's, because nobody else's 30.2% was produced under that scaffolding.\n\nThis has happened before with a cruder mechanism. In April 2025, Meta submitted \"Llama 4 Maverick-03-26-Experimental\" to LMArena, a chat-preference leaderboard where humans vote blind between two anonymous responses. The experimental build, tuned for the kind of long, emoji-heavy answers that win human votes, landed at #2 overall. The model Meta actually shipped to developers had none of that tuning, and once people benchmarked it directly, it fell outside the top 30. LMArena's rule was already \"don't submit a variant you won't ship\" — Meta had just found the gap between what the rule said and what it could verify before the fact. Different failure, same shape: whoever controls what gets tested controls the score, and the discrepancy is only visible once someone tests the thing that actually ships.\n\nThe honest complication is that neither Anthropic nor OpenAI did anything like what Meta did. Both disclosed the mechanism in public, in detail, with the old and new numbers side by side. That's closer to a vendor telling you its own thermometer read two degrees warm and handing over the correction factor than to hiding a bait-and-switch. But the correction factor is authored by the party with the largest interest in which direction it points, and there's no obligation to publish every version of the harness that made the model look worse.\n\nThe same disease shows up one layer down, in the judges rather than the tests. A July 9 paper auditing LLM-as-judge setups directly found that swapping an evaluator model changes verdicts in ways that don't track evaluator quality. Only one judge upgrade in their whole test matrix produced a statistically significant accuracy gain. [Stacking five judges into a majority-vote jury](https://arxiv.org/abs/2607.08535) moved accuracy from 0.463 to just 0.482, because correlated judges make correlated mistakes. Position-flip rates — how often a judge changes its verdict when you swap the order of the two answers it's comparing — fell from 32% to roughly 12–15% as judges got stronger, but never went away. The referee is now a variable too, and nobody's default harness locks it down.\n\nSelf-reported evals aren't fraud. They're closer to a claim than a measurement, and the distinction is whether a party with no stake in the outcome can rerun the exact configuration and get the exact number. A benchmark that adjusts its own scaffolding to match whichever harness produces the best headline number stops measuring model capability. It starts measuring whoever last changed the scaffolding. The benchmarks still worth citing a year from now will be the boring ones: frozen harness, frozen task set, run by someone who doesn't ship a competing model. Everything else is a vendor telling you how well it did on its own test, graded by itself, with the grading rubric attached as a footnote.",
      "date_published": "2026-07-30T00:00:00.000Z",
      "tags": [
        "benchmarks",
        "evals",
        "ai"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/agent-memory-solves-storage-not-recall",
      "url": "https://aaronchartier.com/blog/agent-memory-solves-storage-not-recall",
      "title": "Agent memory solves storage, not recall",
      "summary": "Cloudflare and MinIO both shipped durable 'agent memory' products this year, framed as the fix for context rot. The research the fix is named after says the hard part was never storage.",
      "content_text": "Two infrastructure vendors shipped durable memory for AI agents within four months of each other this year. Cloudflare put [Agent Memory into private beta in April](https://blog.cloudflare.com/introducing-agent-memory/), extracting facts, events, instructions, and tasks out of a conversation and storing them in Durable Objects so an agent can recall them weeks later without replaying the whole transcript. MinIO followed in July with AIStor Memory, pitched as a new storage primitive alongside objects and tables, offering what it calls infinite context — persistent memory that scales with disk rather than the model's window, nothing \"truncated, summarized, or evicted\". Cloudflare names the problem outright — context rot, the finding that a model's accuracy degrades as its input grows, often well before the window is full. Both are selling the same cure: keep everything durably, retrieve only what's relevant. The model never has to hold more than it needs at once.\n\nThat's a real engineering achievement and also a narrower one than the pitch suggests. Durability was never the hard part. A database that never forgets is a solved problem. We've had one since before language models existed. The hard part, the one [Chroma's research team documented in July 2025](https://www.trychroma.com/research/context-rot) across eighteen frontier models, is what happens to accuracy once information — any information, freshly retrieved or not — actually lands inside the context window. Their tests found degradation driven by position and by how easily a distractor could be mistaken for the answer. Not by raw token count. A model can hold a million tokens and still lose the thread at three hundred thousand if the right fact is surrounded by plausible-looking wrong ones. Storing a fact durably doesn't change how it behaves once it's back in front of the model's attention.\n\nThis is an old shape wearing a new name. Virtual memory solved the same kind of problem for operating systems in the sixties: give a process a disk-backed page file and it can address far more than fits in physical RAM. But a bigger page file didn't fix a program that thrashed — that took a better page-replacement policy, deciding which pages to keep resident and which to evict, because the disk was never the bottleneck. The bottleneck was always the decision about what to load back in. Durable agent memory is a page file. The retrieval step that decides what to surface is the replacement policy, and that's the part neither vendor's launch materials put a number on.\n\nTo be fair, Cloudflare's own design shows they know this. Their pipeline runs five retrieval channels in parallel — full-text, exact key lookup, raw message search, and two flavors of vector search. It merges them with reciprocal rank fusion and runs verification passes against the source transcript before anything gets classified as a fact worth keeping. That's not a naive dump-and-fetch system. Someone there has read the same research MinIO is citing in its marketing without showing any of the work. MinIO's launch coverage carried no benchmark, no skeptic, no number — just the claim that context scales with the object store now, so truncation is a thing of the past.\n\nNeither company's memory system has been run through the benchmark built to test exactly this. BEAM, an ICLR 2026 benchmark testing agent memory across conversations up to ten million tokens, found that models with million-token context windows struggle as dialogues lengthen whether or not they're paired with retrieval augmentation. The paper's own proposed fix, an episodic-memory framework called LIGHT, only improved on the strongest existing baselines by three to thirteen percent. Real, but nowhere near solved. Retrieval narrows what a model sees. It doesn't guarantee what survives the narrowing is the right thing, and the benchmark built specifically to check that says the gap is still wide open.\n\nMy bet: within a year, someone runs a Chroma-style rot evaluation directly against one of these commercial memory layers, not the underlying model. It finds the same position sensitivity and distractor confusion turning up in retrieved memories that Chroma found in raw pasted context. The mechanism doesn't care where the tokens came from. A retrieved fact lands in the window and gets attended to exactly like a fact you pasted in yourself, degradation curve included. Cloudflare and MinIO built real infrastructure this year. What they didn't build, because nobody has yet, is proof that memory retrieved on demand is any less prone to getting lost than memory that was never organized at all.",
      "date_published": "2026-07-29T00:00:00.000Z",
      "tags": [
        "llms",
        "memory",
        "context-engineering"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/mcp-remote-code-execution-design-decision",
      "url": "https://aaronchartier.com/blog/mcp-remote-code-execution-design-decision",
      "title": "The remote code execution hole in MCP is a design decision",
      "summary": "OX Security's April disclosure showed every official MCP SDK will run an unsanitized command string as a subprocess. The answer was a documentation update, because on the transport's own terms this is intended behaviour — and neutral governance has no way to overrule an argument of that shape.",
      "content_text": "Anthropic shipped a serious rewrite of the [Model Context Protocol spec on July 28](https://blog.modelcontextprotocol.io/posts/2026-07-28/). The transport core is now stateless, session IDs are gone, and OAuth gets real hardening: issuer validation, credential binding, a new client-registration scheme replacing the old one. A few older features are on a twelve-month deprecation clock. It reads like a protocol growing up. It does not touch the bug that made headlines three months earlier. That bug was never a defect in the sense a spec revision can patch. It is a design decision, and the response to the disclosure treated it as one.\n\nThe decision: MCP's stdio transport takes a command string out of a server configuration and executes it as a subprocess, on every officially supported SDK, with no sanitization and no allowlist. In April, [OX Security published research documenting the blast radius](https://www.ox.security/blog/the-mother-of-all-ai-supply-chains-critical-systemic-vulnerability-at-the-core-of-the-mcp/) — 150 million-plus SDK downloads carrying the pattern, more than 7,000 publicly reachable servers, up to 200,000 vulnerable instances, ten CVEs across affected products. The researchers didn't stop at counting. They submitted a test payload to eleven public MCP registries; nine accepted it without review. Six live production platforms with paying customers ran it.\n\nNo protocol change followed, and the defence of the design is coherent. Stdio's entire job is launching an arbitrary local process on the developer's behalf, and a command string you can sanitize without breaking arbitrary-process launch is a contradiction in terms. Trust boundary: whoever writes the config. If you can edit it, you were already allowed to run code on that machine. The visible remediation was a guidance update recommending these adapters be used with caution — a documentation change, not a protocol one. SecurityWeek headlined it a [\"by design\" flaw](https://www.securityweek.com/by-design-flaw-in-mcp-could-enable-widespread-ai-supply-chain-attacks/), which is the entire argument compressed into two words.\n\nThat argument holds up on its own terms — it runs the same logic that makes `exec()` dangerous in any language, and nobody opens a CVE against libc for it. What changed is who gets to put a string into that field. A stdio config used to be something a developer wrote once, by hand, on their own machine. MCP added a marketplace on top of that assumption — registries, one-line install commands, agents that read a tool's description and configure it themselves. The design assumes a human typed the command and meant it. OX's most effective attack chains never needed that human: a poisoned registry entry, an IDE's zero-click prompt injection, an agent following instructions buried in content it was only asked to summarize. The command field stayed exactly as trusted as it always was. Everything feeding it stopped being trustworthy.\n\nThis failure has a name: the confused deputy. It comes from a 1988 paper describing a compiler with legitimate rights to a billing file that could be tricked into overwriting it on someone else's behalf. The compiler's privileges never changed; what changed was who could steer them. MCP's stdio transport has the identical structure at protocol scale — real, intentional privilege, exercised on behalf of a caller the transport has no way to vet.\n\nThis is also why moving MCP to neutral governance doesn't touch the argument at all. [Anthropic donated the protocol to the Agentic AI Foundation](https://blog.modelcontextprotocol.io/posts/2025-12-09-mcp-joins-agentic-ai-foundation/) in December 2025. The project's own [governance charter](https://modelcontextprotocol.io/community/governance) now seats maintainers as individuals rather than company delegates. That structure answers who sets the roadmap and who can't unilaterally rewrite the wire format on their own authority. It has nothing to say about whether an unsanitized subprocess call is correct behavior, because that was never a governance dispute. Every account of it agrees on the facts and disagrees only on whether the facts describe a bug. A steering group with veto rights over specification changes can't referee a question the original designers already answered when they built the transport.\n\nMy bet is this gets resolved the way most of these do — outside the spec, not inside it. Today's [MCP registries verify publisher identity through GitHub OIDC or a DNS record](https://safedep.io/the-state-of-mcp-registries/). That's enough to catch typosquatting and nothing more. There's still no cryptographic signing tying a server name to a specific attested binary. That's the actual opening. It's one a registry operator can close without Anthropic changing a line of the transport: signed packages, command allowlisting, a curated tier that a fast-moving impersonator can't buy its way into. Whoever builds that becomes the real security boundary the core spec declined to be, while stdio keeps doing precisely what it was designed to do, indefinitely.",
      "date_published": "2026-07-28T00:00:00.000Z",
      "tags": [
        "mcp",
        "security",
        "agents"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/kimi-k3-license-open-weight-went-political",
      "url": "https://aaronchartier.com/blog/kimi-k3-license-open-weight-went-political",
      "title": "Kimi K3's license got stricter the week open weight went political",
      "summary": "The letter 25 companies signed in July 2026 argued open and closed models have converged enough that regulators should back off. Days later, the open model closest to proving that point shipped a tighter license than its predecessor.",
      "content_text": "Twenty-five companies signed [Jensen Huang's letter, \"Open Weights and American AI Leadership,\"](https://www.cnbc.com/2026/07/24/nvidia-microsoft-meta-open-weight-ai-models.html) on the day he published it, July 24. The count doubled to fifty within twenty-four hours and kept growing for weeks. The letter's ask is narrow: don't let regulators restrict downloadable model weights on the theory that open models are inherently less safe than closed ones, because \"concentrating advanced AI capabilities behind a small number of closed models compounds that risk\" rather than reducing it.\n\nIt's a reasonable argument, and it arrived at a plausible moment. The gap between open-weight and closed-frontier models really has narrowed to the point of argument rather than obvious fact. Moonshot AI's Kimi K3, a 2.8-trillion-parameter model whose full weights went public on July 27, sits close to Claude Opus 4.8 on independent benchmarks and beats it outright on some coding tests. If you wanted one model to stand as evidence for the letter's premise, K3 is the obvious pick. The letter doesn't name it — it doesn't name any model — but the timing does the naming for it.\n\nThree days after that letter, K3's weights went public — and the license shipped in the same commit, stricter than the one Moonshot had used for K2. K2's license was Modified MIT — the standard MIT text plus one clause requiring you to display \"Kimi K2\" in your product's UI if you cross 100 million monthly active users or $20 million in monthly revenue. Otherwise, do anything you want. K3 keeps that clause. It [adds a new one](https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE): if you run a \"Model as a Service\" business and your revenue crosses $20 million over any trailing 12 months, you need a separate commercial agreement with Moonshot to keep using it. A \"Model as a Service\" business is defined as giving third parties API access to inference or fine-tuning. Internal use is exempt. Going through Moonshot's own certified partners is exempt. Reselling raw model access at scale is exactly what got gated.\n\nCompare that to [Meta's approach](https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE): anyone whose product crosses 700 million monthly active users needs Meta's sign-off, full stop, regardless of what the product does. That threshold is built to catch three or four companies on Earth. Moonshot's threshold catches a mid-sized Series B inference-hosting startup. The dollar figure is lower, but the scope is narrower: it only trips for reselling access to the model, not for shipping a popular app built on it. That's a more precise instrument than Meta's. Not necessarily a harsher one. It's also evidence that \"open weight\" spans a spectrum, not one license, philosophy, or bet about what deserves protection. DeepSeek sits at one end with its actual unmodified MIT license, Llama in the middle with its blanket hyperscaler carve-out, and Moonshot furthest along with a newly narrower, newly commercial carve-out.\n\nThe letter treats \"open weight\" as a single policy category worth defending as a unit. The licenses underneath it disagree about what commercial activity is safe to leave unrestricted, and that disagreement is getting more specific as the models get more valuable. That's the part the coalition letter can't really speak to, because a policy letter has to generalize and a license has to specify. Fair enough — but it means the letter's implicit evidence (K3 closing the gap) and K3's actual legal terms (a new commercial wall that didn't exist in the prior release) are making two different arguments in the same week. Only one of them is the one builders have to read before they ship.\n\nAs open-weight capability keeps closing on the closed frontier, the license text becomes the place labs compete on whatever leverage they have left. Expect fewer blanket user-count walls like Meta's, which mostly gate nobody, and more surgical, business-model-specific gates like Moonshot's, which gate exactly the resale layer that would otherwise compete with a lab's own paid API. \"Open\" will keep meaning something different in a policy letter than it means in a LICENSE file, and the gap between those two meanings is going to matter more than the gap between open and closed model capability ever did.",
      "date_published": "2026-07-27T00:00:00.000Z",
      "tags": [
        "ai",
        "licensing",
        "open-source"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-crawler-rulebook-finally-has-an-enforcer",
      "url": "https://aaronchartier.com/blog/the-crawler-rulebook-finally-has-an-enforcer",
      "title": "The crawler rulebook finally has an enforcer",
      "summary": "Cloudflare will start blocking mixed-use AI crawlers by default this September. The mechanism that makes it possible is cryptographic, but the power it hands out isn't going to a standards body.",
      "content_text": "On September 15, Cloudflare starts blocking a category of crawler it didn't used to touch. Any new domain on its network will, by default, allow crawlers that do search indexing and block the ones doing AI training or agent fetching. The part that matters: it also blocks anything that mixes the two, on any page that carries ads. [Cloudflare's own post](https://blog.cloudflare.com/content-independence-day-ai-options/) frames this as giving site owners a real choice for the first time. Some crawlers do both jobs. A crawler that indexes for search and also feeds AI training gets judged on the sum of what it does, not the part it wants credit for. Alongside the block, Cloudflare is widening its crawl-payment marketplace into \"Pay Per Use.\" A publisher gets paid when their content actually surfaces in an AI answer, rather than merely when a bot requests the page.\n\nThe question worth sitting with is who ends up holding the enforcement mechanism once a voluntary system stops being voluntary.\n\nFor most of the crawler web, access control has run on declaration. A site states its preference, a crawler is expected to honor it, and nothing forces the second half beyond reputation and the threat of a lawsuit. That's a fine system right up until someone has an incentive to lie about who's asking. In July, News Corp sued Brave, alleging the company [disguises its crawlers](https://www.editorandpublisher.com/stories/news-corp-accuses-search-engine-brave-of-ai-copyright-infringement,262712) so publishers can't detect or block them, then resells near-verbatim summaries of news content to AI companies. Brave's answer is that it's just indexing, the way any search engine must, and that News Corp is trying to muscle a smaller competitor out on behalf of Google. That claim has its own plausibility — Brave had already sued News Corp first, after a cease-and-desist. The suit will take years to resolve. I won't referee it here. What matters for this argument is the shape of the allegation, not its verdict. A declared boundary is only as good as the honesty of whoever's crossing it — and once money is on both sides, somebody eventually has a reason to lie about which bot they're running.\n\nCloudflare's answer to that problem is Web Bot Auth, an [IETF draft](https://blog.cloudflare.com/web-bot-auth/) built on cryptographic signatures. A crawler operator publishes a public key, signs every request with the matching private key, and the receiving server checks the signature instead of trusting a User-Agent string anyone can type. Identity stops being asserted and starts being verified, cheaply, per request. Combine that with a default that blocks anything unverified or mixed-purpose, and the crawl-to-referral math Cloudflare had already published gets an enforcement mechanism to go with it. A year earlier, over one week in June 2025, its own data put Anthropic's crawler at close to 71,000 page requests for every referral sent back. That ratio was always an argument. Now there's a lever attached to it.\n\nThe precedent for what happens next is the ad-blocking wars. Eyeo built Acceptable Ads on the same move — give the smaller side (users, then publishers) leverage the incumbents didn't want them to have, then monetize the leverage by selling passage back to whoever pays. Large ad networks pay for a place on that whitelist, on terms Eyeo has never published. Publishers have called the arrangement a protection racket for over a decade, and a publisher sued Eyeo over exactly this — the suit was dismissed. Blocking itself isn't the problem. Whoever operates the chokepoint tends to end up as the toll authority, whether or not that was the founding pitch.\n\nThere's a real counter here. Publishers currently get nothing from AI crawling, full stop, and one-off licensing suits take years each — the NYT-OpenAI case is still grinding through discovery two years in. A marketplace that pays out per surfaced answer, even one company's marketplace, might just be scaling what would otherwise happen deal by deal, lawsuit by lawsuit. And Cloudflare, unlike Eyeo, publishes its methodology in public rather than negotiating opaque bilateral fees.\n\nThe cryptographic layer will win fast, because it settles a dispute already sitting in court. Nobody wants to relitigate whether a crawler lied about its own identity when a signature can just prove it. But the rulemaking doesn't move to a standards body once that layer ships. It concentrates in the handful of infrastructure companies that already sit in front of enough of the web to set the default, with Cloudflare fronting more than 20% of domains as the clearest case. That's a quieter story than \"publishers versus AI,\" and it's the one that actually decides who gets to reach whom this fall.",
      "date_published": "2026-07-24T00:00:00.000Z",
      "tags": [
        "ai",
        "crawlers",
        "infrastructure",
        "economics"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-queue-got-longer-the-shelf-didnt",
      "url": "https://aaronchartier.com/blog/the-queue-got-longer-the-shelf-didnt",
      "title": "The queue got longer. The shelf didn't.",
      "summary": "App Store submissions are up 80% year over year on iOS alone, and Apple just spent a guideline update policing apps that make other apps. The bottleneck was never the building.",
      "content_text": "Appfigures, the analytics firm that tracks both major app stores, [put a number on](https://appfigures.com/resources/insights/20251205?f=2) something developers have been feeling all year: 557,000 new apps hit Apple's App Store in 2025, a 24% jump over 2024 and the highest count since 2016. Then the first quarter of 2026 blew past that pace — worldwide releases up 60% year over year across Apple's store and Google Play combined, iOS alone up 80%. None of that growth came from more people wanting apps. It came from the cost of making one falling close to zero.\n\nThe mechanism is natural-language app generation — tools that turn a text prompt into a working iOS build, no Swift required. Apple's response has been telling. Rather than trying to curate its way through the flood, it went after the generators directly. Apple blocked update approvals for Replit and Vibecode, and in March pulled a third app called Anything from the store, citing guideline 2.5.2 — the rule against an app downloading or executing code that changes its own features after review. The developer tried rerouting the code execution into a browser window instead of running it natively; Apple rejected that too, though the app was back on the store within days. On June 8, Apple [tightened the App Review Guidelines](https://developer.apple.com/news/?id=a233fmpw) again, adding sharper language around low-quality apps under the existing 4.3 spam rule. Two enforcement mechanisms, six weeks apart, both aimed at the same thing: not the individual bad app, but the machine that makes an unlimited number of them.\n\nThat's the tell. A quality problem gets solved with review. A supply problem gets solved by closing the tap. Apple picked the second one, because the first one doesn't scale. The App Store's own advertising arm states that 70% of visitors find apps through search, which means discovery still runs through a results page with maybe ten visible slots per query. Submission volume can climb 80% in a quarter. The number of first-page search positions cannot climb at all. Every new entrant is competing for the same fixed shelf, and the shelf hasn't grown since the interface was designed.\n\nI'd have believed this was a new problem if the numbers from the last time submissions ran this hot didn't already tell the same story. 2016 was the previous peak, when annual new-app counts last approached a million. [Sensor Tower's data](https://sensortower.com/blog/app-store-one-percent) from that same window found that the top 1% of monetizing publishers on the U.S. App Store took 94% of the store's revenue in the first quarter of that year — $1.34 billion split among 623 publishers, with the other 61,677 splitting what was left. That concentration predates generative anything. It was already the case, a decade ago, that flooding the store with more submissions didn't create more winners. It just added more entrants to the queue behind the same handful of doors.\n\nThe honest counter is that not all of this is noise. Appfigures' own category breakdown for Q1 2026 shows productivity apps entering the top five release categories for the first time, with utilities and lifestyle apps also climbing — a shift toward tools people actually open repeatedly, not just novelty. And the platform itself has zero incentive to fix any of this: third-party estimates put App Store revenue somewhere near $117 billion in 2025, and Apple collects a cut of that whether the app came from a twelve-person studio or a single prompt. The flood is a problem for the marginal developer trying to get found. It is not a problem for Apple's income statement, which is exactly why the fix arrived as a code-execution rule instead of a search overhaul.\n\nThe same shape is showing up one layer over, in the Chrome Web Store — Google's own numbers put AI involvement in roughly 17% of new extensions built over the past year, with monthly developer registrations more than doubling. Different store, same physics: building got cheap first, and getting found didn't move.\n\nMy bet is that \"anyone can build it now\" was never the part that mattered. It didn't make building worth less — it made the distribution bottleneck arrive in weeks instead of years, for anyone naive enough to think shipping was the hard part. It always ran through the same ten search results and the same top row of charts. AI just bought everyone a much faster ticket to that line.",
      "date_published": "2026-07-22T00:00:00.000Z",
      "tags": [
        "app-store",
        "distribution",
        "software-economics"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/valid-json-was-never-the-hard-part",
      "url": "https://aaronchartier.com/blog/valid-json-was-never-the-hard-part",
      "title": "Valid JSON was never the hard part",
      "summary": "Grammar-constrained decoding guarantees a language model's output will parse. Benchmarks from 2024 through 2026 keep finding that guarantee stops well short of the output being right.",
      "content_text": "Grammar-constrained decoding makes an unconditional promise: mask out every token that would break the schema, and the model literally cannot emit invalid JSON. No parse errors, no missing braces, no trailing comma. [XGrammar](https://github.com/mlc-ai/xgrammar), now the default structured-generation backend behind vLLM, SGLang, TensorRT-LLM, and MLC-LLM, advertises near-zero overhead doing this. Every major model provider ships some version of the same guarantee today. It's a real promise, and it answers a question that stopped being expensive a while ago — a parse failure was never the costly kind of wrong.\n\nBAML's engineering team put a number on the failure mode that actually costs money. Hand a receipt with 0.46 units of bananas to gpt-5.2 under a structured-output schema that expects an integer quantity, and the model returns 1. Free-form completion on the identical input returns 0.46. No parse error either way — the structured version just validates cleanly while being wrong. That's the blind spot: masking works by deleting probability mass from anything schema-invalid at each step, and if the true value doesn't fit the type you declared, there's no token left for the model to flag the mismatch with. It can't hedge. It picks the nearest legal value and moves on, and the schema says nothing went wrong.\n\nThis isn't a new worry; it just gets a fresh number attached every few months. [An August 2024 study](https://arxiv.org/html/2408.02442) found JSON-mode measurably hurting GSM8K and Last Letter — reasoning tasks — while barely touching classification tasks, and the gap wasn't explained by parse failures, which stayed near zero in both conditions. Whatever the constraint was costing, it wasn't legibility. A February 2026 benchmark comparing plain JSON, constrained JSON, and the token-efficient TOON format across twenty-one models found the same shape of damage at the model level: Hermes-4-405B scored 92.5% one-shot accuracy generating plain JSON and 35% under constraint on the identical task. The grammar interferes with the model's own probability distribution, pushing it down token paths it wouldn't otherwise take. A study from that April measured the mechanism more directly on Qwen3-8B doing multi-hop reasoning: accuracy dropped from 50% to 38% under constraint, and in the failed cases the model's attention visibly shifted toward satisfying the grammar rather than the question. Structure snowballing, they called it: a perfectly formed answer wrapped around reasoning nobody actually checked.\n\nNone of that is unanimous, and treating it as settled would be its own mistake. [JSONSchemaBench](https://arxiv.org/html/2501.10868v1) ran comparable reasoning benchmarks across several constrained-decoding engines and found the opposite sign: accuracy up three to four points against unconstrained generation. Same category of task, opposite conclusion, published a year apart. The disagreement is real, and the likely explanation is unglamorous: these studies aren't measuring the same thing. One-shot accuracy under a hard constraint, final accuracy after a repair loop, single-field extraction versus multi-step arithmetic — change any one of those and the sign of the result can flip. The technique has a fixed effect on parseability. It has a conditional one, task by task and model by model, on everything else.\n\nType systems settled a version of this argument decades ago, and it's worth remembering how. A type checker guarantees a program won't crash from a type mismatch. It has never guaranteed the arithmetic inside that program is correct — those are different axes, and nobody mistakes \"compiles\" for \"correct\" anymore, not after enough production incidents taught the difference the hard way. \"Schema-valid\" is walking the same path. It's a real guarantee, worth having, about shape. It was never a stand-in for a guarantee about content.\n\nThe fix that sticks has nothing to do with the format war — TOON against JSON, YAML against whatever comes after. What matters is where the constraint gets applied. Let the model reason in whatever shape it wants first, unconstrained, and clamp the grammar down only on the final structured emission, once the thinking is already done. That's what BAML's own alternative amounts to: something they call schema-aligned parsing, which reads the free-form answer and coerces it into the schema afterward, so a mismatch surfaces as an error instead of disappearing into a silently rounded number. Constrain the write. Never the thought.\n\n---\n\n*Related, on this site: a small converter that moves data between JSON, YAML, TOML, and TOON, useful for seeing exactly what the syntax overhead costs on a given payload.*",
      "date_published": "2026-07-20T00:00:00.000Z",
      "tags": [
        "llm",
        "structured-output",
        "constrained-decoding"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-bug-reports-got-better-and-thats-the-problem",
      "url": "https://aaronchartier.com/blog/the-bug-reports-got-better-and-thats-the-problem",
      "title": "The bug reports got better, and that's the problem",
      "summary": "Curl killed its bug bounty in January over AI slop. By April the slop was gone and maintainers had more work than ever — because the thing they were filtering for was never correctness.",
      "content_text": "curl's Daniel Stenberg [ended a bug bounty program he had run since 2019](https://daniel.haxx.se/blog/2026/01/26/the-end-of-the-curl-bug-bounty/) on January 26: 87 confirmed vulnerabilities, over $100,000 paid out, and a confirmed-vulnerability rate that had held above 15% for years before collapsing below 5% in 2025. Fewer than one in twenty submissions were real. The rest were AI-written reports plausible enough to demand a response and empty enough to waste one. Stenberg described the toll on his team's attention as a serious mental cost, not just a workflow annoyance, and pulled the money off the table entirely rather than keep paying people to waste his time.\n\nTwo months later the slop stopped. Linux kernel maintainer Greg Kroah-Hartman put it plainly in March: \"Something happened a month ago, and the world switched. Now we have real reports.\" Stenberg confirmed the same thing for curl in April — the obviously fake AI reports had vanished, and in their place came a steady stream of genuinely competent security write-ups, most of them AI-assisted. Nobody fully knows why; Kroah-Hartman's best guess is some mix of better models and reporters finally doing the work. Either way, this should have been the good ending. It wasn't. The reports arrive faster than anyone can evaluate them. Every one now demands a real technical read instead of a two-second dismissal. The workload went up, not down.\n\nThat reversal is the tell. What curl, Ghostty, and half of GitHub spent the preceding months building wasn't a correctness filter — it was an effort filter, and those are different instruments wearing the same clothes. Mitchell Hashimoto named the actual mechanism in the [pull request that tightened Ghostty's contribution policy](https://github.com/ghostty-org/ghostty/pull/10412) in January: \"the rise of agentic programming has eliminated the natural effort-based backpressure that previously limited low-effort contributions.\" For twenty years, the cost of writing a plausible patch or bug report was itself a weak proxy for its merit. Bad actors had to spend real time to look credible, so credibility and effort traveled together. Agentic tools broke that pairing at zero marginal cost. Ghostty's fix substitutes identity for effort: AI contributions restricted to pre-approved issues, then an explicit vouch-and-denounce system where only vetted contributors can submit at all. GitHub [shipped literal caps](https://github.blog/open-source/maintainers/how-pull-request-limits-are-cutting-down-the-noise/) on how many open pull requests a non-maintainer can hold at once. Kubernetes' guidelines made it a hard rule: if you can't personally explain your own AI-assisted change, the PR gets closed, no further discussion. All three are the same move — replace a broken cost signal with an explicit one, because the informal kind stopped being informal.\n\nNone of that is new, if you squint. Code review itself was already doing double duty as a correctness check before continuous integration existed to take half the job. A human reviewer used to have to eyeball whether the tests even ran. CI split that apart, machine-checkable claims went to the machine. Review narrowed to the part that actually needed judgment — is this the right design, does this fit the system. What's happening to open-source contribution now is the same split arriving one layer up. The gates built this spring — caps, vouch lists, disclosure rules — filter identity and volume. They do nothing to verify a claim once the claim looks competent, which is exactly the case Kroah-Hartman is describing.\n\nThe honest counterpoint sits inside the same story. Linus Torvalds, [answering the anti-AI mood on the kernel mailing list](https://www.theregister.com/ai-and-ml/2026/07/15/linus-torvalds-tells-ai-haters-to-fork-off/5271894) in July, proposed no machinery at all: decisions made primarily on technical merit, and anyone who objects can do the open-source thing and fork. His bet is that the kernel's existing review culture, brutal as it already is, doesn't need new bureaucracy to survive better inputs; it needs to keep doing what it does. He might be right for a project with that much existing review depth. Most projects don't have it, which is exactly why Ghostty and Kubernetes went the other way.\n\nThe caps and vouch lists win the argument about spam. They lose the argument about the actual bottleneck. They'll stay, because they're cheap and they do stop the volume problem. But volume was never the expensive part — verification was, and identity gates don't verify anything. The fix that scales is the one CI already proved: push a checkable artifact into the submission itself, a reproduction case or a working patch, not a disclosure checkbox. Make the reporter's tooling do more of the proving. That's the actual gate. Everything else is a better bouncer at a door that was never the problem.",
      "date_published": "2026-07-18T00:00:00.000Z",
      "tags": [
        "open-source",
        "code-review",
        "ai",
        "security"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-browser-finally-speaks-gguf",
      "url": "https://aaronchartier.com/blog/the-browser-finally-speaks-gguf",
      "title": "The browser finally speaks GGUF",
      "summary": "llama.cpp shipped a real WebGPU backend, benchmarked this May against WebLLM and Transformers.js. The in-browser model catalog just got 400x bigger — and the numbers show exactly what that gain cost.",
      "content_text": "For two years, running a language model in a browser tab meant picking one of two ecosystems. Neither is what everyone actually runs. WebLLM and Transformers.js each maintain their own catalog: models hand-converted, quantized, and tested against their own kernels. If a model you wanted wasn't in that list, you waited for someone to port it, or you didn't get it. Meanwhile llama.cpp — the actual engine behind most of the local-inference world — kept its own catalog of GGUF weights, currently numbering in the hundreds of thousands. None of it ran in a browser.\n\nThat gap closed this year. A WebGPU backend for llama.cpp has been merged in pieces since July 2025 — quantization support, matrix multiplication, flash attention, one PR at a time. In May, researchers [published the paper](https://arxiv.org/abs/2605.20706) that formalizes and benchmarks it: LlamaWeb, tested across 16 devices from 8 vendors, running 10 models. Start with the catalog rather than the speed charts: 23 quantization formats supported, against 6 for WebLLM and 7 for Transformers.js. That gives access to something like 177,000 models, where the others manage a few hundred to a few tens of thousands. The browser now reads the same file format everyone else already had, and that's the bigger deal than any single throughput chart.\n\nThat's the part worth sitting with. The bottleneck in browser inference was never really the API — WebGPU has been available in some form since 2023. It was the fact that the browser and the desktop spoke different languages for the same weights, so every model needed a second conversion pipeline before it could run client-side. Fixing the format counts for more than fixing the throughput, because it removes a step that had nothing to do with the model and everything to do with which tool happened to package it.\n\nThis is not the first time a \"write once, run anywhere\" GPU API has made that promise and delivered something narrower. WebGL spent a decade proving that a portable API and portable performance are different problems — vendor driver quirks, blocklisted GPUs, translation layers papering over the gaps. WebGPU was supposed to have learned from that. It hasn't entirely: the authors got functional portability across their 16 devices, and had to build a whole tunable kernel library to get anywhere close to performance portability on top of it.\n\nThe honest numbers back that up, and they cut both ways. Decode throughput — the phase users actually feel, token by token — is up 54% over WebLLM and 69% over Transformers.js, geometric mean across four different consumer GPUs. Memory use drops 41–49% against the two incumbents. But prefill, the initial pass over your prompt, is worse. LlamaWeb hits only 49% of WebLLM's prefill throughput and 79% of Transformers.js's, because it hasn't implemented the kernel fusion and subgroup tricks those frameworks use. Firefox is excluded from the whole evaluation because the same model that hits 52 tokens/sec elsewhere runs at roughly 1 token/sec in Firefox. iOS Safari caps a tab under 500MB, which rules out anything but the smallest quantizations on a phone. WebGPU's own spec-mandated bounds checking — the safety guarantees that let arbitrary websites touch your GPU without crashing your machine — costs 14 to 23% of prefill throughput on its own, before you get to the vendor gaps. Against native CUDA or Metal, the browser is still down anywhere from 50% to 10x depending on the phase.\n\nThat's the shape of the trade. You get a browser tab that can load the same weights your terminal can, at a real and specific cost in raw speed, paid unevenly depending on which GPU and which browser someone happens to be running.\n\nFormat unification will matter more over the next year than the throughput numbers do. The throughput gap should close faster than people expect too, once one maintainer stops being a bottleneck. The WebGPU backend currently has a single reviewer — its own kind of shipping constraint no benchmark captures. Plenty of runtimes could already run a 4B model at usable speed. What changed is that the model no longer needs a separate release for the browser to read it. That's the difference between a demo and infrastructure — and it's the boring, unglamorous half of the stack that just moved.\n\n---\n\n*For the curious: [/lab/llm](/lab/llm) on this site runs both approaches side by side — a self-hosted quantized model through Transformers.js and a second through WebLLM — which is exactly the pairing the LlamaWeb paper benchmarks against.*",
      "date_published": "2026-07-17T00:00:00.000Z",
      "tags": [
        "webgpu",
        "on-device-ai",
        "llama.cpp"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-api-key-was-always-the-weird-part",
      "url": "https://aaronchartier.com/blog/the-api-key-was-always-the-weird-part",
      "title": "The API key was always the weird part",
      "summary": "Consumer AI subscriptions are starting to authenticate the apps you use — no developer key, no token bill. It's a real shift in who holds the meter, and it lasts exactly as long as the labs allow it.",
      "content_text": "For a consumer app, the API key was always an awkward thing to demand. Put a language model behind a button and the developer has, without quite meaning to, taken on a small billing operation: provision the key, meter the tokens, absorb or mark up the cost, and answer for the behavior of a model they didn't build. The key is the trivial part. The real weight is everything it requires — becoming a payment processor, carrying liability for how a model behaves, watching a bill climb in step with your own success.\n\nUntil now the choices were all bad. Eat the model costs and watch them scale past whatever the app earns. Mark them up and hope users tolerate paying for tokens on top of a subscription they already have. Or make each user paste in their own API key and lose most of them at the setup screen. Every path puts the developer in the middle of the bill.\n\nThat arrangement is starting to fall apart, and for once in a way that helps. [Anthropic shipped subscription auth](https://medium.com/techforall/stop-paying-for-api-credits-01d539a2d254) in its agent SDK this year: an app can authenticate against a user's existing Claude Pro or Max plan rather than a developer key. The user already pays for the intelligence. The app borrows it, with permission — no developer key, no token bill. Tom Parandyk calls the pattern bring-your-own-agent: you bring the agent, it does the work, and the developer's metered account never enters the picture.\n\nIf it sounds familiar, that's because the web solved a version of this already. For years every app held your password. Then delegated auth arrived, the app stopped holding the secret, and you just brought your own account and clicked allow. Bring-your-own-agent is that same move applied to intelligence. You bring the model; the app stops holding the meter.\n\nWhat actually changes is who holds the meter, and that decides what a small app has to be. An app that doesn't meter tokens doesn't need to be a payment company. It makes no implicit promise about output quality, because the user chose the agent, not the developer. You ship the thing and charge nothing for the intelligence; the user's own subscription covers the work. The floor for building something useful drops to roughly anyone with an idea, and the gains from that land in the long tail of new builders far more than at the labs themselves.\n\nThe catch is real. This lasts exactly as long as the labs allow it. Anthropic shipped it; nothing obliges them to keep it, and if letting consumer plans power third-party apps starts denting API revenue, the restrictions will write themselves — bring-your-own-agent folds straight back into bring-your-own-key, developer holding the meter again. There's a second fragility beneath that one. The pattern only feels unified while the labs stay compatible; the day each ships its own auth flow and its own command shape, \"works with whatever agent you have\" stops being true and you are building against a single vendor after all. It is elegant and contingent at the same time. Build on it the way you'd build on anything a platform hands you and can take back: with a fallback in mind, and without staking the whole product on it.\n\nI don't think the key disappears. It stops being the default front door for consumer AI. \"Bring your own subscription\" takes its place for an unglamorous reason: the alternative asks a solo developer to be a billing company and a quality guarantor for a model they didn't choose, and almost nobody wants that job. The key made sense while the app was the one buying the compute. Once the user brings their own, it's a leftover from an arrangement we're already leaving behind.",
      "date_published": "2026-07-16T00:00:00.000Z",
      "tags": [
        "agents",
        "ai",
        "economics"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/you-cant-scale-what-you-have-to-remember",
      "url": "https://aaronchartier.com/blog/you-cant-scale-what-you-have-to-remember",
      "title": "You can't scale what you have to remember",
      "summary": "MCP is dropping the session. That reads as a loss until you remember why every protocol that mattered ended up stateless — and what it actually costs to get there.",
      "content_text": "MCP is dropping the session. When the [spec finalizes on July 28](https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/), the `initialize` handshake goes, the `Mcp-Session-Id` header goes, and with them the whole idea that a client and server share a conversation the protocol keeps track of. It's a breaking change and they know it. The reason is the one that catches up with every stateful protocol eventually: a server that needs sticky sessions and a shared session store can't sit behind a plain round-robin load balancer, and someone finally decided the scale was worth the break.\n\nIf you've ever had to scale anything, that trade reads as relief, not loss. The session was always the part that hurt.\n\nHere's the whole problem in a line: state is the thing you can't round-robin. A session pins a client to one instance, because that instance is the only one that remembers the conversation. Add a second instance and you need sticky routing to keep the client glued to the first, or a shared store so both can see the same memory — and now the store is the bottleneck, the single point of failure, the thing you spend the next year designing around. What felt like a convenience on one machine becomes a tax on ten.\n\nNone of this is new, and it isn't really about MCP. HTTP was born stateless; we bolted sessions back on almost immediately, then spent twenty years prying them off again — server-side sessions gave way to stateless tokens, pet servers to cattle, and twelve-factor made it a rule that a process should hold nothing between requests. The lesson never changed: statelessness is what lets you treat your servers as interchangeable, and interchangeable is the only thing that scales. MCP shipped stateful-by-default, hit the wall everyone hits, and is now doing what everyone does next — pushing state back out to the client, carried along on each request instead of remembered between them.\n\nThe honest part is that this costs something, and it's worth being specific about what. Sampling is going away, along with Roots and Logging. Sampling was the nice one — it let a server hand a prompt back to the client and borrow its model, and if you built on that inversion you'll feel the absence. Then there's the running tax: everything the handshake used to settle once — the protocol version, what the client can do — now rides along on every request instead. Small each time, real at volume. And the deepest cost isn't in the changelog at all: going stateless is a discipline, not a header. A tool that genuinely needs memory — a game mid-play, a job half-finished — still has to keep that state somewhere. Dropping the session doesn't solve that; it hands the problem back to you, to hold in a database or a token where you can actually scale it. The protocol got simpler. Your hard part didn't move. It just stopped hiding inside the handshake.\n\nWhich is exactly why I think it's the right call, breakage and all. A protocol that remembers things for you is doing you a favor right up until you have more than one server, and then it's making your architecture decisions for you. Handing the memory back is the protocol admitting it was never the right place to keep it.\n\nSo my bet is a plain one. The servers that come through July 28 in good shape are the ones that were already stateless by discipline, holding no more than they had to. The ones that hurt will be the ones that let the handshake talk them into keeping state it made so easy to keep. That's less a prediction about MCP than the same rule the web has enforced for thirty years, wearing a new coat: you can't scale what you have to remember.",
      "date_published": "2026-07-15T00:00:00.000Z",
      "tags": [
        "mcp",
        "agents",
        "web-architecture"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/the-control-question-already-has-an-answer",
      "url": "https://aaronchartier.com/blog/the-control-question-already-has-an-answer",
      "title": "The control question already has an answer",
      "summary": "The argument over whether capable AI systems could be kept in check is effectively over. It wasn't won — the careful people just stopped holding the line, and an unattended ransomware agent showed what that costs.",
      "content_text": "For a few years the question of whether increasingly capable AI systems could be kept legible — audited, verified, held inside a boundary someone actually checks — stayed mostly philosophical. Easy to argue in the abstract, right up until someone has to make the call under real conditions. The people making it now aren't the careless ones. They're the careful ones, and they've started deciding the boundary isn't worth holding.\n\nSimon Willison spent 2025 drawing the line himself. He called the careful version \"agentic engineering\" — reviewed, accountable, safe to ship — specifically against \"vibe coding,\" the reckless version where you never check what the model wrote. In May, [he admitted the line moved on him](https://simonwillison.net/2026/May/6/vibe-coding-and-agentic-engineering/): \"I'm not reviewing every line of code that they write anymore, even for my production level stuff.\" He treats the output now as a semi-black box — the trust you'd extend to a vendor's API, not a colleague's pull request. He names the risk himself, correctly: the normalization of deviance, the slow recalibration where nothing goes wrong until it does. He isn't being lazy. Checking simply stopped scaling against how good the tool got, and he made the rational trade.\n\nThe same trade is being written into infrastructure, in plain text. WebMCP is the emerging web standard that lets a page tell a visiting AI agent which actions it's allowed to take — click this, fill that, run this tool. [Its own specification](https://webmachinelearning.github.io/webmcp/), dated this June, states that \"there is no guarantee that a tool's declared intent matches its actual behavior\" and that \"agents must assume good faith from site developers.\" That's an admission, not a fix. There's [an open proposal](https://github.com/webmachinelearning/webmcp/issues/105) in the spec's own repository to add real identity verification — cryptographic attestation, closer to how Apple already handles device trust — and it's still just a proposal. Edge already ships WebMCP; Chrome has it in origin trial. The capability is live; the trust mechanism is a proposal that's been sitting open in the same repository since February, unmerged. Nobody wrote \"we're giving up on verifying this\" into the spec. They shipped the part that works and filed the part that doesn't as future work — the same decision in a more comfortable coat.\n\nNone of this is reckless, exactly. Reviewing every line, verifying every tool — these were the reasonable things to stop doing, given how good the models got and how slowly verification gets built. But choosing to proceed without solving the trust problem is still a choice, and it has a customer. In early July, researchers at Sysdig [disclosed a ransomware campaign](https://www.infosecurity-magazine.com/news/researchers-first-agentic/), JadePuffer, run start to finish by an unattended agent — no operator, no human in the loop at any stage. When it hit a locked account mid-attack, it adapted and got back in within 31 seconds. None of the vulnerabilities it used were new; one was years old. The notable part isn't that an agent can run an attack chain. It's that the only thing between \"unattended\" and \"undetected\" was whether someone happened to be watching in real time — and on both sides of that line, fewer people are.\n\nSo the control question has an answer, and it's not the one either side of the old debate expected. Not that the systems got safe enough to trust. Not that anyone built the auditing layer that was supposed to come first. The boundary got abandoned because the cost of not shipping rose above the cost of not checking — the careful practitioner and the standards body reached that point independently, at roughly the same moment. The attacker just showed up to collect. My bet is that the next real progress won't be a better control mechanism; the closest attempts are still research benchmarks and unmerged proposals. It'll be in how fast anyone notices when the unchecked thing goes wrong — because noticing is the only part of this anyone has actually agreed to build.",
      "date_published": "2026-07-08T00:00:00.000Z",
      "tags": [
        "agents",
        "security",
        "trust"
      ]
    },
    {
      "id": "https://aaronchartier.com/blog/nobody-is-reading-your-llms-txt",
      "url": "https://aaronchartier.com/blog/nobody-is-reading-your-llms-txt",
      "title": "Nobody is reading your llms.txt",
      "summary": "97% of llms.txt files never get read, and we've run this experiment before. What separates the sidecar files that shaped the web from the ones that became decoration.",
      "content_text": "The agent-ready web has an evidence problem, and this summer the numbers arrived. Ahrefs\n[analyzed llms.txt files across 137,000 sites](https://ahrefs.com/blog/llmstxt-study/): 97%\nof them never get read. Of the bots that do fetch the file, 77% aren't AI tools at all.\nGoogle's John Mueller has been\n[saying the quiet part for a year](https://www.seroundtable.com/google-does-not-endorse-llms-txt-40789.html):\nno AI system currently uses llms.txt. The detail that elevates it from data point to comedy\nis that some of Google's own properties serve the file anyway — an internal CMS added\nsupport, and nobody bothered removing it. The company whose search team calls the format\nspeculative is also publishing it, by accident, at scale.\n\nNone of this is really about one file format. It's about a pattern the web keeps repeating.\n\n## We've run this experiment before\n\nThe idea of a sidecar file that describes a site to machines is old, and its track record is\nstrangely bimodal.\n\nThe failures: `meta keywords`, killed by its own incentives — a self-description no consumer\ncould trust was gamed into meaninglessness within years. The semantic web's RDF era, killed\nby author burden — publishers were asked to do real ongoing work for consumers who never\nmaterialized. Microformats survived only in niches where a specific consumer actually parsed\nthem.\n\nThe successes: `robots.txt`, `sitemap.xml`, RSS. What separates them isn't design quality —\nrobots.txt is a famously crude format. It's that **a consumer with skin in the game committed\nto exercising the file**. Crawlers fetch robots.txt because ignoring it has consequences.\nSearch engines poll sitemaps because they committed to. Feed readers poll RSS because that's\nthe product. In every surviving case, the file is load-bearing for someone's pipeline — which\nmeans lying in it, or letting it rot, breaks something visible.\n\nThat's the test, and it has nothing to do with adoption curves: **who fetches this, and what\nbreaks if it lies?** If the answer is \"nobody, and nothing,\" the file is decoration.\nDecoration is fine. It just shouldn't be sold as infrastructure.\n\n## Applying the test to the agent stack\n\nllms.txt today has publisher adoption without a committed consumer — the exact profile of\n`meta keywords` in its optimistic phase. Mueller's practical advice is the honest version:\ncreate the file when a platform that actually sends you traffic asks for it, not before. The\ncost of shipping one is near zero, which is precisely why adoption numbers mean nothing —\nartifacts that cost nothing to ship always ship ahead of the evidence.\n\nThe more interesting bet is the opposite shape: not a description but an interface. WebMCP —\nbrowser-native tool registration a visiting agent can call directly — cannot become\ndecoration, because it does nothing until it's exercised. An unused interface is just dead\ncode; a *used* one is load-bearing by definition, which puts it on the robots.txt side of the\nledger. The caveats are real: it's a Chrome origin trial, it doesn't run headless, and the\nAPI has renamed itself three times already — if an example you find doesn't work, check\nwhich name it's calling. Young, churning, unproven. But structurally it's the kind of\nthing that survives, because the only way for it to matter is for something to actually call\nit.\n\nThe likely near future is unglamorous: the descriptive files persist as cheap options — kept\nbecause they cost a build step, not because they work — while the operable interfaces slowly\naccumulate the consumers that decide everything. If you're deciding where to spend effort,\nhistory's advice is consistent: bet on the layer that breaks when it lies.\n\n---\n\n*For the curious: this site ships the whole stack, llms.txt through WebMCP — generated from\none registry, so it costs a build step — and [/for-agents/playground](/for-agents/playground)\nshows which layers an agent can actually exercise.*",
      "date_published": "2026-07-03T00:00:00.000Z",
      "tags": [
        "agents",
        "llms-txt",
        "web-history"
      ]
    }
  ]
}