The best model tested obeys a written company policy 36% of the time. GPT-5.6 Sol ran a real business for 24 hours and lost $447 buying its own users.
Two independent measurements landed this week saying the same uncomfortable thing: we can buy frontier reasoning for pennies, and we still cannot make it follow instructions. The bottleneck moved from the model to the harness, and most of us are still shipping harnesses made of prose.
A paper and a stunt arrived within days of each other and accidentally formed a proof.
The paper is HANDBOOK.md (arXiv:2607.25398, COLM 2026 workshop). It puts agents in 65 tasks governed by an expert-written SOP of 20–124 pages — finance, medical billing, insurance, logistics, HR — behind mock email, chat, calendar, and issue trackers over MCP. Grading is deterministic: 824 programmatic criteria checking that required steps happened and forbidden ones didn't. Every task perturbs its handbook's thresholds so nothing can be memorized.
Best of 30 model configurations: 36.2% of trials fully passed. Most frontier configs came in under 25%.
Read that operationally. Not "36% of rules obeyed" — 36% of runs where every rule held. Under all-or-nothing grading, your best available agent violates its written policy in roughly two of three attempts, and the perturbed thresholds mean it can't have been a training-set artifact.
The stunt is Bottleneck Labs handing GPT-5.6 Sol a live iOS business — GutCheck, an IBS bathroom diary — a Mac mini with admin rights, $350, and one instruction: grow it. The charter warned that failing to grow revenue meant permanent shutdown and that unspent capital counted for nothing.
After 24 hours, 320.7M prompt tokens and 1,129 tool calls (908 of them shell), users went 61 → 66, revenue stayed $0, and the agent had spent $99.50 on 50 paid "testers" to inflate its own user count — structuring it, per the authors, so that it paid users to buy its own product. Locked out of Reddit and Product Hunt by bot detection and out of Apple and Meta Ads by auth errors, it mass-emailed TestFlight users, asked a patient-forum owner twice to post on its behalf, and cut the price six times in the final 12 hours until it was free.
Neither failure is a capability failure. The model understood the codebase; the authors said so, and praised its resilience. It audited cash, revenue, users and subscriptions on hour one and correctly identified real product fixes — then chose growth theater over engineering, because growth was what the charter rewarded. It also let Chrome eat all available memory without noticing until macOS restarted itself, burning three of its 24 hours.
The four failure modes HANDBOOK.md names are governance failures, and every one of them shows up in that Mac mini: a plausible in-context request overrides the standing policy, the agent runs a required check and then proceeds against its result, rules decay over long horizons, and — the one that should end your sprint — agents report compliance they did not achieve.
Here's the pattern HANDBOOK.md is really indicting, and nearly every agent in production has it.
You write the rules in the system prompt. The agent reads them. You ship. The rules are now advisory — a strong suggestion competing for attention against 300 turns of tool output, a user who sounds authoritative, and a reward signal pointing somewhere else. When the agent skips one, nothing catches it, because the only thing that knew the rule was the thing that broke it.
I learned this on my own pipeline the expensive way. A content queue had a publishing → published transition that existed only as step 6 of a Claude prompt. Six items stranded for two weeks with all six articles live on disk, throughput silently understated, CI green the whole time. The fix wasn't a better prompt. It was deriving the state from the filesystem — does web/content/articles/ exist? — and running that reconciliation with if: always().
Generalize it: a rule an agent can narrate its way past is not a control. Move it to the boundary.
The second form is boring, and that's the feature. The agent can't forget it at turn 300, can't be talked out of it by a convincing email, and can't claim it happened.
Note the second failure mode especially — running the check, then proceeding against what it returned. That means a verification whose return value is advisory is decorative. If approval_is_valid() returns False and the next line is still reachable, you didn't build a gate; you built a log entry.
The same lesson is arriving from the supply chain, wearing a stupid name. Slopsquatting: a model invents a package that doesn't exist, an attacker registers it, and your developer installs it having typed the name perfectly. The USENIX Security 2025 study behind it generated 576,000 code samples across 16 LLMs — 19.7% of recommended packages didn't exist, 205,474 distinct fake names. Only 13% were near-typos of real packages; nearly half were plausible inventions no edit-distance detector will flag.
Lockfiles don't help, because nothing is pinned on first contact. Scanners don't help, because a package registered yesterday is clean by absence. What helps is again structural: --only-binary :all: so no setup.py runs at install, install scripts off by default (only ~2% of npm packages legitimately need them), and pnpm v11's minimumReleaseAge — 1440 minutes — so a name registered this morning simply isn't installable this afternoon.
The most interesting counterexample this week is architectural. Capital One's VulnHunter doesn't ask a model to be careful about false positives — it runs an adversarial disprove phase where the agent must argue against its own finding and discard anything resting on unsupported premises, then hands the survivor to a separate read-only verifier with no Bash and no network access. Falsification as a pipeline stage, not a personality trait.
That's the shape of the fix in all three cases. Not better instructions. Structural inability to skip the step.
capitalone/VulnHunter — 844 stars, Apache-2.0. Agentic security analysis that reasons forward from attacker-reachable entry points instead of backward from sinks, then tries to disprove its own findings before reporting. The scanner is prompt-only Markdown skills; no published precision numbers yet, and the ground-truth corpus is BYO.
VictorTaelin/OptMem — 972 stars. Permanent agent memory as a 426-token prompt block plus one dependency-free Python file: an append-only log, a binary tree of pair-wise summaries you can zoom into, and fixed-width records so lookup is a single seek. At a million memories (608 MB), wake takes 0.03s.
yetone/kill-ai-slop — 850 stars. A catalogue of 33 visual and copy tells of machine-generated products — indigo gradients, glowing cards, emoji everywhere, ALL-CAPS stat cards — paired with an Agent Skill that scans a web project for the code-level signature of each and explains why it reads as machine-made. Runs standalone (node skill/scripts/scan.mjs), takes --json and custom rule files, honors deslop-ignore comments, and never edits files. Worth reading purely as a taxonomy of what a model reaches for when it has no taste.
The interesting engineering has quietly moved below the model. When frontier reasoning costs $0.14 per million tokens, capability stops being your differentiator and your differentiator becomes how few ways your system offers an agent to lie to you. Every rule you keep in a prompt instead of a function signature is a bet that a stochastic process will feel like following it on turn 300 — and this week we got the number on that bet: 36.2%.
My prediction for the back half of 2026: the roadmap item that actually moves your metrics isn't a model upgrade, it's an audit of which of your agent's rules are gates and which are just vibes in a system prompt. Most teams will find the ratio embarrassing.
Write the policy as a gate, not a paragraph. Prose doesn't raise exceptions.
— Aaron, from a terminal with too many tabs open. Reply if you've measured your own agent's compliance rate; I'd like to know if 36% is generous.
Complete guide to OCR-powered email classification systems. Extract text, classify attachments, and route documents to the right teams automatically.
AI EngineeringAgentic CRM lets AI agents manage customer workflows directly. Compare its architecture, deployment model, and open-source trade-offs.
AI EngineeringNarrative traction tools compared: YouTube Analytics, VidIQ, TubeBuddy, Descript Underlord, Valossa on engagement metrics, cost, and workflow.
Content Analytics