There are a lot of interesting ideas in it, mostly none that I have applied to my own workflows. Curious if anyone has used it?
Sounds like a consulting slaughterhouse. Picture Java Enterprise Solutions. As we all know, the pinnacle of software engineering.
The post doesn't include a review in the cost comparison, but I find that immediately doing a review catches lots of mistakes.
Per the OP's technique, I'll use Fable for intensive planning moments, then switch back to opus when things are going well.
To me that was the heart of the article and of what they did - create a self-validating todo list that deterministically forces the agents to stay on task, and then trim out just the right extraneous context (the actual planning) without messing with the read-file context which is where they found subagents burning most of their tokens.
Just a few days ago I was speculating about forking context to implement instead of spawning subagents here on hn. This is similar to that just several more steps more sophisticated. You could even fork the trimmed context here for very large tasks.
It all depends on how much work you're doing; if it's a simple task, there's virtually no need for plans at all; just have the smart agent do it. But if the work is going to take 5, 10 or 100 sessions, there's real value in the "smart agent plans, cheap agent executes" model.
[1]: https://www.stavros.io/posts/how-i-write-software-with-llms/
On the other hand people thought it was fine to use cliched titles like "The unreasonable effectiveness of..", "... for fun and profit", "The rise and fall of ...", "All you need is ...", etc. etc.
So I guess there's just a large portion of writers that are immune to cliche? Pretty annoying anyway.
When it's done, I switch to a smaller model and tell it to start a /goal of calling `kata ready` to get tickets ready to be worked. Work one ticket on each goal iteration, committing changes when done. Stop when all the tickets are either closed out or are blocked on actions from me.
It works fantastically. I can get entire (relatively straightforward) iOS apps done in under my $20/month five hour session window.
I get even better results if I do the QA session with the frontier model with two output artifacts: an implementation spec and a design prompt for Claude Design. I push the design prompt through Claude Design, tweak the results, and get a design spec. When I have the frontier model do the planning, I have it read the implementation spec from before, along with the design spec's overview file.
When I do it this way, I've done A/B tests between Fable 5 and Sol, and the apps they wrote were basically identical. It's so much cheaper than the "let loose the subagent fleet!" form of context management.
I haven't even added any kind of frontier model validation cycle into this loop yet. Everyone keeps talking about a subagent flow in which the big boy reviews the work of the drones, but I've found that if it encodes its acceptance criteria well enough, they do a satisfactory job of it themselves.
That said, it takes me a while to reads plans and often I'll take a break, by when the cache has expired. At that point it may be better to start over
Therefore token sellers have incentive to produce verbose, unreadable code
Even if they understand "slim code = fewer tokens = less spend" they still have to grok that less code is not less functionality.
I still use a plan, because my local model is not as smart as opus, but I can iterate much faster.
I tend to use 5.6-Sol now more for one shot type of tasks where all I want is the answer and I’m not going to read the reasoning.
Once you've done with the pieces, do one "integrate them together" part, then you have a fully formed plan with less chance of wrong stuff in it. Yes, this requires a bit more interactivity than "prompt model then come back after making and drinking a coffee", but personally I prefer that.
Try plan with opus 4.8 and then implement with Deepseek v4 flash - its 35x cheaper for reads, 90x cheaper writes, and 18x cheaper cache reads.
Or plan with Deepseek Pro, or even Flash itself. I've been impressed with both.
It works quite well most of the time. For very deeply technical tasks, where exploration involves multiple agents (and blowing up context limits a few times), I'll have a plan written down to disk that will be much longer than 2k tokens.
> But the expensive part of an agent's day is not the fixing, building, or even the thinking. Opus fixing things does not cost money. Opus reading things costs money.
Heh, another rediscovery that agents and humans are alike. By the time I've researched a Jira ticket and made it unambiguous enough to outsource, I might as well have written the code myself instead.
You could get better/faster at writing good specs and its a higher cap skill now.
For low risk changes you could let llm write its own spec from high level requirements and just validate its assumptions/design decisions.
In case anyone wants a summary: don’t one shot, don’t use plan mode and hand off the plan to cheap executors, ask the frontier model to explore, create a todo list, and then start when it feels confident; stop it after first code edit, then prefill the context to cheap executor to continue.
The build plan should be better than the context that generated it since it strips out wrong turns and other noise. I think?
I guess the way to formalize this would be to add a “make minimal change to confirm approach” instruction to the build planning prompt. Probably can even parallelize that so as the build planning prompt iterates subagents get launched to validate, similar to what research modes do.
I’m a little skeptical, but will give it a try.
I can understand that in their benchmark setting they wouldn't want the model to find an existing solution. But in my day to day work, wouldn't I want the model to search online for the best solution? Am I not shooting myself in the foot by preventing it?
1. As others pointed out, we feed all the same research turns to a smaller model, so we pay the uncached price for all of them.
2. During research, the model typically reads more code than is relevant, to figure out what is relevant and what is not. If that's the expensive part, we keep paying for those turns with the most expensive model?
3. There is no quality comparison of the resulting code. Same plan != same code, and AI tokens during the initial implementation phase isn't the only cost attributable to the task.
I do the opposite:
1. Outsource research to a subagent, or several parallel subagents. Let them output the relevant code paths only. Using gpt-5.6-terra-high on Codex and sonnet on Claude. Merge results into a single per-task research file. This saves tokens and context window of the bigger model, and avoids re-researching after compactions and in subagents.
2. Use the smartest agent (Sol xhigh ultra / Fable max) for both planning and execution. Tell it to use the research file where possible.
3. Switch to dumber agents for verbose substeps, e.g. gpt-5.6-luna-medium / sonnet is enough to drive browser use.
Sadly, got no numbers to back this approach.
PS: Of course, different task complexities and codebase complexities demand different approaches. The most expensive part is actually the code review step, which they don't mention/have at all. We should come up with some sort of complexity grading, so that discussions like this can be contextualized.
> plan deeply, then capture the plan as a todo list, then start.
I do the plan deeply phase by writing a 50 line specification myself and then asking Claude to evaluate it, find what I missed, ask questions. I answer the question and we iterate until I know that we have what I had in mind to do.
Then I ask it to write the implementation plan, that I might end up reviewing because it always asks some more question. Finally it implements the steps.
If the planning was good the implementation is fast.
--- Switch model? Your next response will be slower and use more tokens
This conversation is cached for the current model. Switching to Opus 4.8 (1M context) means the full history gets re-read on your next message. ---
Reminds me a bit of airline booking sites ("We noticed you didn't add insurance").
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Send quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why.
(NB: ranted to long, reposting at top level)
It is, but the article is suggesting something different. The central idea is to use /prewalk to load slightly edited context from the more expensive model into the one, so that the cheaper model doesn't burn tokens reading all the files again.
For those not sufficiently cracked... what is omp? Is there a way I can have this command or workflow in Cursor / Claude Code?
I tried going to the homepage (https://stencil.so/). It didn't really help.
The author is probably only evaluating their own harness with different models, so Claude-Code-specifics like memory aren't part of their evaluation.
The post may have some real insight here, where this 100k working context is actually better than trying to summarize it's findings into a plan. It's also right that the handoff is a perfect time to edit the context (removing planning instructions). But it doesn't mention it's a trade-off: the plan is smaller, so it's a cheaper "on-boarding" of the next model. Seems quite plausible that this is worth it for 1-off tasks. If it's right, this is basically "plans are useless, planning is everything" for LLMs.
My problem is, I think the plans are useful. I want to review and edit them. I want them to give context for the upcoming code review (even if humans aren't reviewing). LLMs are notoriously bad at explaining why they're doing something in the moment. Humans are notoriously bad at accepting there's no reason why. I think the humans have it right here, and to bridge this gap want my PRs, my docs, and my comments teeming with reasons why. Plans help with that.
> Opus reads base.py, signing.py, the test file (twenty cards of gray), then writes its plan and leaves. And what's the first thing Flash does with that beautiful document? It re-reads base.py and the test file, because a plan is not a file and you cannot edit prose. The gray reads just keep stacking, first at Opus prices, then again at Flash prices. There is no version of this where a second reader is the cost optimization.
If you want to interact with plans then I think this technique just isn't for you.
I want an expert opinion, if I just wanted to ask an LLM I have my own.
How can this article not mention the KV cache even once?