9 hours later, I had 12 PRs ready to be merged, and the net result is CI time has dropped from ~10 minutes to ~4 minutes, and billing minutes have dropped around 60%. Less than an hour of my attention.

I've given it some big tasks and asked it to parallelize as much as possible etc.

It did burn through my weekly tokens in about a day (20x max), but the output was completely on point. (I knew there was a "reset token usage - opus 5.5" button in my account.)

I've now come to a point where I even delegate my discovery for new features to it.

You still need to give it methodologies though to get the proper output, but the outcome is way beyond what I would be able to realize with a team of 5 in a month.

So asking it to do things on its own for a long time? Given last week, absolutely not.

What sort of workloads do well with these long tasks? The big labs are optimizing for long run time on their own, but it seems like a terrible thing to optimize on unless you're trying to do something like prove a hard math theorem, which success is clearly defined and the route doesn't matter a ton.

Plan mode has been made increasingly useless. I need to discuss to iterate to get the desired design, explore options, because Claude never gets it right first try and I don't have enough knowledge of options to specify everything up front.

Ah well, the Chinese models will still work well, I guess.

In a few cases it asked me to check some subcircuits and some component values because it couldn't read it right. So instead of just making things up it deferred to me.

It also ran tons of small simulation experiments while doing this to verify claims from the service manual, like that the RC filter it had read off the schematics actually had a cutoff frequency that was sensible in relation to some bandwidth number in the manual.

I had uploaded datasheet PDFs for many of the ICs and it used those to cross-reference and validate.

It kept on working for over an hour. When it asked for the manual verification, I described circuit connections in words, like "from pin 3 on IC 2 there's a series resistor of 3k in parallel with a 10 pF capacitor, it then connects to a 18k resistor to ground, a reverse-biased diode to ground, and then finally into pin 6 of IC 4", and it correctly understood the topology in all the cases. Sometimes it asked me to check again because it though something was off, and indeed I had mis-read the schematics.

I also provided reference articles on the underlying theory. Scannded stuff from the 40s and 50s. It correctly read the equations and cross-validated them across papers, and even caught several typos along the way.

I barely had to do anything apart from providing the PDFs and some occasional manual schematic interpretation.

Claude 5.5 on High. Burned through about 50% of my weekly $20 subscription usage, but I didn't try to optimize much.

I did use Sonnet 5.5 Medium on some datasheets and it also did very well on the extraction, but did have to correct itself more often on the conclusions.

It's been amazing at making sure OOMs for multiple heavy builds on my machine don't happen, adding queues and locks to make sure performance measurements are isolated and gpu stays clean during experiments.

It's also way more able to execute subagent tasks all at once than GPT 6.1 I tried to give it 10 different subtasks all at once that were overlapping and unrelated issues and it did a good job spinning up isolated worktees, agents and then coordinating the merge back together and then verifying them with agents in batches.

What I mean is, for most tasks I do that aren't trivial, by the time I have defined what DONE means, I would've already did the work and walked the path to get there, which is what I would've hoped to not have to do in the first place.

Yes this has held true on Opus 5.5. I checked. It’s a massively better model, peer to Fable but with different strengths and weaknesses. But it still has this issue. Which to be fair, people do too. Planning is a learned skill.

I think what they’re saying is that the harness no longer uses a text search on “think” to engage reasoning modes. Fair, that’s good to know. That doesn’t mean asking the model to think a certain way doesn’t have the intended effect.

I used to have the AI write a planning note with checklists, but this seems good enough nowadays.

This is their "Why it matters" for why asking opus to use subagents matters.

The fact that they think "some other people jumped off the bridge" is a reason to do something does not inspire confidence.

This has zero mention of the success or quality of those efforts. Just because they were done "with little oversight" doesn't mean it went well...

I researched the feasibility of such task about a half year ago and concluded AI wouldn't be able to have good enough spatial and blueprint knowledge, unless you were willing to throw unreasonable amount of money at the task.

[1] : https://www.salahadawi.com/hacker-news-ai-detector/49946567

You’ll get “something” but what you get is certainly not going to be what you wanted.

Look, its not complicated:

1) be precise in what you want

2) have a feedback loop to verify it.

3) check in often, not once a day.

> Migrate the payment endpoints from the old client to the new one. Done means: every endpoint uses the new client, the old client is deleted, and the test suite passes.

What end point? What test suite? What is a client?

Maaaaybe the model can infer it from your code, but look, if a human taking that jira ticket would go: what does this mean? …then your model (yes, even 5.5) is going to make a bunch of assumptions.

Correct assumptions? Maybe. Maybe not. …but do you really want to find that out hours later?

Just say:

> Plan out the following as a set high level tasks: …

Then, review the plan and tell it to execute the plan, maybe something like; after each step, ensure the code base compiles and the tests pass.

Then, come back after one hour and make sure it’s on the right track.

Poorly specified long running tasks is a recipe for “rollback all those changes…”

Do these tips work? Probably. Do we know why they work? Sorta. Could we be just copy/pasting whatever text someone else kinda thought worked one time and now enters every time out of some weird feeling that it helps nudge the LLM, like some remote islanders building airport control towers out of bamboo in the hopes of a new airdrop from the gods?? Totally. But here we are.

Write a “hand off” document? Prohibited. Focus on something else? Prohibited. Eventually every single response was terminated by the classifier before it even began, with the reason being a placeholder string like <thinking quote> or similar. Before it totally forbade any form of response, it agreed with me that it was unfortunate, but stood proudly by its guns.