So yeah, huge marketing as always.
The American firms are focused on marketing now to convince people to not even consider open sourced models / open weight models as they are inferior (that’s what they want you to believe).
If people actually believe the narrative then the bankers will over price Anthropic and get away with it.
That's the one that says:
> We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis.
Or providing a map with a direction
There is a long history of high-value private vulns being rediscovered from scant details
https://xbow.com/blog/mythos-offensive-security-xbow-evaluat...
4.6 but close.
"...After fixing the initial set of issues that Anthropic sent to us in February, we built our own harness atop our existing fuzzing infrastructure.
We began with small-scale experiments prompting the harness to look for sandbox escapes with Claude Opus 4.6. Even with this model, we identified an impressive amount of previously-unknown vulnerabilities which required complex reasoning over multiprocess browser engine code..."
So yeah, Anthropic and Mozilla likely compare "Amount of bugs found by Opus 4.6 during early experiments" vs "Amount of bugs found by Mythos during large-scale codebase scanning".
[1] https://hacks.mozilla.org/2026/05/behind-the-scenes-hardenin...
https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...
And how much with Opus 4.7? 5x?
https://www.flyingpenguin.com/mythos-mystery-in-mozilla-numb...
There is also a pretty big risk that anyone who is not you would leak the answer to the test. We are close to n=1 epistemics here. You’re going to have to do the research yourself.
Yes, Anthropic have said they made Opus 4.7 worse at this on purpose.
> It is entirely possible that Mythos is a different architecture or size
It has 5x the token pricing of Opus 4.7, so it's probably larger.
Is this suspected vulns or actual vulns? If I recall correctly, it produced 5 for curl but only 1 was legit
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity. That means that even if Mythos Preview finds no further vulnerabilities, at our current post-triage true-positive rates, it’s on track to have surfaced nearly 3,900 high- or critical-severity vulnerabilities in open-source code
> Not even half-way through this #curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day.
> 11 CVEs announced in a single release is our record from 2016 after the first-ever security audit (by Cure 53).
> This is the most intense period in #curl that I can remember ever been through.
[1]: https://www.linkedin.com/feed/update/urn:li:activity:7463481...
If you read his own top comment on that LinkedIn post he clarifies:
“The simple reason is: the (AI powered) tools are this good now. And people use these tools against curl source code.They find lots of new problems no one detected before. And none of these new ones used Mythos. Focusing on Mythos is a distraction - there are plenty of good models, and people who can figure out how to get those models and tools to find things.”
I am still a believer that a 100 subagents with good-enough intelligence can get same results as mythos, I am ready for this opinion to be shattered when I eventually try mythos and I believe others here must have tried mythos out too.
I'd say it is about 90% accurate for us. Often even the "Low" findings lead us to dig and realize it is actually exploitable. Everyone makes these mistakes, from the most junior to the most senior. They are just a class of bugs after all.
I expect tools like this to be a regular part of the development lifecycle from here on. We code with AI, we review with AI, we search for vulns with AI. Even if it isn't perfect, it is easily worth the cost IMHO. Highly recommend you get something enabled for your own repos ASAP
The high impact findings have almost all been bang on for me. I was especially surprised by the high-quality documentation it produces as well as how narrow the proposed fixes are.
I’m used to codex producing quite a but more code than it needs to, but the security model proposed fixes that are frequently <10 loc, targeting exactly the correct place.
It’s really quite good. I’m assuming it’ll be pretty expensive once out of beta, but as a business I’d be jumping on this.
So, how is that supposed to work? Claude Code generates security bugs, then Claude Security finds them, then Claude Code generate fix, spend tokens, profit?
1. Ship bugs
2. Fix them
3. You're the hero!
Unless they are not human.
https://english.stackexchange.com/questions/488178/what-does...
Developers create software, which has bugs. Users (including bad guys, pen testers, QA folks, automated scans etc, etc, etc) find bugs, including security bugs, Developers fix bugs and maybe make more. It's an OODA loop, and continues until the developers decide to stop supporting the software.
Whether that fits into the business model, or the value proposition of spending tokens instead of engineer hours or user hours is fundamentally a risk management decision and whether or not the developer (whether OSS contributor, employee, business owner, etc) wants to invest their resources into maintaining the project.
While not evenly distributed, and not perfect, the currently available and behind embargoed tools are absolutely impactful, and yes, they are expensive to operate right now - it may not always be the case, but the "Attacks always get better" adage applies here. The models will get cheaper to run, and if you don't want to pay for engineers or reward volunteers to do the work, then you've got to pay for tokens, or spend some other resource to get the work done.
On other hand, in real world, the developers learn from mistakes and avoid them in the future. However there is no feedback loop with enterprises using LLM with the agreement that the LLM would not use the enterprise code for training purposes
No. Humans learn from mistakes and try to avoid them in the future, but there is a whole pile of other stuff in the bag of neurons between our ears that prevent us from avoiding repetition of errors.
I have seen extremely talented engineers write trivial to avoid memory corruption bugs because they were thinking about the problem they were trying to solve, and not the pitfalls they could fall into. I would argue that the vast majority of software defects in released code are written by people that know better, but the bug introduced was orthogonal to the problem they were trying to solve, or was for an edge case that was not considered in the requirements.
Unless you are writing a software component specifically to be resilient against memory corruption, preventing memory corruption issues aren't top of mind when writing code, and that is ok since humans, like the machines we build, have a limit to the amount of context/content/problem space that we can hold and evaluate at once.
Separately, you don't necessarily need to use different models to generate code vs conduct security checks, but you should be using different prompts, steering, specs, skills and agents for the two tasks because of how the model and agents interpret the instructions given.
https://en.wikipedia.org/wiki/Great_Hanoi_Rat_Massacre
> Today, the events are often used as an example of a perverse incentive, commonly referred to as the cobra effect. The modern discoverer of this event, American historian Michael G. Vann argues that the cobra example from the British Raj cannot be proven, but that the rats in the Vietnam case can be proven, so the term should be changed to the Rat Effect.
Yeah. Presumably as AI code generation gets better, the output gets better. As smaller portions of code are stitched together, human/AI systems analyze it holistically to make sure all its integrations are secure and bug free.
In 2026, different models are better at different things. Cheap models can plan and do small/medium code projects well, more expensive models are even better at architecture and exploit discovery.
On a broader scale, the sheer face-eating-leopards-ness of programmers finally automating away our own jobs and then realising how much this sucks, after automating away so many other kinds of jobs, can feel darkly amusing to me too.
It’s disappointing that Anthropic and OpenAI never responded to the applications to their respective programs for open source maintainers. From my perspective it seems like their offers are primarily for the shiny well-known projects, rather than ones that get only a few million monthly installs but aren’t able to get thousands of stars due to being “hidden” as a dependency of popular tool.
How do you avoid this pitfall?
def run():
with contextlib.suppress(SystemExit):
do_thread_thing()
threading.Thread(target=run, daemon=True).start()
The model's response: "Honestly? Cargo-culting on my part. You should remove it."
Seems you would not need that many tokens to do so and you might find such cases.
Dude is flexing that he's pushing unsecure code every day, that's a skill!
> that's just thousands of vulnerabilities being discovered by our trillion parameter model
> thousands of vulnerabilities and trillions of parameters?! At current energy prices, in this economic climate, isolated entirely within your datacenter?
> yes
> may we see it?
> no
>ya right.
Here's a demonstration of it blowing something up.
>can I have one.
No.
Do we have a sense that projects like OpenBSD/OpenSSH, FreeBSD, ISC[1] and Apache were included in the "blessed" initial participants in Project Glasswing ?
Or is it big name tech companies, banks and fashionable languages and package managers ?
[1] Bind, DHCP
I joke but that is the world we are moving towards. I don’t think many on HN have thought through the second and third order implications.
That means, they intend to make a load of money before a general release. It is a good strategy.
> 1,752 of those high- or critical-rated vulnerabilities have now been carefully assessed by one of six independent security research firms, or in a small number of cases by ourselves. Of these, 90.6% (1,587) have proved to be valid true positives, and 62.4% (1,094) were confirmed as either high- or critical-severity.
for anybody who has applied opus, codex or oss models for vuln scanning - the true positive rate and discovery volume are a clear step change[0]. The ~50 partners in Glasswing have largely all previously run harnesses with other models and many of them have come out and said - essentially - "ye, wow"
Question now is what a second and third phases of access looks like - deciding which class of systems to secure. Routers, firewalls, SaaS, ERP systems, factory controllers, SCADA systems, zero-trust VPN gateways, telecoms gear and networks, medical devices - there's just so much to do
This is why I believe mythos will remain private for the foreseeable future. There's such a large surface that needs to be secured and so much to triage, fix, deploy.
That may suit Anthropic as private models can't be distilled. There's also a runaway effect of model improvement from the discovery, triage and fix data. This is likely already the most potent corpus of curated offensive data ever assembled and will only get better.
I don't see how Chinese companies are given access soon, or ever. We're likely going to see a world soon of CISA mandated audits, and where to buy a mythos-proof VPN gateway or home router - you'll have to buy American[1].
[0] vs ~30% or so in regular audit tools
[1] or allied
sigh I remember the GPT-2 days - when it was the first time OpenAI restricted access to the models citing "humanity is not ready for it". The model was good at writing poetry or something.
Since then, I don't remember a single model announcement from OAI/ANT that didn't use similar wording.
The so-called leak of model announcement was marketing, it being dangerous is marketing, the world not being ready for it is marketing. And yes, the ones that were given access to saying "oh wow", believe or not, is also marketing.
It's all marketing. You can get the same results from any of the top-5/10 models that are generally available already.
Mythos is Anthropic's way to sell the new idea, because the previous one has democratized.
Marketing is like propaganda. It doesn't need to be based on false facts. Of course they're gonna milk it, keep it private and so on. But that doesn't mean the model is bad. Or that others are as good (apparently they're not there yet).
[1] - https://www.aisi.gov.uk/blog/our-evaluation-of-openais-gpt-5...
If that doesn't convince you that both mythos and 5.5 are a step up (several steps, hah) nothing will.
If I was given free access to any frontier model to use on my projects, equivalent of millions of dollars in AI credits, I sure hope people didn't trust anything that came out of my mouth until they were able to verify my claims themselves.
AI industry has even resulted in a new term - benchmaxing - which essentially means we can't even trust the data anymore until we can touch the model ourselves. So this is not at all surprising to me. What's surprising is why am I in the minority here, and since when trusting authorities that have obvious conflicts of interest became normal.
This just seems overly conspiratorial to me. I don't remember Anthropic ever lying in their blog posts. Some other companies have, but they've been about as consistent as Apple when it comes to claims.
But that corpus of data is accessible to all competitors, American or not. I don't believe that this can't be replicated. I'd posit that there's enough annotated data out there (CVE+patch), only increasing thanks to Mythos, that if you specifically RL for this scenario, you can improve your models performance on finding vulnerabilities without access to Mythos.
I guess they forgot to scan Visual Studio Code plugins and their endless npm dependencies.
there is a difference between a stunt and a viable product. diverless cars and agi are the fusion of Silicon Valley.
This is the MoviePass era of language models
Supersonic again is a problem with noise and cost rather than technological.
Self driving is definitely a technological problem.
Here are two experimental exceptions:
We'll like have some standard AI-focused UI libraries that are harnessed into a design gen system where an AI can pull all the real levers, while also developing a large training data set around it.
"Vulnerabilities in the software that makes the internet" is honestly lower priority than "The platform that the software that makes the internet uses to make releases" If buyers of those internal repos find ways to break into GitHub such that they can cut software releases, or poison github actions from a distance, then we're all in a very ugly mess.
Don't forget that in those 3800 repos is likely also npmjs.org itself.
Not to say these things won't catch vulnerabilities static tools cannot, I think they can, it's just we already have the capability to automatically catch a large surface area of common vulns, and have chosen not to, often for expense reasons.
If you're a team that does already apply several layers of analysis and linting, and wants to add this on top, all power to you.
Because most issues are in business logic that static analyzers aren't going to catch.
I'm at a FAANG and even our static analysis tools are not great at identifying how many issues are actually reachable.
Ideally you use both. An AI model that has static analysis as part of the harness, so it can evaluate each potential finding.
Ideally the static analysis tools are improved so that we don't need to piss away yet more tokens like we're competing on Mark's leaderboard just to find vulnerabilities.
Your proposal of relying purely on static analysis is over-idealistic and just not feasible for large, diverse codebases in the real world.
That's where AI comes in.
But I didn't find the most important information (or maybe I missed it): how much did it cost to find 1451 security bugs?
Claude Mythos Preview will be available to participants at $25/$125 per million input/output tokens
...
Anthropic is committing up to $100M in usage credits for Mythos Preview
> For example, at one of our Glasswing partner banks, Mythos Preview helped to detect and prevent a fraudulent $1.5 million wire transfer after a threat actor compromised a customer’s email account and made spoof phone calls.
For some reason I am not able to relate to the concreteness of either of these.
First half of the page was occupied with a image, not sure if it was relevant in any ways other than setting up security scare. The size of code base, number of tokens, $ involved seem to be out of scope of the update for some reason. Personally I am getting skeptical about all these optics at this point, just some money printing scheme at high level.
Security vulnerabilities are one thing, but in legal we offer up a concept of "knowledge security" which goes to protecting the fidelity of the agent's legal context. Software bugs seem much more tractable because they're managed by software engineers, as opposed to the pipeline "vulnerabilities" we're finding. We wrote a little about one vector here where legal documents aren't quite what they seem: https://tritium.legal/blog/noroboto
No doubt there are many such knowledge domains exposed today. These are more concerning because they're understaffed and managed by non-technical people for the most part. No Mythos required.
Great marketing as always, but the rose-tinted view many have seems vicariously misplaced.
These aren't unreachable vulns.
That's convinient.
But wait, don't they have this amazing AI that can fix all the issues itself with a single /goal command? What's the holdup?
I miss the days when HN would RTFA.
> However, this means that disclosed vulnerabilities are a lagging indicator of the accelerating frontier of AI models’ cyber capabilities: we’re not yet at the point where we can fully detail our partners’ findings with Mythos Preview without putting end users at risk. Instead, we provide illustrative examples of the model’s performance, along with aggregate statistics on our progress to date. Once patches for the vulnerabilities that Mythos Preview has discovered are widely deployed, we’ll provide much more detail about what we’ve learned.
So, success is coming not just from the model but also from the harnesses they built around it. The Cloudflare post was more detailed on that front and I wish the rest would share more about it.
The Cisco spec is interesting too, it pretty much describes an architecture of a harness: https://github.com/CiscoDevNet/foundry-security-spec
And at the moment we have reports from like around 5(?) companies. Btw, Palo Alto Networks has found only 26 vulnerabilities [1]. I'm interested what those partners are and why they have such big amount of vulnerabilities.
> For instance, Cloudflare has found 2,000 bugs (400 of which are high- or critical-severity) across their critical-path systems, with a false positive rate that Cloudflare’s team considers better than human testers.
Yet decided not to share that number. I wonder why.
> Mozilla found and fixed 271 vulnerabilities in Firefox 150 while testing Mythos Preview—over ten times more than they found in Firefox 148 with Claude Opus 4.6;
Mozilla tested Opus 4.6 in a very limited setting (i.e. without proper harness and integration into their workflow; likely without large-scale codebase scanning). It's an incorrect comparison.
> The latest Palo Alto Networks release included over five times as many patches as usual.
Yeah, it's better to say "five times as many..." rather than "26 bugs". Btw, they also used GPT-5.5 and Opus 4.7, so the contribution from Mythos there is unclear.
> Microsoft has reported that the number of new patches they’ll release will “continue trending larger for some time.” And Oracle is finding and fixing vulnerabilities across its products and cloud multiple times faster than before.
Both Oracle and Microsoft are talking about "AI and cybersecurity" in general, not about Mythos.
> For the last few months, Anthropic has used Mythos Preview to scan more than 1,000 open-source projects, which collectively underpin much of the internet—and much of our own infrastructure. > So far, Mythos Preview has found what it estimates are 6,202 high- or critical-severity vulnerabilities in these projects (out of 23,019 in total, including those it estimates as medium- or low-severity).
So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.
> One example of an open-source vulnerability that Mythos Preview detected was in wolfSSL, an open-source cryptography library that’s known for its security and is used by billions of devices worldwide. Mythos Preview constructed an exploit that would let an attacker forge certificates that would (for instance) allow them to host a fake website for a bank or email provider. The website would look perfectly legitimate to an end user, despite being controlled by the attacker. We’ll release our full technical analysis of this now-patched vulnerability (assigned CVE-2026-5194) in the coming weeks.
Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.
Overall, it feels like yet another marketing stunt.
[1] https://www.paloaltonetworks.com/blog/2026/05/defenders-guid...
Which is not bad this early in the 90+45 day responsible disclosure window.
> Yet decided not to share that number. I wonder why.
It is absolutely bizarre for you to expect a company to share the false-positive rate of their security engineers, publicly. That does not happen.
> So, ~6 high- and critical- severity bugs per open-source project v.s. hundreds of high- and critical- severity bugs per partner projects. It looks like the math ain't mathing.
It is pretty obvious they're spending more compute on commercial partners. Why is this surprising?
> Of course, they didn't say that Mythos found only 8 bugs in wolfSSL vs 22 CVE fixed in wolfSSL 5.9.1.
WolfSSL is not the only software project in the world. Mozilla also came out with results that paint it as very effective. I don't think Mythos ever claimed to find all bugs anyways.
Drawback of AI: it works fast
“I see no evidence that this setup [Mythos] finds issues to any particular higher or more advanced degree than the other tools have done before Mythos. Maybe this model is a little bit better, but even if it is, it is not better to a degree that seems to make a significant dent in code analyzing.”
https://daniel.haxx.se/blog/2026/05/11/mythos-finds-a-curl-v...
They don't focus on projects where they find nothing. They certainly don't advertise when they find nothing.
Getting a lot of scrutiny is not the recommendation that it appears to be. What is the new standard? Projects that never have bugs are deemed to be suspect because they "have not been scrutinized" (they have, but null results never go public)?
So Mythos only finding one issue after other tools have found 300 this year is embarrassing. Mythos was supposed to be better and novel.
No, it didn't attract a bluepill exploit research.
The fact that 300 bugs found in a year is not a recommendation as the pro-AI mafia suddenly claims ("because it has been analyzed!") still stands. Maybe the AI-mafia should sell "analyzed by Mythos" labels to impress people who don't write public software or find bugs for that matter.
https://en.wikipedia.org/wiki/Blue_Pill_(software)
Now, since you are a literalist, you'll come up with some other nitpick and gain another 1000 Internet points from the AI people. Perhaps a comma is missing somewhere.
In any event, it barely matters. As Anthropic acknowledges, next level models are comings, theirs is only one of them. Current generation models are already good at things like tracing data flow through complex systems and there’s no reason to think that capability has topped out. So within a year it seems very likely we’ll have more than one commercially available model able to find vulnerabilities cheaply.
On the other hand, it seems that they’ve made much less progress on getting it to design solutions to these issues.
Meanwhile from [1]:
"Not even half-way through this #curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day."
"The simple reason is: the (AI powered) tools are this good now. And people use these tools against curl source code.They find lots of new problems no one detected before. And none of these new ones used Mythos. Focusing on Mythos is a distraction - there are plenty of good models, and people who can figure out how to get those models and tools to find things."
Yeah, it looks like there are at least 11 security bugs missed by Mythos.
[1] https://www.linkedin.com/feed/update/urn:li:activity:7463481...
He posted a general update today on LinkedIn which I think gives the wider context:
https://www.linkedin.com/feed/update/urn:li:activity:7463481...
> Not even half-way through this hashtag#curl release cycle we are already at 11 confirmed vulnerabilities - and there are three left in the queue to assess and new reports keep arriving at a pace of more than one/day.
> 11 CVEs announced in a single release is our record from 2016 after the first-ever security audit (by Cure 53).
> This is the most intense period in hashtag#curl that I can remember ever been through.
Why not? TFA says 23 000 findings "of all severities" and then, in the end, only 88 security advisories published.
What we'd really need is how many security advisories not related to Mythos findings have been published in the same time. If it's, say, 500 security advisories (just making a number up), wouldn't Anthropic's update in TFA and Daniel Steinberg's comments reconcile?
Like, yup, we've got a new tool to find exploits. It's a tool. It's new. We already had tools. Let's make the software world a bit more secure.
Now if you tell me that 100 security advisories have been published in that timespan and that 88 were due to Anthropic's Mythos: now I'd have to say that it's hard to reconcile Daniel Steinberg's position with TFA.
This has always been the bottleneck. Automated tools love to flag vulnerabilities, but almost all are false positives. These need to be triaged and evaluated by humans. This is okay. I’d rather close a false positive after a careful review than miss it altogether.
I don’t think it’s appropriate for calling out humans as a bottleneck. They are an essential part of the process, I’m sure Mythos will also become a catalyst in the process.