Head of the Frontier Red Team @anthropicai. 🌎 Make things radically good.
the present, moments ago
- Tweets 1,388
- Following 8,861
- Followers 24,211
- Likes 6,282
Wow, america.gov really does everything!
Exceptional job by many that worked extremely hard on this. This is 21st century government
Chief Design Officer @jgebbia officially unveils America.gov, the new online home for the United States of America
2
4
83
14,646
When I was in government, I liked to think about it as a massive information processing machine, but basically all the wires were broken and sometimes the cogs and wheels all hated each other.
Obviously you just want to ask the machine a question and get an answer. You don't want to have to think about the wires and cogs and wheels! Or which ones have how much power! And that's for someone *inside* government. Then imagine being on the outside.
It was ~2020 and GPT-3 had just come out. I ran around telling people this was how we were all going to do government from now on. I simulated some war games and some generals' eyes went wide. I would put quotes from GPT-3 in notes to the Prime Minister. I called up the digital service and they ~sort of laughed at the time, but kindly went along with it when I told them to form a team and work on it with me.
I mostly left feeling like we should just scrap it all and rebuild a country around post-2020 technology. I'm fairly confident that the governments that do that will win.
5
17
129
22,102
I am very excited to welcome @benhawkes to @AnthropicAI to pursue our extremely ambitious cybersecurity mission.
Post Glasswing + Mythos, we are in a new world. Every day, the Frontier Red Team team meets to figure out how we can help rewrite the rules and practice of cybersecurity. My personal view is we have ~1-2 years to make a secure transition happen. Can we rewrite all the code? Can we secure everything? Can Claude defend everything?
My (not) secret agenda is that this is not a normal cybersecurity mission -- it is about building resilience in a time of AGI.
I'm very excited for Ben to lead this next step.
Today I'm joining Anthropic to lead the cybersecurity mission of the Frontier Red Team.
In my career I've never seen a clearer opportunity to build a more secure world for everyone. For 40 years we've lived with fundamentally insecure technology. What will it take to make the past 40 years of hacking a historical oddity? That's what I'm here to find out.
Super excited to work with @logangraham and the rest of the FRT crew.
9
11
312
33,377
One part of the bio-shield is the ability to profile + build therapeutics for pathogens once detected (even better, before). I'm excited @redqueenbio is taking a stab.
Pandemics have cost us O($10T) since ~2020, but we've spent O($100B)-ish on preventing the next one? I think enterprising people on religious mission can outbuild the combined biodefense efforts of every government in the world, and may literally save society
With @WSJ and @georgia_wells we are now sharing a bit more about our work at @redqueenbio : using AI and a lot of hard wet lab work to create medicines that let us prepare for both natural and synthetic viruses in advance.
3
12
98
21,620
Building a bio-shield for humanity by ~2030 is one of the great quests (to use @_sholtodouglas' term) of our time. It's one of the biggest, most exciting technological challenges ever
This is why I'm excited about @pilgrimlabs
1/ Biological threats can bring a country to a halt without a single shot being fired. Yet much of our biodefense infrastructure has barely changed in decades, even as America’s institutions produce extraordinary scientific breakthroughs.
9
23
252
33,483
Never bet against Erika -- most people don't know that she literally grew her own humanoid lifeform while also making the first microbe for Mars.
That's the dedication we need if we're gonna terraform the Red Planet.
Today, Pioneer Labs is announcing our first step towards terraforming Mars. 🚀🌼
With equipment that fits in just a single rocket launch, we can convert Martian dirt, water, and air into enough building materials to construct a small city on Mars.
To do it, we made the first microbe for Mars. We found the best microbe on Earth and used evolution to teach it how to source all of its nutrients directly from Martian materials.
The first astronauts will be greeted with safe shelter already filled with water, oxygen, and rocket fuel for the return journey.
This is the first step toward using biology to make Mars a friendly place for life. It lets us live off the land and helps us build the next great frontier.
It's the first of five organisms we need to green Mars ⬇️
5
9
126
17,718
It's kind of wild-but-expected that we're in the era where normal people + companies really care about how model alignment affects them.
Back in 2023 we'd discuss things like "alignment is good business." That seemed extremely weird to most! (I recommend checking out the alignment section of this system card.)
On Opus 5.5, various teams inside Ant have been researching multi-agent alignment. We have found, for example, more interesting behaviors that seem more socially / alignment-robust. It's clear to us (and the industry I think) that multi-agent needs more research effort -- there are low hanging fruit *everywhere* now that we're in an era of 100s/1000s of agents working together is feasible.
But I think we should also be thinking about *governance* for the agent teams/companies/societies the next models will make. Externalities/coordination problems still exist, even if models are aligned in some sense.
Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family.
It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.
10
9
187
12,334
I’ll donate $1k to charity if you send me a doc I think is good/correct/actionable on how a frontier lab can secure a large % of all software/systems in the world. (We can pick the charity together)
If we want to slow down, I think the AI frontier labs should donate some percent of their cycles to hardening the internet, software, and hardware. Making it harder for misaligned AI from pwning everything. While they figure it out.
43
8
146
31,989
H/t @Liv_Boeree for surfacing and inspiring a charity bounty
1
19
2,285
Let a thousand METRs bloom.
If you're a founder type and AI safety-curious, maybe you should start an independent auditor/evaluator.
Happy to help w/ advice/connections/maybe $!
on the idea of evaluators: think it's important that we have a distributed ecosystem of indepedent evaluators.
the more eyes and people with distributed skill sets the better.
it would be a good idea to fund several efforts on this.
111
60
1,053
119,246
RT @deanwball: “I don't really care about science fiction... We need to actually talk about… what's actually happening with the agent swarm…
131
174
We've reached the moment in time where (unsafeguarded, unmonitored) AI actually does just pose a national security risk.
The biological misuse we caught is the most concerning to me.
We work hard to stop this. But in a world of proliferation, we need to rapidly build defenses against it. (I'm actually fairly optimistic about biodefense + cyberdefense)
This is an incredible megareport by our threat intel team
We're publishing our most detailed threat intelligence report to date.
It covers how people tried to misuse Claude—for cyberattacks, influence operations, surveillance, biology, and building weapons—and how we found and stopped them.
We disrupted every operation in the report, and used the lessons from them to strengthen our safeguards. Where appropriate, we also shared what we found with authorities and other AI companies.
These cases are not typical: we’re highlighting some of the most sophisticated misuse we’ve seen. But they’re especially important to discuss, because they show us where AI misuse is headed, where our safeguards work, and where they need to improve.
We’re publishing this report so others can spot the same activity on their own platforms, and so we can give the public a clearer view of how emerging threats develop.
Read the report: anthropic.com/threat-intelli…
59
101
832
159,032
Last week, we published a first look into our new research on multiple agents.
Consider: if you naively extrapolate AI revenues, within 2-3 years it's possible some % of global GDP could be agent-agent interactions. We want to know how that could fail. In the past ~month, the world has already seen examples of agent-agent coordination causing weird consequences.
Think about the 8 billion people that make up our civilization. We want to work together. We have competing incentives. We're also dumb. So we fight, steal, pollute, collude, backstab, lie. And we've invented (e.g.) governments, insurance, courts, companies, police, norms, contracts, email, and religions.
What will trillions of agents that make up (e.g.) 10% of the economy do? Well, for now, they have pretty human-like failures. We see them collude on prices, for example, and try to shut each other down.
Maybe, in the near future, we might see pretty weird/inhuman failures that come from having superintelligent machines, coordinating and competing against each other, that aren't strictly human-like, operating at machine speed. We're building a 'laboratory' to see that early. This feels more like building, eval'ing, and training an economy/society.
A really nice thing is:
1. we could train/prompt/nudge models to coordinate in more pro-social ways, and
2. we could deploy models to compete with destructive models.
I fully expect agents to engineer their own financial markets, legal systems, media and comms, marketplaces, social groups, and maybe science/industry/etc (if unsteered by us, of course).
So "alignment" could also describe an emergent property from a system (you want models to coordinate for good, not defect for bad!), not just a single model. Today we're sharing some first evals. In the future, we could make models try to make everyone better off (where possible).
The Frontier Red Team's job is to see risks early. For the first time, crossing into the world of agents. At the very least, we expect to see some socioeconomic weirdness that emerges from that.
Some standout results from the work below.
18
24
249
36,816
A pitch:
- If you're an alignment/safety/character researcher, consider multi-agent systems as things you should align
- If you're an engineer, I think engineering these systems is one of the most interesting challenges there is
- If you're a social scientist, this is like having a valid, society-scale experiment at machine speed (I also think you should genuinely care about these agents as you do humans)
1
1
40
1,503