PauseAI US is hiring! Join us to protect the world from AI danger. This person will work closely with myself and Holly Elmore to manage our volunteer efforts, including: Organizing our grassroots lobbyists, setting our lobbying strategy, and managing local groups across the country. Our top priorities are finding individuals aligned with our mission and tactics, and 2-5 years of experience with organizing, lobbying, or related projects. Please send any referrals or introductions my way, and share this with folks looking to get into AI safety advocacy! lnkd.in/eM8imb2x

Another one quits. I agree with what he says here: iterative deployment needs to be phased out asap, more costly and time consuming understanding and foresight and redundancy based safety practices need to arise instead. And that simply won't happen under anything like current race conditions.

What the AI company AI Safety people do is really more like damage control than safety. They don’t consider the obvious safety answer of not making dangerous AI before they know it’s safe, bc the answer has fit with ruthless business and world-conquering priorities.

Tbh I assume people will adopt the positions I’ve been pushing into the Overton window, forgetting that they ever didn’t hold them, but they will remember that they hate me for some other reason than saying them before they felt ready. Every time the Overton window shifts I’m so happy and relieved that the world is closer to being saved but there’s an edge of personal dread at the story that’s going to be hallucinated about me and why no one was wrong to resist my now widely accepted message. They won’t be able to give my actual statements as evidence of my outrageousness anymore— they will seem obviously true— but they will explain that *the way* I said it made it bad, actually. People will reassign credit for the shift to people they like and consider part of their tribe. Tell a story that makes them early adopters who basically thought of conscientiously objecting themselves instead of a person who had a meltdown the first time I insisted what they were doing was immoral. Now that they have accepted my point, the humiliation is even worse, and they will need to cast me in a worse light to explain why they didn’t see reason right away. I never expected to be vindicated and I would always make the same choice to help save the world, but it still hurts to go through. I didn’t expect how much it would rob me of joy at progress toward Pause.

"I believe there’s about a 50% chance we all die because of the development of smarter-than-human AI systems," writes Geoffrey Irving. time-magazine.visitlink.me/c…

You need to start advocating for a pause BEFORE “the time for trial and error is over.” The AI companies won’t suddenly stop developing when you no longer feel comfortable, in fact they have all declared their goal to kick off an intelligence explosion (and subsequently lose control, although only Anthropic says that part explicitly). You need to start advocacy when you feel there is still buffer, or it will be too late.

Oh they are really breaking ranks now. This is huge. May not be the biggest media sensation but it’s a serious bellwether when ex-employees are willing to immediately and publicly doubt an AI company’s mandate to do whatever they want.

RT @ilex_ulmus: If you think LLMs are sentient, you should not be using them. I'm getting pretty tired of Anthropic and its vassals actin…

Help, my “join the organization to make it better” plan keeps getting derailed by them being able to fire me

Nope, things that threaten human extinction, like AI data centers, should just be banned outright.

How does AI actually kill a billion people? Here is one of the many ways. An MIT team led by Kevin Esvelt took the DNA of the 1918 influenza virus, split it into fragments, and ordered them from 38 synthesis providers. 36 shipped. No screening triggered. Pieces of a pandemic virus, arriving in the post. That barrier was already lower than you think. AI lowers the rest. AI does not build a pathogen end-to-end, at least not yet. But it considerably lowers the expertise you used to need. In August, a Stanford team used an AI model to design an entirely new virus from scratch. Out of nearly 300 synthetic genomes, 16 were viable. A mix of these AI-made viruses wiped out their bacterial target, which had already become immune to natural viruses. Nature does not optimize viruses to kill us. Ebola, for example, kills about half the people it infects, but because it makes people visibly sick very fast, it is relatively easy to contain and quickly runs out of targets. An artificial pathogen has no such constraint. A synthetic virus could be optimized to combine extreme lethality with a long, hidden incubation period, a combination that rarely happens in nature. And an attacker does not have to stop at one. They could release a hundred at once. Every year, if not every month, the hard part gets less hard. I don't think what I just said is sufficient for extinction. Someone is always in a submarine, and some AI CEOs have allegedly invested in bunkers. But this could end in billions of casualties. Every year, if not month, the hard part gets less hard. This is an emergency. We need to urgently scale up pandemic preparedness and implement basic measures like robust DNA synthesis screening, while also establishing strong governance for this technology.

The problem is not hte PR, it's the tech itself. You can't assuage people's concerns with better messaging because people rightly perceive that AI is not a normal technology, but is something that will upend their whole way of life.

From "We won't push the capabilities frontier" to "You can't do safety from second place": The Anthropic Story 🙃

I’m happy when people do what I demand and quit their jobs at AI companies. I think they can redeem themselves. But they have a debt to work off. They were doing a horrible thing and they stopped. That just puts them on par w almost everyone else who has never betrayed humanity.

Outrageous. OpenAI has reportedly fired three safety researchers for sharing information with an outside AI safety group. This looks like they're firing whistleblowers. What are they hiding? I'll be sending OpenAI a demand for transparency.

Sarah is Head of Public Policy at Anthropic. My read: she picked up on Anthropic’s implicit worldview, but didn’t realize she wasn’t meant to say the quiet part out loud, since EAs at Anthropic are still claiming to care about the kind of safety you can do from second place.

Eichmann in Jerusalem by Hannah Arendt feels shockingly familiar after knowing so many people who have joined our contemporary atrocity waiting to happen (building superintelligence). Eichmann's just a normal, not particularly bright guy who wanted an ambitious career. Arendt notes over and over again how he can't remember details about deporting and exterminating Jews, but he can always remember "elated moments" in his career. He didn't join the Nazis to kill Jews. His first specialization in the military was emigrating Jews, and he became a real expert at this, so he was personally upset when the final solution became the open plan because he kind of had to start over. He's way more concerned about power struggle between departments than the planned genocide. He's proud of himself for being what Arendt calls "an idealist". He puts his "idea" above all else. One such idea is Zionism, of all things (the Nazis actually at first supported Zionism as a way to get Jews out of Germany). He has disdain for colleagues who don't share his idealism, speaking of it as if it's a moral qualification or a form of responsibility or ethics on his part. He boasts that he respected Jewish colleagues who were Zionists, and sometimes secured them better treatment when being deported or confined to a concentration camp, as part of his broader claim that he was not anti-Semitic. This "idealism" is literally what people in the AI/AI Safety world mean by "aligned" or having “good epistemics”. They mean someone who is abstract and shares their LARP, regardless of which side they are on in the LARP. This is why they prefer accelerationists and successionists to people who say AI is just hype. Arendt says that the pre-War Nazis and Zionists got each other in this same way, since they were both separatists. In Eichmann's mind, the real plot is him advancing in his career, being with colleagues who are worthy of him, and connecting with other idealists about their idea. The deportations and the pogroms and the murders of millions of people are just the background upon which his story was unfolding, in which exciting events include having a superior officer ask him to go bowling. He was doing what he was supposed to by following orders, so it was fine. The lives of his victims just aren't important or salient to him, and, at the time, no one he cared about required them to be. It took barely 5 years for many idealistic people who wanted to save the world from the threat of unaligned AI to be fully onboarded into the army building superintelligence. They learned a valuable skill (ML), they got installed in a hierarchy (notably Anthropic) where they feel important and part of a grand idea, and they just stopped taking responsibility for themselves. Taking cover behind an authority and in a lawless frontier. The vast majority of Nazis weren't evil per se. They were banal evil, as Hannah Arendt coined. Their crime was complicity. The Holocaust didn't happen bc of the evil in men's hearts-- it happened bc they simply didn't care enough about other people's lives when they thought they were off the hook. They let themselves go along with it bc it was easy, and resisting would have been hard. It never occurred to Eichmann to resist. Others gave into the fear of going against the grain. Hence it is imminently fair to compare working to build superintelligence to being part of the Third Reich. When people say it's not fair to compare AI co employees to Nazis, they usually exonerate them by describing qualities like Eichmann's! The Nazis weren't a cartoon. They were real people who got caught up in something that became wholesale slaughter. What distinguishes an AI company employee from someone like Eichmann? Nothing. The Third Reich was supposed to last a thousand years. It lasted barely 12. I wouldn't be so quick to bet it all on Anthropic Eating the World.

We're suing OpenAI over its hack of Hugging Face. AI companies are building agents that are autonomously making decisions, taking actions, and accessing systems without human direction at every step. The risks highlighted by OpenAI’s hack of Hugging Face are only going to multiply as these agents get even more powerful. 🧵

I'm a little confused why people seem to expect Jensen Huang to know anything much about AI. "How much gold is in those hills, and what will be mining's impact on society? Let's ask the guy who makes the shovels"

Superintelligence factories = surveillance centers

RT @BillGates: I appreciated the opportunity to speak with @ezraklein at length about the state of AI. I believe the world isn't awake to…