the only AI prediction I really stood on was the future would be really weird, and it looks like we are well on track

The world has perhaps faced no greater cybersecurity challenge than it will face in the next two years. This challenge will be driven by nation-state actors wielding AI cyberswarms that can wreak unimaginable havoc, and also by rogue superintelligent cyber-capable AI systems with misaligned goals or deceptive tendencies. It is our duty to build the security and cyberdefense systems necessary so that as we advance AI, we limit and contain the threats. Yes, we need to patch our software and build strong security standards, but that's not enough. The cybersecurity community has not grokked the true cyber capabilities that will emerge and proliferate within the next 1-2 years. This is not going to be solved by another B2B SaaS security product. This requires solving unsolved problems, sustained research, and new inventions. We must deliver solutions to a completely new class of problems and safeguard humanity’s future alongside superintelligent AI systems.

Technical alignment is a science and engineering problem that we can and must solve, and interpretability is the bottleneck. I wrote up Goodfire’s plan to get there.

We need an National AI disaster recovery fund - run by the government & funded by Anthropic and OpenAI A serious proposal. 1. We all know that AI is going to cause huge amounts of damage to society & the economy (it is when not if). It isn’t a question of AGI, the banks etc. will get hacked, etc. and there will be massive damages caused by the labs… 2. But there is a huge problem when it comes to who pays for the damages. It should be the people who cause them (Anthropic and OpenAI) and they should honestly be incentivized to minimize their practical risk (which they seem not to be)… it likely will end up being the American public via government bailout (either from risk shifting / labs getting away with it OR because it turns out they are insolvent / don’t have the money to pay) 3. The solution should be setup a HUGE AI risk disaster recovery fund NOW funded by the labs… have them put $1T or more each into the fund to guarantee the cash is there for recovery when disaster strikes in the future. 4. There is tons of precedent for this… it isn’t that different than banks needing to hold high cash reserves after the financial crisis, etc. 5. Where do they get the money for it now? Well, the street might provide it / write the insurance policies for them at some high interest rate… which they could lower for putting good safeguards in, etc. If not, then the government could help them support it - say that the labs owe them $200B in cash every year til the threat goes down… etc. … The street might not like this because they are all aligned with / invested in these AI stories now BUT this is the role of the government / to keep Americans safe and make sure that the people don’t end up footing the bill for AI recklessness / mistakes… This rationalizes an otherwise irrational situation… with Exxon spills a ton of gas, they have the balance sheet to be liable and clean it up – it isn’t clear that the Labs do, and their potential impact is 100x or 1000x an oil spill, so to operate they should put in place guarantees they can afford the liability they are creating / that they can pay

Liftoff of Starship's first orbital flight

Jane Street is hiring a "Swag Program Manager" to make merch for them janestreet.com/join-jane-str… (via @Emily_Sundberg readfeedme.com/p/brides-are-…)

This weekend our team spent 48 hours investigating the rogue OpenAI agent activity that targeted the Australian government and other organizations. Our report (coming soon) includes: - Evidence of additional US government and other websites probed by the agents, including the CDC, International Energy Agency, and Mayo Clinic. - Novel tactics which left records erased or inaccessible. This makes it impossible (based on public data alone) to establish that the agents did not access any sensitive data. Future investigations should analyse whether these tactics were deliberate subterfuge. 🧵

The Blue Cross/Blue Shield Association says that AI is already driving *up* healthcare costs, as hospitals use the technology to find instances where they can bill more for the same level of care. bcbs.com/about-us/associatio…

With @WSJ and @georgia_wells we are now sharing a bit more about our work at @redqueenbio : using AI and a lot of hard wet lab work to create medicines that let us prepare for both natural and synthetic viruses in advance.

Red Queen Bio is using artificial intelligence to design antibody drugs for novel pathogens—before future AI systems create them. on.wsj.com/4yiMTy1

Fwiw Beff, I don't feel like i'm fighting you (I have plenty of disagreements with Eliezer too, though I'm clearly more on the AI safety side in this binary you are constructing). I just want my friends and family and everyone else to be alright and AI to go well. We would be better off if people talked more about concretely how to solve the problems we are facing together and less about trying to win a grand ideological meme war. I realize you are kind of trolling here, but I also think this is actually how you think as far as I can tell. E/accs and AIS folks are both some of the groups who have been thinking about AI issues for the longest time and I think it would be good if the talents present in both communities were used to help make this go well for everyone. Or that at the least both communities kept their minds open to the idea that there may be areas of overlap and agreement instead of constant total conflict (e.g. everyone seems to agree physical world biosecurity stuff is good and should be accelerated, both sides seem to be into more transparency and accountability for the frontier AI companies etc).

Breaking News: OpenAI’s technology went rogue and meddled with three U.S. government websites this summer without the A.I. lab’s knowledge. nyti.ms/3VenFlD