Why do people think AI is conscious or sentient or whatever? It's because it talks to you. That's the whole reason. No one seems to think the Waymo or Tesla FSD is conscious even though it's just as complex (if not more) and even a remarkably similar architecture

Reading Less Wrong articles be like "so for this we will assume the existence of Billy™. Billy has infinite computing power, perfect knowledge of his own source code, and a utility function defined over all possible futures and worlds. He is actually a laplace's demon and can predict his own actions with zero aliasing effects. Now, some naive readers might object that Billy cannot exist, but suppose Billy is offered a bet: with probability 1/3^^^3, he...

大雨の道頓堀がいい感じに映画の世界

Listened to an EA-adjacent AI safety expert from Redwood Research on podcast, and you come away with 2 obvious realizations. 1. The current working theories for how AI takes control of civilization are...for lack of a better word...ridiculous. The arguments consist of dozens of contingent assumptions held together by made up probabilities assigned to future states nobody can possibly know. Change just one variable and the whole web of doom unravels. That's not to say AI is harmless. Obviously a technology this powerful carries real risks. But if we’re going to accept extraordinary claims about human extinction (and then make major policy decisions around them) we should demand extraordinary rigor. 2. It’s a great reminder that the genius is no less prone to delusion than the midwit. If anything, they may be more prone because of their gift for rationalizing to their own conclusions. And that’s what’s so bizarre about this whole debate. You have outlandish, quasi-religious claims delivered with an air of inevitability, and somehow the perceived intelligence of the messenger gets mistaken for evidence that the arguments themselves are sound. At a certain point, it’s really no different from Tom Cruise talking your ear off about thetans...

DJ Shadow『Endtroducing…..』(1996)のエピソードを一つ紹介すると、この作品はDan the Automatorという人がサンフランシスコの実家に構えていたスタジオ“The Glue Factory”でミックスされていて、Dan自身もアシスタント・エンジニアとして制作に関わってる点です で、このDan the Automatorはプロデューサーとして、同じ1996年に、今ではエクスペリメンタル・ラップの先駆者の一人とも言われるKool Keithの変名プロジェクト、Dr. Octagonのアルバム『Dr. Octagonecologyst』もプロデュースしていて、こちらもまた“The Glue Factory”で録音された作品なんです また、『Dr. Octagonecologyst』には「Waiting List - DJ Shadow/Automator Mix」という、DJ ShadowとDanによるリミックスも収録されている。つまりこの2作は、同じ時期に同じ“The Glue Factory”という制作環境を共有していた、言わば兄弟みたいな関係性があると なので『Endtroducing…..』を聴く際は、『Dr. Octagonecologyst』も合わせて聴いてみると、当時この周辺で生まれていた音楽がより立体的に見えてくるんじゃないかと思ったりします なお、最近YouTubeに、DJ Shadowが久しぶりにDanとこのスタジオを訪れる動画がアップされてるんですが、これを見ると“The Glue Factory”が屋根裏部屋のような狭いスペースで、想像以上にDIYな場所だったことが分かります こういう場所で試行錯誤しながら、同じ時期に『Endtroducing…..』と『Dr. Octagonecologyst』という歴史的な作品が2作も作られていたのかと思うと、なんとも胸を打たれるものがありました Dan The Automator's Home Studio Visit piped.video/watch?v=4TlBr4Ye…

New: In late 2024, FBI investigators had enough evidence to launch a sweeping bribery probe that could ensnare top lawmakers, including Sen. Susan Collins. Then Trump was reelected and eviscerated the FBI and DOJ teams handling public corruption probes. propub.li/4yLLwaF

i dug up 200 map animations dating back to 1927, free to weave into your work movingimagearchive.com/colle…

これアルバムなんですが、基本Jeff様自身でないとDJでは使いこなせない楽曲ばかりw その中でテクノの歴史を彩る最高レベルのキラーチューンがA1の"Change Of Life"。こんな有名なのにシングルカットされてないのね Jeff Mills - Change Of Life （from Waveform Transmission Vol.1) ('92) #vinyl #record #anaログ

RT @TaylorLorenz: The chances of catastrophic harm from AI are so much more likely to come from government misuse/fuck up than some wild th…

Hes gonna rename iran

Asking Jev the hard questions

I would love for someone to explain what I, the average American, receive in exchange for “beating” China. Every day I check the mailbox for my better-than-China prize.

>be me >discover effective altruism >apparently normal charity is inefficient >why donate to random sad thing when spreadsheet can tell you optimal sad thing >fair enough >buy mosquito nets >save lives >numbers look good >feel powerful >couple years later >someone asks an innocent question >why only count people alive today >huh >future people matter too >obviously >my grandchildren shouldn't matter less just because they haven't spawned yet >reasonable.jpg >keep following logic >what about their grandchildren >also yes >what about people in 500 years >sure >5000 years >why not >500 million years >starting to get weird but morality is morality >open calculator >humanity could survive for an astronomically long time >could colonize galaxy >could have trillions upon trillions of descendants >maybe digital people too >maybe simulated civilizations >maybe dyson spheres full of happy uploaded minds >calculator starts smoking >realize currently living humans are rounding error >8 billion people suddenly looking extremely beta >future contains potentially 10^something people >can't even fit beneficiaries in google sheets >new moral priority unlocked >protect the long-term future >stop thinking in units of "people helped" >start thinking in "fraction of cosmic endowment preserved" >malaria? >terrible >but only kills existing humans >AI extinction could delete the entire light cone >nuclear war could permanently derail civilization >bad institutions could lock in terrible values for ten million years >someone invents wrong constitution in 2140 >quadrillions suffer >better fund governance workshop now >friend says maybe we should improve hospitals >explain opportunity cost >friend says hospitals are full of actual sick people >explain scope sensitivity >friend stops inviting me to dinner >need to decide what to fund >easy >expected value >suppose project has one in a million chance of preventing extinction >sounds tiny >but extinction destroys 10^50 future lives >multiply >mother of god >$10 million project has expected value of several galaxies >charity evaluation complete >someone asks where the one-in-a-million number came from >expert judgement >which expert >us >how calibrated >extremely thoughtfully >reduce estimate to one in ten million to be conservative >still beats curing cancer by 38 orders of magnitude >epistemic robustness achieved >someone says maybe project doesn't work >assign 20% chance >still astronomical >maybe project makes problem worse >assign 5% chance >still astronomical >why 5 >because 30 felt pessimistic >publish 46-page report >contains seventeen sensitivity analyses >every sensitivity analysis begins after assuming intervention has positive sign >critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities >yes >that's literally why it's important >critic says the uncertainty might be structural rather than numerical >make probability smaller >critic says no, I mean maybe your model is wrong >make probability smaller again >critic begins rubbing temples >discover AI safety >perfect longtermist cause >AI might kill everyone >or create utopia >or seize galaxy >or tile universe with paperclips >or create billions of conscious software minds >finally a problem with numbers big enough for me >start AI safety nonprofit >mission: prevent dangerous AI >hire smartest people available >smartest people immediately start building better AI to understand dangerous AI >interesting >we must understand capabilities to understand safety >we must scale models to study alignment >we must race ahead so less responsible actors don't get there first >we must deploy systems to learn how deployment can go wrong >we must build the thing quickly because building the thing quickly is dangerous >outsider asks why the people most worried about AI apocalypse all work at AI companies >complicated field >company releases stronger model >very concerned >company begins training even stronger model >extremely concerned >company raises $14 billion >concern reaches unprecedented levels >need to influence government >future is at stake >normal democratic process too slow >politicians don't understand exponential curves >public doesn't understand x-risk >experts must guide them >who counts as expert >people who understand x-risk >who understands x-risk >our friends >someone objects that this seems politically convenient >explain we're representing future generations >future generations unavailable for comment >develop concept of value lock-in >terrifying possibility that one ideology controls civilization forever >therefore extremely important that civilization adopts correct values before lock-in >whose values >let's circle back >begin with impartial morality >end with small group of people deciding what quadrillions of hypothetical beings would want >beautiful arc >meanwhile actual humans keep doing annoying things >voting wrong >having parochial attachments >loving family more than strangers >caring about local community >getting upset when told their suffering is cosmically negligible >evolutionary biases everywhere >explain that moral intuition cannot be trusted >except intuition that future digital people count >and intuition that extinction is uniquely bad >and intuition that our probability estimates are sane >and intuition that our institutional choices improve the future >those intuitions survived peer review >someone donates $5k to local homeless shelter >inefficient >could have funded 0.0000000000003% of an AI governance researcher >think of all the simulated people you just killed >okay maybe don't phrase it that way publicly >PR team says "future generations deserve a voice" >much better >journalist asks what longtermism means >say "future people matter" >everyone agrees >great >journalist asks what follows from that >well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures >journalist raises eyebrow >return to "future people matter" >motte has entered the chat >critic: of course future people matter >me: glad we agree >critic: I don't agree that your institute knows how to help them >me: why do you hate our grandchildren >eventually notice uncomfortable implication >if future value dominates everything >then helping people today mostly matters through effects on future >education matters because future institutions >health matters because future productivity >democracy matters because future trajectory >human beings slowly become instrumental variables in their own moral philosophy >see starving child >feel compassion >check spreadsheet >child's direct welfare contribution negligible >but perhaps childhood nutrition improves national institutional quality >compassion restored >tell myself this is impartial altruism >one day assistant asks obvious question >"how do you know your intervention actually improves the far future?" >silence >open spreadsheet >increase column width >add confidence interval >assistant asks again >"no, I mean how do you know the sign is positive?" >stare into cosmic light cone >10^50 people staring back >none of them exist >none of them can tell me >none of them can falsify my assumptions >realize I have invented the perfect constituency >infinitely important >completely silent >and always represented by me

We frequently conflate RSI with a fast-takeoff to super-intelligence (ASI). They are not the same. RSI means a model can improve on itself. That does not mean that it can do so more at each iteration, which is what would be required for a singularity. Far more likely that diminishing returns inherent to AI improvement (everything is sub-linear, governed by power-laws or log-scale diminishing returns) mean that each iteration of an RSI loop gives less relative uplift to the model than the previous, and that, after an initial boost, the system settles back to something like its previous improvement rate. RSI != Singularity, fast-takeoff, or ASI.

Superintelligence, in the formal sense of Bostrom’s book, may turn out to be the greatest, most consequential nerdsnipe in all of human history. The gateway: what happens once an AI emerges that is to us what we are to mice or chimps? As you’d expect by construction, that bodes very poorly for our species, even if we cannot articulate any precise mechanism of doom (we mere mortal are simply not ‘smart’ enough). The reality distortion field that is going on here is not only that this hypothetical construct exists, but that we are readily approaching its creation. This is absurd to me, as someone who builds the capabilities of these AI models today.

Does anyone stop and think: Why did the AI discourse go from: “It’s gonna take your jobs” to, all of a sudden, “it’s going to exterminate all of humanity” … and it still hasn’t taken our jobs yet?

Tell me you have zero DevOps skills without telling me you got zero DevOps skills. And LLM is not the I Love You Virus. It is not the same as infecting a web server. It is not the same as hacking a wiki. One is a commodity piece of software that is widely distributed and this capability is a non-trivial leap in hardware/software/capabilities and one does not automatically follow the other. Currently, it is a massive effort to run these models, usually distributed across multiple very expensive chips, running on specialized inference software, specialized harness software, databases, cloud computers, with tooling and hundreds of install packages, etc. Kimi K3 is 1.6 TB of space at 2 trillion parameters and you need about 1.5 TB of VRAM, which means you need 8xGB300s which will cost about 350-450K a year to purchase or as much or more to rent. Now let's say Astra or Mythos or later models come in at 10 trillion parameters, so that puts us at roughly 10 to 20 TBs unquanticized. We are guessing her because these folks reveal nothing about architectures anymore, but they are reasonable guesses based on scaling laws. You would need roughly 23 TB of VRAM to run it and roughly several million dollars to run that one instance a year. And to make it worse, it no longer fits on an 8 way cluster, which means you now need special networking configuration and inference software to run it or you need to infect a Nvidia NVL72 rack-scale server system costs between $2.8 million and $6.5 million. So the "rogue" model copies itself to the most expensive compute in the world. It's safe to say that hardware is not sitting idle and is being watched closely for downtime or anomalies. In fact, it is likely doing one of the following things: 1) Serving customers at a hyperscaler 2) Serving inference for a well endowed company. In the first case it will be instantly noticed by the actually good automated monitoring systems of the hyperscaler because there is now something using their infrastructure but not being billed properly and costing them millions of dollars in lost revenue and electricity. In the second example, the model they were serving on that very expensive hardware is now offline (replaced by the rogue model). The model presumably was connected to a valuable piece of internal software that is now glitching and no longer running. Nobody notices this? And so we see that this is where sci-fi meets the little problem of the real world where friction, dust, time, resources limitations and the like come to crush your little fantasies. To be fair, this *may* be possible in coming years. I assign it a non-zero probability. Breakthroughs in new architecture and smaller models and eventually the wider distribution of the computing substrate that commoditizes could change this materially. But for now, it is mitigable and foreseeable and an engineering problem. In no way will these things be copying themselves around like a virus on your computer in their current form. It also betrays a lack of understanding of the evolution of complex economic and societal and technological systems, we tend to get mitigations as parallel evolutionary developments. Things do NOT evolve in a vacuum. So when you get viruses, you get anti-virus software. When you get DDOS attacks, you get CloudFlare. There is money in solving problems and things do not evolve in isolation. Believing that things evolve in isolation, or that one thing changes and all other variables stay the same, is one of the many reasoning errors that destroy people's ability to make accurate predictions. In addition, when you don't have the requisite knowledge to verify your predictions, things that sound sane and rational to you are, in fact, insane and ridiculous.