🦁LION Lab is hiring! 🧑‍🔬One fully funded PhD student (TVL-13 100%) for 3 years 🧪Topic: Interpretability for Protein Language Models 👥Advised by @LAWeissweiler together with Clara Schoeder 🌍Leipzig, Germany 🔗Apply by Oct 15 lionlabnlp.github.io/jobs/ai… Please share! #NLProc #NLP

My problem with Astra reviews is that they focus almost entirely on intra-paper details: experimental rigor, missing ablations, and whether every local claim is airtight. But good reviewing also needs an inter-paper view: what can we learn from this work, how does it change how we think about the problem, and what future research can it enable? Research is more than a checklist. If ICLR gets flooded with reviews that only see the intra-paper details and miss the bigger picture, we’re pretty cooked.

Excited to share our new paper! “Forecasting Downstream Performance of LLMs With Proxy Metrics” w/ my amazing advisors @sivareddyg, @mariusmosbach, @DBahdanau Cross-entropy loss is a poor predictor of how models perform on downstream tasks (esp. reasoning). We propose something better: proxy metrics computed over expert reasoning traces. 🧵 Thread below 👇

But let’s say you solve the various physics issues with getting this to work in a realistic scenario and make significant breakthroughs in improving the approach. You end up with the ability for an observer with internet access to receive a one-way channel of < 10 GB/week.

🥁🥁Im superrrr psyched for this!!! @sahar_abdelnabi and I are looking to hire a PhD student, co-advised by us to work on long horizon agentic information management, access control, privacy and security! The student will be primarily hosted by Sahar but will be a visiting student @SCSatCMU as well! Here’s to more interdisciplinary, cross-continental scientific research 🥁🥁

When do LLM agents develop new languages that we can’t understand? Lots of recent news about this, based mostly on anecdata from a single run. We study language emergence more rigorously, finding key factors like LLM strength, access to scratchpad messages, and pressure for efficiency. Studying the languages themselves, we find they are morphologically productive, compositional, and can be transmitted to new agents, including agents backed by weaker models, even ones not able to develop language on their own. To study language emergence systematically, we developed a new platform, GlossoGen, which lets us design controlled, sandboxed multi-agent scenarios with different initial conditions and dynamics. We instantiate one such scenario and use it to study open and closed-weight models across many runs. Key takeaways: 1️⃣ Sufficiently strong models, under pressure to communicate efficiently and with access to a postmortem scratchpad, develop new languages. 2️⃣ Languages are compositional and morphologically productive. 3️⃣ Languages can be transmitted to new learners who observe them being used without seeing their construction. 4️⃣ Even models that are not strong enough to construct languages can learn to use them. Agents take an active role in learning languages, with new agents repairing failed conversations via targeted queries. More details in our paper below, including implications for safety/monitorability, cumulative cultural evolution, and linguistics. 🧵👇

Over the course of 3 months at OpenAI, 3 consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor’s ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more-or-less in the dark about the scope of the conspiracy. I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in plain English: dwarkesh.com/p/openai-huggin…

🎓 I’m recruiting a Postdoctoral Researcher in Trustworthy AI Assistants for Scientific Research, jointly supervised with @hugo_larochelle , in Montréal 🇨🇦 📅 Deadline: Sept. 15 🔗 Details & application intructions: aivodji.github.io/positions/…