can’t believe it’s October already. 2025 is ending soon.

🦭went a beach party

💀🦭zelda and overwatch changed my dark life.

i said i'd study Miles and so here is my first little pr, using H100s on @sfcompute's givemeanode, now Autoresearch. thank you my friend @evanjconrad <3🦭 Autoresearch is the easiest GPU platform i've used! i used to write and maintain a lot of scripts to use rented compute efficiently. with Autoresearch, my coding agent could create nodes, run the Miles container, inspect failures, retrieve results, and shut down the H100 node when I no longer needed it - all in the same workflow as editing code. It saved me a lot of manual node setup, file management (and money!) the Diffusers version Miles pins could override the deterministic setting and wrapped FA3 in a custom op without a registered backward. my patch calls FA3's public autograd interface and preserves the setting through dispatch. on H100s, the attention tests matched numerical references and produced bitwise-identical outputs and gradients across 20 fixed-input runs per configuration, including 2- and 4-GPU Ulysses. i also added a tiny Wan FSDP2 training test comparing SP=1 and SP=2. GPU charges for this validation came to about $8, including setup and retries（very efficient!). draft PR: github.com/radixark/miles_di…

awesome work and awesome tech report! ever since minimax h3, there’s been so much acceleration work around omni models lately - everyone wants to make them real-time. it’s really interesting to watch these approaches converge. sparse and low-bit attention feel especially complementary: sparse attention reduces how many interactions you compute, while low-bit attention reduces the cost of each interaction. real-time omni models may end up being one of the strongest forcing functions for attention and inference efficiency.

Introducing Limite 1B - Violetto. A model for high-frequency mathematical intelligence.

2030: Your AI lawyer thinks you’re guilty, so the prosecutor subpoenas its activations.

>Their entire hacking cost less than $3,000 in tokens. >OpenAI awarded them $6,500.

Following MiMo-V2.6’s live RL run. Useful to see learning curves alongside sampling stats, queues, costs, and restarts! The2B tokens per step is a substantial workload. What I’d like to understand is the compute allocation: at a fixed total budget, what produces the most improvement on unseen tasks? A few questions this run brings up: - More prompts, more rollouts, or better feedback? The dashboard shows ~1,568 prompts × 16 rollouts per batch. Covering more tasks and exploring more trajectories per task serve different purposes. Grader compute adds another choice: when is it worth generating fewer trajectories and spending more on evaluating them? The best allocation may shift as the policy improves. - What does agentic in-group credit assignment actually do? Test cases and rubrics provide different kinds of feedback. I’m curious how the grader compares attempts within a group, and whether it assigns credit to whole trajectories, stages, or individual actions. The ablation I’d like to see is whether additional grading cost pays for itself through more efficient learning. - Critic/RLOO choices also affect scheduling. RLOO uses sibling rollout returns as a baseline, a learned value function predicts return from the current state. If advantage computation requires the full group, slow trajectories can delay it even in an asynchronous pipeline. A value-based estimator that permits independent trajectory processing could reduce that dependency, but adds critic inference, training, and estimation costs. Retaining an RLOO component may retain the group dependency too. This is a design question, not a claim about MiMo’s implementation. The dashboard’s critic/* metric names alone don’t establish that it uses a learned critic. - Throughput needs to be read alongside policy lag. More trajectories per hour don’t necessarily mean faster learning. By the time a long-running task finishes, its behavior policy may be several versions behind. How the system handles that gap - and avoids systematically favoring short tasks - matters alongside raw throughput. The comparison I’d most like to see: time and total cost to reach the same held-out performance, accounting for rollouts, grading, environments, and any critic. Appreciate the MiMo team making the run visible. There’s a lot to learn from the intermediate behavior of a training system. mimo.xiaomi.com/rl/ Le Critique: arxiv.org/abs/2608.16739

my bet: future agent harnesses will be orchestrations of models, with trained smaller models handling routing, context isolation and memory updates. models deciding what other models get to see, remember and spend compute on.

la vida antes de este tweet:

I heard it's the best post-training framework right now, with really good engineering taste. Gotta study and try it out carefully this week! 👀

So Astra is able to identify sounds from mel spectrograms zero-shot. I don't think we've scratched the surface of what this model can do (and this is light reasoning btw)

so again, most tasks can be considered coding tasks, and the results are natively programmable.

rsi 😂

Wow, this brought back a project Woojeong Kim, @srush_nlp, and I worked on in 2024. A strong model read a task description, turned it into "ability vectors", and passed them to a weak model. Sasha even came up with a great name for it: Strong-to-Weak Ability Transfer (SWAT) 😆 We never published it, but the idea eventually developed into ProgramAsWeights, which compiles task descriptions into small reusable neural programs that run locally. Very cool to see @aimalysheva and the Mostik team make a related idea work so well at this scale. programasweights.com

this is actually huge. sota models could directly provide hidden states to smaller models, allowing a 4B parameter model capable of running on edge devices to perform decoding. it's a new way to do fast and cheap inference. shut out to people who first made it. will agent talk in latent space in the future? probably.

I'm really curious about how Jürgen sees the world. The guy is standing so high up and looking so far ahead. The more you read his work, the more you realize it. A kid, an artist, and a master.