Atlas is live! A spatial foundation model, trained in-house from scratch. After shipping RTFM last year, I was convinced a single camera-conditioned model could unify most spatial generation tasks. Atlas is that bet at scale. The core idea👇

Fantastic job @HaoZhang623 !

sketch-to-simulation with Opus 5.5

Introducing PointZero—a 3D world model pre-trained without robots. Dexterous manipulation requires understanding diverse 3D dynamics, but current approaches rely on expensive robot data. So how can we pre-train a 3D dynamics model without it? PointZero introduces a simple idea: learning to complete 3D point tracks yields transferable 3D dynamics! Our model achieves SOTA results on: 🔥 Zero-shot 3D dynamics 🔥 Action-conditioned 3D dynamics (post-training) 🔥 Imitation learning (post-training) Details and links 👇 1/6

We’re excited to share that @nuance_ai has raised a $50M Series A led by @lightspeedvp, joined by @Accel, @spc, @nvidia, and @definevc! Much of human communication happens beyond words. Before we can speak, we learn to read faces, return smiles, follow gestures, and sense when someone is listening. That visual and emotional vocabulary stays with us for life. At Nuance Labs, we’re building a foundation model that understands and responds to the full spectrum of human emotion. A single full-duplex audio-visual model that sees, listens, and responds in real time, with the natural give-and-take of a conversation with a friend or colleague. We believe the best interface with a machine is the one we’ve practiced since birth. Join us!

An insightful study from @tongzhou_mu and the @RhodaAI team that examines how web video pre-training translates to robot performance. The gist: pre-training doesn’t just help; it scales, i.e. you can continue to reap greater benefits as you put more effort into pre-training. A better pre-trained model will lead to a more robust policy. But how you achieve that pre-training quality is up to you! Whether by scaling the model, or scaling your training tokens, it’s a design choice informed by your compute constraints, training budget, and how much you care about compute optimality.

My favorite use of Atlas is interactive world building / editing. To experiment with this, I hacked frameboy advance to let me supply text prompts on the fly. Atlas’s strong reconstruction and generative capabilities make this a unique experience. You can start from scratch with just a text prompt or from an existing world by loading its views into Atlas’s spatial context, then shape the world while you are flying around in it. The best part is that you can share the resulting worlds between different Atlas sessions simply by loading their views into the model’s spatial context.

To explore large spaces like this, while keeping them consistent is afaik not really possible with any other technique yet. This is the power of deep learning. We just put (real or generated) images into Atlas' context window and let it rip. There is no explicit 3D representation, all the interesting computations happen within Atlas forward pass(es). Atlas-turbo predicts new views that are 3D consistent with its previous context and correspond to whatever camera the frameboy advance requires. In the attached frameboy advance recording I put 48 real views and some Atlas generated anchors in positions where Atlas-turbo was the most uncertain.

Almost a year after the Frameboy, @wendlerch and @_nikunj_gupta bring us Frameboy Advance!

absolute zoomina

a tour of temples in India by atlas

Tsiigaa toi kirkko, Oodis vibet jäätävät, meri hengaa, bro.

europa by atlas

a tour of asia with atlas

atlas world atlas -- north america