Cofounder/CEO @theworldlabs, Prof (CS @Stanford), Co-Director @StanfordHAI, #AI #SpatialIntelligence #GenAI #computervision #robotics #AI-healthcare
Stanford, CA, U.S.A.
- Tweets 3,117
- Following 1,196
- Followers 1,079,616
- Likes 7,872
Sim-to-real aligned simulation also makes evaluation scalable. Here, the same failure behavior and outcome appear in both virtual and physical worlds. The simulation captures more than the task setup or final success label; it reproduces the conditions that push a policy toward success or failure. The result is faster iteration, broader coverage, and substantially lower cost.
4
6
47
11,369
The Sim-to-Real part of R2S2R uses these aligned worlds for training. The policies were trained entirely in simulation with zero real-world data, transferred directly to diverse robot platforms, and operated autonomously for hours without failure or human intervention. Our engine is policy- and embodiment-agnostic, allowing us to serve customers with different robots, sensors, policy stacks, and deployment needs.
2
4
44
5,184
The Real-to-Sim part of R2S2R transforms physical robots, sensors, environments, objects, and interactions into simulations that preserve task-relevant observations and dynamics - not only how the world looks, but how it acts when the robot interacts with it. These results set a new bar for sim-real alignment in robot manipulation.
3
19
144
26,225
With the help of generative world models, a real-to-sim-to-real (R2S2R) simulation engine turns one physical task into many controllable, reusable worlds, helping robotics teams train policy models and test changes faster, uncover failures earlier, and reduce costly experimentation on hardware.
7
24
142
26,500
When SceniX joined World Labs, we said spatial intelligence was never only about perceiving and generating virtual and physical worlds, but also interacting with them. Today, we’re sharing early results from that vision: building worlds that train robots. 🌎🤖↓
40
132
910
335,296
The world is not just made of words, and spatial intelligence was never just about perceiving and generating worlds. It's about interacting with them.
Today, SceniX is joining World Labs. 🌎🤖👇
83
217
2,021
396,983
🔍 2. Larger dataset, better baselines
• 20,000 human teleoperation demos, 1950 hours in total (2 times larger than 2025)
• RGBD observations and Robot proprioception
• Skill/subtask annotations
• Strong baseline support: pi0.5, GR00T N1.7
1
1
16
2,816
💯What’s new in the 2026 BEHAVIOR Challenge?
1. Double the tasks, double the challenge.
We doubled the benchmark from 50 to 100 long-horizon household tasks. These activities average 6 minutes each, requiring navigation, planning, memory, and bimanual coordination. No other robotics benchmark comes close in terms of difficulty.
2
3
27
6,162
1/N Long horizon, complex tasks that truly matter in everyday life are not solved problems by today’s robotics, requiring planning, object detection, object manipulation, and failure recovery.
That's why Stanford's BEHAVIOR Challenge is back for year 2! Last year, the winning solution reached only 12.4% full task success. This year, the BEHAVIOR challenge has more tasks, better evaluation, and is easier to use. 🚨
⏰ Submission deadline: 10/16/2026
📣 Winners announced: 11/04/2026
🏆 Prize pool: $11,000
22
91
531
115,309
It’s a real honor to receive an honorary doctorate of science from @BrownUniversity . 😍
At its 258th Commencement on Sunday, Brown conferred honorary doctorates on six candidates who have achieved great distinction in a variety of fields, including economics, civil rights, artificial intelligence, national security, business and community leadership. buff.ly/cSP89Ff
60
65
1,455
117,438
It’s 11th year and counting! Teaching the first lecture of @cs231n every year has been a highlight of my spring seasons. As usual, I asked students which departments or schools they come from @Stanford . Increasingly, students raise their hands to indicate that they come from all seven schools on campus, from @StanfordEng to @StanfordMed @StanfordHumSci @StanfordGSB @StanfordLaw @StanfordEd @stanforddoerr . AI is truly a horizontal technology that excites students across all backgrounds and disciplines!🤩
51
95
1,246
89,725
We received submissions from 18 teams across academia, industry, and independent participants from around the world, including the US, China, the UK, South Korea, and more.
Amazing work from all participants — thank you for making this challenge truly exceptional! (3/N)
1
2
30
5,897
As a reminder, BEHAVIOR is an open-source benchmark, designed to enable and evaluate embodied AI and robotics solutions. It includes everyday household tasks grounded in human needs.
See the challenge description here: x.lingyaoai.com/drfeifei/status/196297… (2/N)
(1/N) How close are we to enabling robots to solve the long-horizon, complex tasks that matter in everyday life?
🚨 We are thrilled to invite you to join the 1st BEHAVIOR Challenge @NeurIPS 2025, submission deadline: 11/15.
🏆 Prizes:
🥇 $1,000
🥈 $500
🥉 $300
4
12
54
19,284
AI’s next frontier is Spatial Intelligence, a technology that will turn seeing into reasoning, perception into action, and imagination into creation. But what is it? Why does it matter? How do we build it? And how can we use it?
Today, I want to share with you my thoughts on building and using world models to unlock spatial intelligence in this essay below. 1/n
294
745
3,569
929,341
From AI empowered education reforms, to the future of jobs, from spatial intelligence to robotics, it was fun to spend my alumni reunion Saturday with alums from all over the world and my friend @lilysarafan , chair of @Stanford board of trustees. Go Cardinal! @StanfordHAI
22
38
491
46,224
Yes, we have a button for VR, when you're in a world viewer, and click on the 'share' button.
3
1
26
3,489
(7/N) 🧩Feature #4: Diverse State Transitions and Manipulation Skills
• Spatial: next_to, inside, on_top, under, touching
• Particles: covered, uncovered
• Thermal: hot, cooked, on_fire, frozen
• Others: open, closed, on, off, attached, sliced, diced
2
4
28
5,168
(6/N) 🏠Feature #3: Long-Horizon Mobile Manipulation in Realistic Homes
• Task durations range from 1 to 25 minutes (average 6.6 minutes)
• Performed in household-scale scenes
• Requires memory, planning, and reasoning over a long period of time
2
3
32
5,929
(5/N) 💯Feature #2: High-Quality Data
✔️Teleoperated using our JoyLo interface
✔️Near-optimal, clean demos
✔️Consistent manipulation behaviors
✔️Moderate, consistent teleoperation speed
🚫 No sudden accelerations/decelerations
🚫 No failed grasps
🚫 No unintended collisions
2
7
38
6,690
(4/N) 🔍Feature #1: Large-Scale Demonstration Dataset
• 50 tasks, 10,000 demos, a total of ~1,200 hours of data
• Subtask and skill (30+) segmentation
• Spatial relation annotation
• Multi-granularity language annotation
3
5
44
7,804
(1/N) How close are we to enabling robots to solve the long-horizon, complex tasks that matter in everyday life?
🚨 We are thrilled to invite you to join the 1st BEHAVIOR Challenge @NeurIPS 2025, submission deadline: 11/15.
🏆 Prizes:
🥇 $1,000
🥈 $500
🥉 $300
41
276
1,432
456,014
Or stroll in this garden, another permanently consistent 3D world created from a single image no matter how long you hang out in it.
15
20
332
50,038
A picture now is worth more than a thousand words in genAI; it can be turned into a full 3D world! And you can stroll in this garden endlessly long, it will still be there.
144
327
2,596
261,449
Also met up with an artist I’ve admired for a long time, and is now a dear friend of me and @theworldlabs - @refikanadol . We share the belief that AI can empower and augment human creativity if used right. 6/N
19
8
124
29,649
Of course it’s always fun to have a mini reunion of AI friends, some of them are even neighbors but it takes a summit in Paris for us to catch up! @erikbryn James Manyika @demishassabis @ylecun 5/N
10
18
200
28,713
I have also met with a number of global leaders to discuss the state of AI and exchanged ideas on collaborations and partnership, among them are @joteo_ylm (Singapore), @peterkyle (UK), and @ClaraChappaz (France). 4/N
2
5
80
15,935
The Vice President of USA @JDVance just gave a historical speech about being all in in AI minutes ago. I have been hearing AI speeches by heads of state for more than a decade, this is the most forceful speech about the present “opportunities” in building AI, to embrace it to help people and lift society, and to unleash the potential in this profound technology. 2/N
14
21
169
22,639
Live from Paris, at the #ParisAISummit . I gave the opening keynote about AI’s past, present, future and opportunities. “75 years ago, Alan Turing dared humanity to imagine ‘thinking machines’. But perhaps that vision is too narrow and inward-looking. As AI today is poised to become both thinking and doing machines as we unlock language, spatial and embodied intelligence, we should consider a new dare to all of us - a dare to build human-centered AI to empower and augment people.” 1/N
141
211
1,581
220,493
Excited to be at #NeurIPS , still working on my slides for my keynote on Wednesday Dec 11 at 2:30pm! 🤞
20
25
363
32,479
I can't get enough of these 3D worlds... from Monet's garden, to fantasy castle land, to an outdoor bazaar, to floating city, and even underwater coral reef... hope you enjoy them too! @theworldlabs Check out the link below.
26
84
503
115,707
A better link for my and @geoffreyhinton ‘s talk is here:
piped.video/QWWgr2rN45o
8
10
82
12,419
But the real power of seeing is doing. The urge to act is innate for beings with spatial intelligence, which links perception with action. So to advance AI beyond what its capabilities today, we need more than AI that can see or talk. We need AI that can do. 5/
2
16
94
10,311
Three power forces ushered in the Age of Modern AI about a decade ago: A family of algorithms called Neural Network; Fast, specialized hardware called GPUs; Big data, like the collection of 15 million photos my lab spent years curating called ImageNet. 2/
9
29
153
29,358
Some of you can get my book “The Worlds I See: Curiosity, Exploration and Discovery at the Dawn of AI” for $1.99 on Kindle now - it’s a limited promotion! 😍📖
6
12
80
19,265
“Art is the potential within the viewers, within ourselves.” Says @JeffKoons . So is #AI - it’s a tool that can bring out the potential within people, to augment our humanity.
13
17
125
24,929
To help the community navigate the wealth of interconnected information contained in BEHAVIOR-1K’s dataset of 1,000 activities, 50 scenes and 10,000 objects, we also release a web-based knowledgebase browser: 5/n behavior.stanford.edu/knowle…
1
2
1,270
One year ago, we first introduced BEHAVIOR-1K, which we hope will be an important step towards human-centered robotics. After our year-long beta, we’re thrilled to announce its full release, which our team just presented at NVIDIA #GTC2024. 1/n
5
142
685
143,843
I am honored to be invited as the guest of my Congresswoman @RepAnnaEshoo to #SOTU. I'm not only grateful for her hosting me today, but for her leadership on AI in Congress. In particular the CREATE AI Act, a bipartisan bill that will create stronger American public sector AI. 2/
2
9
130
40,951
We owe a debt of gratitude to America's veterans! Properly implemented AI can help advance the healthcare of our veterans. I'm thrilled @SecVetAffairs took the time to meet w/me on behalf of @StanfordHAI and listen to my perspectives on the importance of data.
3
23
163
21,410
Thank you New York friends for supporting my book and a Jersey girl from across the Hudson River 😍🙏 You remind me the human hope and warmth in the era of AI. @ericschmidt @campbell_brown @ariannahuff
2
11
130
26,431
It’s hard to believe tonight was the first time I met @karaswisher in person. But I feel like we are already old friends 😍
12
18
311
113,992
If you want to watch an #AI talk (esp. after some nice #Thanksgiving meal) ranging from the technology itself to its safety, societal and existential issues, I’m proud of this conversation (and friendly debate) I had with @geoffreyhinton 👇 Direct link: piped.video/E14IsFbAbpI?feature…
For more than twenty years, I’ve studied and admired the work of @geoffreyhinton . We’ve had many private conversations. But indeed this is the first time we’ve done a public talk together, thanks to @radicalvcfund @UofT @StanfordHAI . The full talk is now linked below 👇🙏 1/
13
76
346
111,826
Thank you President Obama @BarackObama for the shout-out of my book in your #AI reading list 🤩 barackobama.medium.com/what-…
51
83
593
178,968
The world that I owe my life to is the world of love & support. I give a special thanks to my high school math teacher Bob Sabella &family in New Jersey, who had “shined the light of kindness when I was adrift in a world of strangers” as a 15yo immigrant girl from a world afar 9/
1
3
25
5,626
The world that defines my passion and career is the world of science and technology, in which I have many people to thank 8/
1
3
23
5,677
“This book is about the worlds I see, both physical and intellectual. [...] I want to thank the people who have helped, supported, and loved in all these different dimensions of life.” First, it’s the world of book writing and publishing… 7/
1
3
20
5,993
I’m very grateful to have the endorsements by friends & colleagues such as AI Turing Award winner @geoffreyhinton, Nobel Laureate @doudna_lab, Pixar cofounder @edcatmull, former U.S. Secretary of State @CondoleezzaRice, business leaders @melindagates & @reidhoffman 😍 5/
1
3
37
6,760
“This is no good. Any AI scientist can write this." said John my philosopher friend, “You owe it to all the young ppl of all walks of life out there to tell your story as an AI scientist, a STEM girl, an immigrant, & much more.” Like AI, this book is for all of you.@ai4allorg 4/
1
6
70
8,094
As my awesome book agent @fletcherco says, she “dragged” me into writing the book 3+ years ago, together with my amazing co-writer and friend Alex Michael. But after a year of labor, my colleague and friend John Etchemendy, co-director of @StanfordHAI told us to "re-write"!! 3/
3
3
55
9,476
After 3+ years, today is the day that my book “The Worlds I See” gets to see the world itself. It is a science memoir of the intertwining histories of me becoming an #AI scientist, and the making of the modern AI itself. All versions are now on Amazon a.co/d/fYKf74L 1/
437
576
3,493
1,073,845
Congresswoman, chair of the congressional AI Caucus @RepAnnaEshoo is now here @StanfordHAI discussing w/ us the implications of yesterday’s Executive Order on AI from @POTUS, especially the National AI Research Resource. Panelists include @Rbaltman @kingjen & John Etchemendy 1/2
1
11
43
11,575
I’m so excited that my upcoming AI science memoir book “The Worlds I See” is an Editor’s Pick on Amazon! 🤩 It’s coming out in 10 days on Nov 7, pre-order here: a.co/d/96QLfoE
11
43
315
50,102
AI for Scientific Discovery is one of the most exciting applications of AI! Our GenAI conference is now showcasing such work on climate modeling, sperm whale communications and LLM for protein sequence programing. These speakers are 🤩👇hai.stanford.edu/new-horizon…
4
29
13,057
Three major themes of talks today: GenAI and Scientific Discovery, GenAI and Creativity, GenAI and Society.
1
2
11
11,661
Our fall conference just began with opening remarks by @SuryaGanguli and @percyliang 🤩 It’s livestream now!👇
Generative AI is here – and it’s not just about text and images. Don’t miss the HAI Fall Conference this Tuesday, Oct. 24 as we explore how AI impacts the sciences, human creativity, and our society. Watch the livestream at hai.stanford.edu/events/new-…
ALT New Horizons in Generative AI: Science, Creativity, and Society
3
20
86
40,547
Nothing beats an afternoon with my amazing colleagues and friends @StanfordHAI — @erikbryn @AmyZegart @robreich 😍 AI’s future in the public sector includes doubling down on deepened partnership with and investment in AI+humanities and social sciences.
5
21
215
55,366